蒙特卡洛方法和深度学习的组合最近导致了在高维度中求解部分微分方程(PDE)的有效算法。相关的学习问题通常被称为基于相关随机微分方程(SDE)的变异公式,可以使用基于梯度的优化方法最小化相应损失。因此,在各自的数值实现中,至关重要的是要依靠足够的梯度估计器,这些梯度估计器表现出较低的差异,以便准确,迅速地达到收敛性。在本文中,我们严格研究了在线性Kolmogorov PDE的上下文中出现的相应数值方面。特别是,我们系统地比较了现有的深度学习方法,并为其表演提供了理论解释。随后,我们建议的新方法在理论上和数字上都可以证明更健壮,从而导致了实质性的改进。
translated by 谷歌翻译
求解高维局部微分方程是经济学,科学和工程的反复挑战。近年来,已经开发了大量的计算方法,其中大多数依赖于蒙特卡罗采样和基于深度学习的近似的组合。对于椭圆形和抛物线问题,现有方法可以广泛地分类为依赖于$ \ Texit {向后随机微分方程} $(BSDES)和旨在最小化回归$ L ^ 2 $ -Error( $ \ textit {物理信息的神经网络} $,pinns)。在本文中,我们审查了文献,并提出了一种基于新型$ \ Texit的方法{扩散丢失} $,在BSDES和Pinns之间插值。我们的贡献为对高维PDE的数值方法的统一理解开辟了门,以及结合BSDES和PINNS强度的实施方式。我们还向特征值问题提供概括并进行广泛的数值研究,包括计算非线性SCHR \“odinger运营商的地面状态和分子动态相关的委托功能的计算。
translated by 谷歌翻译
在本文中,我们提出了一种基于深度学习的数值方案,用于强烈耦合FBSDE,这是由随机控制引起的。这是对深度BSDE方法的修改,其中向后方程的初始值不是一个免费参数,并且新的损失函数是控制问题的成本的加权总和,而差异项与与该的差异相吻合终端条件下的平均误差。我们通过一个数值示例表明,经典深度BSDE方法的直接扩展为FBSDE,失败了简单的线性季度控制问题,并激励新方法为何工作。在定期和有限性的假设上,对时间连续和时间离散控制问题的确切控制,我们为我们的方法提供了错误分析。我们从经验上表明,该方法收敛于三个不同的问题,一个方法是直接扩展Deep BSDE方法的问题。
translated by 谷歌翻译
High-dimensional PDEs have been a longstanding computational challenge. We propose to solve highdimensional PDEs by approximating the solution with a deep neural network which is trained to satisfy the differential operator, initial condition, and boundary conditions. Our algorithm is meshfree, which is key since meshes become infeasible in higher dimensions. Instead of forming a mesh, the neural network is trained on batches of randomly sampled time and space points. The algorithm is tested on a class of high-dimensional free boundary PDEs, which we are able to accurately solve in up to 200 dimensions. The algorithm is also tested on a high-dimensional Hamilton-Jacobi-Bellman PDE and Burgers' equation. The deep learning algorithm approximates the general solution to the Burgers' equation for a continuum of different boundary conditions and physical conditions (which can be viewed as a high-dimensional space). We call the algorithm a "Deep Galerkin Method (DGM)" since it is similar in spirit to Galerkin methods, with the solution approximated by a neural network instead of a linear combination of basis functions. In addition, we prove a theorem regarding the approximation power of neural networks for a class of quasilinear parabolic PDEs.
translated by 谷歌翻译
滤波方程控制给定部分,并且可能嘈杂,依次到达的信号过程的条件分布的演变。它们的数值近似在许多真实应用中起着核心作用,包括数字天气预报,金融和工程。近似滤波方程解决方案的一种经典方法是使用由Gyongy,Krylov,Legland,Legland,Legland的PDE启发方法,称为分裂方法,其中包括其他贡献者。该方法和其他基于PDE的方法,具有特别适用性来解决低维问题。在这项工作中,我们将这种方法与神经网络表示相结合。新方法用于产生信号过程的无通知条件分布的近似值。我们进一步开发递归归一化程序,以恢复信号过程的归一化条件分布。新方案可以在多个时间步骤中迭代,同时保持其渐近无偏见属性完整。我们用Kalman和Benes滤波器的数值近似结果测试神经网络近似。
translated by 谷歌翻译
非线性部分差分差异方程成功地用于描述自然科学,工程甚至金融中的广泛时间依赖性现象。例如,在物理系统中,Allen-Cahn方程描述了与相变相关的模式形成。相反,在金融中,黑色 - choles方程描述了衍生投资工具价格的演变。这种现代应用通常需要在经典方法无效的高维度中求解这些方程。最近,E,Han和Jentzen [1] [2]引入了一种有趣的新方法。主要思想是构建一个深网,该网络是根据科尔莫戈罗夫方程式下离散的随机微分方程样本进行训练的。该网络至少能够在数值上近似,在整个空间域中具有多项式复杂性的Kolmogorov方程的解。在这一贡献中,我们通过使用随机微分方程的不同离散方案来研究深网的变体。我们在基准的示例上比较了相关网络的性能,并表明,对于某些离散方案,可以改善准确性,而不会影响观察到的计算复杂性。
translated by 谷歌翻译
连续数据的优化问题出现在,例如强大的机器学习,功能数据分析和变分推理。这里,目标函数被给出为一个(连续)索引目标函数的系列 - 相对于概率测量集成的族聚集。这些问题通常可以通过随机优化方法解决:在随机切换指标执行关于索引目标函数的优化步骤。在这项工作中,我们研究了随机梯度下降算法的连续时间变量,以进行连续数据的优化问题。该所谓的随机梯度过程包括最小化耦合与确定索引的连续时间索引过程的索引目标函数的梯度流程。索引过程是例如,反射扩散,纯跳跃过程或紧凑空间上的其他L evy过程。因此,我们研究了用于连续数据空间的多种采样模式,并允许在算法的运行时进行模拟或流式流的数据。我们分析了随机梯度过程的近似性质,并在恒定下进行了长时间行为和遍历的学习率。我们以噪声功能数据的多项式回归问题以及物理知识的神经网络在多项式回归问题中结束了随机梯度过程的适用性。
translated by 谷歌翻译
本论文主要涉及解决深层(时间)高斯过程(DGP)回归问题的状态空间方法。更具体地,我们代表DGP作为分层组合的随机微分方程(SDES),并且我们通过使用状态空间过滤和平滑方法来解决DGP回归问题。由此产生的状态空间DGP(SS-DGP)模型生成丰富的电视等级,与建模许多不规则信号/功能兼容。此外,由于他们的马尔可道结构,通过使用贝叶斯滤波和平滑方法可以有效地解决SS-DGPS回归问题。本论文的第二次贡献是我们通过使用泰勒力矩膨胀(TME)方法来解决连续离散高斯滤波和平滑问题。这诱导了一类滤波器和SmooThers,其可以渐近地精确地预测随机微分方程(SDES)解决方案的平均值和协方差。此外,TME方法和TME过滤器和SmoOthers兼容模拟SS-DGP并解决其回归问题。最后,本文具有多种状态 - 空间(深)GPS的应用。这些应用主要包括(i)来自部分观察到的轨迹的SDES的未知漂移功能和信号的光谱 - 时间特征估计。
translated by 谷歌翻译
Developing algorithms for solving high-dimensional partial differential equations (PDEs) has been an exceedingly difficult task for a long time, due to the notoriously difficult problem known as the "curse of dimensionality". This paper introduces a deep learning-based approach that can handle general high-dimensional parabolic PDEs. To this end, the PDEs are reformulated using backward stochastic differential equations and the gradient of the unknown solution is approximated by neural networks, very much in the spirit of deep reinforcement learning with the gradient acting as the policy function. Numerical results on examples including the nonlinear Black-Scholes equation, the Hamilton-Jacobi-Bellman equation, and the Allen-Cahn equation suggest that the proposed algorithm is quite effective in high dimensions, in terms of both accuracy and cost. This opens up new possibilities in economics, finance, operational research, and physics, by considering all participating agents, assets, resources, or particles together at the same time, instead of making ad hoc assumptions on their inter-relationships.
translated by 谷歌翻译
计算科学和统计推断中的许多应用都需要计算有关具有未知归一化常数的复杂高维分布以及这些常数的估计。在这里,我们开发了一种基于从简单的基本分布生成样品,沿着速度场生成的流量运输的方法,并沿这些流程线执行平均值。这种非平衡重要性采样(NEIS)策略是直接实施的,可用于具有任意目标分布的计算。在理论方面,我们讨论了如何将速度场定制到目标,并建立所提出的估计器是一个完美的估计器,具有零变化。我们还通过将基本分布映射到目标上,通过传输图绘制了NEIS和方法之间的连接。在计算方面,我们展示了如何使用深度学习来代表神经网络,并将其训练为零方差最佳。这些结果在高维示例上进行了数值说明,我们表明训练速度场可以将NEIS估计量的方差降低至6个数量级,而不是Vanilla估计量。我们还表明,NEIS在这些示例上的表现要比NEAL的退火重要性采样(AIS)更好。
translated by 谷歌翻译
在本文中,我们研究了针对泊松方程的解决方案的概率和神经网络近似,但在$ \ mathbb {r}^d $的一般边界域中,较旧或$ c^2 $数据。我们的目标是两个基本目标。首先,也是最重要的是,我们证明了泊松方程的解决方案可以通过蒙特卡洛方法在sup-norm中进行数值近似,但基于球形算法的步行略有变化。这提供了相对于相对于相对于相对于有效的估计值规定的近似误差且没有维度的诅咒。此外,样品的总数不取决于执行近似的点。作为第二个目标,我们表明获得的蒙特卡洛求解器renders relu relu深层神经网络(DNN)解决泊松问题的解决方案,其大小在尺寸$ d $以及所需的错误中大多数取决于多项式。和低多项式复杂性。
translated by 谷歌翻译
The purpose of this paper is to explore the use of deep learning for the solution of the nonlinear filtering problem. This is achieved by solving the Zakai equation by a deep splitting method, previously developed for approximate solution of (stochastic) partial differential equations. This is combined with an energy-based model for the approximation of functions by a deep neural network. This results in a computationally fast filter that takes observations as input and that does not require re-training when new observations are received. The method is tested on four examples, two linear in one and twenty dimensions and two nonlinear in one dimension. The method shows promising performance when benchmarked against the Kalman filter and the bootstrap particle filter.
translated by 谷歌翻译
连续的时间加强学习提供了一种吸引人的形式主义,用于描述控制问题,其中时间的流逝并不自然地分为离散的增量。在这里,我们考虑了预测在连续时间随机环境中相互作用的代理商获得的回报分布的问题。准确的回报预测已被证明可用于确定对风险敏感的控制,学习状态表示,多基因协调等的最佳策略。我们首先要建立汉密尔顿 - 雅各布人(HJB)方程的分布模拟,以扩散和更广泛的feller-dynkin过程。然后,我们将此方程式专注于返回分布近似于$ n $均匀加权粒子的设置,这是分销算法中常见的设计选择。我们的派生突出显示了由于统计扩散率而引起的其他术语,这是由于在连续时间设置中正确处理分布而产生的。基于此,我们提出了一种可访问算法,用于基于JKO方案近似求解分布HJB,该方案可以在在线控制算法中实现。我们证明了这种算法在合成控制问题中的有效性。
translated by 谷歌翻译
本文研究了使用神经跳跃(NJ-ODE)框架扩展的一般随机过程的问题。虽然NJ-ODE是为预测不规则观察到的时间序列而建立收敛保证的第一个框架,但这些结果仅限于从中\^o-diffusions的数据,特别是Markov过程,特别是在其中同时观察到所有坐标。。在这项工作中,我们通过利用签名变换的重建属性,将这些结果推广到具有不完整观察结果的通用,可能是非马克维亚或不连续的随机过程。这些理论结果得到了经验研究的支持,在该研究中,在非马克维亚数据的情况下,依赖路径依赖性的NJ-ode优于原始的NJ-ode框架。
translated by 谷歌翻译
The Physics-Informed Neural Network (PINN) approach is a new and promising way to solve partial differential equations using deep learning. The $L^2$ Physics-Informed Loss is the de-facto standard in training Physics-Informed Neural Networks. In this paper, we challenge this common practice by investigating the relationship between the loss function and the approximation quality of the learned solution. In particular, we leverage the concept of stability in the literature of partial differential equation to study the asymptotic behavior of the learned solution as the loss approaches zero. With this concept, we study an important class of high-dimensional non-linear PDEs in optimal control, the Hamilton-Jacobi-Bellman(HJB) Equation, and prove that for general $L^p$ Physics-Informed Loss, a wide class of HJB equation is stable only if $p$ is sufficiently large. Therefore, the commonly used $L^2$ loss is not suitable for training PINN on those equations, while $L^{\infty}$ loss is a better choice. Based on the theoretical insight, we develop a novel PINN training algorithm to minimize the $L^{\infty}$ loss for HJB equations which is in a similar spirit to adversarial training. The effectiveness of the proposed algorithm is empirically demonstrated through experiments. Our code is released at https://github.com/LithiumDA/L_inf-PINN.
translated by 谷歌翻译
神经网络的经典发展主要集中在有限维欧基德空间或有限组之间的学习映射。我们提出了神经网络的概括,以学习映射无限尺寸函数空间之间的运算符。我们通过一类线性积分运算符和非线性激活函数的组成制定运营商的近似,使得组合的操作员可以近似复杂的非线性运算符。我们证明了我们建筑的普遍近似定理。此外,我们介绍了四类运算符参数化:基于图形的运算符,低秩运算符,基于多极图形的运算符和傅里叶运算符,并描述了每个用于用每个计算的高效算法。所提出的神经运营商是决议不变的:它们在底层函数空间的不同离散化之间共享相同的网络参数,并且可以用于零击超分辨率。在数值上,与现有的基于机器学习的方法,达西流程和Navier-Stokes方程相比,所提出的模型显示出卓越的性能,而与传统的PDE求解器相比,与现有的基于机器学习的方法有关的基于机器学习的方法。
translated by 谷歌翻译
Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics, and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than classical feed-forward neural networks, recurrent neural networks, and convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering, In this work, we review such methods and extend them for parametric studies as well as for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.
translated by 谷歌翻译
Linear partial differential equations (PDEs) are an important, widely applied class of mechanistic models, describing physical processes such as heat transfer, electromagnetism, and wave propagation. In practice, specialized numerical methods based on discretization are used to solve PDEs. They generally use an estimate of the unknown model parameters and, if available, physical measurements for initialization. Such solvers are often embedded into larger scientific models or analyses with a downstream application such that error quantification plays a key role. However, by entirely ignoring parameter and measurement uncertainty, classical PDE solvers may fail to produce consistent estimates of their inherent approximation error. In this work, we approach this problem in a principled fashion by interpreting solving linear PDEs as physics-informed Gaussian process (GP) regression. Our framework is based on a key generalization of a widely-applied theorem for conditioning GPs on a finite number of direct observations to observations made via an arbitrary bounded linear operator. Crucially, this probabilistic viewpoint allows to (1) quantify the inherent discretization error; (2) propagate uncertainty about the model parameters to the solution; and (3) condition on noisy measurements. Demonstrating the strength of this formulation, we prove that it strictly generalizes methods of weighted residuals, a central class of PDE solvers including collocation, finite volume, pseudospectral, and (generalized) Galerkin methods such as finite element and spectral methods. This class can thus be directly equipped with a structured error estimate and the capability to incorporate uncertain model parameters and observations. In summary, our results enable the seamless integration of mechanistic models as modular building blocks into probabilistic models.
translated by 谷歌翻译
在这项工作中,我们提出了一种基于深度学习的新方案,用于解决高维非线性后向随机微分方程(BSDES)。这个想法是将问题重新重新制定为包括本地损失功能的全球优化。本质上,我们使用深神网络及其具有自动分化的梯度近似BSDE的未知解。通过在每个时间步骤定义的二次局部损耗函数中最小化近似值来执行近似值,该局部损失函数始终包括终端条件。这种损失函数是通过用终端条件迭代时间积分的Euler离散化来获得的。我们的公式可以促使随机梯度下降算法不仅要考虑到每个时间层的准确性,而且会收敛到良好的局部最小值。为了证明我们的算法的性能,提供了几种高维非线性BSDE,包括金融中的定价问题。
translated by 谷歌翻译
我们提出了对使用Rademacher和Vapnik-Chervonenkis边界学习有条件的价值(VAR)和预期短缺的两步方法的非反应收敛分析。我们的VAR方法扩展到了一次学习的问题,该问题对应于不同的分数水平。这导致基于神经网络分位数和最小二乘回归的有效学习方案。引入了一个后验蒙特卡洛(非巢)程序,以估计地面真相和ES的距离,而无需访问后者。使用高斯玩具模型中的数值实验和财务案例研究中的目标是学习动态初始边缘的情况。
translated by 谷歌翻译