Stochastic Gradient Descent Langevin Dynamics (SGLD) algorithms, which add noise to the classic gradient descent, are known to improve the training of neural networks in some cases where the neural network is very deep. In this paper we study the possibilities of training acceleration for the numerical resolution of stochastic control problems through gradient descent, where the control is parametrized by a neural network. If the control is applied at many discretization times then solving the stochastic control problem reduces to minimizing the loss of a very deep neural network. We numerically show that Langevin algorithms improve the training on various stochastic control problems like hedging and resource management, and for different choices of gradient descent methods.

Training a very deep neural network is a challenging task, as the deeper a neural network is, the more non-linear it is. We compare the performances of various preconditioned Langevin algorithms with their non-Langevin counterparts for the training of neural networks of increasing depth. For shallow neural networks, Langevin algorithms do not lead to any improvement, however the deeper the network is and the greater are the gains provided by Langevin algorithms. Adding noise to the gradient descent allows to escape from local traps, which are more frequent for very deep neural networks. Following this heuristic we introduce a new Langevin algorithm called Layer Langevin, which consists in adding Langevin noise only to the weights associated to the deepest layers. We then prove the benefits of Langevin and Layer Langevin algorithms for the training of popular deep residual architectures for image classification.

在本文中，我们主要专注于用边界条件求解高维随机汉密尔顿系统，并从随机对照的角度提出一种新的方法。为了获得哈密顿系统的近似解，我们首先引入了一个相应的随机最佳控制问题，使得汉密尔顿控制问题的系统正是我们需要解决的，然后开发两种不同的算法适合不同的控制问题。深神经网络近似随机控制。从数值结果中，与先前从求解FBSDES开发的深度FBSDE方法相比，新颖的算法会聚得更快，这意味着它们需要更少的训练步骤，并展示不同哈密顿系统的更稳定的收敛。

在本文中，我们提出了一种基于深度学习的数值方案，用于强烈耦合FBSDE，这是由随机控制引起的。这是对深度BSDE方法的修改，其中向后方程的初始值不是一个免费参数，并且新的损失函数是控制问题的成本的加权总和，而差异项与与该的差异相吻合终端条件下的平均误差。我们通过一个数值示例表明，经典深度BSDE方法的直接扩展为FBSDE，失败了简单的线性季度控制问题，并激励新方法为何工作。在定期和有限性的假设上，对时间连续和时间离散控制问题的确切控制，我们为我们的方法提供了错误分析。我们从经验上表明，该方法收敛于三个不同的问题，一个方法是直接扩展Deep BSDE方法的问题。

This paper is devoted to the numerical resolution of McKean-Vlasov control problems via the class of mean-field neural networks introduced in our companion paper [25] in order to learn the solution on the Wasserstein space. We propose several algorithms either based on dynamic programming with control learning by policy or value iteration, or backward SDE from stochastic maximum principle with global or local loss functions. Extensive numerical results on different examples are presented to illustrate the accuracy of each of our eight algorithms. We discuss and compare the pros and cons of all the tested methods.

Developing algorithms for solving high-dimensional partial differential equations (PDEs) has been an exceedingly difficult task for a long time, due to the notoriously difficult problem known as the "curse of dimensionality". This paper introduces a deep learning-based approach that can handle general high-dimensional parabolic PDEs. To this end, the PDEs are reformulated using backward stochastic differential equations and the gradient of the unknown solution is approximated by neural networks, very much in the spirit of deep reinforcement learning with the gradient acting as the policy function. Numerical results on examples including the nonlinear Black-Scholes equation, the Hamilton-Jacobi-Bellman equation, and the Allen-Cahn equation suggest that the proposed algorithm is quite effective in high dimensions, in terms of both accuracy and cost. This opens up new possibilities in economics, finance, operational research, and physics, by considering all participating agents, assets, resources, or particles together at the same time, instead of making ad hoc assumptions on their inter-relationships.

已知生成对抗网络（GANS）的培训以难以收敛。本文首先确认了这一收敛问题背后的罪魁祸首之一：缺乏凸起的GANS目标功能，因此GANS模型的良好问题。然后，它提出了一种随机控制方法，用于GAN训练中的超参数调整。In particular, it presents an optimal solution for adaptive learning rate which depends on the convexity of the objective function, and builds a precise relation between improper choices of learning rate and explosion in GANs training.最后，经验研究表明，培训算法包含这种选择方法优于标准的训练算法。

The optimal stopping problem is one of the core problems in financial markets, with broad applications such as pricing American and Bermudan options. The deep BSDE method [Han, Jentzen and E, PNAS, 115(34):8505-8510, 2018] has shown great power in solving high-dimensional forward-backward stochastic differential equations (FBSDEs), and inspired many applications. However, the method solves backward stochastic differential equations (BSDEs) in a forward manner, which can not be used for optimal stopping problems that in general require running BSDE backwardly. To overcome this difficulty, a recent paper [Wang, Chen, Sudjianto, Liu and Shen, arXiv:1807.06622, 2018] proposed the backward deep BSDE method to solve the optimal stopping problem. In this paper, we provide the rigorous theory for the backward deep BSDE method. Specifically, 1. We derive the a posteriori error estimation, i.e., the error of the numerical solution can be bounded by the training loss function; and; 2. We give an upper bound of the loss function, which can be sufficiently small subject to universal approximations. We give two numerical examples, which present consistent performance with the proved theory.

连续数据的优化问题出现在，例如强大的机器学习，功能数据分析和变分推理。这里，目标函数被给出为一个（连续）索引目标函数的系列 - 相对于概率测量集成的族聚集。这些问题通常可以通过随机优化方法解决：在随机切换指标执行关于索引目标函数的优化步骤。在这项工作中，我们研究了随机梯度下降算法的连续时间变量，以进行连续数据的优化问题。该所谓的随机梯度过程包括最小化耦合与确定索引的连续时间索引过程的索引目标函数的梯度流程。索引过程是例如，反射扩散，纯跳跃过程或紧凑空间上的其他L evy过程。因此，我们研究了用于连续数据空间的多种采样模式，并允许在算法的运行时进行模拟或流式流的数据。我们分析了随机梯度过程的近似性质，并在恒定下进行了长时间行为和遍历的学习率。我们以噪声功能数据的多项式回归问题以及物理知识的神经网络在多项式回归问题中结束了随机梯度过程的适用性。

我们提出了一种深层签名/对数符号FBSDE算法，以求解具有状态和路径依赖性特征的前回向随机微分方程（FBSDE）。通过将深度签名/对数签名转换纳入复发性神经网络（RNN）模型，我们的算法缩短了训练时间，提高了准确性，并扩展了与现有文献中方法相比的时间范围。此外，我们的算法可以应用于涉及高频数据，模型歧义和随机游戏等广泛的应用程序和路径依赖的选项定价，这些定价与抛物线偏差方程（PDES）以及路径依赖性依赖性链接有关PDE（PPDE）。最后，我们还得出了深度签名/对数签名FBSDE算法的收敛分析。

定量金融中经典问题的许多现代计算方法被提出为经验损失最小化（ERM），从而可以直接应用统计机器学习的经典结果。这些方法旨在直接构建对冲或投资决策的最佳反馈表示，在此框架中分析了它们的有效性以及它们对概括错误的敏感性。使用古典技术表明，过度训练的渲染仪训练有素的投资决策成为预期，并证明了大型假设空间的过度学习。另一方面，基于Rademacher复杂性的非反应估计显示了足够大的训练集的收敛性。这些结果强调了合成数据生成的重要性以及复杂模型对市场数据的适当校准。一个数值研究的风格化示例说明了这些可能性，包括问题维度在过度学习程度上的重要性以及该方法的有效性。

We investigate the asymptotic properties of deep Residual networks (ResNets) as the number of layers increases. We first show the existence of scaling regimes for trained weights markedly different from those implicitly assumed in the neural ODE literature. We study the convergence of the hidden state dynamics in these scaling regimes, showing that one may obtain an ODE, a stochastic differential equation (SDE) or neither of these. In particular, our findings point to the existence of a diffusive regime in which the deep network limit is described by a class of stochastic differential equations (SDEs). Finally, we derive the corresponding scaling limits for the backpropagation dynamics.

蒙特卡洛方法和深度学习的组合最近导致了在高维度中求解部分微分方程（PDE）的有效算法。相关的学习问题通常被称为基于相关随机微分方程（SDE）的变异公式，可以使用基于梯度的优化方法最小化相应损失。因此，在各自的数值实现中，至关重要的是要依靠足够的梯度估计器，这些梯度估计器表现出较低的差异，以便准确，迅速地达到收敛性。在本文中，我们严格研究了在线性Kolmogorov PDE的上下文中出现的相应数值方面。特别是，我们系统地比较了现有的深度学习方法，并为其表演提供了理论解释。随后，我们建议的新方法在理论上和数字上都可以证明更健壮，从而导致了实质性的改进。

In this thesis, we consider two simple but typical control problems and apply deep reinforcement learning to them, i.e., to cool and control a particle which is subject to continuous position measurement in a one-dimensional quadratic potential or in a quartic potential. We compare the performance of reinforcement learning control and conventional control strategies on the two problems, and show that the reinforcement learning achieves a performance comparable to the optimal control for the quadratic case, and outperforms conventional control strategies for the quartic case for which the optimal control strategy is unknown. To our knowledge, this is the first time deep reinforcement learning is applied to quantum control problems in continuous real space. Our research demonstrates that deep reinforcement learning can be used to control a stochastic quantum system in real space effectively as a measurement-feedback closed-loop controller, and our research also shows the ability of AI to discover new control strategies and properties of the quantum systems that are not well understood, and we can gain insights into these problems by learning from the AI, which opens up a new regime for scientific research.

非线性部分差分差异方程成功地用于描述自然科学，工程甚至金融中的广泛时间依赖性现象。例如，在物理系统中，Allen-Cahn方程描述了与相变相关的模式形成。相反，在金融中，黑色 - choles方程描述了衍生投资工具价格的演变。这种现代应用通常需要在经典方法无效的高维度中求解这些方程。最近，E，Han和Jentzen [1] [2]引入了一种有趣的新方法。主要思想是构建一个深网，该网络是根据科尔莫戈罗夫方程式下离散的随机微分方程样本进行训练的。该网络至少能够在数值上近似，在整个空间域中具有多项式复杂性的Kolmogorov方程的解。在这一贡献中，我们通过使用随机微分方程的不同离散方案来研究深网的变体。我们在基准的示例上比较了相关网络的性能，并表明，对于某些离散方案，可以改善准确性，而不会影响观察到的计算复杂性。

我们为可交易仪器的市场模拟器提供了一种数值有效的方法，用于学习最少的等效鞅措施，例如，可交易仪器的市场模拟器。出于在同一底层写入的现货价格和选择。在存在交易成本和交易限制的情况下，我们放松了对学习最低等同的“近马丁措施”的结果，其中预期的回报仍然存在于普遍的出价/询问差价中。我们在高维复杂空间中“去除漂移”的方法完全是无模型的，并且可以应用于任何不展示经典套用的市场模拟器。所产生的模型可用于风险中性定价，或者在交易成本或交易限制的情况下，“深度套期保值”。我们通过将其应用于两个市场模拟器，自动回归离散时间随机隐含的波动率模型和基于生成的对冲网络（GAN）的模拟器来展示我们的方法，这些模拟器都在统计测量下的选项价格的历史数据上培训产生现货和期权价格的现实样本。关于原始市场模拟器的估计误差，我们评论了鲁棒性。

我们以封闭的形式分析了随机梯度下降（SGD）的学习动态，用于分类每个群集的高位高斯混合的单层神经网络，其中每个群集分配两个标签中的一个。该问题提供了具有内插制度的非凸损景观的原型和大的概括间隙。我们定义了一个特定的随机过程，其中SGD可以扩展到我们称呼随机梯度流的连续时间限制。在全批处理中，我们恢复标准梯度流。我们将动态平均场理论从统计物理应用于通过自成的随机过程跟踪高维极限中算法的动态。我们探讨了算法的性能，作为控制参数脱落灯的函数，它如何导航损耗横向。

Conventional wisdom in deep learning states that increasing depth improves expressiveness but complicates optimization. This paper suggests that, sometimes, increasing depth can speed up optimization. The effect of depth on optimization is decoupled from expressiveness by focusing on settings where additional layers amount to overparameterization -linear neural networks, a wellstudied model. Theoretical analysis, as well as experiments, show that here depth acts as a preconditioner which may accelerate convergence. Even on simple convex problems such as linear regression with p loss, p > 2, gradient descent can benefit from transitioning to a non-convex overparameterized objective, more than it would from some common acceleration schemes. We also prove that it is mathematically impossible to obtain the acceleration effect of overparametrization via gradients of any regularizer.

计算科学和统计推断中的许多应用都需要计算有关具有未知归一化常数的复杂高维分布以及这些常数的估计。在这里，我们开发了一种基于从简单的基本分布生成样品，沿着速度场生成的流量运输的方法，并沿这些流程线执行平均值。这种非平衡重要性采样（NEIS）策略是直接实施的，可用于具有任意目标分布的计算。在理论方面，我们讨论了如何将速度场定制到目标，并建立所提出的估计器是一个完美的估计器，具有零变化。我们还通过将基本分布映射到目标上，通过传输图绘制了NEIS和方法之间的连接。在计算方面，我们展示了如何使用深度学习来代表神经网络，并将其训练为零方差最佳。这些结果在高维示例上进行了数值说明，我们表明训练速度场可以将NEIS估计量的方差降低至6个数量级，而不是Vanilla估计量。我们还表明，NEIS在这些示例上的表现要比NEAL的退火重要性采样（AIS）更好。

High-dimensional PDEs have been a longstanding computational challenge. We propose to solve highdimensional PDEs by approximating the solution with a deep neural network which is trained to satisfy the differential operator, initial condition, and boundary conditions. Our algorithm is meshfree, which is key since meshes become infeasible in higher dimensions. Instead of forming a mesh, the neural network is trained on batches of randomly sampled time and space points. The algorithm is tested on a class of high-dimensional free boundary PDEs, which we are able to accurately solve in up to 200 dimensions. The algorithm is also tested on a high-dimensional Hamilton-Jacobi-Bellman PDE and Burgers' equation. The deep learning algorithm approximates the general solution to the Burgers' equation for a continuum of different boundary conditions and physical conditions (which can be viewed as a high-dimensional space). We call the algorithm a "Deep Galerkin Method (DGM)" since it is similar in spirit to Galerkin methods, with the solution approximated by a neural network instead of a linear combination of basis functions. In addition, we prove a theorem regarding the approximation power of neural networks for a class of quasilinear parabolic PDEs.