这项工作与发现物理系统的偏微分方程（PDE）有关。现有方法证明了有限观察结果的PDE识别，但未能保持令人满意的噪声性能，部分原因是由于次优估计衍生物并发现了PDE系数。我们通过引入噪音吸引物理学的机器学习（NPIML）框架来解决问题，以在任意分布后从数据中发现管理PDE。我们的建议是双重的。首先，我们提出了几个神经网络，即求解器和预选者，这些神经网络对隐藏的物理约束产生了可解释的神经表示。在经过联合训练之后，求解器网络将近似潜在的候选物，例如部分衍生物，然后将其馈送到稀疏的回归算法中，该算法最初公布了最有可能的PERSIMISIAL PDE，根据信息标准决定。其次，我们提出了基于离散的傅立叶变换（DFT）的Denoising物理信息信息网络（DPINNS），以提供一组最佳的鉴定PDE系数，以符合降低降噪变量。 Denoising Pinns的结构被划分为前沿投影网络和PINN，以前学到的求解器初始化。我们对五个规范PDE的广泛实验确认，该拟议框架为PDE发现提供了一种可靠，可解释的方法，适用于广泛的系统，可能会因噪声而复杂。

数据驱动的PDE的发现最近取得了巨大进展，许多规范的PDE已成功地发现了概念验证。但是，在没有事先参考的情况下，确定最合适的PDE在实际应用方面仍然具有挑战性。在这项工作中，提出了物理信息的信息标准（PIC），以合成发现的PDE的简约和精度。所提出的PIC可在不同的物理场景中七个规范的PDE上获得最新的鲁棒性，并稀疏的数据，这证实了其处理困难情况的能力。该图片还用于从实际的物理场景中从微观模拟数据中发现未开采的宏观管理方程。结果表明，发现的宏观PDE精确且简约，并满足基础的对称性，从而有助于对物理过程的理解和模拟。 PIC的命题可以在发现更广泛的物理场景中发现未透视的管理方程式中PDE发现的实际应用。

物理信息的神经网络（PINN）是神经网络（NNS），它们作为神经网络本身的组成部分编码模型方程，例如部分微分方程（PDE）。如今，PINN是用于求解PDE，分数方程，积分分化方程和随机PDE的。这种新颖的方法已成为一个多任务学习框架，在该框架中，NN必须在减少PDE残差的同时拟合观察到的数据。本文对PINNS的文献进行了全面的综述：虽然该研究的主要目标是表征这些网络及其相关的优势和缺点。该综述还试图将出版物纳入更广泛的基于搭配的物理知识的神经网络，这些神经网络构成了香草·皮恩（Vanilla Pinn）以及许多其他变体，例如物理受限的神经网络（PCNN），各种HP-VPINN，变量HP-VPINN，VPINN，VPINN，变体。和保守的Pinn（CPINN）。该研究表明，大多数研究都集中在通过不同的激活功能，梯度优化技术，神经网络结构和损耗功能结构来定制PINN。尽管使用PINN的应用范围广泛，但通过证明其在某些情况下比有限元方法（FEM）等经典数值技术更可行的能力，但仍有可能的进步，最著名的是尚未解决的理论问题。

In this paper, we introduce PDE-LEARN, a novel PDE discovery algorithm that can identify governing partial differential equations (PDEs) directly from noisy, limited measurements of a physical system of interest. PDE-LEARN uses a Rational Neural Network, $U$, to approximate the system response function and a sparse, trainable vector, $\xi$, to characterize the hidden PDE that the system response function satisfies. Our approach couples the training of $U$ and $\xi$ using a loss function that (1) makes $U$ approximate the system response function, (2) encapsulates the fact that $U$ satisfies a hidden PDE that $\xi$ characterizes, and (3) promotes sparsity in $\xi$ using ideas from iteratively reweighted least-squares. Further, PDE-LEARN can simultaneously learn from several data sets, allowing it to incorporate results from multiple experiments. This approach yields a robust algorithm to discover PDEs directly from realistic scientific data. We demonstrate the efficacy of PDE-LEARN by identifying several PDEs from noisy and limited measurements.

PDE发现显示了揭示复杂物理系统的预测模型，但在测量稀疏和嘈杂时难以困难。我们介绍了一种新方法，用于PDE发现，它使用两个合理的神经网络和原始的稀疏回归算法来识别管理系统响应的隐藏动态。第一网络了解系统响应函数，而第二个网络了解一个驱动系统演进的隐藏PDE。然后，我们使用无参数稀疏回归算法从第二网络中提取隐藏PDE的人类可读形式。我们在名为PDE-读取的开源库中实现了我们的方法。我们的方法成功地识别了热，汉堡和KorteDeg-de Vries方程，具有显着的一致性。我们表明，我们的方法对稀疏性和噪音都是前所未有的强大，因此适用于现实世界的观察数据。

Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics, and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than classical feed-forward neural networks, recurrent neural networks, and convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering, In this work, we review such methods and extend them for parametric studies as well as for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.

Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Here, we present an overview of physics-informed neural networks (PINNs), which embed a PDE into the loss of the neural network using automatic differentiation. The PINN algorithm is simple, and it can be applied to different types of PDEs, including integro-differential equations, fractional PDEs, and stochastic PDEs. Moreover, from the implementation point of view, PINNs solve inverse problems as easily as forward problems. We propose a new residual-based adaptive refinement (RAR) method to improve the training efficiency of PINNs. For pedagogical reasons, we compare the PINN algorithm to a standard finite element method. We also present a Python library for PINNs, DeepXDE, which is designed to serve both as an education tool to be used in the classroom as well as a research tool for solving problems in computational science and engineering. Specifically, DeepXDE can solve forward problems given initial and boundary conditions, as well as inverse problems given some extra measurements. DeepXDE supports complex-geometry domains based on the technique of constructive solid geometry, and enables the user code to be compact, resembling closely the mathematical formulation. We introduce the usage of DeepXDE and its customizability, and we also demonstrate the capability of PINNs and the user-friendliness of DeepXDE for five different examples. More broadly, DeepXDE contributes to the more rapid development of the emerging Scientific Machine Learning field.

拟合科学数据的部分微分方程（PDE）可以用可解释的机制来代表各种以数学为导向的受试者的物理定律。从科学数据中发现PDE的数据驱动的发现蓬勃发展，作为对自然界中复杂现象进行建模的新尝试，但是当前实践的有效性通常受数据的稀缺性和现象的复杂性的限制。尤其是，从低质量数据中发现具有高度非线性系数的PDE在很大程度上已经不足。为了应对这一挑战，我们提出了一种新颖的物理学指导学习方法，该方法不仅可以编码观察知识，例如初始和边界条件，而且还包含了基本的物理原理和法律来指导模型优化。我们从经验上证明，所提出的方法对数据噪声和稀疏性更为强大，并且可以将估计误差较大。此外，我们第一次能够发现具有高度非线性系数的PDE。凭借有希望的性能，提出的方法推动了PDE的边界，这可以通过机器学习模型来进行科学发现。

Physics-Informed Neural Networks (PINN) are algorithms from deep learning leveraging physical laws by including partial differential equations together with a respective set of boundary and initial conditions as penalty terms into their loss function. In this work, we observe the significant role of correctly weighting the combination of multiple competitive loss functions for training PINNs effectively. To this end, we implement and evaluate different methods aiming at balancing the contributions of multiple terms of the PINNs loss function and their gradients. After reviewing of three existing loss scaling approaches (Learning Rate Annealing, GradNorm and SoftAdapt), we propose a novel self-adaptive loss balancing scheme for PINNs named \emph{ReLoBRaLo} (Relative Loss Balancing with Random Lookback). We extensively evaluate the performance of the aforementioned balancing schemes by solving both forward as well as inverse problems on three benchmark PDEs for PINNs: Burgers' equation, Kirchhoff's plate bending equation and Helmholtz's equation. The results show that ReLoBRaLo is able to consistently outperform the baseline of existing scaling methods in terms of accuracy, while also inducing significantly less computational overhead.

两个不混溶的流体的位移是多孔介质中流体流动的常见问题。这种问题可以作为局部微分方程（PDE）构成通常被称为Buckley-Leverett（B-L）问题。 B-L问题是一种非线性双曲守护法，众所周知，使用传统的数值方法难以解决。在这里，我们使用物理信息的神经网络（Pinns）使用非凸版通量函数来解决前向双曲线B-L问题。本文的贡献是双重的。首先，我们通过将Oleinik熵条件嵌入神经网络残差来提出一种Pinn方法来解决双曲线B-L问题。我们不使用扩散术语（人工粘度）在残留损失中，但我们依靠PDE的强形式。其次，我们使用ADAM优化器与基于残留的自适应细化（RAR）算法，实现不加权的超低损耗。我们的解决方案方法可以精确地捕获冲击前并产生精确的整体解决方案。我们报告了一个2 x 10-2的L2验证误差和1x 10-6的L2损耗。所提出的方法不需要任何额外的正则化或加权损失以获得这种准确的解决方案。

在本文中，我们提出了用于求解非线性微分方程（NDE）的神经网络的物理知情训练（PIAT）。众所周知，神经网络的标准培训会导致非平滑函数。对抗训练（AT）是针对对抗攻击的既定防御机制，这也可能有助于使解决方案平滑。 AT包括通过扰动增强训练迷你批量，使网络输出不匹配所需的输出对手。与正式AT仅依靠培训数据不同，在这里，我们使用对抗网络体系结构中的自动差异来以非线性微分方程的形式编码管理物理定律。我们将PIAT与PIAT进行了比较，以指示我们方法在求解多达10个维度方面的有效性。此外，我们提出了重量衰减和高斯平滑，以证明PIAT的优势。代码存储库可从https://github.com/rohban-lab/piat获得。

这本数字本书包含在物理模拟的背景下与深度学习相关的一切实际和全面的一切。尽可能多，所有主题都带有Jupyter笔记本的形式的动手代码示例，以便快速入门。除了标准的受监督学习的数据中，我们将看看物理丢失约束，更紧密耦合的学习算法，具有可微分的模拟，以及加强学习和不确定性建模。我们生活在令人兴奋的时期：这些方法具有从根本上改变计算机模拟可以实现的巨大潜力。

科学机器学习已成功应用于计算物理中的逆问题和PDE发现。一个警告有关当前方法的需要是需要大量的（“清洁”）数据，以表征完整的系统响应并发现底层物理模型。贝叶斯方法可能特别有希望克服这些挑战，因为它们对稀疏和嘈杂数据的负面影响自然敏感。在本文中，我们建议使用贝叶斯神经网络（BNN），以便：1）从测量数据（例如，温度，速度场等）恢复完整的系统状态。我们使用Hamiltonian Monte-Carlo来对深层和致密的BNN的后部分布进行样本，并表明可以精确地捕获不同复杂性的物理学，而不会过度拟合。 2）恢复实例化管理物理系统的底层部分微分方程（PDE）的参数。使用训练的BNN作为系统响应的代理，我们生成可能包括控制观察到的系统的潜在PDE的衍生物的数据集，然后在空间和时间的连续衍生物之间执行顺序阈值贝叶斯线性回归（StBLR） ，恢复原始PDE参数。我们利用了BNN输出内的置信区间，并将空间衍生物累积方差引入了Stblr可能性，以减轻高度不确定的衍生数据点的影响;因此，允许更准确的参数发现。我们在应用物理和非线性动力学中逐渐展示了我们的方法。

我们考虑从高噪声限制的时间序列数据中控制方程的数据驱动发现。该算法开发描述了在非线性动力学（SINDY）框架的稀疏识别的背景下避免噪声的广泛影响的方法的广泛工具包。我们提供了两个主要贡献，都集中在系统x'= f（x）中获取的嘈杂数据。首先，我们提出用于高噪声设置的广泛工具包，这是一个批判性的回归方法的扩展，从完整的库中逐步剔除剔除功能，并产生一组稀疏方程，其回归到衍生x' 。这些创新可以从高噪声时间序列数据中提取稀疏控制方程和系数（例如，增加噪声300％）。例如，它发现洛伦茨系统中的正确稀疏文库，中值系数估计误差等于1％ - 3％（50％噪声），6％ - 8％（100％噪声）;和23％ - 25％（噪音300％）。工具包中的启用模块组合成单个方法，但各个模块可以在其他方程发现方法（Sindy或不）中进行战术，以改善高噪声数据的结果。其次，我们提出了一种技术，适用于基于X'= F（X）的任何模型发现方法，以评估由于噪声数据而在非唯一解决方案的上下文中发现模型的准确性。目前，这种非唯一性可以模糊发现模型的准确性，从而造成发现方法的有效性。我们描述了一种使用线性依赖性的技术，该技术将发现的模型转换为最接近真实模型的等效形式，从而能够更准确地评估发现的模型的准确性。

在科学的背景下，众所周知的格言“一张图片胜过千言万语”可能是“一个型号胜过一千个数据集”。在本手稿中，我们将Sciml软件生态系统介绍作为混合物理法律和科学模型的信息，并使用数据驱动的机器学习方法。我们描述了一个数学对象，我们表示通用微分方程（UDE），作为连接生态系统的统一框架。我们展示了各种各样的应用程序，从自动发现解决高维汉密尔顿 - Jacobi-Bellman方程的生物机制，可以通过UDE形式主义和工具进行措辞和有效地处理。我们展示了软件工具的一般性，以处理随机性，延迟和隐式约束。这使得各种SCIML应用程序变为核心训练机构的核心集，这些训练机构高度优化，稳定硬化方程，并与分布式并行性和GPU加速器兼容。

These notes were compiled as lecture notes for a course developed and taught at the University of the Southern California. They should be accessible to a typical engineering graduate student with a strong background in Applied Mathematics. The main objective of these notes is to introduce a student who is familiar with concepts in linear algebra and partial differential equations to select topics in deep learning. These lecture notes exploit the strong connections between deep learning algorithms and the more conventional techniques of computational physics to achieve two goals. First, they use concepts from computational physics to develop an understanding of deep learning algorithms. Not surprisingly, many concepts in deep learning can be connected to similar concepts in computational physics, and one can utilize this connection to better understand these algorithms. Second, several novel deep learning algorithms can be used to solve challenging problems in computational physics. Thus, they offer someone who is interested in modeling a physical phenomena with a complementary set of tools.

Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and high-dimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often prohibitively expensive and requires different formulations and elaborate computer codes. Machine learning has emerged as a promising alternative, but training deep neural networks requires big data, not always available for scientific problems. Instead, such networks can be trained from additional information obtained by enforcing the physical laws (for example, at random points in the continuous space-time domain). Such physics-informed learning integrates (noisy) data and mathematical models, and implements them through neural networks or other kernel-based regression networks. Moreover, it may be possible to design specialized network architectures that automatically satisfy some of the physical invariants for better accuracy, faster training and improved generalization. Here, we review some of the prevailing trends in embedding physics into machine learning, present some of the current capabilities and limitations and discuss diverse applications of physics-informed learning both for forward and inverse problems, including discovering hidden physics and tackling high-dimensional problems.

作为深度学习的典型{Application}，物理知识的神经网络（PINN）{已成功用于找到部分微分方程（PDES）的数值解决方案（PDES），但是如何提高有限准确性仍然是PINN的巨大挑战。 。在这项工作中，我们引入了一种新方法，对称性增强物理学知情的神经网络（SPINN），其中PDE的谎言对称性诱导的不变表面条件嵌入PINN的损失函数中，以提高PINN的准确性。我们分别通过两组十组独立数值实验来测试SPINN的有效性，分别用于热方程，Korteweg-De Vries（KDV）方程和潜在的汉堡{方程式}，这表明Spinn的性能比PINN更好，而PINN的训练点和更简单的结构都更好神经网络。此外，我们讨论了Spinn的计算开销，以PINN的相对计算成本，并表明Spinn的训练时间没有明显的增加，甚至在某些情况下还不是PINN。

Data-driven identification of differential equations is an interesting but challenging problem, especially when the given data are corrupted by noise. When the governing differential equation is a linear combination of various differential terms, the identification problem can be formulated as solving a linear system, with the feature matrix consisting of linear and nonlinear terms multiplied by a coefficient vector. This product is equal to the time derivative term, and thus generates dynamical behaviors. The goal is to identify the correct terms that form the equation to capture the dynamics of the given data. We propose a general and robust framework to recover differential equations using a weak formulation, for both ordinary and partial differential equations (ODEs and PDEs). The weak formulation facilitates an efficient and robust way to handle noise. For a robust recovery against noise and the choice of hyper-parameters, we introduce two new mechanisms, narrow-fit and trimming, for the coefficient support and value recovery, respectively. For each sparsity level, Subspace Pursuit is utilized to find an initial set of support from the large dictionary. Then, we focus on highly dynamic regions (rows of the feature matrix), and error normalize the feature matrix in the narrow-fit step. The support is further updated via trimming of the terms that contribute the least. Finally, the support set of features with the smallest Cross-Validation error is chosen as the result. A comprehensive set of numerical experiments are presented for both systems of ODEs and PDEs with various noise levels. The proposed method gives a robust recovery of the coefficients, and a significant denoising effect which can handle up to $100\%$ noise-to-signal ratio for some equations. We compare the proposed method with several state-of-the-art algorithms for the recovery of differential equations.

我们制定了一类由物理驱动的深层变量模型（PDDLVM），以学习参数偏微分方程（PDES）的参数到解决方案（正向）和解决方案到参数（逆）图。我们的公式利用有限元方法（FEM），深神经网络和概率建模来组装一个深层概率框架，在该框架中，向前和逆图通过连贯的不确定性量化近似。我们的概率模型明确合并了基于参数PDE的密度和可训练的解决方案到参数网络，而引入的摊销变异家庭假定参数到解决方案网络，所有这些网络均经过联合培训。此外，所提出的方法不需要任何昂贵的PDE解决方案，并且仅在训练时间内对物理信息进行了信息，该方法允许PDE的实时仿真和培训后的逆问题解决方案的产生，绕开了对FEM操作的需求，以相当的准确性，以便于FEM解决方案。提出的框架进一步允许无缝集成观察到的数据，以解决反问题和构建生成模型。我们证明了方法对非线性泊松问题，具有复杂3D几何形状的弹性壳以及整合通用物理信息信息的神经网络（PINN）体系结构的有效性。与传统的FEM求解器相比，训练后，我们最多达到了三个数量级的速度，同时输出连贯的不确定性估计值。