最近在科学机器学习的工作已经开发出所谓的物理信息的神经网络（Pinn）模型。典型方法是将物理域知识纳入经验丢失功能的软限制，并使用现有的机器学习方法来培训模型。我们展示了，虽然现有的Pinn方法可以学习良好的模型，但它们可以轻松地未能学习相关的物理现象，甚至更复杂的问题。特别是，我们分析了众多不同的普遍物理兴趣的情况，包括使用对流，反应和扩散运营商学习微分方程。我们提供了证据表明Pinns中的软正规化，涉及基于PDE的差分运营商，可以引入许多微妙的问题，包括使问题更加不良。重要的是，我们表明，这些可能的失败模式不是由于NN架构中缺乏富有效力，但Pinn的设置使得损失景观很难优化。然后，我们描述了两个有希望的解决方案来解决这些故障模式。第一种方法是使用课程正则化，其中Pinn的丢失项从简单的PDE正则化开始，并且随着NN训练而变得逐渐变得更加复杂。第二种方法是将问题构成为序列到序列的学习任务，而不是学习一次性地预测整个时空。广泛的测试表明，与常规Pinn训练相比，我们可以通过这些方法实现最多1-2个数量级。

物理知识的神经网络（PINN）将问题领域的物理知识作为对损失函数的软限制，但最近的工作表明这可能导致优化困难。在这里，我们研究了搭配点的位置对这些模型训练性的影响。我们发现，随着训练的进行，可以通过适应搭配点的位置来显着提高香草·皮恩的性能。具体而言，我们提出了一种新型的自适应搭配方案，该方案逐渐将更多的搭配点（不增加数量）分配给模型正在造成更高误差的区域（基于域中损失函数的梯度）。加上在任何优化失速过程中对训练的明智重新启动（通过简单地重新采样搭配点以调整损失景观）会导致预测错误的更好估计。我们提出了一些问题的结果，包括具有不同强迫函数的2D泊松和扩散 - 辅助系统。我们发现，针对这些问题的训练香草PINN可以导致解决方案中的预测误差高达70％，尤其是在低搭配点的状态下。相比之下，我们的自适应方案可以达到较小误差的顺序，其计算复杂性与基线相似。此外，我们发现自适应方法始终如一地执行PAR或比香草Pinn方法稍好，即使对于大型搭配点方案也是如此。所有实验的代码都是开源的。

深入学习被证明是通过物理信息的神经网络（PINNS）求解部分微分方程（PDE）的有效工具。 Pinns将PDE残差嵌入到神经网络的损耗功能中，已成功用于解决各种前向和逆PDE问题。然而，第一代Pinns的一个缺点是它们通常具有许多训练点即使具有有限的准确性。在这里，我们提出了一种新的方法，梯度增强的物理信息的神经网络（GPInns），用于提高Pinns的准确性和培训效率。 GPInns利用PDE残差的梯度信息，并将梯度嵌入损耗功能。我们广泛地测试了GPinns，并证明了GPInns在前进和反向PDE问题中的有效性。我们的数值结果表明，GPInn比贴图更好地表现出较少的训练点。此外，我们将GPIn与基于残留的自适应细化（RAR）的方法组合，一种用于在训练期间自适应地改善训练点分布的方法，以进一步提高GPInn的性能，尤其是具有陡峭梯度的溶液的PDE。

物理信息的神经网络（PINN）是神经网络（NNS），它们作为神经网络本身的组成部分编码模型方程，例如部分微分方程（PDE）。如今，PINN是用于求解PDE，分数方程，积分分化方程和随机PDE的。这种新颖的方法已成为一个多任务学习框架，在该框架中，NN必须在减少PDE残差的同时拟合观察到的数据。本文对PINNS的文献进行了全面的综述：虽然该研究的主要目标是表征这些网络及其相关的优势和缺点。该综述还试图将出版物纳入更广泛的基于搭配的物理知识的神经网络，这些神经网络构成了香草·皮恩（Vanilla Pinn）以及许多其他变体，例如物理受限的神经网络（PCNN），各种HP-VPINN，变量HP-VPINN，VPINN，VPINN，变体。和保守的Pinn（CPINN）。该研究表明，大多数研究都集中在通过不同的激活功能，梯度优化技术，神经网络结构和损耗功能结构来定制PINN。尽管使用PINN的应用范围广泛，但通过证明其在某些情况下比有限元方法（FEM）等经典数值技术更可行的能力，但仍有可能的进步，最著名的是尚未解决的理论问题。

机器学习方法最近在求解部分微分方程（PDE）中的承诺。它们可以分为两种广泛类别：近似解决方案功能并学习解决方案操作员。物理知识的神经网络（PINN）是前者的示例，而傅里叶神经操作员（FNO）是后者的示例。这两种方法都有缺点。 Pinn的优化是具有挑战性，易于发生故障，尤其是在多尺度动态系统上。 FNO不会遭受这种优化问题，因为它在给定的数据集上执行了监督学习，但获取此类数据可能太昂贵或无法使用。在这项工作中，我们提出了物理知识的神经运营商（Pino），在那里我们结合了操作学习和功能优化框架。这种综合方法可以提高PINN和FNO模型的收敛速度和准确性。在操作员学习阶段，Pino在参数PDE系列的多个实例上学习解决方案操作员。在测试时间优化阶段，Pino优化预先训练的操作员ANSATZ，用于PDE的查询实例。实验显示Pino优于许多流行的PDE家族的先前ML方法，同时保留与求解器相比FNO的非凡速度。特别是，Pino准确地解决了挑战的长时间瞬态流量，而其他基线ML方法无法收敛的Kolmogorov流程。

在本文中，我们演示并调查了一些挑战，这些挑战阻碍了使用物理知识的神经网络解决复杂问题的方式。特别是，我们可视化受过训练的模型的损失景观，并在存在物理学的情况下对反向传播梯度进行灵敏度分析。我们的发现表明，现有的方法产生了难以导航的高度非凸损失景观。此外，高阶PDE污染了可能阻碍或防止收敛的反向传播梯度。然后，我们提出了一种新的方法，该方法绕过了高阶PDE操作员的计算并减轻反向传播梯度的污染。为此，我们降低了解决方案搜索空间的维度，并通过非平滑解决方案促进学习问题。我们的配方还提供了一种反馈机制，可帮助我们的模型适应地专注于难以学习的领域的复杂区域。然后，我们通过调整Lagrange乘数方法来提出一个无约束的二重问题。我们运用我们的方法来解决由线性和非线性PDE控制的几个具有挑战性的基准问题。

已经提出了物理信息神经网络（PINN）来学习偏微分方程（PDE）的解决方案。在PINN中，感兴趣的PDE及其边界条件的残余形式被归为复合目标函数，作为软惩罚。在这里，我们表明，将目标函数制定的这种特定方式是应用于不同种类PDE的PINN方法中严重限制的来源。为了解决这些局限性，我们提出了一个基于约束优化问题公式的多功能框架，在该框架中，我们使用增强的拉格朗日方法（ALM）来限制PDE的解决方案，并具有其边界条件和任何可能可用的高保真数据。我们的方法擅长于具有多保真数据融合的转发和反问题。我们通过将其应用于涉及多维PDE的几个远期和反向问题来证明物理和相等性约束深度学习框架的功效和多功能性。您的框架与最先进的框架相比，与最先进的框架提高了幅度的提高顺序。 ART物理信息的神经网络。

我们引入了一种实用方法，以实施由神经网络（NNS）定义的功能的线性偏微分方程（PDE）约束，直至所需的公差。通过将隐式函数定理的可区分物理和应用中的应用结合到NN模型中，我们开发了可区分的PDE受限的NN层。在培训期间，我们的模型学习了一个功能系列，每个功能都定义了从PDE参数到PDE解决方案的映射。在推理时，该模型通过解决PDE受限的优化问题来发现学到家族中功能的最佳线性组合。我们的方法提供了有关感兴趣领域的连续解决方案，这些解决方案完全满足了所需的物理约束。我们的结果表明，与对无约束目标的训练相比，将硬约束直接纳入NN体系结构的测试错误要低得多。

物理知识的神经网络（PINN）算法在解决涉及部分微分方程（PDES）的各种问题方面显示出令人鼓舞的结果。但是，由于一种称为光谱偏置的现象，当目标函数包含高频特征时，它们通常无法收敛到理想的溶液。在目前的工作中，我们利用神经切线内核（NTK）来研究用动量（SGDM）在随机梯度下降下进化的Pinns的训练动力学。这表明SGDM显着降低了光谱偏置的影响。我们还研究了为什么通过ADAM优化器训练模型可以在减少频谱偏置的同时加速收敛。此外，我们的数值实验已经证实，即使在存在高频特征的情况下，使用SGDM的广泛网络仍会收敛到理想的解决方案。实际上，我们表明网络的宽度在收敛中起着至关重要的作用。

科学机器学习（Sciml）的出现在思路科学领域开辟了一个新的领域，通过在基于物理和数据建模的界面的界面中开发方法。为此，近年来介绍了物理知识的神经网络（Pinns），通过在所谓的焊点上纳入物理知识来应对培训数据的稀缺。在这项工作中，我们研究了Pinns关于用于强制基于物理惩罚术语的配偶数量的预测性能。我们表明Pinns可能会失败，学习通过定义来满足物理惩罚术语的琐碎解决方案。我们制定了一种替代的采样方法和新的惩罚术语，使我们能够在具有竞争性结果的数据稀缺设置中纠正Pinns中的核心问题，同时减少最多80 \％的基准问题所需的搭配数量。

Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics, and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than classical feed-forward neural networks, recurrent neural networks, and convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering, In this work, we review such methods and extend them for parametric studies as well as for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.

科学和工程学中的一个基本问题是设计最佳的控制政策，这些政策将给定的系统转向预期的结果。这项工作提出了同时求解给定系统状态和最佳控制信号的控制物理信息的神经网络（控制PINNS），在符合基础物理定律的一个阶段框架中。先前的方法使用两个阶段的框架，该框架首先建模然后按顺序控制系统。相比之下，控制PINN将所需的最佳条件纳入其体系结构和损耗函数中。通过解决以下开环的最佳控制问题来证明控制PINN的成功：（i）一个分析问题，（ii）一维热方程，以及（iii）二维捕食者捕食者问题。

We propose characteristic-informed neural networks (CINN), a simple and efficient machine learning approach for solving forward and inverse problems involving hyperbolic PDEs. Like physics-informed neural networks (PINN), CINN is a meshless machine learning solver with universal approximation capabilities. Unlike PINN, which enforces a PDE softly via a multi-part loss function, CINN encodes the characteristics of the PDE in a general-purpose deep neural network trained with the usual MSE data-fitting regression loss and standard deep learning optimization methods. This leads to faster training and can avoid well-known pathologies of gradient descent optimization of multi-part PINN loss functions. If the characteristic ODEs can be solved exactly, which is true in important cases, the output of a CINN is an exact solution of the PDE, even at initialization, preventing the occurrence of non-physical outputs. Otherwise, the ODEs must be solved approximately, but the CINN is still trained only using a data-fitting loss function. The performance of CINN is assessed empirically in forward and inverse linear hyperbolic problems. These preliminary results indicate that CINN is able to improve on the accuracy of the baseline PINN, while being nearly twice as fast to train and avoiding non-physical solutions. Future extensions to hyperbolic PDE systems and nonlinear PDEs are also briefly discussed.

Solute transport in porous media is relevant to a wide range of applications in hydrogeology, geothermal energy, underground CO2 storage, and a variety of chemical engineering systems. Due to the complexity of solute transport in heterogeneous porous media, traditional solvers require high resolution meshing and are therefore expensive computationally. This study explores the application of a mesh-free method based on deep learning to accelerate the simulation of solute transport. We employ Physics-informed Neural Networks (PiNN) to solve solute transport problems in homogeneous and heterogeneous porous media governed by the advection-dispersion equation. Unlike traditional neural networks that learn from large training datasets, PiNNs only leverage the strong form mathematical models to simultaneously solve for multiple dependent or independent field variables (e.g., pressure and solute concentration fields). In this study, we construct PiNN using a periodic activation function to better represent the complex physical signals (i.e., pressure) and their derivatives (i.e., velocity). Several case studies are designed with the intention of investigating the proposed PiNN's capability to handle different degrees of complexity. A manual hyperparameter tuning method is used to find the best PiNN architecture for each test case. Point-wise error and mean square error (MSE) measures are employed to assess the performance of PiNNs' predictions against the ground truth solutions obtained analytically or numerically using the finite element method. Our findings show that the predictions of PiNN are in good agreement with the ground truth solutions while reducing computational complexity and cost by, at least, three orders of magnitude.

Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Here, we present an overview of physics-informed neural networks (PINNs), which embed a PDE into the loss of the neural network using automatic differentiation. The PINN algorithm is simple, and it can be applied to different types of PDEs, including integro-differential equations, fractional PDEs, and stochastic PDEs. Moreover, from the implementation point of view, PINNs solve inverse problems as easily as forward problems. We propose a new residual-based adaptive refinement (RAR) method to improve the training efficiency of PINNs. For pedagogical reasons, we compare the PINN algorithm to a standard finite element method. We also present a Python library for PINNs, DeepXDE, which is designed to serve both as an education tool to be used in the classroom as well as a research tool for solving problems in computational science and engineering. Specifically, DeepXDE can solve forward problems given initial and boundary conditions, as well as inverse problems given some extra measurements. DeepXDE supports complex-geometry domains based on the technique of constructive solid geometry, and enables the user code to be compact, resembling closely the mathematical formulation. We introduce the usage of DeepXDE and its customizability, and we also demonstrate the capability of PINNs and the user-friendliness of DeepXDE for five different examples. More broadly, DeepXDE contributes to the more rapid development of the emerging Scientific Machine Learning field.

Non-equilibrium chemistry is a key process in the study of the InterStellar Medium (ISM), in particular the formation of molecular clouds and thus stars. However, computationally it is among the most difficult tasks to include in astrophysical simulations, because of the typically high (>40) number of reactions, the short evolutionary timescales (about $10^4$ times less than the ISM dynamical time) and the characteristic non-linearity and stiffness of the associated Ordinary Differential Equations system (ODEs). In this proof of concept work, we show that Physics Informed Neural Networks (PINN) are a viable alternative to traditional ODE time integrators for stiff thermo-chemical systems, i.e. up to molecular hydrogen formation (9 species and 46 reactions). Testing different chemical networks in a wide range of densities ($-2< \log n/{\rm cm}^{-3}< 3$) and temperatures ($1 < \log T/{\rm K}< 5$), we find that a basic architecture can give a comfortable convergence only for simplified chemical systems: to properly capture the sudden chemical and thermal variations a Deep Galerkin Method is needed. Once trained ($\sim 10^3$ GPUhr), the PINN well reproduces the strong non-linear nature of the solutions (errors $\lesssim 10\%$) and can give speed-ups up to a factor of $\sim 200$ with respect to traditional ODE solvers. Further, the latter have completion times that vary by about $\sim 30\%$ for different initial $n$ and $T$, while the PINN method gives negligible variations. Both the speed-up and the potential improvement in load balancing imply that PINN-powered simulations are a very palatable way to solve complex chemical calculation in astrophysical and cosmological problems.

Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and high-dimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often prohibitively expensive and requires different formulations and elaborate computer codes. Machine learning has emerged as a promising alternative, but training deep neural networks requires big data, not always available for scientific problems. Instead, such networks can be trained from additional information obtained by enforcing the physical laws (for example, at random points in the continuous space-time domain). Such physics-informed learning integrates (noisy) data and mathematical models, and implements them through neural networks or other kernel-based regression networks. Moreover, it may be possible to design specialized network architectures that automatically satisfy some of the physical invariants for better accuracy, faster training and improved generalization. Here, we review some of the prevailing trends in embedding physics into machine learning, present some of the current capabilities and limitations and discuss diverse applications of physics-informed learning both for forward and inverse problems, including discovering hidden physics and tackling high-dimensional problems.

两个不混溶的流体的位移是多孔介质中流体流动的常见问题。这种问题可以作为局部微分方程（PDE）构成通常被称为Buckley-Leverett（B-L）问题。 B-L问题是一种非线性双曲守护法，众所周知，使用传统的数值方法难以解决。在这里，我们使用物理信息的神经网络（Pinns）使用非凸版通量函数来解决前向双曲线B-L问题。本文的贡献是双重的。首先，我们通过将Oleinik熵条件嵌入神经网络残差来提出一种Pinn方法来解决双曲线B-L问题。我们不使用扩散术语（人工粘度）在残留损失中，但我们依靠PDE的强形式。其次，我们使用ADAM优化器与基于残留的自适应细化（RAR）算法，实现不加权的超低损耗。我们的解决方案方法可以精确地捕获冲击前并产生精确的整体解决方案。我们报告了一个2 x 10-2的L2验证误差和1x 10-6的L2损耗。所提出的方法不需要任何额外的正则化或加权损失以获得这种准确的解决方案。

深度学习方法的应用加快了挑战性电流问题的分辨率，最近显示出令人鼓舞的结果。但是，电力系统动力学不是快照，稳态操作。必须考虑这些动力学，以确保这些模型提供的最佳解决方案遵守实用的动力约束，避免频率波动和网格不稳定性。不幸的是，由于其高计算成本，基于普通或部分微分方程的动态系统模型通常不适合在控制或状态估计中直接应用。为了应对这些挑战，本文介绍了一种机器学习方法，以近乎实时近似电力系统动态的行为。该拟议的框架基于梯度增强的物理知识的神经网络（GPINNS），并编码有关电源系统的基本物理定律。拟议的GPINN的关键特征是它的训练能力而无需生成昂贵的培训数据。该论文说明了在单机无限总线系统中提出的方法在预测转子角度和频率的前进和反向问题中的潜力，以及不确定的参数，例如惯性和阻尼，以展示其在一系列电力系统应用中的潜力。

Machine learning-based modeling of physical systems has experienced increased interest in recent years. Despite some impressive progress, there is still a lack of benchmarks for Scientific ML that are easy to use but still challenging and representative of a wide range of problems. We introduce PDEBench, a benchmark suite of time-dependent simulation tasks based on Partial Differential Equations (PDEs). PDEBench comprises both code and data to benchmark the performance of novel machine learning models against both classical numerical simulations and machine learning baselines. Our proposed set of benchmark problems contribute the following unique features: (1) A much wider range of PDEs compared to existing benchmarks, ranging from relatively common examples to more realistic and difficult problems; (2) much larger ready-to-use datasets compared to prior work, comprising multiple simulation runs across a larger number of initial and boundary conditions and PDE parameters; (3) more extensible source codes with user-friendly APIs for data generation and baseline results with popular machine learning models (FNO, U-Net, PINN, Gradient-Based Inverse Method). PDEBench allows researchers to extend the benchmark freely for their own purposes using a standardized API and to compare the performance of new models to existing baseline methods. We also propose new evaluation metrics with the aim to provide a more holistic understanding of learning methods in the context of Scientific ML. With those metrics we identify tasks which are challenging for recent ML methods and propose these tasks as future challenges for the community. The code is available at https://github.com/pdebench/PDEBench.