大规模复杂动力系统的实时精确解决方案非常需要控制,优化,不确定性量化以及实践工程和科学应用中的决策。本文朝着这个方向做出了贡献,模型限制了切线流形学习(MCTANGENT)方法。 McTangent的核心是几种理想策略的协同作用:i)切线的学术学习,以利用神经网络速度和线条方法的准确性; ii)一种模型限制的方法,将神经网络切线与基础管理方程式进行编码; iii)促进长时间稳定性和准确性的顺序学习策略;和iv)数据随机方法,以隐式强制执行神经网络切线的平滑度及其对真相切线的可能性,以进一步提高麦克氏解决方案的稳定性和准确性。提供了半启发式和严格的论点,以分析和证明拟议的方法是合理的。提供了几个用于传输方程,粘性汉堡方程和Navier Stokes方程的数值结果,以研究和证明所提出的MCTANGENT学习方法的能力。
translated by 谷歌翻译
深度学习(DL),尤其是深神经网络(DNN),默认情况下纯粹是数据驱动的,通常不需要物理。这是DL的优势,但在应用于科学和工程问题时,它的主要局限性之一就是必不可少的物理特性和所需的准确性。其原始形式的DL方法也无法尊重基本的数学模型或即使在大数据制度中也可以达到所需的准确性。但是,许多数据驱动的科学和工程问题(例如反问题)通常具有有限的实验或观察数据,而在这种情况下,DL会过分拟合数据。我们认为,利用基础数学模型中编码的信息,不仅可以补偿低数据制度中缺少的信息,而且还提供了将DL方法与基础物理学配备的机会,从而促进了更好的概括。本文开发了一种模型受限的深度学习方法及其变体TNET,该方法能够学习隐藏在培训数据和基础数学模型中的信息,以解决由部分微分方程控制的反问题。我们为提出的方法提供了构造和一些理论结果。我们表明,数据随机化可以增强网络的平滑度及其概括。全面的数值结果不仅确认了理论发现,而且还表明,即使仅20个训练数据样本,一维卷积的训练数据样本,50次反向2D热电导率问题,100和50对于时间依赖的2D汉堡方程和逆初始条件和50 2D Navier-Stokes方程。 TNET溶液可以像Tikhonov溶液一样准确,同时几个数量级。由于模型受限项,复制和随机化,这可能是可能的。
translated by 谷歌翻译
Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics, and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than classical feed-forward neural networks, recurrent neural networks, and convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering, In this work, we review such methods and extend them for parametric studies as well as for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.
translated by 谷歌翻译
标准的神经网络可以近似一般的非线性操作员,要么通过数学运算符的组合(例如,在对流 - 扩散反应部分微分方程中)的组合,要么仅仅是黑匣子,例如黑匣子,例如一个系统系统。第一个神经操作员是基于严格的近似理论于2019年提出的深层操作员网络(DeepOnet)。从那时起,已经发布了其他一些较少的一般操作员,例如,基于图神经网络或傅立叶变换。对于黑匣子系统,对神经操作员的培训仅是数据驱动的,但是如果知道管理方程式可以在培训期间将其纳入损失功能,以开发物理知识的神经操作员。神经操作员可以用作设计问题,不确定性量化,自主系统以及几乎任何需要实时推断的应用程序中的代替代物。此外,通过将它们与相对轻的训练耦合,可以将独立的预训练deponets用作复杂多物理系统的组成部分。在这里,我们介绍了Deponet,傅立叶神经操作员和图神经操作员的评论,以及适当的扩展功能扩展,并突出显示它们在计算机械师中的各种应用中的实用性,包括多孔媒体,流体力学和固体机制, 。
translated by 谷歌翻译
物理信息的神经网络(PINN)是神经网络(NNS),它们作为神经网络本身的组成部分编码模型方程,例如部分微分方程(PDE)。如今,PINN是用于求解PDE,分数方程,积分分化方程和随机PDE的。这种新颖的方法已成为一个多任务学习框架,在该框架中,NN必须在减少PDE残差的同时拟合观察到的数据。本文对PINNS的文献进行了全面的综述:虽然该研究的主要目标是表征这些网络及其相关的优势和缺点。该综述还试图将出版物纳入更广泛的基于搭配的物理知识的神经网络,这些神经网络构成了香草·皮恩(Vanilla Pinn)以及许多其他变体,例如物理受限的神经网络(PCNN),各种HP-VPINN,变量HP-VPINN,VPINN,VPINN,变体。和保守的Pinn(CPINN)。该研究表明,大多数研究都集中在通过不同的激活功能,梯度优化技术,神经网络结构和损耗功能结构来定制PINN。尽管使用PINN的应用范围广泛,但通过证明其在某些情况下比有限元方法(FEM)等经典数值技术更可行的能力,但仍有可能的进步,最著名的是尚未解决的理论问题。
translated by 谷歌翻译
这本数字本书包含在物理模拟的背景下与深度学习相关的一切实际和全面的一切。尽可能多,所有主题都带有Jupyter笔记本的形式的动手代码示例,以便快速入门。除了标准的受监督学习的数据中,我们将看看物理丢失约束,更紧密耦合的学习算法,具有可微分的模拟,以及加强学习和不确定性建模。我们生活在令人兴奋的时期:这些方法具有从根本上改变计算机模拟可以实现的巨大潜力。
translated by 谷歌翻译
尽管在整个科学和工程中都无处不在,但只有少数部分微分方程(PDE)具有分析或封闭形式的解决方案。这激发了有关PDE的数值模拟的大量经典工作,最近,对数据驱动技术的研究旋转了机器学习(ML)。最近的一项工作表明,与机器学习的经典数值技术的混合体可以对任何一种方法提供重大改进。在这项工作中,我们表明,在纳入基于物理学的先验时,数值方案的选择至关重要。我们以基于傅立叶的光谱方法为基础,这些光谱方法比其他数值方案要高得多,以模拟使用平滑且周期性解决方案的PDE。具体而言,我们为流体动力学的三个模型PDE开发了ML增强的光谱求解器,从而提高了标准光谱求解器在相同分辨率下的准确性。我们还展示了一些关键设计原则,用于将机器学习和用于解决PDE的数值方法结合使用。
translated by 谷歌翻译
我们介绍了一种用于学习时空平流扩散过程的组成物理学意识的神经网络(FINN)。 FINN实现了一种新的方式,通过以组成方式模拟部分微分方程(PDE)的成分来实现与数值模拟的物理和结构知识结合人工神经网络的学习能力。导致单维和二维PDE(汉堡,扩散,扩散反应,Allen-Cahn)展示了FinN的卓越的建模精度和超出初始和边界条件的优异分配概率。只有十分之一的参数数量平均,Finn在所有情况下占纯机学习和其他最先进的物理知识模型 - 通常甚至通过多个数量级。此外,在扩散吸附场景中近似稀疏的实际数据时,Finn优于校准的物理模型,通过揭示观察过程的未知延迟因子来确认其泛化能力并显示出说明潜力。
translated by 谷歌翻译
神经网络的经典发展主要集中在有限维欧基德空间或有限组之间的学习映射。我们提出了神经网络的概括,以学习映射无限尺寸函数空间之间的运算符。我们通过一类线性积分运算符和非线性激活函数的组成制定运营商的近似,使得组合的操作员可以近似复杂的非线性运算符。我们证明了我们建筑的普遍近似定理。此外,我们介绍了四类运算符参数化:基于图形的运算符,低秩运算符,基于多极图形的运算符和傅里叶运算符,并描述了每个用于用每个计算的高效算法。所提出的神经运营商是决议不变的:它们在底层函数空间的不同离散化之间共享相同的网络参数,并且可以用于零击超分辨率。在数值上,与现有的基于机器学习的方法,达西流程和Navier-Stokes方程相比,所提出的模型显示出卓越的性能,而与传统的PDE求解器相比,与现有的基于机器学习的方法有关的基于机器学习的方法。
translated by 谷歌翻译
Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and high-dimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often prohibitively expensive and requires different formulations and elaborate computer codes. Machine learning has emerged as a promising alternative, but training deep neural networks requires big data, not always available for scientific problems. Instead, such networks can be trained from additional information obtained by enforcing the physical laws (for example, at random points in the continuous space-time domain). Such physics-informed learning integrates (noisy) data and mathematical models, and implements them through neural networks or other kernel-based regression networks. Moreover, it may be possible to design specialized network architectures that automatically satisfy some of the physical invariants for better accuracy, faster training and improved generalization. Here, we review some of the prevailing trends in embedding physics into machine learning, present some of the current capabilities and limitations and discuss diverse applications of physics-informed learning both for forward and inverse problems, including discovering hidden physics and tackling high-dimensional problems.
translated by 谷歌翻译
Data-driven modeling has become a key building block in computational science and engineering. However, data that are available in science and engineering are typically scarce, often polluted with noise and affected by measurement errors and other perturbations, which makes learning the dynamics of systems challenging. In this work, we propose to combine data-driven modeling via operator inference with the dynamic training via roll outs of neural ordinary differential equations. Operator inference with roll outs inherits interpretability, scalability, and structure preservation of traditional operator inference while leveraging the dynamic training via roll outs over multiple time steps to increase stability and robustness for learning from low-quality and noisy data. Numerical experiments with data describing shallow water waves and surface quasi-geostrophic dynamics demonstrate that operator inference with roll outs provides predictive models from training trajectories even if data are sampled sparsely in time and polluted with noise of up to 10%.
translated by 谷歌翻译
科学和工程学中的一个基本问题是设计最佳的控制政策,这些政策将给定的系统转向预期的结果。这项工作提出了同时求解给定系统状态和最佳控制信号的控制物理信息的神经网络(控制PINNS),在符合基础物理定律的一个阶段框架中。先前的方法使用两个阶段的框架,该框架首先建模然后按顺序控制系统。相比之下,控制PINN将所需的最佳条件纳入其体系结构和损耗函数中。通过解决以下开环的最佳控制问题来证明控制PINN的成功:(i)一个分析问题,(ii)一维热方程,以及(iii)二维捕食者捕食者问题。
translated by 谷歌翻译
事实证明,神经操作员是无限维函数空间之间非线性算子的强大近似值,在加速偏微分方程(PDE)的溶液方面是有希望的。但是,它需要大量的模拟数据,这些数据可能成本高昂,从而导致鸡肉 - 蛋的困境并限制其在求解PDE中的使用。为了摆脱困境,我们提出了一个无数据的范式,其中神经网络直接从由离散的PDE构成的平方平方残留(MSR)损失中学习物理。我们研究了MSR损失中的物理信息,并确定神经网络必须具有对PDE空间域中的远距离纠缠建模的挑战,PDE的空间域中的模式在不同的PDE中有所不同。因此,我们提出了低级分解网络(Lordnet),该网络可调节,并且也有效地建模各种纠缠。具体而言,Lordnet通过简单的完全连接的层学习了与全球纠缠的低级别近似值,从而以降低的计算成本来提取主要模式。关于解决泊松方程和纳维尔 - 长方式方程的实验表明,MSR损失的物理约束可以提高神经网络的精确度和泛化能力。此外,Lordnet在PDE中的其他现代神经网络体系结构都优于最少的参数和最快的推理速度。对于Navier-Stokes方程式,学习的运算符的速度比具有相同计算资源的有限差异解决方案快50倍。
translated by 谷歌翻译
Deep operator network (DeepONet) has demonstrated great success in various learning tasks, including learning solution operators of partial differential equations. In particular, it provides an efficient approach to predict the evolution equations in a finite time horizon. Nevertheless, the vanilla DeepONet suffers from the issue of stability degradation in the long-time prediction. This paper proposes a {\em transfer-learning} aided DeepONet to enhance the stability. Our idea is to use transfer learning to sequentially update the DeepONets as the surrogates for propagators learned in different time frames. The evolving DeepONets can better track the varying complexities of the evolution equations, while only need to be updated by efficient training of a tiny fraction of the operator networks. Through systematic experiments, we show that the proposed method not only improves the long-time accuracy of DeepONet while maintaining similar computational cost but also substantially reduces the sample size of the training set.
translated by 谷歌翻译
在本文中,我们提出了一种深度学习技术,用于数据驱动的流体介质中波传播的预测。该技术依赖于基于注意力的卷积复发自动编码器网络(AB-CRAN)。为了构建波传播数据的低维表示,我们采用了基于转化的卷积自动编码器。具有基于注意力的长期短期记忆细胞的AB-CRAN体系结构构成了我们的深度神经网络模型,用于游行低维特征的时间。我们评估了针对标准复发性神经网络的拟议的AB-Cran框架,用于波传播的低维学习。为了证明AB-Cran模型的有效性,我们考虑了三个基准问题,即一维线性对流,非线性粘性汉堡方程和二维圣人浅水系统。我们的新型AB-CRAN结构使用基准问题的空间 - 时空数据集,可以准确捕获波幅度,并在长期范围内保留溶液的波特性。与具有长期短期记忆细胞的标准复发性神经网络相比,基于注意力的序列到序列网络增加了预测的时间莫。 Denoising自动编码器进一步减少了预测的平方平方误差,并提高了参数空间中的概括能力。
translated by 谷歌翻译
科学机器学习(Sciml)的出现在思路科学领域开辟了一个新的领域,通过在基于物理和数据建模的界面的界面中开发方法。为此,近年来介绍了物理知识的神经网络(Pinns),通过在所谓的焊点上纳入物理知识来应对培训数据的稀缺。在这项工作中,我们研究了Pinns关于用于强制基于物理惩罚术语的配偶数量的预测性能。我们表明Pinns可能会失败,学习通过定义来满足物理惩罚术语的琐碎解决方案。我们制定了一种替代的采样方法和新的惩罚术语,使我们能够在具有竞争性结果的数据稀缺设置中纠正Pinns中的核心问题,同时减少最多80 \%的基准问题所需的搭配数量。
translated by 谷歌翻译
These notes were compiled as lecture notes for a course developed and taught at the University of the Southern California. They should be accessible to a typical engineering graduate student with a strong background in Applied Mathematics. The main objective of these notes is to introduce a student who is familiar with concepts in linear algebra and partial differential equations to select topics in deep learning. These lecture notes exploit the strong connections between deep learning algorithms and the more conventional techniques of computational physics to achieve two goals. First, they use concepts from computational physics to develop an understanding of deep learning algorithms. Not surprisingly, many concepts in deep learning can be connected to similar concepts in computational physics, and one can utilize this connection to better understand these algorithms. Second, several novel deep learning algorithms can be used to solve challenging problems in computational physics. Thus, they offer someone who is interested in modeling a physical phenomena with a complementary set of tools.
translated by 谷歌翻译
机器学习方法最近在求解部分微分方程(PDE)中的承诺。它们可以分为两种广泛类别:近似解决方案功能并学习解决方案操作员。物理知识的神经网络(PINN)是前者的示例,而傅里叶神经操作员(FNO)是后者的示例。这两种方法都有缺点。 Pinn的优化是具有挑战性,易于发生故障,尤其是在多尺度动态系统上。 FNO不会遭受这种优化问题,因为它在给定的数据集上执行了监督学习,但获取此类数据可能太昂贵或无法使用。在这项工作中,我们提出了物理知识的神经运营商(Pino),在那里我们结合了操作学习和功能优化框架。这种综合方法可以提高PINN和FNO模型的收敛速度和准确性。在操作员学习阶段,Pino在参数PDE系列的多个实例上学习解决方案操作员。在测试时间优化阶段,Pino优化预先训练的操作员ANSATZ,用于PDE的查询实例。实验显示Pino优于许多流行的PDE家族的先前ML方法,同时保留与求解器相比FNO的非凡速度。特别是,Pino准确地解决了挑战的长时间瞬态流量,而其他基线ML方法无法收敛的Kolmogorov流程。
translated by 谷歌翻译
Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Here, we present an overview of physics-informed neural networks (PINNs), which embed a PDE into the loss of the neural network using automatic differentiation. The PINN algorithm is simple, and it can be applied to different types of PDEs, including integro-differential equations, fractional PDEs, and stochastic PDEs. Moreover, from the implementation point of view, PINNs solve inverse problems as easily as forward problems. We propose a new residual-based adaptive refinement (RAR) method to improve the training efficiency of PINNs. For pedagogical reasons, we compare the PINN algorithm to a standard finite element method. We also present a Python library for PINNs, DeepXDE, which is designed to serve both as an education tool to be used in the classroom as well as a research tool for solving problems in computational science and engineering. Specifically, DeepXDE can solve forward problems given initial and boundary conditions, as well as inverse problems given some extra measurements. DeepXDE supports complex-geometry domains based on the technique of constructive solid geometry, and enables the user code to be compact, resembling closely the mathematical formulation. We introduce the usage of DeepXDE and its customizability, and we also demonstrate the capability of PINNs and the user-friendliness of DeepXDE for five different examples. More broadly, DeepXDE contributes to the more rapid development of the emerging Scientific Machine Learning field.
translated by 谷歌翻译
We apply Physics Informed Neural Networks (PINNs) to the problem of wildfire fire-front modelling. The PINN is an approach that integrates a differential equation into the optimisation loss function of a neural network to guide the neural network to learn the physics of a problem. We apply the PINN to the level-set equation, which is a Hamilton-Jacobi partial differential equation that models a fire-front with the zero-level set. This results in a PINN that simulates a fire-front as it propagates through a spatio-temporal domain. We demonstrate the agility of the PINN to learn physical properties of a fire under extreme changes in external conditions (such as wind) and show that this approach encourages continuity of the PINN's solution across time. Furthermore, we demonstrate how data assimilation and uncertainty quantification can be incorporated into the PINN in the wildfire context. This is significant contribution to wildfire modelling as the level-set method -- which is a standard solver to the level-set equation -- does not naturally provide this capability.
translated by 谷歌翻译