物理知识的神经网络(PINN)最近成为基于部分微分方程模型的广泛工程和科学问题的有前途的深度学习应用。然而,有证据表明,梯度下降的PINN训练显示出病理和梯度流动动力学的刚度。在本文中,我们建议使用杂交粒子群优化和梯度下降方法来训练PINN。所得的PSO-PINN算法不仅减轻了经过标准梯度下降训练的PINN的不希望的行为,而且还为PINN提供了合奏方法,可以提供具有量化不确定性的强大预测的可能性。线性和非线性PDE模型的实验证明了所提出的方法的功效。
translated by 谷歌翻译
We propose characteristic-informed neural networks (CINN), a simple and efficient machine learning approach for solving forward and inverse problems involving hyperbolic PDEs. Like physics-informed neural networks (PINN), CINN is a meshless machine learning solver with universal approximation capabilities. Unlike PINN, which enforces a PDE softly via a multi-part loss function, CINN encodes the characteristics of the PDE in a general-purpose deep neural network trained with the usual MSE data-fitting regression loss and standard deep learning optimization methods. This leads to faster training and can avoid well-known pathologies of gradient descent optimization of multi-part PINN loss functions. If the characteristic ODEs can be solved exactly, which is true in important cases, the output of a CINN is an exact solution of the PDE, even at initialization, preventing the occurrence of non-physical outputs. Otherwise, the ODEs must be solved approximately, but the CINN is still trained only using a data-fitting loss function. The performance of CINN is assessed empirically in forward and inverse linear hyperbolic problems. These preliminary results indicate that CINN is able to improve on the accuracy of the baseline PINN, while being nearly twice as fast to train and avoiding non-physical solutions. Future extensions to hyperbolic PDE systems and nonlinear PDEs are also briefly discussed.
translated by 谷歌翻译
物理信息的神经网络(PINN)是神经网络(NNS),它们作为神经网络本身的组成部分编码模型方程,例如部分微分方程(PDE)。如今,PINN是用于求解PDE,分数方程,积分分化方程和随机PDE的。这种新颖的方法已成为一个多任务学习框架,在该框架中,NN必须在减少PDE残差的同时拟合观察到的数据。本文对PINNS的文献进行了全面的综述:虽然该研究的主要目标是表征这些网络及其相关的优势和缺点。该综述还试图将出版物纳入更广泛的基于搭配的物理知识的神经网络,这些神经网络构成了香草·皮恩(Vanilla Pinn)以及许多其他变体,例如物理受限的神经网络(PCNN),各种HP-VPINN,变量HP-VPINN,VPINN,VPINN,变体。和保守的Pinn(CPINN)。该研究表明,大多数研究都集中在通过不同的激活功能,梯度优化技术,神经网络结构和损耗功能结构来定制PINN。尽管使用PINN的应用范围广泛,但通过证明其在某些情况下比有限元方法(FEM)等经典数值技术更可行的能力,但仍有可能的进步,最著名的是尚未解决的理论问题。
translated by 谷歌翻译
Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Here, we present an overview of physics-informed neural networks (PINNs), which embed a PDE into the loss of the neural network using automatic differentiation. The PINN algorithm is simple, and it can be applied to different types of PDEs, including integro-differential equations, fractional PDEs, and stochastic PDEs. Moreover, from the implementation point of view, PINNs solve inverse problems as easily as forward problems. We propose a new residual-based adaptive refinement (RAR) method to improve the training efficiency of PINNs. For pedagogical reasons, we compare the PINN algorithm to a standard finite element method. We also present a Python library for PINNs, DeepXDE, which is designed to serve both as an education tool to be used in the classroom as well as a research tool for solving problems in computational science and engineering. Specifically, DeepXDE can solve forward problems given initial and boundary conditions, as well as inverse problems given some extra measurements. DeepXDE supports complex-geometry domains based on the technique of constructive solid geometry, and enables the user code to be compact, resembling closely the mathematical formulation. We introduce the usage of DeepXDE and its customizability, and we also demonstrate the capability of PINNs and the user-friendliness of DeepXDE for five different examples. More broadly, DeepXDE contributes to the more rapid development of the emerging Scientific Machine Learning field.
translated by 谷歌翻译
This paper introduces the use of evolutionary algorithms for solving differential equations. The solution is obtained by optimizing a deep neural network whose loss function is defined by the residual terms from the differential equations. Recent studies have used stochastic gradient descent (SGD) variants to train these physics-informed neural networks (PINNs), but these methods can struggle to find accurate solutions due to optimization challenges. When solving differential equations, it is important to find the globally optimum parameters of the network, rather than just finding a solution that works well during training. SGD only searches along a single gradient direction, so it may not be the best approach for training PINNs with their accompanying complex optimization landscapes. In contrast, evolutionary algorithms perform a parallel exploration of different solutions in order to avoid getting stuck in local optima and can potentially find more accurate solutions. However, evolutionary algorithms can be slow, which can make them difficult to use in practice. To address this, we provide a set of five benchmark problems with associated performance metrics and baseline results to support the development of evolutionary algorithms for enhanced PINN training. As a baseline, we evaluate the performance and speed of using the widely adopted Covariance Matrix Adaptation Evolution Strategy (CMA-ES) for solving PINNs. We provide the loss and training time for CMA-ES run on TensorFlow, and CMA-ES and SGD run on JAX (with GPU acceleration) for the five benchmark problems. Our results show that JAX-accelerated evolutionary algorithms, particularly CMA-ES, can be a useful approach for solving differential equations. We hope that our work will support the exploration and development of alternative optimization algorithms for the complex task of optimizing PINNs.
translated by 谷歌翻译
科学和工程学中的一个基本问题是设计最佳的控制政策,这些政策将给定的系统转向预期的结果。这项工作提出了同时求解给定系统状态和最佳控制信号的控制物理信息的神经网络(控制PINNS),在符合基础物理定律的一个阶段框架中。先前的方法使用两个阶段的框架,该框架首先建模然后按顺序控制系统。相比之下,控制PINN将所需的最佳条件纳入其体系结构和损耗函数中。通过解决以下开环的最佳控制问题来证明控制PINN的成功:(i)一个分析问题,(ii)一维热方程,以及(iii)二维捕食者捕食者问题。
translated by 谷歌翻译
Although physics-informed neural networks(PINNs) have progressed a lot in many real applications recently, there remains problems to be further studied, such as achieving more accurate results, taking less training time, and quantifying the uncertainty of the predicted results. Recent advances in PINNs have indeed significantly improved the performance of PINNs in many aspects, but few have considered the effect of variance in the training process. In this work, we take into consideration the effect of variance and propose our VI-PINNs to give better predictions. We output two values in the final layer of the network to represent the predicted mean and variance respectively, and the latter is used to represent the uncertainty of the output. A modified negative log-likelihood loss and an auxiliary task are introduced for fast and accurate training. We perform several experiments on a wide range of different problems to highlight the advantages of our approach. The results convey that our method not only gives more accurate predictions but also converges faster.
translated by 谷歌翻译
Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics, and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than classical feed-forward neural networks, recurrent neural networks, and convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering, In this work, we review such methods and extend them for parametric studies as well as for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.
translated by 谷歌翻译
随着计算能力的增加和机器学习的进步,基于数据驱动的学习方法在解决PDE方面引起了极大的关注。物理知识的神经网络(PINN)最近出现并成功地在各种前进和逆PDES问题中取得了成功,其优异的特性,例如灵活性,无网格解决方案和无监督的培训。但是,它们的收敛速度较慢和相对不准确的解决方案通常会限制其在许多科学和工程领域中的更广泛适用性。本文提出了一种新型的数据驱动的PDES求解器,物理知识的细胞表示(Pixel),优雅地结合了经典数值方法和基于学习的方法。我们采用来自数值方法的网格结构,以提高准确性和收敛速度并克服PINN中呈现的光谱偏差。此外,所提出的方法在PINN中具有相同的好处,例如,使用相同的优化框架来解决前进和逆PDE问题,并很容易通过现代自动分化技术强制执行PDE约束。我们为原始Pinn所努力的各种具有挑战性的PDE提供了实验结果,并表明像素达到了快速收敛速度和高精度。
translated by 谷歌翻译
深度学习方法的应用加快了挑战性电流问题的分辨率,最近显示出令人鼓舞的结果。但是,电力系统动力学不是快照,稳态操作。必须考虑这些动力学,以确保这些模型提供的最佳解决方案遵守实用的动力约束,避免频率波动和网格不稳定性。不幸的是,由于其高计算成本,基于普通或部分微分方程的动态系统模型通常不适合在控制或状态估计中直接应用。为了应对这些挑战,本文介绍了一种机器学习方法,以近乎实时近似电力系统动态的行为。该拟议的框架基于梯度增强的物理知识的神经网络(GPINNS),并编码有关电源系统的基本物理定律。拟议的GPINN的关键特征是它的训练能力而无需生成昂贵的培训数据。该论文说明了在单机无限总线系统中提出的方法在预测转子角度和频率的前进和反向问题中的潜力,以及不确定的参数,例如惯性和阻尼,以展示其在一系列电力系统应用中的潜力。
translated by 谷歌翻译
Solute transport in porous media is relevant to a wide range of applications in hydrogeology, geothermal energy, underground CO2 storage, and a variety of chemical engineering systems. Due to the complexity of solute transport in heterogeneous porous media, traditional solvers require high resolution meshing and are therefore expensive computationally. This study explores the application of a mesh-free method based on deep learning to accelerate the simulation of solute transport. We employ Physics-informed Neural Networks (PiNN) to solve solute transport problems in homogeneous and heterogeneous porous media governed by the advection-dispersion equation. Unlike traditional neural networks that learn from large training datasets, PiNNs only leverage the strong form mathematical models to simultaneously solve for multiple dependent or independent field variables (e.g., pressure and solute concentration fields). In this study, we construct PiNN using a periodic activation function to better represent the complex physical signals (i.e., pressure) and their derivatives (i.e., velocity). Several case studies are designed with the intention of investigating the proposed PiNN's capability to handle different degrees of complexity. A manual hyperparameter tuning method is used to find the best PiNN architecture for each test case. Point-wise error and mean square error (MSE) measures are employed to assess the performance of PiNNs' predictions against the ground truth solutions obtained analytically or numerically using the finite element method. Our findings show that the predictions of PiNN are in good agreement with the ground truth solutions while reducing computational complexity and cost by, at least, three orders of magnitude.
translated by 谷歌翻译
深入学习被证明是通过物理信息的神经网络(PINNS)求解部分微分方程(PDE)的有效工具。 Pinns将PDE残差嵌入到神经网络的损耗功能中,已成功用于解决各种前向和逆PDE问题。然而,第一代Pinns的一个缺点是它们通常具有许多训练点即使具有有限的准确性。在这里,我们提出了一种新的方法,梯度增强的物理信息的神经网络(GPInns),用于提高Pinns的准确性和培训效率。 GPInns利用PDE残差的梯度信息,并将梯度嵌入损耗功能。我们广泛地测试了GPinns,并证明了GPInns在前进和反向PDE问题中的有效性。我们的数值结果表明,GPInn比贴图更好地表现出较少的训练点。此外,我们将GPIn与基于残留的自适应细化(RAR)的方法组合,一种用于在训练期间自适应地改善训练点分布的方法,以进一步提高GPInn的性能,尤其是具有陡峭梯度的溶液的PDE。
translated by 谷歌翻译
使用深层学习方法来解决PDE是完全扩张的领域。特别是,物理知识的神经网络,其实现物理域的采样并使用惩罚偏差方程的违反违反部分微分方程的丢失函数。然而,为了解决实际应用中遇到的大规模问题并与PDE的现有数值方法竞争,重要的是设计具有良好可扩展性的平行算法。在传统领域分解方法(DDM)的静脉中,我们认为最近提出的深层DDM方法。我们展示了这种方法的扩展,依赖于使用粗糙空间校正,类似于传统DDM求解器中所做的内容。我们的研究表明,当由于每个迭代时子域之间的瞬时信息交换而增加,当子域的数量增加时,粗校正能够缓解求解器的收敛性的恶化。实验结果表明,我们的方法引起了原始的深度DDM方法的显着加速,降低了额外的计算成本。
translated by 谷歌翻译
两个不混溶的流体的位移是多孔介质中流体流动的常见问题。这种问题可以作为局部微分方程(PDE)构成通常被称为Buckley-Leverett(B-L)问题。 B-L问题是一种非线性双曲守护法,众所周知,使用传统的数值方法难以解决。在这里,我们使用物理信息的神经网络(Pinns)使用非凸版通量函数来解决前向双曲线B-L问题。本文的贡献是双重的。首先,我们通过将Oleinik熵条件嵌入神经网络残差来提出一种Pinn方法来解决双曲线B-L问题。我们不使用扩散术语(人工粘度)在残留损失中,但我们依靠PDE的强形式。其次,我们使用ADAM优化器与基于残留的自适应细化(RAR)算法,实现不加权的超低损耗。我们的解决方案方法可以精确地捕获冲击前并产生精确的整体解决方案。我们报告了一个2 x 10-2的L2验证误差和1x 10-6的L2损耗。所提出的方法不需要任何额外的正则化或加权损失以获得这种准确的解决方案。
translated by 谷歌翻译
在本文中,我们提出了用于求解非线性微分方程(NDE)的神经网络的物理知情训练(PIAT)。众所周知,神经网络的标准培训会导致非平滑函数。对抗训练(AT)是针对对抗攻击的既定防御机制,这也可能有助于使解决方案平滑。 AT包括通过扰动增强训练迷你批量,使网络输出不匹配所需的输出对手。与正式AT仅依靠培训数据不同,在这里,我们使用对抗网络体系结构中的自动差异来以非线性微分方程的形式编码管理物理定律。我们将PIAT与PIAT进行了比较,以指示我们方法在求解多达10个维度方面的有效性。此外,我们提出了重量衰减和高斯平滑,以证明PIAT的优势。代码存储库可从https://github.com/rohban-lab/piat获得。
translated by 谷歌翻译
Deep neural operators can learn nonlinear mappings between infinite-dimensional function spaces via deep neural networks. As promising surrogate solvers of partial differential equations (PDEs) for real-time prediction, deep neural operators such as deep operator networks (DeepONets) provide a new simulation paradigm in science and engineering. Pure data-driven neural operators and deep learning models, in general, are usually limited to interpolation scenarios, where new predictions utilize inputs within the support of the training set. However, in the inference stage of real-world applications, the input may lie outside the support, i.e., extrapolation is required, which may result to large errors and unavoidable failure of deep learning models. Here, we address this challenge of extrapolation for deep neural operators. First, we systematically investigate the extrapolation behavior of DeepONets by quantifying the extrapolation complexity via the 2-Wasserstein distance between two function spaces and propose a new behavior of bias-variance trade-off for extrapolation with respect to model capacity. Subsequently, we develop a complete workflow, including extrapolation determination, and we propose five reliable learning methods that guarantee a safe prediction under extrapolation by requiring additional information -- the governing PDEs of the system or sparse new observations. The proposed methods are based on either fine-tuning a pre-trained DeepONet or multifidelity learning. We demonstrate the effectiveness of the proposed framework for various types of parametric PDEs. Our systematic comparisons provide practical guidelines for selecting a proper extrapolation method depending on the available information, desired accuracy, and required inference speed.
translated by 谷歌翻译
科学机器学习(Sciml)的出现在思路科学领域开辟了一个新的领域,通过在基于物理和数据建模的界面的界面中开发方法。为此,近年来介绍了物理知识的神经网络(Pinns),通过在所谓的焊点上纳入物理知识来应对培训数据的稀缺。在这项工作中,我们研究了Pinns关于用于强制基于物理惩罚术语的配偶数量的预测性能。我们表明Pinns可能会失败,学习通过定义来满足物理惩罚术语的琐碎解决方案。我们制定了一种替代的采样方法和新的惩罚术语,使我们能够在具有竞争性结果的数据稀缺设置中纠正Pinns中的核心问题,同时减少最多80 \%的基准问题所需的搭配数量。
translated by 谷歌翻译
当通过差异模型研究流行动力学时,要了解现象并模拟预测场景所需的参数需要微妙的校准阶段,通常会因官方来源报告的稀缺性和不确定性而变得更加挑战。在这种情况下,通过嵌入控制物理现象在学习过程中的差异模型的知识,可以有效解决数据驱动的学习的逆问题,并解决相应的流行病问题,从而使物理知识的神经网络(PINN)(PINN)(PINN)(PINNS)。 。然而,在许多情况下,传染病的空间传播的特征是在多尺度PDE的不同尺度上的个体运动。这反映了与城市和邻近区域内动态有关的区域或领域的异质性。在存在多个量表的情况下,PINN的直接应用通常会导致由于神经网络损失函数中差异模型的多尺度性质而导致的结果差。为了使神经网络相对于小规模统一运行,希望神经网络满足学习过程中的渐近保护(AP)特性。为此,我们考虑了一类新的AP神经网络(APNNS),用于多尺度双曲线传输模型的流行病扩散模型,由于损失函数的适当配方,它能够在系统的不同尺度上均匀地工作。一系列针对不同流行病的数值测试证实了所提出的方法的有效性,在处理多尺度问题时,突出了AP在神经网络中的重要性,尤其是在存在稀疏和部分观察到的系统的情况下。
translated by 谷歌翻译
基于神经网络的求解部分微分方程的方法由于其简单性和灵活性来表示偏微分方程的解决方案而引起了相当大的关注。在训练神经网络时,网络倾向于学习与低频分量相对应的全局特征,而高频分量以较慢的速率(F原理)近似。对于解决方案包含广泛尺度的一类等式,由于无法捕获高频分量,网络训练过程可能会遭受缓慢的收敛性和低精度。在这项工作中,我们提出了一种分层方法来提高神经网络解决方案的收敛速率和准确性。所提出的方法包括多训练水平,其中引导新引入的神经网络来学习先前级别近似的残余。通过神经网络训练过程的性​​质,高级校正倾向于捕获高频分量。我们通过一套线性和非线性部分微分方程验证所提出的分层方法的效率和稳健性。
translated by 谷歌翻译
微分方程用于多种学科,描述了物理世界的复杂行为。这些方程式的分析解决方案通常很难求解,从而限制了我们目前求解复杂微分方程的能力,并需要将复杂的数值方法近似于解决方案。训练有素的神经网络充当通用函数近似器,能够以新颖的方式求解微分方程。在这项工作中,探索了神经网络算法在数值求解微分方程方面的方法和应用,重点是不同的损失函数和生物应用。传统损失函数和训练参数的变化显示出使神经网络辅助解决方案更有效的希望,从而可以调查更复杂的方程式管理生物学原理。
translated by 谷歌翻译