在这项工作中，我们提出了一个新的高斯进程回归（GPR）方法：物理信息辅助Kriging（PHIK）。在标准数据驱动的Kriging中，感兴趣的未知功能通常被视为高斯过程，其中具有假定的静止协方差，其具有从数据估计的QuandEdmente。在PHIK中，我们从可用随机模型的实现中计算平均值和协方差函数，例如，从管理随机部分微分方程解决方案的实现。这种构造的高斯过程通常是非静止的，并且不承担特定形式的协方差。我们的方法避免了数据驱动的GPR方法中的优化步骤来识别超参数。更重要的是，我们证明了确定性线性操作员形式的物理约束在得到的预测中保证。当在随机模型实现中包含错误时，我们还提供了保留物理约束时的误差估计。为了降低获取随机模型的计算成本，我们提出了一种多级蒙特卡罗估计的平均和协方差函数。此外，我们介绍了一种有源学习算法，指导选择附加观察位置。 PHIK的效率和准确性被证明重建部分已知的修饰的Branin功能，研究三维传热问题，并从稀疏浓度测量学习保守的示踪剂分布。

Partial differential equations (PDEs) are widely used for description of physical and engineering phenomena. Some key parameters involved in PDEs, which represents certain physical properties with important scientific interpretations, are difficult or even impossible to be measured directly. Estimation of these parameters from noisy and sparse experimental data of related physical quantities is an important task. Many methods for PDE parameter inference involve a large number of evaluations of numerical solution of PDE through algorithms such as finite element method, which can be time-consuming especially for nonlinear PDEs. In this paper, we propose a novel method for estimating unknown parameters in PDEs, called PDE-Informed Gaussian Process Inference (PIGPI). Through modeling the PDE solution as a Gaussian process (GP), we derive the manifold constraints induced by the (linear) PDE structure such that under the constraints, the GP satisfies the PDE. For nonlinear PDEs, we propose an augmentation method that transfers the nonlinear PDE into an equivalent PDE system linear in all derivatives that our PIGPI can handle. PIGPI can be applied to multi-dimensional PDE systems and PDE systems with unobserved components. The method completely bypasses the numerical solver for PDE, thus achieving drastic savings in computation time, especially for nonlinear PDEs. Moreover, the PIGPI method can give the uncertainty quantification for both the unknown parameters and the PDE solution. The proposed method is demonstrated by several application examples from different areas.

心房颤动的计算模型已成功地用于预测最佳消融部位。评估消融模式的效果的关键步骤是从不同，潜在的随机的位置加速模型以确定是否可以在ATRIA中诱导心律失常。在这项工作中，我们建议使用黎曼歧管的多保真高斯过程分类，以有效地确定心律失常是诱导性诱导的区域内的区域。我们构建一个直接在心房表面上运行的概率分类器。我们利用较低的分辨率模型来探索心房表面，并与高分辨率模型无缝结合，以识别诱导区域。当用40个样本培训时，我们的多保真性分级器显示了比使用作为基线心房颤动模型的最近邻分类器的均衡精度，并且在心房颤动的情况下具有9％。我们希望这种新技术将允许更快，更精确地对心房颤动的计算模型临床应用。

We present the GPry algorithm for fast Bayesian inference of general (non-Gaussian) posteriors with a moderate number of parameters. GPry does not need any pre-training, special hardware such as GPUs, and is intended as a drop-in replacement for traditional Monte Carlo methods for Bayesian inference. Our algorithm is based on generating a Gaussian Process surrogate model of the log-posterior, aided by a Support Vector Machine classifier that excludes extreme or non-finite values. An active learning scheme allows us to reduce the number of required posterior evaluations by two orders of magnitude compared to traditional Monte Carlo inference. Our algorithm allows for parallel evaluations of the posterior at optimal locations, further reducing wall-clock times. We significantly improve performance using properties of the posterior in our active learning scheme and for the definition of the GP prior. In particular we account for the expected dynamical range of the posterior in different dimensionalities. We test our model against a number of synthetic and cosmological examples. GPry outperforms traditional Monte Carlo methods when the evaluation time of the likelihood (or the calculation of theoretical observables) is of the order of seconds; for evaluation times of over a minute it can perform inference in days that would take months using traditional methods. GPry is distributed as an open source Python package (pip install gpry) and can also be found at https://github.com/jonaselgammal/GPry.

我们开发了一个计算程序，以估计具有附加噪声的半摩托车高斯过程回归模型的协方差超参数。也就是说，提出的方法可用于有效估计相关误差的方差，以及基于最大化边际似然函数的噪声方差。我们的方法涉及适当地降低超参数空间的维度，以简化单变量的根发现问题的估计过程。此外，我们得出了边际似然函数及其衍生物的边界和渐近线，这对于缩小高参数搜索的初始范围很有用。使用数值示例，我们证明了与传统参数优化相比，提出方法的计算优势和鲁棒性。

社会和自然中的极端事件，例如大流行尖峰，流氓波浪或结构性失败，可能会带来灾难性的后果。极端的表征很困难，因为它们很少出现，这似乎是由良性的条件引起的，并且属于复杂且通常是未知的无限维系统。这种挑战使他们将其描述为“毫无意义”。我们通过将贝叶斯实验设计（BED）中的新型训练方案与深神经操作员（DNOS）合奏结合在一起来解决这些困难。这个模型不足的框架配对了一个床方案，该床方案积极选择数据以用近似于无限二二维非线性运算符的DNO集合来量化极端事件。我们发现，这个框架不仅清楚地击败了高斯流程（GPS），而且只有两个成员的浅色合奏表现最好； 2）无论初始数据的状态如何（即有或没有极端），都会发现极端； 3）我们的方法消除了“双研究”现象； 4）与逐步全球Optima相比，使用次优的采集点的使用不会阻碍床的性能； 5）蒙特卡洛的获取优于高量级的标准优化器。这些结论共同构成了AI辅助实验基础设施的基础，该基础设施可以有效地推断并查明从物理到社会系统的许多领域的关键情况。

线性系统发生在整个工程和科学中，最著名的是差分方程。在许多情况下，系统的强迫函数尚不清楚，兴趣在于使用对系统的嘈杂观察来推断强迫以及其他未知参数。在微分方程中，强迫函数是自变量（通常是时间和空间）的未知函数，可以建模为高斯过程（GP）。在本文中，我们展示了如何使用GP内核的截断基础扩展，如何使用线性系统的伴随有效地推断成GP的功能。我们展示了如何实现截短的GP的确切共轭贝叶斯推断，在许多情况下，计算的计算大大低于使用MCMC方法所需的计算。我们证明了普通和部分微分方程系统的方法，并表明基础扩展方法与数量适中的基础向量相近。最后，我们展示了如何使用贝叶斯优化来推断非线性模型参数（例如内核长度尺度）的点估计值。

映射近场污染物的浓度对于跟踪城市地区意外有毒羽状分散体至关重要。通过求解大部分湍流谱，大型模拟（LES）具有准确表示污染物浓度空间变异性的潜力。找到一种合成大量信息的方法，以提高低保真操作模型的准确性（例如，提供更好的湍流封闭条款）特别有吸引力。这是一个挑战，在多质量环境中，LES的部署成本高昂，以了解羽流和示踪剂分散如何随着各种大气和源参数的变化。为了克服这个问题，我们提出了一个合并正交分解（POD）和高斯过程回归（GPR）的非侵入性降低阶模型，以预测与示踪剂浓度相关的LES现场统计。通过最大的后验（MAP）过程，GPR HyperParameter是通过POD告知的最大后验（MAP）过程来优化组件的。我们在二维案例研究上提供了详细的分析，该案例研究对应于表面安装的障碍物上的湍流大气边界层流。我们表明，障碍物上游的近源浓度异质性需要大量的POD模式才能得到充分捕获。我们还表明，逐组分的优化允许捕获POD模式中的空间尺度范围，尤其是高阶模式中较短的浓度模式。如果学习数据库由至少五十至100个LES快照制成，则可以首先估算所需的预算，以朝着更逼真的大气分散应用程序迈进，因此减少订单模型的预测仍然可以接受。

物理建模对于许多现代科学和工程应用至关重要。从数据科学或机器学习的角度来看，更多的域 - 不可吻合，数据驱动的模型是普遍的，物理知识 - 通常表示为微分方程 - 很有价值，因为它与数据是互补的，并且可能有可能帮助克服问题例如数据稀疏性，噪音和不准确性。在这项工作中，我们提出了一个简单但功能强大且通用的框架 - 自动构建物理学，可以将各种微分方程集成到高斯流程（GPS）中，以增强预测准确性和不确定性量化。这些方程可以是线性或非线性，空间，时间或时空，与未知的源术语完全或不完整，等等。基于内核分化，我们在示例目标函数，方程相关的衍生物和潜在源函数之前构建了GP，这些函数全部来自多元高斯分布。采样值被馈送到两个可能性：一个以适合观测值，另一个符合方程式。我们使用美白方法来逃避采样函数值和内核参数之间的强依赖性，并开发出一种随机变分学习算法。在模拟和几个现实世界应用中，即使使用粗糙的，不完整的方程式，自动元素都显示出对香草GPS的改进。

逆问题本质上是普遍存在的，几乎在科学和工程的几乎所有领域都出现，从地球物理学和气候科学到天体物理学和生物力学。解决反问题的核心挑战之一是解决他们的不良天性。贝叶斯推论提供了一种原则性的方法来克服这一方法，通过将逆问题提出为统计框架。但是，当推断具有大幅度的离散表示的字段（所谓的“维度的诅咒”）和/或仅以先前获取的解决方案的形式可用时。在这项工作中，我们提出了一种新的方法，可以使用深层生成模型进行有效，准确的贝叶斯反转。具体而言，我们证明了如何使用生成对抗网络（GAN）在贝叶斯更新中学到的近似分布，并在GAN的低维度潜在空间中重新解决所得的推断问题，从而有效地解决了大规模的解决方案。贝叶斯逆问题。我们的统计框架保留了潜在的物理学，并且被证明可以通过可靠的不确定性估计得出准确的结果，即使没有有关基础噪声模型的信息，这对于许多现有方法来说都是一个重大挑战。我们证明了提出方法对各种反问题的有效性，包括合成和实验观察到的数据。

封闭曲线的建模和不确定性量化是形状分析领域的重要问题，并且可以对随后的统计任务产生重大影响。这些任务中的许多涉及封闭曲线的集合，这些曲线通常在多个层面上表现出结构相似性。以有效融合这种曲线间依赖性的方式对多个封闭曲线进行建模仍然是一个具有挑战性的问题。在这项工作中，我们提出并研究了一个多数输出（又称多输出），多维高斯流程建模框架。我们说明了提出的方法学进步，并在几个曲线和形状相关的任务上证明了有意义的不确定性量化的实用性。这种基于模型的方法不仅解决了用内核构造对封闭曲线（及其形状）的推断问题，而且还为通常对功能对象的多层依赖性的非参数建模打开了门。

我们考虑了使用显微镜或X射线散射技术产生的图像数据自组装的模型的贝叶斯校准。为了说明BCP平衡结构中的随机远程疾病，我们引入了辅助变量以表示这种不确定性。然而，这些变量导致了高维图像数据的综合可能性，通常可以评估。我们使用基于测量运输的可能性方法以及图像数据的摘要统计数据来解决这一具有挑战性的贝叶斯推理问题。我们还表明，可以计算出有关模型参数的数据中的预期信息收益（EIG），而无需额外的成本。最后，我们介绍了基于二嵌段共聚物薄膜自组装和自上而下显微镜表征的ohta-kawasaki模型的数值案例研究。为了进行校准，我们介绍了一些基于域的能量和傅立叶的摘要统计数据，并使用EIG量化了它们的信息性。我们证明了拟议方法研究数据损坏和实验设计对校准结果的影响的力量。

In a fissile material, the inherent multiplicity of neutrons born through induced fissions leads to correlations in their detection statistics. The correlations between neutrons can be used to trace back some characteristics of the fissile material. This technique known as neutron noise analysis has applications in nuclear safeguards or waste identification. It provides a non-destructive examination method for an unknown fissile material. This is an example of an inverse problem where the cause is inferred from observations of the consequences. However, neutron correlation measurements are often noisy because of the stochastic nature of the underlying processes. This makes the resolution of the inverse problem more complex since the measurements are strongly dependent on the material characteristics. A minor change in the material properties can lead to very different outputs. Such an inverse problem is said to be ill-posed. For an ill-posed inverse problem the inverse uncertainty quantification is crucial. Indeed, seemingly low noise in the data can lead to strong uncertainties in the estimation of the material properties. Moreover, the analytical framework commonly used to describe neutron correlations relies on strong physical assumptions and is thus inherently biased. This paper addresses dual goals. Firstly, surrogate models are used to improve neutron correlations predictions and quantify the errors on those predictions. Then, the inverse uncertainty quantification is performed to include the impact of measurement error alongside the residual model bias.

Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics, and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than classical feed-forward neural networks, recurrent neural networks, and convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering, In this work, we review such methods and extend them for parametric studies as well as for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.

Bayesian optimization provides sample-efficient global optimization for a broad range of applications, including automatic machine learning, engineering, physics, and experimental design. We introduce BOTORCH, a modern programming framework for Bayesian optimization that combines Monte-Carlo (MC) acquisition functions, a novel sample average approximation optimization approach, autodifferentiation, and variance reduction techniques. BOTORCH's modular design facilitates flexible specification and optimization of probabilistic models written in PyTorch, simplifying implementation of new acquisition functions. Our approach is backed by novel theoretical convergence results and made practical by a distinctive algorithmic foundation that leverages fast predictive distributions, hardware acceleration, and deterministic optimization. We also propose a novel "one-shot" formulation of the Knowledge Gradient, enabled by a combination of our theoretical and software contributions. In experiments, we demonstrate the improved sample efficiency of BOTORCH relative to other popular libraries.34th Conference on Neural Information Processing Systems (NeurIPS 2020),

The saddle point (SP) calculation is a grand challenge for computationally intensive energy function in computational chemistry area, where the saddle point may represent the transition state (TS). The traditional methods need to evaluate the gradients of the energy function at a very large number of locations. To reduce the number of expensive computations of the true gradients, we propose an active learning framework consisting of a statistical surrogate model, Gaussian process regression (GPR) for the energy function, and a single-walker dynamics method, gentle accent dynamics (GAD), for the saddle-type transition states. SP is detected by the GAD applied to the GPR surrogate for the gradient vector and the Hessian matrix. Our key ingredient for efficiency improvements is an active learning method which sequentially designs the most informative locations and takes evaluations of the original model at these locations to train GPR. We formulate this active learning task as the optimal experimental design problem and propose a very efficient sample-based sub-optimal criterion to construct the optimal locations. We show that the new method significantly decreases the required number of energy or force evaluations of the original model.

Linear partial differential equations (PDEs) are an important, widely applied class of mechanistic models, describing physical processes such as heat transfer, electromagnetism, and wave propagation. In practice, specialized numerical methods based on discretization are used to solve PDEs. They generally use an estimate of the unknown model parameters and, if available, physical measurements for initialization. Such solvers are often embedded into larger scientific models or analyses with a downstream application such that error quantification plays a key role. However, by entirely ignoring parameter and measurement uncertainty, classical PDE solvers may fail to produce consistent estimates of their inherent approximation error. In this work, we approach this problem in a principled fashion by interpreting solving linear PDEs as physics-informed Gaussian process (GP) regression. Our framework is based on a key generalization of a widely-applied theorem for conditioning GPs on a finite number of direct observations to observations made via an arbitrary bounded linear operator. Crucially, this probabilistic viewpoint allows to (1) quantify the inherent discretization error; (2) propagate uncertainty about the model parameters to the solution; and (3) condition on noisy measurements. Demonstrating the strength of this formulation, we prove that it strictly generalizes methods of weighted residuals, a central class of PDE solvers including collocation, finite volume, pseudospectral, and (generalized) Galerkin methods such as finite element and spectral methods. This class can thus be directly equipped with a structured error estimate and the capability to incorporate uncertain model parameters and observations. In summary, our results enable the seamless integration of mechanistic models as modular building blocks into probabilistic models.

从卫星图像中提取的大气运动向量（AMV）是唯一具有良好全球覆盖范围的风观测。它们是进食数值天气预测（NWP）模型的重要特征。已经提出了几种贝叶斯模型来估计AMV。尽管对于正确同化NWP模型至关重要，但很少有方法可以彻底表征估计误差。估计误差的困难源于后验分布的特异性，这既是很高的维度，又是由于奇异的可能性而导致高度不良的条件，这在缺少数据（未观察到的像素）的情况下特别重要。这项工作研究了使用基于梯度的Markov链Monte Carlo（MCMC）算法评估AMV的预期误差。我们的主要贡献是提出一种回火策略，这相当于在点估计值附近的AMV和图像变量的联合后验分布的局部近似。此外，我们提供了与先前家庭本身有关的协方差（分数布朗运动），并具有不同的超参数。从理论的角度来看，我们表明，在规律性假设下，随着温度降低到{optimal}高斯近似值，在最大a后验（MAP）对数密度给出的点估计下，温度降低到{optimal}高斯近似值。从经验的角度来看，我们根据一些定量的贝叶斯评估标准评估了提出的方法。我们对合成和真实气象数据进行的数值模拟揭示了AMV点估计的准确性及其相关的预期误差估计值的显着提高，但在MCMC算法的收敛速度方面也有很大的加速度。

我们引入了一个计算有效的数据驱动框架，适合量化物理参数中的不确定性和计算机模型的模型公式，以微分方程为代表。我们构建了物理知识的先验，它们是多输出的GP先验，它们在协方差函数中编码模型的结构。我们将其扩展到一个完全贝叶斯的框架中，该框架量化了物理参数和模型预测的不确定性。由于物理模型通常是对实际过程的不完美描述，因此我们允许该模型通过考虑差异函数来偏离观察到的数据。为了获得后验分布，我们使用汉密尔顿蒙特卡洛采样。我们在使用血液动力学模型的仿真研究中证明了我们的方法，这些模型是时间依赖的微分方程。与我们的建模选择更复杂的模型模拟数据，目的是根据已知的数学连接学习物理参数。为了证明我们的方法的灵活性（使用热方程式的示例），还包括一个时空依赖的微分方程，其中还包括我们考虑偏见的数据收购过程的情况。最后，我们使用医学试验中获得的实际数据符合血液动力学模型。

Surrogate models have shown to be an extremely efficient aid in solving engineering problems that require repeated evaluations of an expensive computational model. They are built by sparsely evaluating the costly original model and have provided a way to solve otherwise intractable problems. A crucial aspect in surrogate modelling is the assumption of smoothness and regularity of the model to approximate. This assumption is however not always met in reality. For instance in civil or mechanical engineering, some models may present discontinuities or non-smoothness, e.g., in case of instability patterns such as buckling or snap-through. Building a single surrogate model capable of accounting for these fundamentally different behaviors or discontinuities is not an easy task. In this paper, we propose a three-stage approach for the approximation of non-smooth functions which combines clustering, classification and regression. The idea is to split the space following the localized behaviors or regimes of the system and build local surrogates that are eventually assembled. A sequence of well-known machine learning techniques are used: Dirichlet process mixtures models (DPMM), support vector machines and Gaussian process modelling. The approach is tested and validated on two analytical functions and a finite element model of a tensile membrane structure.