我们提出了一种自适应方法来构建贝叶斯推理的高斯过程,并使用昂贵的评估正演模型。我们的方法依赖于完全贝叶斯方法来训练高斯过程模型,并利用贝叶斯全局优化的预期改进思想。我们通过最大化高斯过程模型与噪声观测数据拟合的预期改进来自适应地构建训练设计。对合成数据模型问题的数值实验证明了所获得的自适应设计与固定非自适应设计相比,在前向模型推断成本的精确后验估计方面的有效性。
translated by 谷歌翻译
当只能获得有限数量的noisylog-likelihood评估时,我们考虑贝叶斯推断。例如,当基于复杂模拟器的统计模型适合于数据时,这发生,并且使用合成似然(SL)来使用计算成本高的前向模拟来形成噪声对数似然估计。我们将推理任务构建为贝叶斯序列设计问题,其中对数似然函数使用分层高斯过程(GP)代理模型进行建模,该模型用于有效地选择其他对数似然评估位置。最近在批处理贝叶斯优化中取得了进展,我们开发了各种顺序策略,其中自适应地选择多个模拟以最小化预期或中值损失函数,从而测量所得到的后验中的不确定性。我们从理论上和经验上分析了所得方法的性质。玩具问题和三个模拟模型的实验表明我们的方法是稳健的,高度可并行的,并且样本有效。
translated by 谷歌翻译
We develop an efficient, Bayesian Uncertainty Quantification framework using a novel treed Gaussian process model. The tree is adaptively constructed using information conveyed by the observed data about the length scales of the underlying process. On each leaf of the tree, we utilize Bayesian Experimental Design techniques in order to learn a multi-output Gaussian process. The constructed surrogate can provide analytical point estimates, as well as error bars, for the statistics of interest. We numerically demonstrate the effectiveness of the suggested framework in identifying discontinuities, local features and unimportant dimensions in the solution of stochastic differential equations.
translated by 谷歌翻译
许多对科学计算和机器学习感兴趣的概率模型具有昂贵的黑盒可能性,这些可能性阻止了贝叶斯推理的标准技术的应用,例如MCMC,其需要接近梯度或大量可能性评估。我们在这里介绍一种新颖的样本有效推理框架,VariationalBayesian Monte Carlo(VBMC)。 VBMC将变分推理与基于高斯过程的有源采样贝叶斯积分结合起来,使用latterto有效逼近变分目标中的难以求的积分。我们的方法产生了后验分布的非参数近似和模型证据的近似下界,对模型选择很有用。我们在几种合成可能性和神经元模型上展示VBMC,其中包含来自真实神经元的数据。在所有测试的问题和维度(高达$ D = 10 $)中,VBMC始终如一地通过有限的可能性评估预算重建后验证和模型证据,而不像其他仅在非常低维度下工作的方法。我们的框架作为一种新颖的工具,具有昂贵的黑盒可能性,可用于后期模型推理。
translated by 谷歌翻译
近似贝叶斯计算(ABC)是贝叶斯推理的一种方法,当可能性不可用时,但是可以从模型中进行模拟。然而,许多ABC算法需要大量的模拟,这可能是昂贵的。为了降低计算成本,已经提出了贝叶斯优化(BO)和诸如高斯过程的模拟模型。贝叶斯优化使人们可以智能地决定在哪里评估模型下一个,但是常见的BO策略不是为了估计后验分布而设计的。我们的论文解决了文献中的这一差距。我们建议计算ABC后验密度的不确定性,这是因为缺乏模拟来准确估计这个数量,并且定义了测量这种不确定性的aloss函数。然后,我们建议选择下一个评估位置,以尽量减少预期的损失。实验表明,与普通BO策略相比,所提出的方法通常产生最准确的近似。
translated by 谷歌翻译
使用高斯过程的贝叶斯优化是处理昂贵的黑盒功能优化的流行方法。然而,由于经典GaussianProcesses的协方差矩阵的平稳性的先验,该方法可能不适用于优化问题中涉及的非平稳函数。为了克服这个问题,提出了一种新的贝叶斯优化方法。它基于深度高斯过程的assurrogate模型而不是经典的高斯过程。该建模技术通过简单地考虑静态高斯过程的功能组合来提高表示的能力以捕获非平稳性,从而提供多层结构。本文提出了一种新的全局优化算法,通过耦合深度高斯过程和贝叶斯优化算法。通过学术测试案例讨论并突出了这种优化方法的特殊性。所提出的算法的性能在分析测试用例和航空设计优化问题上进行评估,并与最先进的固定和非静态贝叶斯优化方法进行比较。
translated by 谷歌翻译
高效全局优化(EGO)广泛用于优化计算上昂贵的黑盒功能。它使用基于高斯过程(克里金法)的替代建模技术。然而,由于使用了静态协方差,克里金不太适合近似非平稳函数。本文探讨了深度高斯过程(DGP)在EGO框架中的整合,以处理非平稳问题,并研究诱发的挑战和机遇。对分析问题进行数值实验以突出DGP和EGO的不同方面。
translated by 谷歌翻译
我们考虑学习嘈杂的黑盒功能超过给定阈值的水平集的问题。为了有效地重建水平集,我们研究了高斯过程(GP)元模型。我们的重点是强随机采样器,特别是重尾模拟噪声和低信噪比。为了防止噪声错误指定,我们评估了三个变量的性能:(i)具有Student-$ t $观察值的GP; (ii)学生 - $ t $流程(TP); (iii)分类GP对响应的符号进行建模。作为第四个扩展,我们研究具有单调性约束的GP代理,这些约束在已知连接的级别集时是相关的。结合这些模型,我们分析了几个采集函数,用于指导顺序实验设计,将现有的逐步不确定性减少标准扩展到随机轮廓发现环境。这也促使我们开发(近似)更新公式以有效地计算取代函数。我们的方案通过在1-6维度中使用各种合成实验进行基准测试。我们还考虑应用水平集估计来确定最佳的运动政策和百慕大金融期权的估值。
translated by 谷歌翻译
我们开发了一种自动变分方法,用于推导具有高斯过程(GP)先验和一般可能性的模型。该方法支持多个输出和多个潜在函数,不需要条件似然的详细知识,只需将其评估为ablack-box函数。使用高斯混合作为变分分布,我们表明使用来自单变量高斯分布的样本可以有效地估计证据下界及其梯度。此外,该方法可扩展到大数据集,这是通过使用诱导变量使用增广先验来实现的。支持最稀疏GP近似的方法,以及并行计算和随机优化。我们在小数据集,中等规模数据集和大型数据集上定量和定性地评估我们的方法,显示其在不同似然模型和稀疏性水平下的竞争力。在涉及航空延误预测和手写数字分类的大规模实验中,我们表明我们的方法与可扩展的GP回归和分类的最先进的硬编码方法相同。
translated by 谷歌翻译
Our paper deals with inferring simulator-based statistical models given some observed data. A simulator-based model is a parametrized mechanism which specifies how data are generated. It is thus also referred to as generative model. We assume that only a finite number of parameters are of interest and allow the generative process to be very general; it may be a noisy nonlinear dynamical system with an unrestricted number of hidden variables. This weak assumption is useful for devising realistic models but it renders statistical inference very difficult. The main challenge is the intractability of the likelihood function. Several likelihood-free inference methods have been proposed which share the basic idea of identifying the parameters by finding values for which the discrepancy between simulated and observed data is small. A major obstacle to using these methods is their computational cost. The cost is largely due to the need to repeatedly simulate data sets and the lack of knowledge about how the parameters affect the discrepancy. We propose a strategy which combines probabilistic modeling of the discrepancy with optimization to facilitate likelihood-free inference. The strategy is implemented using Bayesian optimization and is shown to accelerate the inference through a reduction in the number of required simulations by several orders of magnitude.
translated by 谷歌翻译
Computer codes simulating physical systems usually have responses that consist of a set of distinct outputs (e.g., velocity and pressure) that evolve also in space and time and depend on many unknown input parameters (e.g., physical constants, initial/boundary conditions, etc.). Furthermore, essential engineering procedures such as uncertainty quantification, inverse problems or design are notoriously difficult to carry out mostly due to the limited simulations available. The aim of this work is to introduce a fully Bayesian approach for treating these problems which accounts for the uncertainty induced by the finite number of observations. Our model is built on a multi-dimensional Gaussian process that explicitly treats correlations between distinct output variables as well as space and/or time. The proper use of a separable covariance function enables us to describe the huge covariance matrix as a Kronecker product of smaller matrices leading to efficient algorithms for carrying out inference and predictions. The novelty of this work, is the recognition that the Gaussian process model defines a posterior probability measure on the function space of possible surrogates for the computer code and the derivation of an algorithmic procedure that allows us to sample it efficiently. We demonstrate how the scheme can be used in uncertainty quantification tasks in order to obtain error bars for the statistics of interest that account for the finite number of observations.
translated by 谷歌翻译
Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalized) linear models, (gener-alized) additive models, smoothing spline models, state space models, semiparametric regression , spatial and spatiotemporal models, log-Gaussian Cox processes and geostatistical and geoadditive models. We consider approximate Bayesian inference in a popular subset of struc-tured additive regression models, latent Gaussian models, where the latent field is Gaussian, controlled by a few hyperparameters and with non-Gaussian response variables. The posterior marginals are not available in closed form owing to the non-Gaussian response variables. For such models, Markov chain Monte Carlo methods can be implemented, but they are not without problems, in terms of both convergence and computational time. In some practical applications, the extent of these problems is such that Markov chain Monte Carlo sampling is simply not an appropriate tool for routine analysis. We show that, by using an integrated nested Laplace approximation and its simplified version, we can directly compute very accurate approximations to the posterior marginals. The main benefit of these approximations is computational: where Markov chain Monte Carlo algorithms need hours or days to run, our approximations provide more precise estimates in seconds or minutes. Another advantage with our approach is its generality , which makes it possible to perform Bayesian analysis in an automatic, streamlined way, and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged.
translated by 谷歌翻译
概率二分算法基于从嘈杂的oracle响应中获得的知识来执行根查找。我们考虑广义PBA设置(G-PBA),其中oracle的统计分布是未知的和位置依赖的,因此模型推理和贝叶斯知识更新必须同时进行。为此,我们建议通过构建基础逻辑回归步骤的统计代理来利用典型oracle的空间结构。我们研究了几个非参数代理,包括二项式高斯过程(B-GP),多项式,核和样条Logistic回归。与此同时,我们开发了自适应平衡学习oracle分布和学习根的策略。我们的一个建议模仿了B-GP的主动学习,并提供了一种新颖的前瞻预测方差公式。我们用空间PBA算法得到的相对于早期G-PBA模型的增益用合成的例子和来自贝尔丹期权定价的具有挑战性的随机根发现问题来说明。
translated by 谷歌翻译
我们考虑在概率数据模型中推断潜在函数的问题。当通过高斯过程指定潜在函数的依赖性并且数据似然是复杂的时,有效计算涉及马尔可夫链蒙特卡罗采样,其对大数据集的适用性有限。我们扩展了其中一些技术,以便在问题呈现顺序结构时有效地扩展。我们提出了对潜在变量和相关参数的近似可重复序列采样。我们证明了在生长数据设置中的强大性能,否则这些设置对于幼稚的非顺序采样是不可行的。
translated by 谷歌翻译
Standard Gaussian processes (GPs) model observations' noise as constant throughout input space. This is often a too restrictive assumption, but one that is needed for GP inference to be tractable. In this work we present a non-standard variational approximation that allows accurate inference in heteroscedastic GPs (i.e., under input-dependent noise conditions). Computational cost is roughly twice that of the standard GP, and also scales as O(n 3). Accuracy is verified by comparing with the golden standard MCMC and its effectiveness is illustrated on several synthetic and real datasets of diverse characteristics. An application to volatility forecasting is also considered.
translated by 谷歌翻译
We propose a novel sampling framework for inference in probabilistic models: an active learning approach that converges more quickly (in wall-clock time) than Markov chain Monte Carlo (MCMC) benchmarks. The central challenge in proba-bilistic inference is numerical integration, to average over ensembles of models or unknown (hyper-)parameters (for example to compute the marginal likelihood or a partition function). MCMC has provided approaches to numerical integration that deliver state-of-the-art inference, but can suffer from sample inefficiency and poor convergence diagnostics. Bayesian quadrature techniques offer a model-based solution to such problems, but their uptake has been hindered by prohibitive computation costs. We introduce a warped model for probabilistic integrands (like-lihoods) that are known to be non-negative, permitting a cheap active learning scheme to optimally select sample locations. Our algorithm is demonstrated to offer faster convergence (in seconds) relative to simple Monte Carlo and annealed importance sampling on both synthetic and real-world examples.
translated by 谷歌翻译
在本文中,我们提出了一种使用规则模糊推理系统从数据中学习的新方法,其中使用贝叶斯推理和马尔可夫链蒙特卡罗(MCMC)技术估计模型参数。我们使用合理的数据集展示了该方法在回归和分类任务中的适用性,并且也是金融服务行业的现实例子。然后,我们演示如何扩展该方法以进行知识提取,以贝叶斯方式选择单个规则,从而最好地解释给定数据。最后,我们讨论了使用这种方法优于最先进技术的优点和缺陷,并突出了有用的特定问题类别。
translated by 谷歌翻译
Bayesian optimization is a sample-efficient method for black-box global optimization. However , the performance of a Bayesian optimization method very much depends on its exploration strategy, i.e. the choice of acquisition function , and it is not clear a priori which choice will result in superior performance. While portfolio methods provide an effective, principled way of combining a collection of acquisition functions, they are often based on measures of past performance which can be misleading. To address this issue, we introduce the Entropy Search Portfolio (ESP): a novel approach to portfolio construction which is motivated by information theoretic considerations. We show that ESP outperforms existing portfolio methods on several real and synthetic problems, including geostatistical datasets and simulated control tasks. We not only show that ESP is able to offer performance as good as the best, but unknown, acquisition function, but surprisingly it often gives better performance. Finally , over a wide range of conditions we find that ESP is robust to the inclusion of poor acquisition functions.
translated by 谷歌翻译
Numerical integration is a key component of many problems in scientific computing , statistical modelling, and machine learning. Bayesian Quadrature is a model-based method for numerical integration which, relative to standard Monte Carlo methods, offers increased sample efficiency and a more robust estimate of the uncertainty in the estimated integral. We propose a novel Bayesian Quadrature approach for numerical integration when the integrand is non-negative, such as the case of computing the marginal likelihood, predictive distribution, or normal-ising constant of a probabilistic model. Our approach approximately marginalises the quadrature model's hyperparameters in closed form, and introduces an active learning scheme to optimally select function evaluations, as opposed to using Monte Carlo samples. We demonstrate our method on both a number of synthetic benchmarks and a real scientific problem from astronomy.
translated by 谷歌翻译
由于通用性,使用简单性和贝叶斯预测的效用,高斯过程(GP)回归已广泛应用于机器人技术。特别是,GP回归的主要实现是基于内核的,因为通过利用内核函数作为无限维特征,可以拟合任意非线性函数。虽然结合先前信息有可能大大提高基于内核的GP回归的数据效率,但通过选择内核函数和相关的超参数来表达复杂的先验通常具有挑战性且不直观。此外,基于内核的GPregression的计算复杂度随着样本数量的不足而缩小,限制了其在可获得大量数据的情况下的应用。在这项工作中,我们提出了ALPaCA,一种有效的贝叶斯回归算法,可以解决这些问题。 ALPaCA使用样本函数的数据集来学习特定于域的有限维特征编码,以及相关权重之前的先验,使得该特征空间中的贝叶斯线性回归产生对后密度的准确在线预测。这些功能是神经网络,通过元学习方法进行训练。 ALPaCA从数据集中提取所有先行信息,而不是依赖于任意限制性内核超参数的选择。此外,它大大降低了样本的复杂性,并允许扩展到大型系统。我们研究了ALPaCA在两个简单回归问题,两个模拟机器人系统以及人类执行的车道变换驾驶任务上的表现。我们发现,我们的方法优于基于内核的GP回归,以及theart元学习方法的状态,从而为机器人中的多种回归任务提供了一种有前途的插件工具,其中可扩展性和数据效率是重要的。
translated by 谷歌翻译