Simulator-based models are models for which the likelihood is intractable but simulation of synthetic data is possible. They are often used to describe complex real-world phenomena, and as such can often be misspecified in practice. Unfortunately, existing Bayesian approaches for simulators are known to perform poorly in those cases. In this paper, we propose a novel algorithm based on the posterior bootstrap and maximum mean discrepancy estimators. This leads to a highly-parallelisable Bayesian inference algorithm with strong robustness properties. This is demonstrated through an in-depth theoretical study which includes generalisation bounds and proofs of frequentist consistency and robustness of our posterior. The approach is then assessed on a range of examples including a g-and-k distribution and a toggle-switch model.

广义贝叶斯推理使用损失函数而不是可能性的先前信仰更新，因此可以用于赋予鲁棒性，以防止可能的错误规范的可能性。在这里，我们认为广泛化的贝叶斯推论斯坦坦差异作为损失函数的损失，由应用程序的可能性含有难治性归一化常数。在这种情况下，斯坦因差异来避免归一化恒定的评估，并产生封闭形式或使用标准马尔可夫链蒙特卡罗的通用后出版物。在理论层面上，我们显示了一致性，渐近的正常性和偏见 - 稳健性，突出了这些物业如何受到斯坦因差异的选择。然后，我们提供关于一系列棘手分布的数值实验，包括基于内核的指数家庭模型和非高斯图形模型的应用。

We introduce and study a novel model-selection strategy for Bayesian learning, based on optimal transport, along with its associated predictive posterior law: the Wasserstein population barycenter of the posterior law over models. We first show how this estimator, termed Bayesian Wasserstein barycenter (BWB), arises naturally in a general, parameter-free Bayesian model-selection framework, when the considered Bayesian risk is the Wasserstein distance. Examples are given, illustrating how the BWB extends some classic parametric and non-parametric selection strategies. Furthermore, we also provide explicit conditions granting the existence and statistical consistency of the BWB, and discuss some of its general and specific properties, providing insights into its advantages compared to usual choices, such as the model average estimator. Finally, we illustrate how this estimator can be computed using the stochastic gradient descent (SGD) algorithm in Wasserstein space introduced in a companion paper arXiv:2201.04232v2 [math.OC], and provide a numerical example for experimental validation of the proposed method.

离散状态空间代表了对统计推断的主要计算挑战，因为归一化常数的计算需要在大型或可能的无限集中进行求和，这可能是不切实际的。本文通过开发适合离散可怜的可能性的新型贝叶斯推理程序来解决这一计算挑战。受到连续数据的最新方法学进步的启发，主要思想是使用离散的Fisher Divergence更新有关模型参数的信念，以代替有问题的棘手的可能性。结果是可以使用标准计算工具（例如Markov Chain Monte Carlo）进行采样的广义后部，从而规避了棘手的归一化常数。分析了广义后验的统计特性，并具有足够的后验一致性和渐近正态性的条件。此外，提出了一种新颖的通用后代校准方法。应用程序在离散空间数据的晶格模型和计数数据的多元模型上介绍，在每种情况下，方法论都以低计算成本促进通用的贝叶斯推断。

概率分布之间的差异措施，通常被称为统计距离，在概率理论，统计和机器学习中普遍存在。为了在估计这些距离的距离时，对维度的诅咒，最近的工作已经提出了通过带有高斯内核的卷积在测量的分布中平滑局部不规则性。通过该框架的可扩展性至高维度，我们研究了高斯平滑$ P $ -wassersein距离$ \ mathsf {w} _p ^ {（\ sigma）} $的结构和统计行为，用于任意$ p \ GEQ 1 $。在建立$ \ mathsf {w} _p ^ {（\ sigma）} $的基本度量和拓扑属性之后，我们探索$ \ mathsf {w} _p ^ {（\ sigma）}（\ hat {\ mu} _n，\ mu）$，其中$ \ hat {\ mu} _n $是$ n $独立观察的实证分布$ \ mu $。我们证明$ \ mathsf {w} _p ^ {（\ sigma）} $享受$ n ^ { - 1/2} $的参数经验融合速率，这对比$ n ^ { - 1 / d} $率对于未平滑的$ \ mathsf {w} _p $ why $ d \ geq 3 $。我们的证明依赖于控制$ \ mathsf {w} _p ^ {（\ sigma）} $ by $ p $ th-sting spoollow sobolev restion $ \ mathsf {d} _p ^ {（\ sigma）} $并导出限制$ \ sqrt {n} \，\ mathsf {d} _p ^ {（\ sigma）}（\ hat {\ mu} _n，\ mu）$，适用于所有尺寸$ d $。作为应用程序，我们提供了使用$ \ mathsf {w} _p ^ {（\ sigma）} $的两个样本测试和最小距离估计的渐近保证，使用$ p = 2 $的实验使用$ \ mathsf {d} _2 ^ {（\ sigma）} $。

在没有明确或易于处理的可能性的情况下，贝叶斯人经常诉诸于贝叶斯计算（ABC）进行推理。我们的工作基于生成的对抗网络（GAN）和对抗性变分贝叶斯（GAN），为ABC桥接了ABC。 ABC和GAN都比较了观察到的数据和假数据的各个方面，分别从后代和似然模拟。我们开发了一个贝叶斯gan（B-GAN）采样器，该采样器通过解决对抗性优化问题直接靶向后部。 B-GAN是由有条件gan在ABC参考上学习的确定性映射驱动的。一旦训练了映射，就可以通过以可忽略的额外费用过滤噪声来获得IID后样品。我们建议使用（1）数据驱动的提案和（2）变化贝叶斯提出两项后处理的本地改进。我们通过常见的bayesian结果支持我们的发现，表明对于某些神经网络发生器和歧视器，真实和近似后骨之间的典型总变化距离收敛到零。我们对模拟数据的发现相对于一些最新的无可能后验模拟器显示出竞争激烈的性能。

在概率空间或分销回归方面的学习功能的问题正在对机器学习社区产生重大兴趣。此问题背后的一个关键挑战是确定捕获基础功能映射的所有相关属性的合适表示形式。内核平均嵌入式提供了一种原则性的分布回归方法，该方法在概率水平上提高了内核诱导的输入域的相似性。该策略有效地解决了问题的两阶段抽样性质，使人们能够得出具有强大统计保证的估计器，例如普遍的一致性和过度的风险界限。但是，内核平均值嵌入在最大平均差异（MMD）上隐含地铰接，这是概率的度量，可能无法捕获分布之间的关键几何关系。相反，最佳运输（OT）指标可能更具吸引力。在这项工作中，我们提出了一个基于OT的分布回归估计器。我们建立在切成薄片的Wasserstein距离上，以获得基于OT的表示。我们基于这种表示，我们研究了内核脊回归估计量的理论特性，我们证明了普遍的一致性和过多的风险界限。初步实验通过显示提出方法的有效性并将其与基于MMD的估计器进行比较，以补充我们的理论发现。

The kernel Maximum Mean Discrepancy~(MMD) is a popular multivariate distance metric between distributions that has found utility in two-sample testing. The usual kernel-MMD test statistic is a degenerate U-statistic under the null, and thus it has an intractable limiting distribution. Hence, to design a level-$\alpha$ test, one usually selects the rejection threshold as the $(1-\alpha)$-quantile of the permutation distribution. The resulting nonparametric test has finite-sample validity but suffers from large computational cost, since every permutation takes quadratic time. We propose the cross-MMD, a new quadratic-time MMD test statistic based on sample-splitting and studentization. We prove that under mild assumptions, the cross-MMD has a limiting standard Gaussian distribution under the null. Importantly, we also show that the resulting test is consistent against any fixed alternative, and when using the Gaussian kernel, it has minimax rate-optimal power against local alternatives. For large sample sizes, our new cross-MMD provides a significant speedup over the MMD, for only a slight loss in power.

We propose a framework for analyzing and comparing distributions, which we use to construct statistical tests to determine if two samples are drawn from different distributions. Our test statistic is the largest difference in expectations over functions in the unit ball of a reproducing kernel Hilbert space (RKHS), and is called the maximum mean discrepancy (MMD). We present two distributionfree tests based on large deviation bounds for the MMD, and a third test based on the asymptotic distribution of this statistic. The MMD can be computed in quadratic time, although efficient linear time approximations are available. Our statistic is an instance of an integral probability metric, and various classical metrics on distributions are obtained when alternative function classes are used in place of an RKHS. We apply our two-sample tests to a variety of problems, including attribute matching for databases using the Hungarian marriage method, where they perform strongly. Excellent performance is also obtained when comparing distributions over graphs, for which these are the first such tests.

We investigate the training and performance of generative adversarial networks using the Maximum Mean Discrepancy (MMD) as critic, termed MMD GANs. As our main theoretical contribution, we clarify the situation with bias in GAN loss functions raised by recent work: we show that gradient estimators used in the optimization process for both MMD GANs and Wasserstein GANs are unbiased, but learning a discriminator based on samples leads to biased gradients for the generator parameters. We also discuss the issue of kernel choice for the MMD critic, and characterize the kernel corresponding to the energy distance used for the Cramér GAN critic. Being an integral probability metric, the MMD benefits from training strategies recently developed for Wasserstein GANs. In experiments, the MMD GAN is able to employ a smaller critic network than the Wasserstein GAN, resulting in a simpler and faster-training algorithm with matching performance. We also propose an improved measure of GAN convergence, the Kernel Inception Distance, and show how to use it to dynamically adapt learning rates during GAN training.

本文介绍了一种新的基于仿真的推理程序，以对访问I.I.D. \ samples的多维概率分布进行建模和样本，从而规避明确建模密度函数或设计Markov Chain Monte Carlo的通常方法。我们提出了一个称为可逆的Gromov-monge（RGM）距离的新概念的距离和同构的动机，并研究了RGM如何用于设计新的转换样本，以执行基于模拟的推断。我们的RGM采样器还可以估计两个异质度量度量空间之间的最佳对齐$（\ cx，\ mu，c _ {\ cx}）$和$（\ cy，\ cy，\ nu，c _ {\ cy}）$从经验数据集中，估计的地图大约将一个量度$ \ mu $推向另一个$ \ nu $，反之亦然。我们研究了RGM距离的分析特性，并在轻度条件下得出RGM等于经典的Gromov-Wasserstein距离。奇怪的是，与Brenier的两极分解结合了连接，我们表明RGM采样器以$ C _ {\ cx} $和$ C _ {\ cy} $的正确选择诱导了强度同构的偏见。研究了有关诱导采样器的收敛，表示和优化问题的统计率。还展示了展示RGM采样器有效性的合成和现实示例。

我们提出了一种统一的技术，用于顺序估计分布之间的凸面分歧，包括内核最大差异等积分概率度量，$ \ varphi $ - 像Kullback-Leibler发散，以及最佳运输成本，例如Wassersein距离的权力。这是通过观察到经验凸起分歧（部分有序）反向半角分离的实现来实现的，而可交换过滤耦合，其具有这些方法的最大不等式。这些技术似乎是对置信度序列和凸分流的现有文献的互补和强大的补充。我们构建一个离线到顺序设备，将各种现有的离线浓度不等式转换为可以连续监测的时间均匀置信序列，在任意停止时间提供有效的测试或置信区间。得到的顺序边界仅在相应的固定时间范围内支付迭代对数价格，保留对问题参数的相同依赖性（如适用的尺寸或字母大小）。这些结果也适用于更一般的凸起功能，如负差分熵，实证过程的高度和V型统计。

Empirical risk minimization (ERM) and distributionally robust optimization (DRO) are popular approaches for solving stochastic optimization problems that appear in operations management and machine learning. Existing generalization error bounds for these methods depend on either the complexity of the cost function or dimension of the uncertain parameters; consequently, the performance of these methods is poor for high-dimensional problems with objective functions under high complexity. We propose a simple approach in which the distribution of uncertain parameters is approximated using a parametric family of distributions. This mitigates both sources of complexity; however, it introduces a model misspecification error. We show that this new source of error can be controlled by suitable DRO formulations. Our proposed parametric DRO approach has significantly improved generalization bounds over existing ERM / DRO methods and parametric ERM for a wide variety of settings. Our method is particularly effective under distribution shifts. We also illustrate the superior performance of our approach on both synthetic and real-data portfolio optimization and regression tasks.

我们提出了一种基于最大平均差异（MMD）的新型非参数两样本测试，该测试是通过具有不同核带宽的聚合测试来构建的。这种称为MMDAGG的聚合过程可确保对所使用的内核的收集最大化测试能力，而无需持有核心选择的数据（这会导致测试能力损失）或任意内核选择，例如中位数启发式。我们在非反应框架中工作，并证明我们的聚集测试对Sobolev球具有最小自适应性。我们的保证不仅限于特定的内核，而是符合绝对可集成的一维翻译不变特性内核的任何产品。此外，我们的结果适用于流行的数值程序来确定测试阈值，即排列和野生引导程序。通过对合成数据集和现实世界数据集的数值实验，我们证明了MMDAGG优于MMD内核适应的替代方法，用于两样本测试。

Scientists continue to develop increasingly complex mechanistic models to reflect their knowledge more realistically. Statistical inference using these models can be highly challenging, since the corresponding likelihood function is often intractable, and model simulation may be computationally burdensome or infeasible. Fortunately, in many of these situations, it is possible to adopt a surrogate model or approximate likelihood function. It may be convenient to base Bayesian inference directly on the surrogate, but this can result in bias and poor uncertainty quantification. In this paper we propose a new method for adjusting approximate posterior samples to reduce bias and produce more accurate uncertainty quantification. We do this by optimising a transform of the approximate posterior that minimises a scoring rule. Our approach requires only a (fixed) small number of complex model simulations and is numerically stable. We demonstrate good performance of the new method on several examples of increasing complexity.

比较概率分布是许多机器学习算法的关键。最大平均差异（MMD）和最佳运输距离（OT）是在过去几年吸引丰富的关注的概率措施之间的两类距离。本文建立了一些条件，可以通过MMD规范控制Wassersein距离。我们的作品受到压缩统计学习（CSL）理论的推动，资源有效的大规模学习的一般框架，其中训练数据总结在单个向量（称为草图）中，该训练数据捕获与所考虑的学习任务相关的信息。在CSL中的现有结果启发，我们介绍了H \“较旧的较低限制的等距属性（H \”较旧的LRIP）并表明这家属性具有有趣的保证对压缩统计学习。基于MMD与Wassersein距离之间的关系，我们通过引入和研究学习任务的Wassersein可读性的概念来提供压缩统计学习的保证，即概率分布之间的某些特定于特定的特定度量，可以由Wassersein界定距离。

利用启发式来评估收敛性和压缩马尔可夫链蒙特卡罗的输出可以在生产的经验逼近时是次优。通常，许多初始状态归因于“燃烧”并移除，而链条的其余部分是“变薄”，如果还需要压缩。在本文中，我们考虑回顾性地从样本路径中选择固定基数的状态的问题，使得由其经验分布提供的近似接近最佳。提出了一种基于核心稳定性差异的贪婪最小化的新方法，这适用于需要重压力的问题。理论结果保障方法的一致性及其有效性在常微分方程的参数推理的具体背景下证明了该效果。软件可在Python，R和Matlab中的Stein细化包中提供。

在包括生成建模的各种机器学习应用中的两个概率措施中，已经证明了切片分歧的想法是成功的，并且包括计算两种测量的一维随机投影之间的“基地分歧”的预期值。然而，这种技术的拓扑，统计和计算后果尚未完整地确定。在本文中，我们的目标是弥合这种差距并导出切片概率分歧的各种理论特性。首先，我们表明切片保留了公制公理和分歧的弱连续性，这意味着切片分歧将共享相似的拓扑性质。然后，我们在基本发散属于积分概率度量类别的情况下精确结果。另一方面，我们在轻度条件下建立了切片分歧的样本复杂性并不依赖于问题尺寸。我们终于将一般结果应用于几个基地分歧，并说明了我们对合成和实际数据实验的理论。

近年来目睹了采用灵活的机械学习模型进行乐器变量（IV）回归的兴趣，但仍然缺乏不确定性量化方法的发展。在这项工作中，我们为IV次数回归提出了一种新的Quasi-Bayesian程序，建立了最近开发的核化IV模型和IV回归的双/极小配方。我们通过在$ l_2 $和sobolev规范中建立最低限度的最佳收缩率，并讨论可信球的常见有效性来分析所提出的方法的频繁行为。我们进一步推出了一种可扩展的推理算法，可以扩展到与宽神经网络模型一起工作。实证评价表明，我们的方法对复杂的高维问题产生了丰富的不确定性估计。

We develop an online kernel Cumulative Sum (CUSUM) procedure, which consists of a parallel set of kernel statistics with different window sizes to account for the unknown change-point location. Compared with many existing sliding window-based kernel change-point detection procedures, which correspond to the Shewhart chart-type procedure, the proposed procedure is more sensitive to small changes. We further present a recursive computation of detection statistics, which is crucial for online procedures to achieve a constant computational and memory complexity, such that we do not need to calculate and remember the entire Gram matrix, which can be a computational bottleneck otherwise. We obtain precise analytic approximations of the two fundamental performance metrics, the Average Run Length (ARL) and Expected Detection Delay (EDD). Furthermore, we establish the optimal window size on the order of $\log ({\rm ARL})$ such that there is nearly no power loss compared with an oracle procedure, which is analogous to the classic result for window-limited Generalized Likelihood Ratio (GLR) procedure. We present extensive numerical experiments to validate our theoretical results and the competitive performance of the proposed method.