In this paper, we propose Wasserstein Isometric Mapping (Wassmap), a nonlinear dimensionality reduction technique that provides solutions to some drawbacks in existing global nonlinear dimensionality reduction algorithms in imaging applications. Wassmap represents images via probability measures in Wasserstein space, then uses pairwise Wasserstein distances between the associated measures to produce a low-dimensional, approximately isometric embedding. We show that the algorithm is able to exactly recover parameters of some image manifolds including those generated by translations or dilations of a fixed generating measure. Additionally, we show that a discrete version of the algorithm retrieves parameters from manifolds generated from discrete measures by providing a theoretical bridge to transfer recovery results from functional data to discrete data. Testing of the proposed algorithms on various image data manifolds show that Wassmap yields good embeddings compared with other global and local techniques.

我们将最初在多维扩展和降低多元数据的降低领域发展为功能设置。我们专注于经典缩放和ISOMAP - 在这些领域中起重要作用的原型方法 - 并在功能数据分析的背景下展示它们的使用。在此过程中，我们强调了环境公制扮演的关键作用。

比较概率分布是许多机器学习算法的关键。最大平均差异（MMD）和最佳运输距离（OT）是在过去几年吸引丰富的关注的概率措施之间的两类距离。本文建立了一些条件，可以通过MMD规范控制Wassersein距离。我们的作品受到压缩统计学习（CSL）理论的推动，资源有效的大规模学习的一般框架，其中训练数据总结在单个向量（称为草图）中，该训练数据捕获与所考虑的学习任务相关的信息。在CSL中的现有结果启发，我们介绍了H \“较旧的较低限制的等距属性（H \”较旧的LRIP）并表明这家属性具有有趣的保证对压缩统计学习。基于MMD与Wassersein距离之间的关系，我们通过引入和研究学习任务的Wassersein可读性的概念来提供压缩统计学习的保证，即概率分布之间的某些特定于特定的特定度量，可以由Wassersein界定距离。

在此备忘录中，我们开发了一般框架，它允许同时研究$ \ MathBB R ^ D $和惠特尼在$ \ Mathbb r的离散和非离散子集附近的insoctry扩展问题附近的标签和未标记的近对准数据问题。^ d $与某些几何形状。此外，我们调查了与集群，维度减少，流形学习，视觉以及最小的能量分区，差异和最小最大优化的相关工作。给出了谐波分析，计算机视觉，歧管学习和与我们工作的信号处理中的众多开放问题。本发明内容中的一部分工作基于纸张中查尔斯Fefferman的联合研究[48]，[49]，[50]，[51]。

假设我们在$ \ mathbb {r} ^ d $和predictor x中的响应变量y在$ \ mathbb {r} ^ d $，以便为$ d \ geq 1 $。在置换或未解释的回归中，我们可以访问x和y上的单独无序数据，而不是在通常回归中的（x，y）-pabes上的数据。到目前为止，在文献中，案件$ d = 1 $已收到关注，请参阅例如近期的纸张和杂草[信息和推理，8,619--717]和Balabdaoui等人。 [J.马赫。学习。 res，22（172），1-60]。在本文中，我们考虑使用$ d \ geq 1 $的一般多变量设置。我们表明回归函数的周期性单调性的概念足以用于置换/未解释的回归模型中的识别和估计。我们在允许的回归设置中研究置换恢复，并在基于Kiefer-WolfoItz的基于代索的计算高效且易用算法[ANN。数学。统计部。，27,887--906]非参数最大似然估计和来自最佳运输理论的技术。我们在高斯噪声的相关均方方向误差误差上提供显式上限。与之前的案件的工作$ d = 1 $一样，置换/未解释的设置涉及潜在的解卷积问题的慢速（对数）收敛率。数值研究证实了我们的理论分析，并表明所提出的方法至少根据上述事先工作中的方法进行了比例，同时在计算复杂性方面取得了大量减少。

We introduce and study a novel model-selection strategy for Bayesian learning, based on optimal transport, along with its associated predictive posterior law: the Wasserstein population barycenter of the posterior law over models. We first show how this estimator, termed Bayesian Wasserstein barycenter (BWB), arises naturally in a general, parameter-free Bayesian model-selection framework, when the considered Bayesian risk is the Wasserstein distance. Examples are given, illustrating how the BWB extends some classic parametric and non-parametric selection strategies. Furthermore, we also provide explicit conditions granting the existence and statistical consistency of the BWB, and discuss some of its general and specific properties, providing insights into its advantages compared to usual choices, such as the model average estimator. Finally, we illustrate how this estimator can be computed using the stochastic gradient descent (SGD) algorithm in Wasserstein space introduced in a companion paper arXiv:2201.04232v2 [math.OC], and provide a numerical example for experimental validation of the proposed method.

我们研究了随着正则化参数的消失，差异调节的最佳转运的收敛性消失。一般差异的尖锐费率包括相对熵或$ l^{p} $正则化，一般运输成本和多边界问题。使用量化和Martingale耦合的新方法适用于非紧密的边际和实现，特别是对于所有有限$（2+ \ delta）$ - 时刻的边缘的熵正规化2-wasserstein距离的尖锐前阶项。

我们研究了由覆盖在R ^ M中的N维歧管支持的概率措施的近似 - 由可逆流和单层注射部件组成的神经网络。当M <= 3N时，我们显示R ^ n和r ^ m之间的注射流量在可扩展的嵌入物图像中支持的普遍近似措施，这是标准嵌入的适当子集。在这个制度拓扑障碍物中，拓扑障碍能够作为可允许的目标。当m> = 3n + 1时，我们使用称为*清洁技巧*的代数拓扑的论点来证明拓扑障碍物消失和注射般的流动普遍近似任何可分辨率的嵌入。沿途，我们表明，可以在Brehmer et Cranmer 2020中的猜想中建立“反向”可以建立铭刻流动网络的最优性。此外，设计的网络可以简单，它们可以配备其他属性，例如一个新的投影结果。

我们使用运输公制（Delon和Desolneux 2020）中的单变量高斯混合物中的任意度量空间$ \ MATHCAL {X} $研究数据表示。我们得出了由称为\ emph {Probabilistic Transfersers}的小神经网络实现的特征图的保证。我们的保证是记忆类型：我们证明了深度约为$ n \ log（n）$的概率变压器和大约$ n^2 $ can bi-h \'{o} lder嵌入任何$ n $ - 点数据集从低度量失真的$ \ Mathcal {x} $，从而避免了维数的诅咒。我们进一步得出了概率的bi-lipschitz保证，可以兑换失真量和随机选择的点与该失真的随机选择点的可能性。如果$ \ MATHCAL {X} $的几何形状足够规律，那么我们可以为数据集中的所有点获得更强的Bi-Lipschitz保证。作为应用程序，我们从Riemannian歧管，指标和某些类型的数据集中获得了神经嵌入保证金组合图。

本文通过引入几何深度学习（GDL）框架来构建通用馈电型型模型与可区分的流形几何形状兼容的通用馈电型模型，从而解决了对非欧国人数据进行处理的需求。我们表明，我们的GDL模型可以在受控最大直径的紧凑型组上均匀地近似任何连续目标函数。我们在近似GDL模型的深度上获得了最大直径和上限的曲率依赖性下限。相反，我们发现任何两个非分类紧凑型歧管之间始终都有连续的函数，任何“局部定义”的GDL模型都不能均匀地近似。我们的最后一个主要结果确定了数据依赖性条件，确保实施我们近似的GDL模型破坏了“维度的诅咒”。我们发现，任何“现实世界”（即有限）数据集始终满足我们的状况，相反，如果目标函数平滑，则任何数据集都满足我们的要求。作为应用，我们确认了以下GDL模型的通用近似功能：Ganea等。 （2018）的双波利馈电网络，实施Krishnan等人的体系结构。 （2015年）的深卡尔曼 - 滤波器和深度玛克斯分类器。我们构建了：Meyer等人的SPD-Matrix回归剂的通用扩展/变体。 （2011）和Fletcher（2003）的Procrustean回归剂。在欧几里得的环境中，我们的结果暗示了Kidger和Lyons（2020）的近似定理和Yarotsky和Zhevnerchuk（2019）无估计近似率的数据依赖性版本的定量版本。

Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets.This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed-either explicitly or implicitly-to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, speed, and robustness. These claims are supported by extensive numerical experiments and a detailed error analysis.The specific benefits of randomized techniques depend on the computational environment. Consider the model problem of finding the k dominant components of the singular value decomposition of an m × n matrix. (i) For a dense input matrix, randomized algorithms require O(mn log(k)) floating-point operations (flops) in contrast with O(mnk) for classical algorithms. (ii) For a sparse input matrix, the flop count matches classical Krylov subspace methods, but the randomized approach is more robust and can easily be reorganized to exploit multi-processor architectures. (iii) For a matrix that is too large to fit in fast memory, the randomized techniques require only a constant number of passes over the data, as opposed to O(k) passes for classical algorithms. In fact, it is sometimes possible to perform matrix approximation with a single pass over the data.

概率分布之间的差异措施，通常被称为统计距离，在概率理论，统计和机器学习中普遍存在。为了在估计这些距离的距离时，对维度的诅咒，最近的工作已经提出了通过带有高斯内核的卷积在测量的分布中平滑局部不规则性。通过该框架的可扩展性至高维度，我们研究了高斯平滑$ P $ -wassersein距离$ \ mathsf {w} _p ^ {（\ sigma）} $的结构和统计行为，用于任意$ p \ GEQ 1 $。在建立$ \ mathsf {w} _p ^ {（\ sigma）} $的基本度量和拓扑属性之后，我们探索$ \ mathsf {w} _p ^ {（\ sigma）}（\ hat {\ mu} _n，\ mu）$，其中$ \ hat {\ mu} _n $是$ n $独立观察的实证分布$ \ mu $。我们证明$ \ mathsf {w} _p ^ {（\ sigma）} $享受$ n ^ { - 1/2} $的参数经验融合速率，这对比$ n ^ { - 1 / d} $率对于未平滑的$ \ mathsf {w} _p $ why $ d \ geq 3 $。我们的证明依赖于控制$ \ mathsf {w} _p ^ {（\ sigma）} $ by $ p $ th-sting spoollow sobolev restion $ \ mathsf {d} _p ^ {（\ sigma）} $并导出限制$ \ sqrt {n} \，\ mathsf {d} _p ^ {（\ sigma）}（\ hat {\ mu} _n，\ mu）$，适用于所有尺寸$ d $。作为应用程序，我们提供了使用$ \ mathsf {w} _p ^ {（\ sigma）} $的两个样本测试和最小距离估计的渐近保证，使用$ p = 2 $的实验使用$ \ mathsf {d} _2 ^ {（\ sigma）} $。

We reformulate unsupervised dimension reduction problem (UDR) in the language of tempered distributions, i.e. as a problem of approximating an empirical probability density function by another tempered distribution, supported in a $k$-dimensional subspace. We show that this task is connected with another classical problem of data science -- the sufficient dimension reduction problem (SDR). In fact, an algorithm for the first problem induces an algorithm for the second and vice versa. In order to reduce an optimization problem over distributions to an optimization problem over ordinary functions we introduce a nonnegative penalty function that ``forces'' the support of the model distribution to be $k$-dimensional. Then we present an algorithm for the minimization of the penalized objective, based on the infinite-dimensional low-rank optimization, which we call the alternating scheme. Also, we design an efficient approximate algorithm for a special case of the problem, where the distance between the empirical distribution and the model distribution is measured by Maximum Mean Discrepancy defined by a Mercer kernel of a certain type. We test our methods on four examples (three UDR and one SDR) using synthetic data and standard datasets.

我们研究基于度量传输的非参数密度估计器的收敛性和相关距离。这些估计量代表了利息的度量，作为传输图下选择的参考分布的推动力，其中地图是通过最大似然目标选择（等效地，将经验性的kullback-leibler损失）或其受惩罚版本选择。我们通过将M估计的技术与基于运输的密度表示的分析性能相结合，为一般惩罚措施估计量的一般类别的措施运输估计器建立了浓度不平等。然后，我们证明了我们的理论对三角形knothe-rosenblatt（kr）在$ d $维单元方面的运输的含义，并表明该估计器的惩罚和未化的版本都达到了Minimax最佳收敛速率，超过了H \ \ \'“较旧的密度类别。具体来说，我们建立了在有限的h \“较旧型球上，未确定的非参数最大似然估计，然后在某些sobolev-penalate的估计器和筛分的小波估计器中建立了最佳速率。

In this work we study statistical properties of graph-based algorithms for multi-manifold clustering (MMC). In MMC the goal is to retrieve the multi-manifold structure underlying a given Euclidean data set when this one is assumed to be obtained by sampling a distribution on a union of manifolds $\mathcal{M} = \mathcal{M}_1 \cup\dots \cup \mathcal{M}_N$ that may intersect with each other and that may have different dimensions. We investigate sufficient conditions that similarity graphs on data sets must satisfy in order for their corresponding graph Laplacians to capture the right geometric information to solve the MMC problem. Precisely, we provide high probability error bounds for the spectral approximation of a tensorized Laplacian on $\mathcal{M}$ with a suitable graph Laplacian built from the observations; the recovered tensorized Laplacian contains all geometric information of all the individual underlying manifolds. We provide an example of a family of similarity graphs, which we call annular proximity graphs with angle constraints, satisfying these sufficient conditions. We contrast our family of graphs with other constructions in the literature based on the alignment of tangent planes. Extensive numerical experiments expand the insights that our theory provides on the MMC problem.

Experimental sciences have come to depend heavily on our ability to organize, interpret and analyze high-dimensional datasets produced from observations of a large number of variables governed by natural processes. Natural laws, conservation principles, and dynamical structure introduce intricate inter-dependencies among these observed variables, which in turn yield geometric structure, with fewer degrees of freedom, on the dataset. We show how fine-scale features of this structure in data can be extracted from \emph{discrete} approximations to quantum mechanical processes given by data-driven graph Laplacians and localized wavepackets. This data-driven quantization procedure leads to a novel, yet natural uncertainty principle for data analysis induced by limited data. We illustrate the new approach with algorithms and several applications to real-world data, including the learning of patterns and anomalies in social distancing and mobility behavior during the COVID-19 pandemic.

We consider the problem of estimating a multivariate function $f_0$ of bounded variation (BV), from noisy observations $y_i = f_0(x_i) + z_i$ made at random design points $x_i \in \mathbb{R}^d$, $i=1,\ldots,n$. We study an estimator that forms the Voronoi diagram of the design points, and then solves an optimization problem that regularizes according to a certain discrete notion of total variation (TV): the sum of weighted absolute differences of parameters $\theta_i,\theta_j$ (which estimate the function values $f_0(x_i),f_0(x_j)$) at all neighboring cells $i,j$ in the Voronoi diagram. This is seen to be equivalent to a variational optimization problem that regularizes according to the usual continuum (measure-theoretic) notion of TV, once we restrict the domain to functions that are piecewise constant over the Voronoi diagram. The regression estimator under consideration hence performs (shrunken) local averaging over adaptively formed unions of Voronoi cells, and we refer to it as the Voronoigram, following the ideas in Koenker (2005), and drawing inspiration from Tukey's regressogram (Tukey, 1961). Our contributions in this paper span both the conceptual and theoretical frontiers: we discuss some of the unique properties of the Voronoigram in comparison to TV-regularized estimators that use other graph-based discretizations; we derive the asymptotic limit of the Voronoi TV functional; and we prove that the Voronoigram is minimax rate optimal (up to log factors) for estimating BV functions that are essentially bounded.

聚类是基于它们的相似性对组对象的重要探索性数据分析技术。广泛使用的$ k $ -MEANS聚类方法依赖于一些距离的概念将数据划分为较少数量的组。在欧几里得空间中，$ k $ -Means的基于质心和基于距离的公式相同。在现代机器学习应用中，数据通常是作为概率分布而出现的，并且可以使用最佳运输指标来处理测量值数据。由于瓦斯坦斯坦空间的非负亚历山德罗夫曲率，巴里中心遭受了规律性和非舒适性问题。 Wasserstein Barycenters的特殊行为可能使基于质心的配方无法代表集群内的数据点，而基于距离的$ K $ -MEANS方法及其半决赛计划（SDP）可以恢复真实的方法集群标签。在聚集高斯分布的特殊情况下，我们表明SDP放松的Wasserstein $ k $ - 金钱可以实现精确的恢复，因为这些集群按照$ 2 $ - WASSERSTEIN MERTRIC进行了良好的分离。我们的仿真和真实数据示例还表明，基于距离的$ K $ -Means可以比基于标准的基于质心的$ k $ -Means获得更好的分类性能，用于聚类概率分布和图像。

由编码器和解码器组成的自动编码器被广泛用于机器学习，以缩小高维数据的尺寸。编码器将输入数据歧管嵌入到较低的潜在空间中，而解码器表示反向映射，从而提供了潜在空间中的歧管的数据歧管的参数化。嵌入式歧管的良好规律性和结构可以实质性地简化进一步的数据处理任务，例如群集分析或数据插值。我们提出并分析了一种新的正则化，以学习自动编码器的编码器组件：一种损失功能，可倾向于等距，外层平坦的嵌入，并允许自行训练编码器。为了进行训练，假定对于输入歧管上的附近点，他们的本地riemannian距离及其本地riemannian平均水平可以评估。损失函数是通过蒙特卡洛集成计算的，具有不同的采样策略，用于输入歧管上的一对点。我们的主要定理将嵌入图的几何损失函数识别为$ \ gamma $ - 依赖于采样损失功能的限制。使用编码不同明确给定的数据歧管的图像数据的数值测试表明，将获得平滑的歧管嵌入到潜在空间中。由于促进了外部平坦度，这些嵌入足够规律，因此在潜在空间中线性插值可以作为一种可能的后处理。

本文介绍了一种新的基于仿真的推理程序，以对访问I.I.D. \ samples的多维概率分布进行建模和样本，从而规避明确建模密度函数或设计Markov Chain Monte Carlo的通常方法。我们提出了一个称为可逆的Gromov-monge（RGM）距离的新概念的距离和同构的动机，并研究了RGM如何用于设计新的转换样本，以执行基于模拟的推断。我们的RGM采样器还可以估计两个异质度量度量空间之间的最佳对齐$（\ cx，\ mu，c _ {\ cx}）$和$（\ cy，\ cy，\ nu，c _ {\ cy}）$从经验数据集中，估计的地图大约将一个量度$ \ mu $推向另一个$ \ nu $，反之亦然。我们研究了RGM距离的分析特性，并在轻度条件下得出RGM等于经典的Gromov-Wasserstein距离。奇怪的是，与Brenier的两极分解结合了连接，我们表明RGM采样器以$ C _ {\ cx} $和$ C _ {\ cy} $的正确选择诱导了强度同构的偏见。研究了有关诱导采样器的收敛，表示和优化问题的统计率。还展示了展示RGM采样器有效性的合成和现实示例。