虽然在矩阵完成文献中广泛研究了均匀的采样，但CUR采样近似于通过行样品和列样品近似矩阵。不幸的是，在现实世界应用中，这两种采样模型在各种情况下都缺乏灵活性。在这项工作中，我们提出了一种新颖且易于实现的采样策略，即跨浓缩采样（CCS）。通过桥接统一的采样和CUR采样，CCS提供了额外的灵活性，可以节省应用程序中的采样成本。此外，我们还为基于CCS的矩阵完成提供了足够的条件。此外，我们建议针对拟议的CCS模型，提出了一种高效的非凸算法，称为迭代CUR完成（ICURC）。数值实验验证了CCS和ICURC针对均匀采样及其基线算法的经验优势，这些实验在合成数据集和实际数据集上都验证了基线算法。

This paper is about a curious phenomenon. Suppose we have a data matrix, which is the superposition of a low-rank component and a sparse component. Can we recover each component individually? We prove that under some suitable assumptions, it is possible to recover both the low-rank and the sparse components exactly by solving a very convenient convex program called Principal Component Pursuit; among all feasible decompositions, simply minimize a weighted combination of the nuclear norm and of the 1 norm. This suggests the possibility of a principled approach to robust principal component analysis since our methodology and results assert that one can recover the principal components of a data matrix even though a positive fraction of its entries are arbitrarily corrupted. This extends to the situation where a fraction of the entries are missing as well. We discuss an algorithm for solving this optimization problem, and present applications in the area of video surveillance, where our methodology allows for the detection of objects in a cluttered background, and in the area of face recognition, where it offers a principled way of removing shadows and specularities in images of faces.

This article explores and analyzes the unsupervised clustering of large partially observed graphs. We propose a scalable and provable randomized framework for clustering graphs generated from the stochastic block model. The clustering is first applied to a sub-matrix of the graph's adjacency matrix associated with a reduced graph sketch constructed using random sampling. Then, the clusters of the full graph are inferred based on the clusters extracted from the sketch using a correlation-based retrieval step. Uniform random node sampling is shown to improve the computational complexity over clustering of the full graph when the cluster sizes are balanced. A new random degree-based node sampling algorithm is presented which significantly improves upon the performance of the clustering algorithm even when clusters are unbalanced. This framework improves the phase transitions for matrix-decomposition-based clustering with regard to computational complexity and minimum cluster size, which are shown to be nearly dimension-free in the low inter-cluster connectivity regime. A third sampling technique is shown to improve balance by randomly sampling nodes based on spatial distribution. We provide analysis and numerical results using a convex clustering algorithm based on matrix completion.

We consider the nonlinear inverse problem of learning a transition operator $\mathbf{A}$ from partial observations at different times, in particular from sparse observations of entries of its powers $\mathbf{A},\mathbf{A}^2,\cdots,\mathbf{A}^{T}$. This Spatio-Temporal Transition Operator Recovery problem is motivated by the recent interest in learning time-varying graph signals that are driven by graph operators depending on the underlying graph topology. We address the nonlinearity of the problem by embedding it into a higher-dimensional space of suitable block-Hankel matrices, where it becomes a low-rank matrix completion problem, even if $\mathbf{A}$ is of full rank. For both a uniform and an adaptive random space-time sampling model, we quantify the recoverability of the transition operator via suitable measures of incoherence of these block-Hankel embedding matrices. For graph transition operators these measures of incoherence depend on the interplay between the dynamics and the graph topology. We develop a suitable non-convex iterative reweighted least squares (IRLS) algorithm, establish its quadratic local convergence, and show that, in optimal scenarios, no more than $\mathcal{O}(rn \log(nT))$ space-time samples are sufficient to ensure accurate recovery of a rank-$r$ operator $\mathbf{A}$ of size $n \times n$. This establishes that spatial samples can be substituted by a comparable number of space-time samples. We provide an efficient implementation of the proposed IRLS algorithm with space complexity of order $O(r n T)$ and per-iteration time complexity linear in $n$. Numerical experiments for transition operators based on several graph models confirm that the theoretical findings accurately track empirical phase transitions, and illustrate the applicability and scalability of the proposed algorithm.

本文涉及低级矩阵恢复问题的$ \ ell_ {2,0} $ \ ell_ {2,0} $ - 正则化分解模型及其计算。引入了Qual $ \ ell_ {2,0} $ - 因子矩阵的规范，以促进因素和低级别解决方案的柱稀疏性。对于这种不透露的不连续优化问题，我们开发了一种具有外推的交替的多种化 - 最小化（AMM）方法，以及一个混合AMM，其中提出了一种主要的交替的近端方法，以寻找与较少的非零列和带外推的AMM的初始因子对。然后用于最小化平滑的非凸损失。我们为所提出的AMM方法提供全局收敛性分析，并使用非均匀采样方案将它们应用于矩阵完成问题。数值实验是用综合性和实际数据示例进行的，并且与核形态正则化分解模型的比较结果和MAX-NORM正则化凸模型显示柱$ \ ell_ {2,0} $ - 正则化分解模型具有优势在更短的时间内提供较低误差和排名的解决方案。

提供了一种强大而灵活的模型，可用于代表多属数据和多种方式相互作用，在科学和工程中的各个领域中发挥着现代数据科学中的不可或缺的作用。基本任务是忠实地以统计和计算的有效方式从高度不完整的测量中恢复张量。利用Tucker分解中的张量的低级别结构，本文开发了一个缩放的梯度下降（Scaledgd）算法，可以直接恢复具有定制频谱初始化的张量因子，并表明它以与条件号无关的线性速率收敛对于两个规范问题的地面真理张量 - 张量完成和张量回归 - 一旦样本大小高于$ n ^ {3/2} $忽略其他参数依赖项，$ n $是维度张量。这导致与现有技术相比的低秩张力估计的极其可扩展的方法，这些方法具有以下至少一个缺点：对记忆和计算方面的对不良，偏移成本高的极度敏感性，或差样本复杂性保证。据我们所知，Scaledgd是第一算法，它可以同时实现近最佳统计和计算复杂性，以便与Tucker分解进行低级张力完成。我们的算法突出了加速非耦合统计估计在加速非耦合统计估计中的适当预处理的功率，其中迭代改复的预处理器促进轨迹的所需的不变性属性相对于低级张量分解中的底层对称性。

We consider a problem of considerable practical interest: the recovery of a data matrix from a sampling of its entries. Suppose that we observe m entries selected uniformly at random from a matrix M . Can we complete the matrix and recover the entries that we have not seen?We show that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries. We prove that if the number m of sampled entries obeys m ≥ C n 1.2 r log n for some positive numerical constant C, then with very high probability, most n × n matrices of rank r can be perfectly recovered by solving a simple convex optimization program. This program finds the matrix with minimum nuclear norm that fits the data. The condition above assumes that the rank is not too large. However, if one replaces the 1.2 exponent with 1.25, then the result holds for all values of the rank. Similar results hold for arbitrary rectangular matrices as well. Our results are connected with the recent literature on compressed sensing, and show that objects other than signals and images can be perfectly reconstructed from very limited information.

矩阵的完成问题旨在从对其个别元素的观察中恢复低级$ r \ ll d $的$ d \ times d $地面真相矩阵。现实世界中的矩阵完成通常是一个巨大的优化问题，$ d $如此之大，以至于即使是$ O（d）$ o（d）$ o（d）$ o（d）$ o（d）$ o（d）$ o（d）$ o（d）$ o（d）$ o（d）$ o（d）$ o（d）$ o（d）$ d $的昂贵。随机梯度下降（SGD）是少数能够大规模求解矩阵完成的算法之一，也可以自然地通过不断发展的地面真相处理流数据。不幸的是，当底层地面真理不足时，SGD经历了戏剧性的减速。它至少需要$ o（\ kappa \ log（1/\ epsilon））$迭代才能获得$ \ epsilon $ -close $ \ epsilon $ -Close以接地真相矩阵，条件号$ \ kappa $。在本文中，我们提出了一个预处理的SGD版本，该版本保留了SGD的所有有利的实践素质用于大规模的在线优化，同时也使其不可知到$ \ kappa $。对于对称地面真相和根平方错误（RMSE）损失，我们证明预处理的SGD收敛到$ \ epsilon $ -Accuracy in $ o（\ log（1/\ epsilon））$ tererations $迭代，并具有快速的线性线性融合率好像地面真相是完美的条件，$ \ kappa = 1 $。在我们的数值实验中，我们观察到在1位跨透明拷贝损失下进行的不条件矩阵完成的加速度，以及贝叶斯个性化排名（BPR）损失等成对损失。

Tensor完成是矩阵完成的自然高阶泛化，其中目标是从其条目的稀疏观察中恢复低级张量。现有算法在没有可证明的担保的情况下是启发式，基于解决运行不切实际的大型半纤维程序，或者需要强大的假设，例如需要因素几乎正交。在本文中，我们介绍了交替最小化的新变型，其又通过了解如何对矩阵设置中的交替最小化的收敛性的进展措施来调整到张量设置的启发。我们展示了强大的可证明的保证，包括表明我们的算法即使当因素高度相关时，我们的算法也会在真正的张量线上会聚，并且可以在几乎线性的时间内实现。此外，我们的算法也非常实用，我们表明我们可以完成具有千维尺寸的三阶张量，从观察其条目的微小一部分。相比之下，有些令人惊讶的是，我们表明，如果没有我们的新扭曲，则表明交替最小化的标准版本可以在实践中以急剧速度收敛。

我们研究了\ textit {在线}低率矩阵完成的问题，并使用$ \ mathsf {m} $用户，$ \ mathsf {n} $项目和$ \ mathsf {t} $ rounds。在每回合中，我们建议每个用户一项。对于每个建议，我们都会从低级别的用户项目奖励矩阵中获得（嘈杂的）奖励。目的是设计一种以下遗憾的在线方法（以$ \ mathsf {t} $）。虽然该问题可以映射到标准的多臂强盗问题，其中每个项目都是\ textit {独立}手臂，但由于没有利用武器和用户之间的相关性，因此遗憾会导致遗憾。相比之下，由于低级别的歧管的非凸度，利用奖励矩阵的低排列结构是具有挑战性的。我们使用探索-Commit（etc）方法克服了这一挑战，该方法确保了$ O（\ Mathsf {polylog}（\ Mathsf {m}+\ \ \ \ \ Mathsf {n}）\ Mathsf {t}^{2/2/ 3}）$。 That is, roughly only $\mathsf{polylog} (\mathsf{M}+\mathsf{N})$ item recommendations are required per user to get non-trivial solution.我们进一步改善了排名$ 1 $设置的结果。在这里，我们提出了一种新颖的算法八进制（使用迭代用户群集的在线协作过滤），以确保$ O（\ Mathsf {polylog}（\ Mathsf {M}+\ Mathsf {N}）几乎最佳的遗憾。 ^{1/2}）$。我们的算法使用了一种新颖的技术，可以共同和迭代地消除项目，这使我们能够在$ \ Mathsf {t} $中获得几乎最小的最佳速率。

The affine rank minimization problem consists of finding a matrix of minimum rank that satisfies a given system of linear equality constraints. Such problems have appeared in the literature of a diverse set of fields including system identification and control, Euclidean embedding, and collaborative filtering. Although specific instances can often be solved with specialized algorithms, the general affine rank minimization problem is NP-hard, because it contains vector cardinality minimization as a special case.In this paper, we show that if a certain restricted isometry property holds for the linear transformation defining the constraints, the minimum rank solution can be recovered by solving a convex optimization problem, namely the minimization of the nuclear norm over the given affine space. We present several random ensembles of equations where the restricted isometry property holds with overwhelming probability, provided the codimension of the subspace is Ω(r(m + n) log mn), where m, n are the dimensions of the matrix, and r is its rank.The techniques used in our analysis have strong parallels in the compressed sensing framework. We discuss how affine rank minimization generalizes this pre-existing concept and outline a dictionary relating concepts from cardinality minimization to those of rank minimization. We also discuss several algorithmic approaches to solving the norm minimization relaxations, and illustrate our results with numerical examples.

We study inductive matrix completion (matrix completion with side information) under an i.i.d. subgaussian noise assumption at a low noise regime, with uniform sampling of the entries. We obtain for the first time generalization bounds with the following three properties: (1) they scale like the standard deviation of the noise and in particular approach zero in the exact recovery case; (2) even in the presence of noise, they converge to zero when the sample size approaches infinity; and (3) for a fixed dimension of the side information, they only have a logarithmic dependence on the size of the matrix. Differently from many works in approximate recovery, we present results both for bounded Lipschitz losses and for the absolute loss, with the latter relying on Talagrand-type inequalities. The proofs create a bridge between two approaches to the theoretical analysis of matrix completion, since they consist in a combination of techniques from both the exact recovery literature and the approximate recovery literature.

低秩矩阵恢复的现有结果在很大程度上专注于二次损失，这享有有利的性质，例如限制强的强凸/平滑度（RSC / RSM）以及在所有低等级矩阵上的良好调节。然而，许多有趣的问题涉及更一般，非二次损失，这不满足这些属性。对于这些问题，标准的非耦合方法，例如秩约为秩约为预定的梯度下降（A.K.A.迭代硬阈值）和毛刺蒙特罗分解可能具有差的经验性能，并且没有令人满意的理论保证了这些算法的全球和快速收敛。在本文中，我们表明，具有非二次损失的可证实低级恢复中的关键组成部分是规律性投影oracle。该Oracle限制在适当的界限集中迭代到低级矩阵，损耗功能在其上表现良好并且满足一组近似RSC / RSM条件。因此，我们分析配备有这样的甲骨文的（平均）投影的梯度方法，并证明它在全球和线性地收敛。我们的结果适用于广泛的非二次低级估计问题，包括一个比特矩阵感测/完成，个性化排名聚集，以及具有等级约束的更广泛的广义线性模型。

随机奇异值分解（RSVD）是用于计算大型数据矩阵截断的SVD的一类计算算法。给定A $ n \ times n $对称矩阵$ \ mathbf {m} $，原型RSVD算法输出通过计算$ \ mathbf {m mathbf {m} $的$ k $引导singular vectors的近似m}^{g} \ mathbf {g} $;这里$ g \ geq 1 $是一个整数，$ \ mathbf {g} \ in \ mathbb {r}^{n \ times k} $是一个随机的高斯素描矩阵。在本文中，我们研究了一般的“信号加上噪声”框架下的RSVD的统计特性，即，观察到的矩阵$ \ hat {\ mathbf {m}} $被认为是某种真实但未知的加法扰动信号矩阵$ \ mathbf {m} $。我们首先得出$ \ ell_2 $（频谱规范）和$ \ ell_ {2 \ to \ infty} $（最大行行列$ \ ell_2 $ norm）$ \ hat {\ hat {\ Mathbf {M}} $和信号矩阵$ \ Mathbf {M} $的真实单数向量。这些上限取决于信噪比（SNR）和功率迭代$ g $的数量。观察到一个相变现象，其中较小的SNR需要较大的$ g $值以保证$ \ ell_2 $和$ \ ell_ {2 \ to \ fo \ infty} $ distances的收敛。我们还表明，每当噪声矩阵满足一定的痕量生长条件时，这些相变发生的$ g $的阈值都会很清晰。最后，我们得出了近似奇异向量的行波和近似矩阵的进入波动的正常近似。我们通过将RSVD的几乎最佳性能保证在应用于三个统计推断问题的情况下，即社区检测，矩阵完成和主要的组件分析，并使用缺失的数据来说明我们的理论结果。

We investigate the problem of recovering a partially observed high-rank matrix whose columns obey a nonlinear structure such as a union of subspaces, an algebraic variety or grouped in clusters. The recovery problem is formulated as the rank minimization of a nonlinear feature map applied to the original matrix, which is then further approximated by a constrained non-convex optimization problem involving the Grassmann manifold. We propose two sets of algorithms, one arising from Riemannian optimization and the other as an alternating minimization scheme, both of which include first- and second-order variants. Both sets of algorithms have theoretical guarantees. In particular, for the alternating minimization, we establish global convergence and worst-case complexity bounds. Additionally, using the Kurdyka-Lojasiewicz property, we show that the alternating minimization converges to a unique limit point. We provide extensive numerical results for the recovery of union of subspaces and clustering under entry sampling and dense Gaussian sampling. Our methods are competitive with existing approaches and, in particular, high accuracy is achieved in the recovery using Riemannian second-order methods.

在这项工作中，我们估计具有高概率的张量的随机选择元素的数量，保证了黎曼梯度下降的局部收敛性，以便张力列车完成。基于展开奇异值的谐波平均值，我们从正交投影的正交投影推导出一个新的界限，并引入张力列车的核心相干概念。我们还将结果扩展到张力列车完成与侧面信息，并获得相应的本地收敛保证。

我们考虑一个矩阵完成问题，用于将社交或项目相似性图形作为侧面信息。我们开发了一种普遍的，无参数和计算的有效算法，该算法以分层图形聚类开始，然后迭代地改进图形聚类和矩阵额定值。在一个层次的随机块模型，尊重实际相关的社交图和低秩评级矩阵模型（要详细），我们证明了我们的算法实现了观察到的矩阵条目数量的信息 - 理论限制（即，最佳通过与较低的不可能结果一起导出的样本复杂性）通过最大似然估计。该结果的一个结果是利用社交图的层次结构，相对于简单地识别不同组的情况，在不诉诸于它们的情况下，可以产生相对于不同组的样本复杂性的大量增益。我们对合成和现实世界数据集进行了广泛的实验，以证实我们的理论结果，并展示了利用图形侧信息的其他矩阵完成算法的显着性能改进。

Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets.This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed-either explicitly or implicitly-to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, speed, and robustness. These claims are supported by extensive numerical experiments and a detailed error analysis.The specific benefits of randomized techniques depend on the computational environment. Consider the model problem of finding the k dominant components of the singular value decomposition of an m × n matrix. (i) For a dense input matrix, randomized algorithms require O(mn log(k)) floating-point operations (flops) in contrast with O(mnk) for classical algorithms. (ii) For a sparse input matrix, the flop count matches classical Krylov subspace methods, but the randomized approach is more robust and can easily be reorganized to exploit multi-processor architectures. (iii) For a matrix that is too large to fit in fast memory, the randomized techniques require only a constant number of passes over the data, as opposed to O(k) passes for classical algorithms. In fact, it is sometimes possible to perform matrix approximation with a single pass over the data.

In this paper, we study the trace regression when a matrix of parameters B* is estimated via the convex relaxation of a rank-regularized regression or via regularized non-convex optimization. It is known that these estimators satisfy near-optimal error bounds under assumptions on the rank, coherence, and spikiness of B*. We start by introducing a general notion of spikiness for B* that provides a generic recipe to prove the restricted strong convexity of the sampling operator of the trace regression and obtain near-optimal and non-asymptotic error bounds for the estimation error. Similar to the existing literature, these results require the regularization parameter to be above a certain theory-inspired threshold that depends on observation noise that may be unknown in practice. Next, we extend the error bounds to cases where the regularization parameter is chosen via cross-validation. This result is significant in that existing theoretical results on cross-validated estimators (Kale et al., 2011; Kumar et al., 2013; Abou-Moustafa and Szepesvari, 2017) do not apply to our setting since the estimators we study are not known to satisfy their required notion of stability. Finally, using simulations on synthetic and real data, we show that the cross-validated estimator selects a near-optimal penalty parameter and outperforms the theory-inspired approach of selecting the parameter.

本文向许多受访者调查了同时的偏好和度量学习。一组由$ d $二维功能向量和表格的配对比较``项目$ i $都比item $ j $更可取'的项目。我们的模型共同学习了一个距离指标，该指标表征了人群对项目相似性的一般度量，以及每个用户反映其个人喜好的潜在理想点。该模型具有捕获个人喜好的灵活性，同时享受在人群中摊销的度量学习样本成本。我们首先以无声的，连续的响应设置（即等于项目距离的差异）来研究这个问题，以了解学习的基本限制。接下来，我们建立了嘈杂的预测错误保证，可以从人类受访者那里收集诸如二进制测量值，并显示样品复杂性在基础度量较低时如何提高。最后，我们根据响应分布的假设建立恢复保证。我们在模拟数据和大量用户的颜色偏好判断数据集上演示了模型的性能。