个性化Pagerank（PPR）是无监督学习图表（例如节点排名，标签和图形嵌入）的基本工具。但是，尽管数据隐私是最近的最重要问题之一，但现有的PPR算法并非旨在保护用户隐私。 PPR对输入图边缘高度敏感：仅一个边缘的差异可能会导致PPR矢量发生很大变化，并可能泄漏私人用户数据。在这项工作中，我们提出了一种输出近似PPR的算法，并证明对输入边缘的敏感性有界限。此外，我们证明，当输入图具有较大的程度时，我们的算法与非私有算法相似。我们敏感性的PPR直接暗示了用于几种图形学习工具的私有算法，例如差异私有（DP）PPR排名，DP节点分类和DP节点嵌入。为了补充我们的理论分析，我们还经验验证了算法的实际性能。

在使用提供明确定义的隐私保证的用户数据时，至关重要。在这项工作中，我们旨在与第三方私下操纵和分享整个稀疏数据集。实际上，差异隐私已成为隐私的黄金标准，但是，当涉及到稀疏数据集时，作为我们的主要结果之一，我们证明\ emph {any}与最初的私人机制有差异化的私人机制数据集注定要拥有非常薄弱的隐私保证。因此，我们需要选择其他隐私概念，例如$ k $ - 匿名性更好地在这种情况下保存实用程序。在这项工作中，我们介绍了$ k $ - 匿名的变体，我们称之为平滑$ k $ - 匿名和设计简单算法，可有效地提供平滑的$ k $ - 匿名性。我们进一步执行经验评估以支持我们的理论保证，并表明我们的算法改善了匿名数据下游机器学习任务的性能。

最大信息系数（MIC）是一个强大的统计量，可以识别变量之间的依赖性。但是，它可以应用于敏感数据，并且发布可能会泄漏私人信息。作为解决方案，我们提出算法以提供差异隐私的方式近似麦克风。我们表明，经典拉普拉斯机制的自然应用产生的精度不足。因此，我们介绍了MICT统计量，这是一种新的MIC近似值，与差异隐私更加兼容。我们证明MICS是麦克风的一致估计器，我们提供了两个差异性私有版本。我们对各种真实和合成数据集进行实验。结果表明，私人微统计数据极大地超过了拉普拉斯机制的直接应用。此外，对现实世界数据集的实验显示出准确性，当样本量至少适中时可用。

In this work, we give efficient algorithms for privately estimating a Gaussian distribution in both pure and approximate differential privacy (DP) models with optimal dependence on the dimension in the sample complexity. In the pure DP setting, we give an efficient algorithm that estimates an unknown $d$-dimensional Gaussian distribution up to an arbitrary tiny total variation error using $\widetilde{O}(d^2 \log \kappa)$ samples while tolerating a constant fraction of adversarial outliers. Here, $\kappa$ is the condition number of the target covariance matrix. The sample bound matches best non-private estimators in the dependence on the dimension (up to a polylogarithmic factor). We prove a new lower bound on differentially private covariance estimation to show that the dependence on the condition number $\kappa$ in the above sample bound is also tight. Prior to our work, only identifiability results (yielding inefficient super-polynomial time algorithms) were known for the problem. In the approximate DP setting, we give an efficient algorithm to estimate an unknown Gaussian distribution up to an arbitrarily tiny total variation error using $\widetilde{O}(d^2)$ samples while tolerating a constant fraction of adversarial outliers. Prior to our work, all efficient approximate DP algorithms incurred a super-quadratic sample cost or were not outlier-robust. For the special case of mean estimation, our algorithm achieves the optimal sample complexity of $\widetilde O(d)$, improving on a $\widetilde O(d^{1.5})$ bound from prior work. Our pure DP algorithm relies on a recursive private preconditioning subroutine that utilizes the recent work on private mean estimation [Hopkins et al., 2022]. Our approximate DP algorithms are based on a substantial upgrade of the method of stabilizing convex relaxations introduced in [Kothari et al., 2022].

分散的优化在机器学习方面越来越受欢迎，其可伸缩性和效率。直观地，它也应提供更好的隐私保证，因为节点只能观察到网络图中其邻居发送的消息。但是，正式化和量化这一收益是具有挑战性的：现有结果通常仅限于当地差异隐私（LDP）保证忽略权力下放的优势。在这项工作中，我们介绍了成对网络差异隐私，这是一种放松的LDP，该隐藏率捕获了一个事实，即从节点$ u $到节点$ v $的隐私泄漏可能取决于它们在图中的相对位置。然后，我们分析局部噪声注入与固定和随机通信图上的（简单或随机）八卦方案的组合。我们还得出了一种差异化的分散优化算法，该算法在局部梯度下降步骤和八卦平均之间进行交替。我们的结果表明，我们的算法放大隐私保证是图表中节点之间距离的函数，与受信任策展人的隐私性权衡取舍相匹配，直到明确取决于图形拓扑的因素。最后，我们通过有关合成和现实世界数据集的实验来说明我们的隐私收益。

Differentially private algorithms for common metric aggregation tasks, such as clustering or averaging, often have limited practicality due to their complexity or to the large number of data points that is required for accurate results. We propose a simple and practical tool, $\mathsf{FriendlyCore}$, that takes a set of points ${\cal D}$ from an unrestricted (pseudo) metric space as input. When ${\cal D}$ has effective diameter $r$, $\mathsf{FriendlyCore}$ returns a "stable" subset ${\cal C} \subseteq {\cal D}$ that includes all points, except possibly few outliers, and is {\em certified} to have diameter $r$. $\mathsf{FriendlyCore}$ can be used to preprocess the input before privately aggregating it, potentially simplifying the aggregation or boosting its accuracy. Surprisingly, $\mathsf{FriendlyCore}$ is light-weight with no dependence on the dimension. We empirically demonstrate its advantages in boosting the accuracy of mean estimation and clustering tasks such as $k$-means and $k$-GMM, outperforming tailored methods.

聚类是数据分析中的一个根本问题。在差别私有聚类中，目标是识别$ k $群集中心，而不披露各个数据点的信息。尽管研究进展显着，但问题抵制了实际解决方案。在这项工作中，我们的目的是提供简单的可实现的差异私有聚类算法，当数据“简单”时，提供实用程序，例如，当簇之间存在显着的分离时。我们提出了一个框架，允许我们将非私有聚类算法应用于简单的实例，并私下结合结果。在高斯混合的某些情况下，我们能够改善样本复杂性界限，并获得$ k $ -means。我们与合成数据的实证评估补充了我们的理论分析。

差异隐私通常使用比理论更大的隐私参数应用于理想的理想。已经提出了宽大隐私参数的各种非正式理由。在这项工作中，我们考虑了部分差异隐私（DP），该隐私允许以每个属性为基础量化隐私保证。在此框架中，我们研究了几个基本数据分析和学习任务，并设计了其每个属性隐私参数的算法，其较小的人（即所有属性）的最佳隐私参数比最佳的隐私参数。

We establish a simple connection between robust and differentially-private algorithms: private mechanisms which perform well with very high probability are automatically robust in the sense that they retain accuracy even if a constant fraction of the samples they receive are adversarially corrupted. Since optimal mechanisms typically achieve these high success probabilities, our results imply that optimal private mechanisms for many basic statistics problems are robust. We investigate the consequences of this observation for both algorithms and computational complexity across different statistical problems. Assuming the Brennan-Bresler secret-leakage planted clique conjecture, we demonstrate a fundamental tradeoff between computational efficiency, privacy leakage, and success probability for sparse mean estimation. Private algorithms which match this tradeoff are not yet known -- we achieve that (up to polylogarithmic factors) in a polynomially-large range of parameters via the Sum-of-Squares method. To establish an information-computation gap for private sparse mean estimation, we also design new (exponential-time) mechanisms using fewer samples than efficient algorithms must use. Finally, we give evidence for privacy-induced information-computation gaps for several other statistics and learning problems, including PAC learning parity functions and estimation of the mean of a multivariate Gaussian.

Kernel matrices, as well as weighted graphs represented by them, are ubiquitous objects in machine learning, statistics and other related fields. The main drawback of using kernel methods (learning and inference using kernel matrices) is efficiency -- given $n$ input points, most kernel-based algorithms need to materialize the full $n \times n$ kernel matrix before performing any subsequent computation, thus incurring $\Omega(n^2)$ runtime. Breaking this quadratic barrier for various problems has therefore, been a subject of extensive research efforts. We break the quadratic barrier and obtain $\textit{subquadratic}$ time algorithms for several fundamental linear-algebraic and graph processing primitives, including approximating the top eigenvalue and eigenvector, spectral sparsification, solving linear systems, local clustering, low-rank approximation, arboricity estimation and counting weighted triangles. We build on the recent Kernel Density Estimation framework, which (after preprocessing in time subquadratic in $n$) can return estimates of row/column sums of the kernel matrix. In particular, we develop efficient reductions from $\textit{weighted vertex}$ and $\textit{weighted edge sampling}$ on kernel graphs, $\textit{simulating random walks}$ on kernel graphs, and $\textit{importance sampling}$ on matrices to Kernel Density Estimation and show that we can generate samples from these distributions in $\textit{sublinear}$ (in the support of the distribution) time. Our reductions are the central ingredient in each of our applications and we believe they may be of independent interest. We empirically demonstrate the efficacy of our algorithms on low-rank approximation (LRA) and spectral sparsification, where we observe a $\textbf{9x}$ decrease in the number of kernel evaluations over baselines for LRA and a $\textbf{41x}$ reduction in the graph size for spectral sparsification.

We study fine-grained error bounds for differentially private algorithms for counting under continual observation. Our main insight is that the matrix mechanism when using lower-triangular matrices can be used in the continual observation model. More specifically, we give an explicit factorization for the counting matrix $M_\mathsf{count}$ and upper bound the error explicitly. We also give a fine-grained analysis, specifying the exact constant in the upper bound. Our analysis is based on upper and lower bounds of the {\em completely bounded norm} (cb-norm) of $M_\mathsf{count}$. Along the way, we improve the best-known bound of 28 years by Mathias (SIAM Journal on Matrix Analysis and Applications, 1993) on the cb-norm of $M_\mathsf{count}$ for a large range of the dimension of $M_\mathsf{count}$. Furthermore, we are the first to give concrete error bounds for various problems under continual observation such as binary counting, maintaining a histogram, releasing an approximately cut-preserving synthetic graph, many graph-based statistics, and substring and episode counting. Finally, we note that our result can be used to get a fine-grained error bound for non-interactive local learning {and the first lower bounds on the additive error for $(\epsilon,\delta)$-differentially-private counting under continual observation.} Subsequent to this work, Henzinger et al. (SODA2023) showed that our factorization also achieves fine-grained mean-squared error.

Concentrated differential privacy" was recently introduced by Dwork and Rothblum as a relaxation of differential privacy, which permits sharper analyses of many privacy-preserving computations. We present an alternative formulation of the concept of concentrated differential privacy in terms of the Rényi divergence between the distributions obtained by running an algorithm on neighboring inputs. With this reformulation in hand, we prove sharper quantitative results, establish lower bounds, and raise a few new questions. We also unify this approach with approximate differential privacy by giving an appropriate definition of "approximate concentrated differential privacy."

We study the task of training regression models with the guarantee of label differential privacy (DP). Based on a global prior distribution on label values, which could be obtained privately, we derive a label DP randomization mechanism that is optimal under a given regression loss function. We prove that the optimal mechanism takes the form of a ``randomized response on bins'', and propose an efficient algorithm for finding the optimal bin values. We carry out a thorough experimental evaluation on several datasets demonstrating the efficacy of our algorithm.

随机漫游是许多机器学习算法中使用的基本原语，其中包括聚类和半监督学习中的几种应用。尽管他们的相关性，但最近推出了第一个计算随机散步的有效并行算法（Lacki等人）。不幸的是，他们的方法具有基本缺点：它们的算法是非本地的，因为它严重依赖于计算随机从输入图中的所有节点中散布，即使在许多实际应用中只对计算随机只能从一个小子集中散步感兴趣图中的节点。在本文中，我们介绍了一种新的算法，通过同时建立随机和本地的随机行走来克服这种限制。我们表明我们的技术既存储器也又高效，特别是产生有效的并行本地聚类算法。最后，我们将我们的理论分析补充了实验结果，表明我们的算法比以前的方法更可扩展。

我们考虑如何私下分享客观扰动，使用每个实例差异隐私（PDP）所产生的个性化隐私损失。标准差异隐私（DP）为我们提供了一个最坏的绑定，可能是相对于固定数据集的特定个人的隐私丢失的数量级。PDP框架对目标个人的隐私保障提供了更细粒度的分析，但每个实例隐私损失本身可能是敏感数据的函数。在本文中，我们分析了通过客观扰动释放私人经验风险最小化器的每案隐私丧失，并提出一组私下和准确地公布PDP损失的方法，没有额外的隐私费用。

We study the relationship between adversarial robustness and differential privacy in high-dimensional algorithmic statistics. We give the first black-box reduction from privacy to robustness which can produce private estimators with optimal tradeoffs among sample complexity, accuracy, and privacy for a wide range of fundamental high-dimensional parameter estimation problems, including mean and covariance estimation. We show that this reduction can be implemented in polynomial time in some important special cases. In particular, using nearly-optimal polynomial-time robust estimators for the mean and covariance of high-dimensional Gaussians which are based on the Sum-of-Squares method, we design the first polynomial-time private estimators for these problems with nearly-optimal samples-accuracy-privacy tradeoffs. Our algorithms are also robust to a constant fraction of adversarially-corrupted samples.

我们启动差异私有（DP）估计的研究，并访问少量公共数据。为了对D维高斯人进行私人估计，我们假设公共数据来自高斯人，该高斯与私人数据的基础高斯人的总变化距离可能消失了。我们表明，在纯或集中DP的约束下，D+1个公共数据样本足以从私人样本复杂性中删除对私人数据分布的范围参数的任何依赖性，而在没有公共数据的情况下，这是必不可少的。对于分离的高斯混合物，我们假设基本的公共和私人分布是相同的，我们考虑两个设置：（1）当给出独立于维度的公共数据时，可以根据多种方式改善私人样本复杂性混合组件的数量以及对分布范围参数的任何依赖性都可以在近似DP情况下去除； （2）当在维度上给出了一定数量的公共数据线性时，即使在集中的DP下，也可以独立于范围参数使私有样本复杂性使得可以对整体样本复杂性进行其他改进。

作为标准本地模型和中央模型之间的中间信任模型，差异隐私的洗牌模型已引起了人们的极大兴趣[EFMRTT19；CSUZZ19]。该模型的关键结果是，随机洗牌本地随机数据放大了差异隐私保证。这种放大意味着对数据匿名贡献的系统提供了更大的隐私保证[BEMMRLRKTS17]。在这项工作中，我们通过在理论和数字上逐渐改造结果来改善最新隐私放大的状态。我们的第一个贡献是对LDP Randomizers洗牌输出的R \'enyi差异隐私参数的首次渐近最佳分析。我们的第二个贡献是通过改组对隐私放大的新分析。该分析改进了[FMT20]的技术，并导致所有参数设置中的数值范围更紧密。

分析若干缔约方拥有的数据，同时在效用和隐私之间实现良好的权衡是联邦学习和分析的关键挑战。在这项工作中，我们介绍了一种新颖的差异隐私（LDP）的放松，自然地出现在完全分散的算法中，即，当参与者通过沿着网络图的边缘传播没有中央协调员的边缘交换信息时。我们呼叫网络DP的这种放松捕获了用户只有系统的本地视图。为了展示网络DP的相关性，我们研究了一个分散的计算模型，其中令牌在网络图上执行散步，并由接收它的方顺序更新。对于诸如实际求和，直方图计算和具有梯度下降的优化等任务，我们提出了在环和完整拓扑上的简单算法。我们证明，网络DP下我们算法的隐私式实用权折衷显着提高了LDP下可实现的内容（有时甚至与可信赖的策展人模型的效用）的可实现，首次显示正式隐私收益可以从中获得完全分散。我们的实验说明了通过随机梯度下降的分散训练方法的改进效用。

我们为其非私人对准减少$（\ varepsilon，\ delta）$差异私人（dp）统计估计，提供了一个相当一般的框架。作为本框架的主要应用，我们提供多项式时间和$（\ varepsilon，\ delta）$ - DP算法用于学习（不受限制的）高斯分布在$ \ mathbb {r} ^ d $。我们学习高斯的方法的样本复杂度高斯距离总变化距离$ \ alpha $是$ \ widetilde {o} \ left（\ frac {d ^ 2} {\ alpha ^ 2} + \ frac {d ^ 2 \ sqrt {\ ln {1 / \ delta}} {\ alpha \ varepsilon} \右）$，匹配（最多为对数因子）最佳已知的信息理论（非高效）样本复杂性上限的aden-ali， Ashtiani，Kamath〜（alt'21）。在一个独立的工作中，Kamath，Mouzakis，Singhal，Steinke和Ullman〜（Arxiv：2111.04609）使用不同的方法证明了类似的结果，并以$ O（d ^ {5/2}）$样本复杂性依赖于$ d $ 。作为我们的框架的另一个应用，我们提供了第一次多项式时间$（\ varepsilon，\ delta）$-dp算法，用于鲁棒学习（不受限制的）高斯。