结合神经网络是一种长期的技术，可以通过委员会决定通过将网络与正交性结合到正交属性来改善神经网络的概括错误。我们表明，该技术非常适合在医疗数据上进行机器学习：首先，合奏可以平行和异步学习，从而有效地培训患者特定的组件神经网络。其次，基于选择不相关的患者特定网络来最大程度地减少概括错误的想法，我们表明可以建立一些选定的特定于患者特定模型的合奏，以优于在更大的合并数据集中训练的单个模型。第三，非著作集合组合步骤是一个最佳的低维入口点，用于应用输出扰动以确保患者特定的网络的隐私。我们使用临床专家标记的现实生活中的重症监护病房数据来体现差异私人合奏在败血症早期预测任务上的框架。

Differential privacy is a strong notion for privacy that can be used to prove formal guarantees, in terms of a privacy budget, , about how much information is leaked by a mechanism. However, implementations of privacy-preserving machine learning often select large values of in order to get acceptable utility of the model, with little understanding of the impact of such choices on meaningful privacy. Moreover, in scenarios where iterative learning procedures are used, differential privacy variants that offer tighter analyses are used which appear to reduce the needed privacy budget but present poorly understood trade-offs between privacy and utility. In this paper, we quantify the impact of these choices on privacy in experiments with logistic regression and neural network models. Our main finding is that there is a huge gap between the upper bounds on privacy loss that can be guaranteed, even with advanced mechanisms, and the effective privacy loss that can be measured using current inference attacks. Current mechanisms for differentially private machine learning rarely offer acceptable utility-privacy trade-offs with guarantees for complex learning tasks: settings that provide limited accuracy loss provide meaningless privacy guarantees, and settings that provide strong privacy guarantees result in useless models.

数十年来，计算机系统持有大量个人数据。一方面，这种数据丰度允许在人工智能（AI），尤其是机器学习（ML）模型中突破。另一方面，它可能威胁用户的隐私并削弱人类与人工智能之间的信任。最近的法规要求，可以从一般情况下从计算机系统中删除有关用户的私人信息，特别是根据要求从ML模型中删除（例如，“被遗忘的权利”）。虽然从后端数据库中删除数据应该很简单，但在AI上下文中，它不够，因为ML模型经常“记住”旧数据。现有的对抗攻击证明，我们可以从训练有素的模型中学习私人会员或培训数据的属性。这种现象要求采用新的范式，即机器学习，以使ML模型忘记了特定的数据。事实证明，由于缺乏共同的框架和资源，最近在机器上学习的工作无法完全解决问题。在本调查文件中，我们试图在其定义，场景，机制和应用中对机器进行彻底的研究。具体而言，作为最先进的研究的类别集合，我们希望为那些寻求机器未学习的入门及其各种表述，设计要求，删除请求，算法和用途的人提供广泛的参考。 ML申请。此外，我们希望概述范式中的关键发现和趋势，并突出显示尚未看到机器无法使用的新研究领域，但仍可以受益匪浅。我们希望这项调查为ML研究人员以及寻求创新隐私技术的研究人员提供宝贵的参考。我们的资源是在https://github.com/tamlhp/awesome-machine-unlearning上。

从公共机器学习（ML）模型中泄漏数据是一个越来越重要的领域，因为ML的商业和政府应用可以利用多个数据源，可能包括用户和客户的敏感数据。我们对几个方面的当代进步进行了全面的调查，涵盖了非自愿数据泄漏，这对ML模型很自然，潜在的恶毒泄漏是由隐私攻击引起的，以及目前可用的防御机制。我们专注于推理时间泄漏，这是公开可用模型的最可能场景。我们首先在不同的数据，任务和模型体系结构的背景下讨论什么是泄漏。然后，我们提出了跨非自愿和恶意泄漏的分类法，可用的防御措施，然后进行当前可用的评估指标和应用。我们以杰出的挑战和开放性的问题结束，概述了一些有希望的未来研究方向。

We initiate the study of privacy in pharmacogenetics, wherein machine learning models are used to guide medical treatments based on a patient's genotype and background. Performing an in-depth case study on privacy in personalized warfarin dosing, we show that suggested models carry privacy risks, in particular because attackers can perform what we call model inversion: an attacker, given the model and some demographic information about a patient, can predict the patient's genetic markers.As differential privacy (DP) is an oft-proposed solution for medical settings such as this, we evaluate its effectiveness for building private versions of pharmacogenetic models. We show that DP mechanisms prevent our model inversion attacks when the privacy budget is carefully selected. We go on to analyze the impact on utility by performing simulated clinical trials with DP dosing models. We find that for privacy budgets effective at preventing attacks, patients would be exposed to increased risk of stroke, bleeding events, and mortality. We conclude that current DP mechanisms do not simultaneously improve genomic privacy while retaining desirable clinical efficacy, highlighting the need for new mechanisms that should be evaluated in situ using the general methodology introduced by our work.

联合学习允许许多设备在机器学习模型的培训中进行协作。与传统的机器学习一样，越来越关注的是，接受联合学习的模型可能会对不同的人群组表现出不同的表现。现有的解决方案来衡量和确保跨小组的平等模型绩效需要访问有关小组成员的信息，但是此访问并不总是可用或可取的，尤其是在联邦学习的隐私愿望下。我们研究了衡量此类性能差异的可行性，同时保护用户组成员资格的隐私以及联合模型在用户数据上的性能。保护两者对于隐私至关重要，因为它们可能是相关的，因此学习一个可能会揭示另一个。另一方面，从公用事业的角度来看，保留隐私的数据应保持相关性，以确保能够对性能差异进行准确的测量。我们通过开发当地差异化的私人机制来实现这两个目标，从而保留小组成员和模型绩效之间的相关性。为了分析机制的有效性，我们在对给定隐私预算进行优化时估算差异时的错误，并在合成数据上验证这些界限。我们的结果表明，对于参与的客户数量的实际数量，错误迅速减少，这表明，与先前的工作相反，保护受保护属性的隐私不一定与确定联合模型性能的差异相抵触。

Applying machine learning (ML) to sensitive domains requires privacy protection of the underlying training data through formal privacy frameworks, such as differential privacy (DP). Yet, usually, the privacy of the training data comes at the cost of the resulting ML models' utility. One reason for this is that DP uses one uniform privacy budget epsilon for all training data points, which has to align with the strictest privacy requirement encountered among all data holders. In practice, different data holders have different privacy requirements and data points of data holders with lower requirements can contribute more information to the training process of the ML models. To account for this need, we propose two novel methods based on the Private Aggregation of Teacher Ensembles (PATE) framework to support the training of ML models with individualized privacy guarantees. We formally describe the methods, provide a theoretical analysis of their privacy bounds, and experimentally evaluate their effect on the final model's utility using the MNIST, SVHN, and Adult income datasets. Our empirical results show that the individualized privacy methods yield ML models of higher accuracy than the non-individualized baseline. Thereby, we improve the privacy-utility trade-off in scenarios in which different data holders consent to contribute their sensitive data at different individual privacy levels.

Electronic Health Records (EHRs) are a valuable asset to facilitate clinical research and point of care applications; however, many challenges such as data privacy concerns impede its optimal utilization. Deep generative models, particularly, Generative Adversarial Networks (GANs) show great promise in generating synthetic EHR data by learning underlying data distributions while achieving excellent performance and addressing these challenges. This work aims to review the major developments in various applications of GANs for EHRs and provides an overview of the proposed methodologies. For this purpose, we combine perspectives from healthcare applications and machine learning techniques in terms of source datasets and the fidelity and privacy evaluation of the generated synthetic datasets. We also compile a list of the metrics and datasets used by the reviewed works, which can be utilized as benchmarks for future research in the field. We conclude by discussing challenges in GANs for EHRs development and proposing recommended practices. We hope that this work motivates novel research development directions in the intersection of healthcare and machine learning.

我们审查在机器学习（ML）中使用差异隐私（DP）对隐私保护的使用。我们表明，在维护学习模型的准确性的驱动下，基于DP的ML实现非常宽松，以至于它们不提供DP的事前隐私保证。取而代之的是，他们提供的基本上是与传统（经常受到批评的）统计披露控制方法相似的噪声。由于缺乏正式的隐私保证，因此所提供的实际隐私水平必须经过实验评估，这很少进行。在这方面，我们提出的经验结果表明，ML中的标准反拟合技术可以比DP实现更好的实用性/隐私/效率权衡。

Covid-19的早期检测是一个持续的研究领域，可以帮助潜在患者的潜在患者进行分类，监测和一般健康评估，并可能降低应对冠状病毒大流行病的医院的运营压力。在文献中使用了不同的机器学习技术，用于使用常规临床数据（血液测试和生命体征）来检测冠状病毒。使用这些型号时，数据漏洞和信息泄漏可以带来声誉损害并导致医院的法律问题。尽管如此，保护避免潜在敏感信息泄漏的医疗保健模型是一个被人吸引人的研究区。在这项工作中，我们检查了两种机器学习方法，旨在预测使用常规收集和易于使用的临床数据的患者的Covid-19状态。我们雇用对抗性培训来探索强大的深度学习架构，保护与有关患者的人口统计信息相关的属性。我们在这项工作中检查的两种模型旨在保持对抗对抗攻击和信息泄漏的敏感信息。在一系列使用来自牛津大学医院的数据集，Bedfordshire医院NHS Foundation Trust，大学医院伯明翰NHS基金会信托，而朴茨茅斯医院大学NHS信任我们训练并测试两个神经网络，以使用来自基本实验室血液的信息预测PCR测试结果的神经网络对患者到达医院的测试和生命体征。我们评估其每个模型的隐私水平可以提供和展示我们提出的架构对可比基线的效力和稳健性。我们的主要贡献之一是，我们专门针对具有内置机制的有效Covid-19检测模型的开发，以便选择性地保护对抗对抗攻击的敏感属性。

Distributing machine learning predictors enables the collection of large-scale datasets while leaving sensitive raw data at trustworthy sites. We show that locally training support vector machines (SVMs) and computing their averages leads to a learning technique that is scalable to a large number of users, satisfies differential privacy, and is applicable to non-trivial tasks, such as CIFAR-10. For a large number of participants, communication cost is one of the main challenges. We achieve a low communication cost by requiring only a single invocation of an efficient secure multiparty summation protocol. By relying on state-of-the-art feature extractors (SimCLR), we are able to utilize differentially private convex learners for non-trivial tasks such as CIFAR-10. Our experimental results illustrate that for $1{,}000$ users with $50$ data points each, our scheme outperforms state-of-the-art scalable distributed learning methods (differentially private federated learning, short DP-FL) while requiring around $500$ times fewer communication costs: For CIFAR-10, we achieve a classification accuracy of $79.7\,\%$ for an $\varepsilon = 0.59$ while DP-FL achieves $57.6\,\%$. More generally, we prove learnability properties for the average of such locally trained models: convergence and uniform stability. By only requiring strongly convex, smooth, and Lipschitz-continuous objective functions, locally trained via stochastic gradient descent (SGD), we achieve a strong utility-privacy tradeoff.

差异隐私（DP）已被出现为严格的形式主义，以推理可量化的隐私泄漏。在机器学习（ML）中，已采用DP限制推理/披露训练示例。在现有的工作中杠杆横跨ML管道，尽管隔离，通常专注于梯度扰动等机制。在本文中，我们展示了DP-util，DP整体实用分析框架，跨越ML管道，重点是输入扰动，客观扰动，梯度扰动，输出扰动和预测扰动。在隐私敏感数据上给出ML任务，DP-Util使ML隐私从业者能够对DP在这五个扰动点中的影响，以模型公用事业丢失，隐私泄漏和真正透露的数量来测量DP的影响。训练样本。我们在视觉，医疗和金融数据集上使用两个代表性学习算法（Logistic回归和深神经网络）来评估DP-Uts，以防止会员资格推论攻击作为案例研究攻击。我们结果的一个亮点是，预测扰动一致地在所有数据集中始终如一地实现所有模型的最低实用损耗。在Logistic回归模型中，与其他扰动技术相比，客观扰动导致最低的隐私泄漏。对于深度神经网络，梯度扰动导致最低的隐私泄漏。此外，我们的结果揭示了记录的结果表明，由于隐私泄漏增加，差异私有模型揭示了更多数量的成员样本。总体而言，我们的研究结果表明，为了使使用的扰动机制有明智的决定，ML隐私从业者需要检查优化技术（凸与非凸），扰动机制，课程数量和隐私预算之间的动态。

深度神经网络（DNNS）铰接对大型数据集的可用性的最新成功;但是，对此类数据集的培训经常为敏感培训信息构成隐私风险。在本文中，我们的目标是探讨生成模型和梯度稀疏性的力量，并提出了一种可扩展的隐私保留生成模型数据标准。与标准展示隐私保留框架相比，允许教师对一维预测进行投票，在高维梯度向量上投票在隐私保存方面具有挑战性。随着需要尺寸减少技术，我们需要在（1）之间的改进之间导航精致的权衡空间，并进行SGD收敛的放缓。为了解决这一点，我们利用通信高效学习，并通过将顶-K压缩与相应的噪声注入机构相结合，提出一种新的噪声压缩和聚集方法TopAGG。理论上，我们证明了DataLens框架保证了其生成数据的差异隐私，并提供了其收敛性的分析。为了展示DataLens的实际使用情况，我们对不同数据集进行广泛的实验，包括Mnist，Fashion-Mnist和高维Celeba，并且我们表明，DataLens显着优于其他基线DP生成模型。此外，我们改进了所提出的Topagg方法，该方法是DP SGD培训的主要构建块之一，并表明它能够在大多数情况下实现比最先进的DP SGD方法更高的效用案件。我们的代码在HTTPS://github.com/ai-secure/datalens公开提供。

预测组合在预测社区中蓬勃发展，近年来，已经成为预测研究和活动主流的一部分。现在，由单个（目标）系列产生的多个预测组合通过整合来自不同来源收集的信息，从而提高准确性，从而减轻了识别单个“最佳”预测的风险。组合方案已从没有估计的简单组合方法演变为涉及时间变化的权重，非线性组合，组件之间的相关性和交叉学习的复杂方法。它们包括结合点预测和结合概率预测。本文提供了有关预测组合的广泛文献的最新评论，并参考可用的开源软件实施。我们讨论了各种方法的潜在和局限性，并突出了这些思想如何随着时间的推移而发展。还调查了有关预测组合实用性的一些重要问题。最后，我们以当前的研究差距和未来研究的潜在见解得出结论。

Federated learning facilitates the collaborative training of models without the sharing of raw data. However, recent attacks demonstrate that simply maintaining data locality during training processes does not provide sufficient privacy guarantees. Rather, we need a federated learning system capable of preventing inference over both the messages exchanged during training and the final trained model while ensuring the resulting model also has acceptable predictive accuracy. Existing federated learning approaches either use secure multiparty computation (SMC) which is vulnerable to inference or differential privacy which can lead to low accuracy given a large number of parties with relatively small amounts of data each. In this paper, we present an alternative approach that utilizes both differential privacy and SMC to balance these trade-offs. Combining differential privacy with secure multiparty computation enables us to reduce the growth of noise injection as the number of parties increases without sacrificing privacy while maintaining a pre-defined rate of trust. Our system is therefore a scalable approach that protects against inference threats and produces models with high accuracy. Additionally, our system can be used to train a variety of machine learning models, which we validate with experimental results on 3 different machine learning algorithms. Our experiments demonstrate that our approach out-performs state of the art solutions. CCS CONCEPTS• Security and privacy → Privacy-preserving protocols; Trust frameworks; • Computing methodologies → Learning settings.

合奏的基本分支混合合奏在许多机器学习问题，尤其是回归中蓬勃发展。几项研究证实了多样性的重要性。但是，以前的合奏仅考虑在子模型训练阶段的多样性，与单个模型相比，改进有限。相反，本研究从异质模型池中选择和权重子模型。它使用内点过滤线性搜索算法解决了优化问题。这种优化问题创新地将负相关学习作为惩罚项，可以选择多种模型子集。实验结果显示了一些有意义的观点。模型池构造需要不同类别的模型，每个类别都作为子模型为所有可能的参数集。选择每个类的最佳子模型以构建基于NCL的合奏，该集合比子模型的平均值要好得多。此外，与经典常数和非恒定加权方法相比，基于NCL的合奏在几种预测指标中具有重要优势。实际上，由于模型不确定性，很难在事先结论数据集的最佳子模型。但是，我们的方法将获得可比较的精度作为RMSE度量的潜在最佳子模型。总之，这项研究的价值在于它的易用性和有效性，使混合团合奏可以接受多样性和准确性。

Although query-based systems (QBS) have become one of the main solutions to share data anonymously, building QBSes that robustly protect the privacy of individuals contributing to the dataset is a hard problem. Theoretical solutions relying on differential privacy guarantees are difficult to implement correctly with reasonable accuracy, while ad-hoc solutions might contain unknown vulnerabilities. Evaluating the privacy provided by QBSes must thus be done by evaluating the accuracy of a wide range of privacy attacks. However, existing attacks require time and expertise to develop, need to be manually tailored to the specific systems attacked, and are limited in scope. In this paper, we develop QuerySnout (QS), the first method to automatically discover vulnerabilities in QBSes. QS takes as input a target record and the QBS as a black box, analyzes its behavior on one or more datasets, and outputs a multiset of queries together with a rule to combine answers to them in order to reveal the sensitive attribute of the target record. QS uses evolutionary search techniques based on a novel mutation operator to find a multiset of queries susceptible to lead to an attack, and a machine learning classifier to infer the sensitive attribute from answers to the queries selected. We showcase the versatility of QS by applying it to two attack scenarios, three real-world datasets, and a variety of protection mechanisms. We show the attacks found by QS to consistently equate or outperform, sometimes by a large margin, the best attacks from the literature. We finally show how QS can be extended to QBSes that require a budget, and apply QS to a simple QBS based on the Laplace mechanism. Taken together, our results show how powerful and accurate attacks against QBSes can already be found by an automated system, allowing for highly complex QBSes to be automatically tested "at the pressing of a button".

机器学习（ML）模型已广泛应用于各种应用，包括图像分类，文本生成，音频识别和图形数据分析。然而，最近的研究表明，ML模型容易受到隶属推导攻击（MIS），其目的是推断数据记录是否用于训练目标模型。 ML模型上的MIA可以直接导致隐私违规行为。例如，通过确定已经用于训练与某种疾病相关的模型的临床记录，攻击者可以推断临床记录的所有者具有很大的机会。近年来，MIS已被证明对各种ML模型有效，例如，分类模型和生成模型。同时，已经提出了许多防御方法来减轻米西亚。虽然ML模型上的MIAS形成了一个新的新兴和快速增长的研究区，但还没有对这一主题进行系统的调查。在本文中，我们对会员推论和防御进行了第一个全面调查。我们根据其特征提供攻击和防御的分类管理，并讨论其优点和缺点。根据本次调查中确定的限制和差距，我们指出了几个未来的未来研究方向，以激发希望遵循该地区的研究人员。这项调查不仅是研究社区的参考，而且还为该研究领域之外的研究人员带来了清晰的照片。为了进一步促进研究人员，我们创建了一个在线资源存储库，并与未来的相关作品继续更新。感兴趣的读者可以在https://github.com/hongshenghu/membership-inference-machine-learning-literature找到存储库。

从机器学习模型中删除指定的培训数据子集的影响可能需要解决隐私，公平和数据质量等问题。删除子集后剩余数据从头开始对模型进行重新审查是有效但通常是不可行的，因为其计算费用。因此，在过去的几年中，已经看到了几种有效拆除的新方法，形成了“机器学习”领域，但是，到目前为止，出版的文献的许多方面都是不同的，缺乏共识。在本文中，我们总结并比较了七个最先进的机器学习算法，合并对现场中使用的核心概念的定义，调和不同的方法来评估算法，并讨论与在实践中应用机器相关的问题。

Federated learning is a collaborative method that aims to preserve data privacy while creating AI models. Current approaches to federated learning tend to rely heavily on secure aggregation protocols to preserve data privacy. However, to some degree, such protocols assume that the entity orchestrating the federated learning process (i.e., the server) is not fully malicious or dishonest. We investigate vulnerabilities to secure aggregation that could arise if the server is fully malicious and attempts to obtain access to private, potentially sensitive data. Furthermore, we provide a method to further defend against such a malicious server, and demonstrate effectiveness against known attacks that reconstruct data in a federated learning setting.