垂直协作学习系统也被称为垂直联合学习(VFL)系统最近成为一个概念,以处理在许多个人来源上分布的数据,而无需集中它。多个参与者以隐私保留方式基于其本地数据协作培训模型。迄今为止,VFL已成为一项事实上的解决方案,以便在组织之间安全地学习模型,允许在不影响任何个人组织的隐私的情况下共享知识。尽管VFL系统的发展繁荣发展,但我们发现参与者的某些投入,名叫对抗的主导投入(ADIS),可以将联合推断占主持旨在的意志的方向,并迫使其他(受害者)参与者进行可忽略不计的捐款,失败奖励通常提供他们在合作学习情景中的贡献的重要性。通过首先在典型的VFL系统中证明其存在,我们对ADI进行了系统研究。然后,我们提出基于梯度的方法来综合各种格式的ADI并利用公共VFL系统。我们进一步推出了Greybox Fuzz测试,以“受害者”参与者的弹性分数为指导,以扰乱对抗控制的输入,并以隐私保存方式系统地探索VFL攻击表面。我们对临界参数和环境在合成ADIS中的影响进行了深入的研究。我们的研究揭示了新的VFL攻击机会,在违反之前促进了未知威胁的识别,并建立了更安全的VFL系统。
translated by 谷歌翻译
对网络攻击的现代防御越来越依赖于主动的方法,例如,基于过去的事件来预测对手的下一个行动。建立准确的预测模型需要许多组织的知识; las,这需要披露敏感信息,例如网络结构,安全姿势和政策,这些信息通常是不受欢迎的或完全不可能的。在本文中,我们探讨了使用联合学习(FL)预测未来安全事件的可行性。为此,我们介绍了Cerberus,这是一个系统,可以为参与组织的复发神经网络(RNN)模型进行协作培训。直觉是,FL可能会在非私有方法之间提供中间地面,在非私有方法中,训练数据在中央服务器上合并,而仅训练本地模型的较低性替代方案。我们将Cerberus实例化在从一家大型安全公司的入侵预防产品中获得的数据集上,并评估其有关实用程序,鲁棒性和隐私性,以及参与者如何从系统中贡献和受益。总体而言,我们的工作阐明了将FL执行此任务的积极方面和挑战,并为部署联合方法以进行预测安全铺平了道路。
translated by 谷歌翻译
已经提出了安全的多方计算(MPC),以允许多个相互不信任的数据所有者在其合并数据上共同训练机器学习(ML)模型。但是,通过设计,MPC协议忠实地计算了训练功能,对抗性ML社区已证明该功能泄漏了私人信息,并且可以在中毒攻击中篡改。在这项工作中,我们认为在我们的框架中实现的模型合奏是一种称为Safenet的框架,是MPC的高度无限方法,可以避免许多对抗性ML攻击。 MPC培训中所有者之间数据的自然分区允许这种方法在训练时间高度可扩展,可证明可保护免受中毒攻击的保护,并证明可以防御许多隐私攻击。我们展示了Safenet对在端到端和转移学习方案训练的几个机器学习数据集和模型上中毒的效率,准确性和韧性。例如,Safenet可显着降低后门攻击的成功,同时获得$ 39 \ times $ $的培训,$ 36 \ times $ $ $少于达尔斯科夫(Dalskov)等人的四方MPC框架。我们的实验表明,即使在许多非IID设置中,结合也能保留这些好处。结合的简单性,廉价的设置和鲁棒性属性使其成为MPC私下培训ML模型的强大首选。
translated by 谷歌翻译
Vertical federated learning (VFL) is an emerging paradigm that enables collaborators to build machine learning models together in a distributed fashion. In general, these parties have a group of users in common but own different features. Existing VFL frameworks use cryptographic techniques to provide data privacy and security guarantees, leading to a line of works studying computing efficiency and fast implementation. However, the security of VFL's model remains underexplored.
translated by 谷歌翻译
In recent years, mobile devices are equipped with increasingly advanced sensing and computing capabilities. Coupled with advancements in Deep Learning (DL), this opens up countless possibilities for meaningful applications, e.g., for medical purposes and in vehicular networks. Traditional cloudbased Machine Learning (ML) approaches require the data to be centralized in a cloud server or data center. However, this results in critical issues related to unacceptable latency and communication inefficiency. To this end, Mobile Edge Computing (MEC) has been proposed to bring intelligence closer to the edge, where data is produced. However, conventional enabling technologies for ML at mobile edge networks still require personal data to be shared with external parties, e.g., edge servers. Recently, in light of increasingly stringent data privacy legislations and growing privacy concerns, the concept of Federated Learning (FL) has been introduced. In FL, end devices use their local data to train an ML model required by the server. The end devices then send the model updates rather than raw data to the server for aggregation. FL can serve as an enabling technology in mobile edge networks since it enables the collaborative training of an ML model and also enables DL for mobile edge network optimization. However, in a large-scale and complex mobile edge network, heterogeneous devices with varying constraints are involved. This raises challenges of communication costs, resource allocation, and privacy and security in the implementation of FL at scale. In this survey, we begin with an introduction to the background and fundamentals of FL. Then, we highlight the aforementioned challenges of FL implementation and review existing solutions. Furthermore, we present the applications of FL for mobile edge network optimization. Finally, we discuss the important challenges and future research directions in FL.
translated by 谷歌翻译
Today's AI still faces two major challenges. One is that in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated learning framework, which includes horizontal federated learning, vertical federated learning and federated transfer learning. We provide definitions, architectures and applications for the federated learning framework, and provide a comprehensive survey of existing works on this subject. In addition, we propose building data networks among organizations based on federated mechanisms as an effective solution to allow knowledge to be shared without compromising user privacy.
translated by 谷歌翻译
Federated learning is a collaborative method that aims to preserve data privacy while creating AI models. Current approaches to federated learning tend to rely heavily on secure aggregation protocols to preserve data privacy. However, to some degree, such protocols assume that the entity orchestrating the federated learning process (i.e., the server) is not fully malicious or dishonest. We investigate vulnerabilities to secure aggregation that could arise if the server is fully malicious and attempts to obtain access to private, potentially sensitive data. Furthermore, we provide a method to further defend against such a malicious server, and demonstrate effectiveness against known attacks that reconstruct data in a federated learning setting.
translated by 谷歌翻译
Federated learning enables thousands of participants to construct a deep learning model without sharing their private training data with each other. For example, multiple smartphones can jointly train a next-word predictor for keyboards without revealing what individual users type.Federated models are created by aggregating model updates submitted by participants. To protect confidentiality of the training data, the aggregator by design has no visibility into how these updates are generated. We show that this makes federated learning vulnerable to a model-poisoning attack that is significantly more powerful than poisoning attacks that target only the training data.A malicious participant can use model replacement to introduce backdoor functionality into the joint model, e.g., modify an image classifier so that it assigns an attacker-chosen label to images with certain features, or force a word predictor to complete certain sentences with an attacker-chosen word. These attacks can be performed by a single participant or multiple colluding participants. We evaluate model replacement under different assumptions for the standard federated-learning tasks and show that it greatly outperforms training-data poisoning.Federated learning employs secure aggregation to protect confidentiality of participants' local models and thus cannot prevent our attack by detecting anomalies in participants' contributions to the joint model. To demonstrate that anomaly detection would not have been effective in any case, we also develop and evaluate a generic constrain-and-scale technique that incorporates the evasion of defenses into the attacker's loss function during training. ! "#$%" train & % '() * '()! +%$,-##.
translated by 谷歌翻译
作为服务的云计算和机器学习的繁荣发展导致媒体软件的广泛使用来处理机密媒体数据。本文探讨了对媒体软件启动侧通道分析(SCA)以重建机密介质输入的侵略性的能力。代表学习和感知学习的最新进展激发了我们考虑从侧通道迹线的媒体输入的重建作为跨模式歧管学习任务,可以以统一的方式通过训练的自动介质框架来寻址,以便学习媒体输入之间的映射和侧沟道观测。我们进一步提升了自动统计学家,注意本地化对SCA的主要贡献的程序点,从而自动查明媒体软件中的信息泄漏点。我们还提出了一种新颖且非常有效的防御技术,称为感知致盲,可以使媒体输入具有感知掩模和减轻基于多种学习的SCA。我们的评估利用三个流行的媒体软件重建图像,音频和文本格式的输入。我们分析了三个常见的侧面通道 - 缓存库,缓存行和页面表 - 以及仅由标准Prime +探针记录的用户空间缓存设置访问。我们的框架成功地从评估的媒体软件重建了高质量的机密输入,并自动查明了他们脆弱的程序点,其中许多是公众所未知的。我们进一步表明,感知致盲可以减轻基于流形的学习的SCA,额外的成本可忽略不计。
translated by 谷歌翻译
Differentially private federated learning (DP-FL) has received increasing attention to mitigate the privacy risk in federated learning. Although different schemes for DP-FL have been proposed, there is still a utility gap. Employing central Differential Privacy in FL (CDP-FL) can provide a good balance between the privacy and model utility, but requires a trusted server. Using Local Differential Privacy for FL (LDP-FL) does not require a trusted server, but suffers from lousy privacy-utility trade-off. Recently proposed shuffle DP based FL has the potential to bridge the gap between CDP-FL and LDP-FL without a trusted server; however, there is still a utility gap when the number of model parameters is large. In this work, we propose OLIVE, a system that combines the merits from CDP-FL and LDP-FL by leveraging Trusted Execution Environment (TEE). Our main technical contributions are the analysis and countermeasures against the vulnerability of TEE in OLIVE. Firstly, we theoretically analyze the memory access pattern leakage of OLIVE and find that there is a risk for sparsified gradients, which is common in FL. Secondly, we design an inference attack to understand how the memory access pattern could be linked to the training data. Thirdly, we propose oblivious yet efficient algorithms to prevent the memory access pattern leakage in OLIVE. Our experiments on real-world data demonstrate that OLIVE is efficient even when training a model with hundreds of thousands of parameters and effective against side-channel attacks on TEE.
translated by 谷歌翻译
联邦学习一直是一个热门的研究主题,使不同组织的机器学习模型的协作培训在隐私限制下。随着研究人员试图支持更多具有不同隐私方法的机器学习模型,需要开发系统和基础设施,以便于开发各种联合学习算法。类似于Pytorch和Tensorflow等深度学习系统,可以增强深度学习的发展,联邦学习系统(FLSS)是等效的,并且面临各个方面的面临挑战,如有效性,效率和隐私。在本调查中,我们对联合学习系统进行了全面的审查。为实现流畅的流动和引导未来的研究,我们介绍了联合学习系统的定义并分析了系统组件。此外,我们根据六种不同方面提供联合学习系统的全面分类,包括数据分布,机器学习模型,隐私机制,通信架构,联合集市和联合的动机。分类可以帮助设计联合学习系统,如我们的案例研究所示。通过系统地总结现有联合学习系统,我们展示了设计因素,案例研究和未来的研究机会。
translated by 谷歌翻译
我们调查分裂学习的安全 - 一种新颖的协作机器学习框架,通过需要最小的资源消耗来实现峰值性能。在本文中,我们通过介绍客户私人培训集重建的一般攻击策略来揭示议定书的脆弱性并展示其固有的不安全。更突出地,我们表明恶意服务器可以积极地劫持分布式模型的学习过程,并将其纳入不安全状态,从而为客户端提供推动攻击。我们实施不同的攻击调整,并在各种数据集中测试它们以及现实的威胁方案。我们证明我们的攻击能够克服最近提出的防御技术,旨在提高分裂学习议定书的安全性。最后,我们还通过扩展以前设计的联合学习的攻击来说明协议对恶意客户的不安全性。要使我们的结果可重复,我们会在https://github.com/pasquini-dario/splitn_fsha提供的代码。
translated by 谷歌翻译
联合学习(FL)和分裂学习(SL)是两种新兴的协作学习方法,可能会极大地促进物联网(IoT)中无处不在的智能。联合学习使机器学习(ML)模型在本地培训的模型使用私人数据汇总为全球模型。分裂学习使ML模型的不同部分可以在学习框架中对不同工人进行协作培训。联合学习和分裂学习,每个学习都有独特的优势和各自的局限性,可能会相互补充,在物联网中无处不在的智能。因此,联合学习和分裂学习的结合最近成为一个活跃的研究领域,引起了广泛的兴趣。在本文中,我们回顾了联合学习和拆分学习方面的最新发展,并介绍了有关最先进技术的调查,该技术用于将这两种学习方法组合在基于边缘计算的物联网环境中。我们还确定了一些开放问题,并讨论了该领域未来研究的可能方向,希望进一步引起研究界对这个新兴领域的兴趣。
translated by 谷歌翻译
联邦学习的出现在维持隐私的同时,促进了机器学习模型之间的大规模数据交换。尽管历史悠久,但联邦学习正在迅速发展,以使更广泛的使用更加实用。该领域中最重要的进步之一是将转移学习纳入联邦学习,这克服了主要联合学习的基本限制,尤其是在安全方面。本章从安全的角度进行了有关联合和转移学习的交集的全面调查。这项研究的主要目标是发现可能损害使用联合和转移学习的系统的隐私和性能的潜在脆弱性和防御机制。
translated by 谷歌翻译
Federated Learning (FL) is a scheme for collaboratively training Deep Neural Networks (DNNs) with multiple data sources from different clients. Instead of sharing the data, each client trains the model locally, resulting in improved privacy. However, recently so-called targeted poisoning attacks have been proposed that allow individual clients to inject a backdoor into the trained model. Existing defenses against these backdoor attacks either rely on techniques like Differential Privacy to mitigate the backdoor, or analyze the weights of the individual models and apply outlier detection methods that restricts these defenses to certain data distributions. However, adding noise to the models' parameters or excluding benign outliers might also reduce the accuracy of the collaboratively trained model. Additionally, allowing the server to inspect the clients' models creates a privacy risk due to existing knowledge extraction methods. We propose CrowdGuard, a model filtering defense, that mitigates backdoor attacks by leveraging the clients' data to analyze the individual models before the aggregation. To prevent data leaks, the server sends the individual models to secure enclaves, running in client-located Trusted Execution Environments. To effectively distinguish benign and poisoned models, even if the data of different clients are not independently and identically distributed (non-IID), we introduce a novel metric called HLBIM to analyze the outputs of the DNN's hidden layers. We show that the applied significance-based detection algorithm combined can effectively detect poisoned models, even in non-IID scenarios. We show in our extensive evaluation that CrowdGuard can effectively mitigate targeted poisoning attacks and achieve in various scenarios a True-Positive-Rate of 100% and a True-Negative-Rate of 100%.
translated by 谷歌翻译
我们考虑垂直逻辑回归(VLR)接受了迷你批次梯度下降训练,这种环境吸引了行业日益增长的兴趣,并被证明在包括金融和医学研究在内的广泛应用中很有用。我们在一系列开源联合学习框架中提供了对VLR的全面和严格的隐私分析,其中协议之间可能会有所不同,但是获得了获得本地梯度的过程。我们首先考虑了诚实而有趣的威胁模型,其中忽略了协议的详细实施,并且仅假定共享过程,我们将其作为甲骨文提取。我们发现,即使在这种一般环境下,在适当的批处理大小约束下,仍然可以从另一方恢复单维功能和标签,从而证明了遵循相同理念的所有框架的潜在脆弱性。然后,我们研究基于同态加密(HE)的协议的流行实例。我们提出了一种主动攻击,该攻击通过生成和压缩辅助密文来显着削弱对先前分析中批处理大小的约束。为了解决基于HE的协议中的隐私泄漏,我们基于差异隐私(DP)开发了一种简单的对策,并为更新的算法提供实用程序和隐私保证。最后,我们从经验上验证了我们对基准数据集的攻击和防御的有效性。总之,我们的发现表明,仅依靠他的所有垂直联合学习框架可能包含严重的隐私风险,而DP已经证明了其在水平联合学习中的力量,也可以在垂直环境中起着至关重要的作用,尤其是当耦合时使用HE或安全的多方计算(MPC)技术。
translated by 谷歌翻译
联合学习允许一组用户在私人训练数据集中培训深度神经网络。在协议期间,数据集永远不会留下各个用户的设备。这是通过要求每个用户向中央服务器发送“仅”模型更新来实现,从而汇总它们以更新深神经网络的参数。然而,已经表明,每个模型更新都具有关于用户数据集的敏感信息(例如,梯度反转攻击)。联合学习的最先进的实现通过利用安全聚合来保护这些模型更新:安全监控协议,用于安全地计算用户的模型更新的聚合。安全聚合是关键,以保护用户的隐私,因为它会阻碍服务器学习用户提供的个人模型更新的源,防止推断和数据归因攻击。在这项工作中,我们表明恶意服务器可以轻松地阐明安全聚合,就像后者未到位一样。我们设计了两种不同的攻击,能够在参与安全聚合的用户数量上,独立于参与安全聚合的用户数。这使得它们在大规模现实世界联邦学习应用中的具体威胁。攻击是通用的,不瞄准任何特定的安全聚合协议。即使安全聚合协议被其理想功能替换为提供完美的安全性的理想功能,它们也同样有效。我们的工作表明,安全聚合与联合学习相结合,当前实施只提供了“虚假的安全感”。
translated by 谷歌翻译
拜占庭式联合学习(FL)旨在对抗恶意客户并培训准确的全球模型,同时保持极低的攻击成功率。然而,大多数现有系统仅在诚实/半hon最达克的多数设置中都具有强大的功能。 FLTRUST(NDSS '21)将上下文扩展到对客户的恶意多数,但在训练之前,应在训练之前为服务器提供辅助数据集,以便过滤恶意输入。私人火焰/flguard(Usenix '22)提供了一种解决方案,以确保在半多数上下文中既有稳健性和更新机密性。到目前为止,不可能平衡恶意背景,鲁棒性和更新机密性之间的权衡。为了解决这个问题,我们提出了一种新颖的拜占庭式bybust和隐私的FL系统,称为简介,以捕获恶意的少数群体和多数服务器和客户端。具体而言,基于DBSCAN算法,我们设计了一种通过成对调整的余弦相似性聚类的新方法,以提高聚类结果的准确性。为了阻止多数攻击恶意的攻击,我们开发了一种称为模型分割的算法,在该算法中,同一集群中的本地更新聚集在一起,并且将聚合正确地发送回相应的客户端。我们还利用多种密码工具来执行聚类任务,而无需牺牲培训正确性并更新机密性。我们介绍了详细的安全证明和经验评估以及简要的收敛分析。实验结果表明,简介的测试精度实际上接近FL基线(平均为0.8%的差距)。同时,攻击成功率约为0%-5%。我们进一步优化了设计,以便可以分别降低{67%-89.17%和66.05%-68.75%}的通信开销和运行时。
translated by 谷歌翻译
Federated Learning (FL) has emerged as a promising distributed learning paradigm with an added advantage of data privacy. With the growing interest in having collaboration among data owners, FL has gained significant attention of organizations. The idea of FL is to enable collaborating participants train machine learning (ML) models on decentralized data without breaching privacy. In simpler words, federated learning is the approach of ``bringing the model to the data, instead of bringing the data to the mode''. Federated learning, when applied to data which is partitioned vertically across participants, is able to build a complete ML model by combining local models trained only using the data with distinct features at the local sites. This architecture of FL is referred to as vertical federated learning (VFL), which differs from the conventional FL on horizontally partitioned data. As VFL is different from conventional FL, it comes with its own issues and challenges. In this paper, we present a structured literature review discussing the state-of-the-art approaches in VFL. Additionally, the literature review highlights the existing solutions to challenges in VFL and provides potential research directions in this domain.
translated by 谷歌翻译
网络威胁情报(CTI)共享是减少攻击者和捍卫者之间信息不对称的重要活动。但是,由于数据共享和机密性之间的紧张关系,这项活动带来了挑战,这导致信息保留通常会导致自由骑士问题。因此,共享的信息仅代表冰山一角。当前的文献假设访问包含所有信息的集中数据库,但是由于上述张力,这并不总是可行的。这会导致不平衡或不完整的数据集,需要使用技术扩展它们。我们展示了这些技术如何导致结果和误导性能期望。我们提出了一个新颖的框架,用于从分布式数据中提取有关事件,漏洞和妥协指标的分布式数据,并与恶意软件信息共享平台(MISP)一起证明其在几种实际情况下的使用。提出和讨论了CTI共享的政策影响。拟议的系统依赖于隐私增强技术和联合处理的有效组合。这使组织能够控制其CTI,并最大程度地减少暴露或泄漏的风险,同时为共享的好处,更准确和代表性的结果以及更有效的预测性和预防性防御能力。
translated by 谷歌翻译