随着面部识别系统的广泛应用,人们担心原始的面部图像可能会暴露于恶意意图并因此导致个人隐私漏洞。本文介绍了Duetface,这是一种新型的隐私面部识别方法,该方法采用了频域中的协作推断。从违反直觉的发现开始,即面部识别只能通过视觉上无法区分的高频通道就可以实现出人意料的良好性能,此方法通过其可视化的关键性设计了可信的频道划分,并在非重要通道上操作服务器端模型。但是,由于缺少的视觉信息,该模型在注意力特征上的注意力降低了。为了补偿,该方法引入了插件交互式块,以通过产生功能掩码来从客户端转移注意力。通过得出和覆盖感兴趣的面部区域(ROI),进一步完善了面具。在多个数据集上进行的广泛实验验证了所提出的方法在保护面部图像免受不希望的视觉检查,重建和识别的同时保持高任务可用性和性能的有效性。结果表明,所提出的方法实现了对未受保护的弧形的可比识别精度和计算成本,并优于最先进的隐私保护方法。源代码可在https://github.com/tencent/tcace/tree/master/recognition/tasks/duetface上获得。
translated by 谷歌翻译
由于其高识别精度,包括移动设备的面部解锁,社区访问控制系统和城市监视,因此在许多领域都使用了面部识别技术。由于非常深的网络结构可以保证当前的高精度,因此通常需要将面部图像传输到具有高计算能力以进行推理的第三方服务器。但是,面部图像在视觉上揭示了用户的身份信息。在此过程中,不受信任的服务提供商和恶意用户都可以显着增加个人隐私漏洞的风险。当前的隐私识别方法通常伴随着许多副作用,例如推理时间的显着增加或明显的识别准确性下降。本文提出了使用频域中使用差异隐私的保护隐私面部识别方法。由于利用了差异隐私,它在理论上提供了隐私的保证。同时,准确性的丧失非常小。该方法首先将原始图像转换为频域,并删除称为DC的直接组件。然后,可以根据差异隐私框架内的后端面部识别网络的丢失来学习隐私预算分配方法。最后,它为频域特征添加了相应的噪声。根据广泛的实验,我们的方法在几个经典的面部识别测试集中表现出色。
translated by 谷歌翻译
Recent years witnessed the breakthrough of face recognition with deep convolutional neural networks. Dozens of papers in the field of FR are published every year. Some of them were applied in the industrial community and played an important role in human life such as device unlock, mobile payment, and so on. This paper provides an introduction to face recognition, including its history, pipeline, algorithms based on conventional manually designed features or deep learning, mainstream training, evaluation datasets, and related applications. We have analyzed and compared state-of-the-art works as many as possible, and also carefully designed a set of experiments to find the effect of backbone size and data distribution. This survey is a material of the tutorial named The Practical Face Recognition Technology in the Industrial World in the FG2023.
translated by 谷歌翻译
深度学习模型越来越多地部署在现实世界中。这些模型通常在服务器端部署,并在信息丰富的表示中接收用户数据,以求解特定任务,例如图像分类。由于图像可以包含敏感信息,而用户可能不愿意共享,因此隐私保护变得越来越重要。对抗表示学习(ARL)是一种训练在客户端运行并混淆图像的编码器的常见方法。假定可以安全地将混淆的图像安全地传输并用于服务器上的任务,而无需隐私问题。但是,在这项工作中,我们发现培训重建攻击者可以成功恢复现有ARL方法的原始图像。为此,我们通过低通滤波引入了一种新颖的ARL方法,从而限制了要在频域中编码的可用信息量。我们的实验结果表明,我们的方法可以承受重建攻击,同时超过了先前有关隐私 - 实用性权衡的最先进方法。我们进一步进行用户研究,以定性评估我们对重建攻击的防御。
translated by 谷歌翻译
Split学习(SL)通过允许客户在不共享原始数据的情况下协作培训深度学习模型来实现数据隐私保护。但是,SL仍然有限制,例如潜在的数据隐私泄漏和客户端的高计算。在这项研究中,我们建议将SL局部层进行二线以进行更快的计算(在移动设备上的培训和推理阶段的前进时间少17.5倍)和减少内存使用情况(最多减少32倍的内存和带宽要求) 。更重要的是,二进制的SL(B-SL)模型可以减少SL污染数据中的隐私泄漏,而模型精度的降解仅小。为了进一步增强隐私保护,我们还提出了两种新颖的方法:1)培训额外的局部泄漏损失,2)应用差异隐私,可以单独或同时集成到B-SL模型中。与多种基准模型相比,使用不同数据集的实验结果肯定了B-SL模型的优势。还说明了B-SL模型针对功能空间劫持攻击(FSHA)的有效性。我们的结果表明,B-SL模型对于具有高隐私保护要求(例如移动医疗保健应用程序)的轻巧的物联网/移动应用程序很有希望。
translated by 谷歌翻译
Federated learning is a collaborative method that aims to preserve data privacy while creating AI models. Current approaches to federated learning tend to rely heavily on secure aggregation protocols to preserve data privacy. However, to some degree, such protocols assume that the entity orchestrating the federated learning process (i.e., the server) is not fully malicious or dishonest. We investigate vulnerabilities to secure aggregation that could arise if the server is fully malicious and attempts to obtain access to private, potentially sensitive data. Furthermore, we provide a method to further defend against such a malicious server, and demonstrate effectiveness against known attacks that reconstruct data in a federated learning setting.
translated by 谷歌翻译
Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images, which is a domain-specific image restoration problem in the low-level computer vision area. The early face restoration methods mainly use statistic priors and degradation models, which are difficult to meet the requirements of real-world applications in practice. In recent years, face restoration has witnessed great progress after stepping into the deep learning era. However, there are few works to study deep learning-based face restoration methods systematically. Thus, this paper comprehensively surveys recent advances in deep learning techniques for face restoration. Specifically, we first summarize different problem formulations and analyze the characteristic of the face image. Second, we discuss the challenges of face restoration. Concerning these challenges, we present a comprehensive review of existing FR methods, including prior based methods and deep learning-based methods. Then, we explore developed techniques in the task of FR covering network architectures, loss functions, and benchmark datasets. We also conduct a systematic benchmark evaluation on representative methods. Finally, we discuss future directions, including network designs, metrics, benchmark datasets, applications,etc. We also provide an open-source repository for all the discussed methods, which is available at https://github.com/TaoWangzj/Awesome-Face-Restoration.
translated by 谷歌翻译
当前用于面部识别的模型(FR)中存在人口偏见。我们在野外(BFW)数据集中平衡的面孔是衡量种族和性别亚组偏见的代理,使一个人可以表征每个亚组的FR表现。当单个分数阈值确定样本对是真实还是冒名顶替者时,我们显示的结果是非最佳选择的。在亚组中,性能通常与全球平均水平有很大差异。因此,仅适用于与验证数据相匹配的人群的特定错误率。我们使用新的域适应性学习方案来减轻性能不平衡,以使用最先进的神经网络提取的面部特征。该技术平衡了性能,但也可以提高整体性能。该建议的好处是在面部特征中保留身份信息,同时减少其所包含的人口统计信息。人口统计学知识的去除阻止了潜在的未来偏见被注入决策。由于对个人的可用信息或推断,因此此删除可改善隐私。我们定性地探索这一点;我们还定量地表明,亚组分类器不再从提出的域适应方案的特征中学习。有关源代码和数据描述,请参见https://github.com/visionjo/facerec-bias-bfw。
translated by 谷歌翻译
联邦机器学习利用边缘计算来开发网络用户数据的模型,但联合学习的隐私仍然是一个重大挑战。已经提出了使用差异隐私的技术来解决这一点,但是带来了自己的挑战 - 许多人需要一个值得信赖的第三方,或者增加了太多的噪音来生产有用的模型。使用多方计算的\ EMPH {SERVE聚合}的最新进步消除了对第三方的需求,但是在计算上尤其在规模上昂贵。我们提出了一种新的联合学习协议,利用了一种基于与错误学习的技术的新颖差异私有的恶意安全聚合协议。我们的协议优于当前最先进的技术,并且经验结果表明它缩放到大量方面,具有任何差别私有联合学习方案的最佳精度。
translated by 谷歌翻译
姿势变异是人脸识别的关键挑战之一。常规技术主要关注面部正利化或图像空间中的面部增强。然而,不保证在图像空间中转换面部图像以保留原始图像的无损身份特征。此外,由于附加模型,这些方法遭受更多的计算成本和内存要求。我们认为,更希望在分层特​​征空间而不是图像空间中执行特征转换,这可以利用不同的特征级别,并利益与表示学习的联合学习。为此,我们提出了一种轻量级且易于实施的注意力块,名为Pose Intence Module(PAM),用于姿势不变的人脸识别。具体地,PAM通过使用软栅极机制的姿势变化之间的剩余来执行分层特征空间中的前型特征变换。我们通过广泛的消融研究验证了PAM块设计的有效性,并验证了几种流行基准的性能,包括LFW,CFP-FP,Agedb-30,CPLFW和Calfw。实验结果表明,我们的方法不仅优于最先进的方法,而且有效地降低了超过75倍的内存要求。值得注意的是,我们的方法不仅限于与大的姿态变化的面对识别。通过将PAM的软栅极机构调整为特定系数,这种语义关注块可以容易地扩展到面部识别中的其他类别的不平衡问题,包括年龄,照明,表达等的大变化。
translated by 谷歌翻译
联合学习(FL)和分裂学习(SL)是两种新兴的协作学习方法,可能会极大地促进物联网(IoT)中无处不在的智能。联合学习使机器学习(ML)模型在本地培训的模型使用私人数据汇总为全球模型。分裂学习使ML模型的不同部分可以在学习框架中对不同工人进行协作培训。联合学习和分裂学习,每个学习都有独特的优势和各自的局限性,可能会相互补充,在物联网中无处不在的智能。因此,联合学习和分裂学习的结合最近成为一个活跃的研究领域,引起了广泛的兴趣。在本文中,我们回顾了联合学习和拆分学习方面的最新发展,并介绍了有关最先进技术的调查,该技术用于将这两种学习方法组合在基于边缘计算的物联网环境中。我们还确定了一些开放问题,并讨论了该领域未来研究的可能方向,希望进一步引起研究界对这个新兴领域的兴趣。
translated by 谷歌翻译
随着最近深度卷积神经网络的进步,一般面临的概念取得了重大进展。然而,最先进的一般面部识别模型对遮挡面部图像没有概括,这正是现实世界场景中的常见情况。潜在原因是用于训练和特定设计的大规模遮挡面部数据,用于解决闭塞所带来的损坏功能。本文提出了一种新颖的面部识别方法,其基于单端到端的深神经网络的闭塞是强大的。我们的方法(使用遮挡掩码)命名(面部识别),学会发现深度卷积神经网络的损坏功能,并通过动态学习的面具清洁它们。此外,我们构建了大规模的遮挡面部图像,从有效且有效地培训。与现有方法相比,依靠外部探测器发现遮挡或采用较少鉴别的浅模型的现有方法,从简单且功能强大。 LFW,Megaface挑战1,RMF2,AR数据集和其他模拟遮挡/掩蔽数据集的实验结果证实,从大幅提高了遮挡下的准确性,并概括了一般面部识别。
translated by 谷歌翻译
横梁面部识别(CFR)旨在识别个体,其中比较面部图像源自不同的感测模式,例如红外与可见的。虽然CFR由于与模态差距相关的面部外观的显着变化,但CFR具有比经典的面部识别更具挑战性,但它在具有有限或挑战的照明的场景中,以及在呈现攻击的情况下,它是优越的。与卷积神经网络(CNNS)相关的人工智能最近的进展使CFR的显着性能提高了。由此激励,这项调查的贡献是三倍。我们提供CFR的概述,目标是通过首先正式化CFR然后呈现具体相关的应用来比较不同光谱中捕获的面部图像。其次,我们探索合适的谱带进行识别和讨论最近的CFR方法,重点放在神经网络上。特别是,我们提出了提取和比较异构特征以及数据集的重新访问技术。我们枚举不同光谱和相关算法的优势和局限性。最后,我们讨论了研究挑战和未来的研究线。
translated by 谷歌翻译
通信技术和互联网的最新进展与人工智能(AI)启用了智能医疗保健。传统上,由于现代医疗保健网络的高性性和日益增长的数据隐私问题,AI技术需要集中式数据收集和处理,这可能在现实的医疗环境中可能是不可行的。作为一个新兴的分布式协作AI范例,通过协调多个客户(例如,医院)来执行AI培训而不共享原始数据,对智能医疗保健特别有吸引力。因此,我们对智能医疗保健的使用提供了全面的调查。首先,我们在智能医疗保健中展示了近期进程,动机和使用FL的要求。然后讨论了近期智能医疗保健的FL设计,从资源感知FL,安全和隐私感知到激励FL和个性化FL。随后,我们对关键医疗领域的FL新兴应用提供了最先进的综述,包括健康数据管理,远程健康监测,医学成像和Covid-19检测。分析了几个最近基于智能医疗保健项目,并突出了从调查中学到的关键经验教训。最后,我们讨论了智能医疗保健未来研究的有趣研究挑战和可能的指示。
translated by 谷歌翻译
我们设计可扩展的算法,以私下生成从数百万用户设备的分散数据的位置热量。它旨在确保在服务提供商对服务提供商可见之前的差异隐私,同时保持高数据准确性和最小化用户设备的资源消耗。为实现这一目标,我们根据安全多方计算领域的最新结果重新审视分布式差异隐私概念,并设计用于位置分析的可扩展和自适应分布式差分隐私方法。关于公共位置数据集的评估表明,该方法成功地从数百万用户样本中成功地生成了大量的客户样本,最坏的客户端通信开销明显小于现有的类似准确性的现有最先进的私有协议。
translated by 谷歌翻译
Differentially private federated learning (DP-FL) has received increasing attention to mitigate the privacy risk in federated learning. Although different schemes for DP-FL have been proposed, there is still a utility gap. Employing central Differential Privacy in FL (CDP-FL) can provide a good balance between the privacy and model utility, but requires a trusted server. Using Local Differential Privacy for FL (LDP-FL) does not require a trusted server, but suffers from lousy privacy-utility trade-off. Recently proposed shuffle DP based FL has the potential to bridge the gap between CDP-FL and LDP-FL without a trusted server; however, there is still a utility gap when the number of model parameters is large. In this work, we propose OLIVE, a system that combines the merits from CDP-FL and LDP-FL by leveraging Trusted Execution Environment (TEE). Our main technical contributions are the analysis and countermeasures against the vulnerability of TEE in OLIVE. Firstly, we theoretically analyze the memory access pattern leakage of OLIVE and find that there is a risk for sparsified gradients, which is common in FL. Secondly, we design an inference attack to understand how the memory access pattern could be linked to the training data. Thirdly, we propose oblivious yet efficient algorithms to prevent the memory access pattern leakage in OLIVE. Our experiments on real-world data demonstrate that OLIVE is efficient even when training a model with hundreds of thousands of parameters and effective against side-channel attacks on TEE.
translated by 谷歌翻译
Holistic methods using CNNs and margin-based losses have dominated research on face recognition. In this work, we depart from this setting in two ways: (a) we employ the Vision Transformer as an architecture for training a very strong baseline for face recognition, simply called fViT, which already surpasses most state-of-the-art face recognition methods. (b) Secondly, we capitalize on the Transformer's inherent property to process information (visual tokens) extracted from irregular grids to devise a pipeline for face recognition which is reminiscent of part-based face recognition methods. Our pipeline, called part fViT, simply comprises a lightweight network to predict the coordinates of facial landmarks followed by the Vision Transformer operating on patches extracted from the predicted landmarks, and it is trained end-to-end with no landmark supervision. By learning to extract discriminative patches, our part-based Transformer further boosts the accuracy of our Vision Transformer baseline achieving state-of-the-art accuracy on several face recognition benchmarks.
translated by 谷歌翻译
The emergence of COVID-19 has had a global and profound impact, not only on society as a whole, but also on the lives of individuals. Various prevention measures were introduced around the world to limit the transmission of the disease, including face masks, mandates for social distancing and regular disinfection in public spaces, and the use of screening applications. These developments also triggered the need for novel and improved computer vision techniques capable of (i) providing support to the prevention measures through an automated analysis of visual data, on the one hand, and (ii) facilitating normal operation of existing vision-based services, such as biometric authentication schemes, on the other. Especially important here, are computer vision techniques that focus on the analysis of people and faces in visual data and have been affected the most by the partial occlusions introduced by the mandates for facial masks. Such computer vision based human analysis techniques include face and face-mask detection approaches, face recognition techniques, crowd counting solutions, age and expression estimation procedures, models for detecting face-hand interactions and many others, and have seen considerable attention over recent years. The goal of this survey is to provide an introduction to the problems induced by COVID-19 into such research and to present a comprehensive review of the work done in the computer vision based human analysis field. Particular attention is paid to the impact of facial masks on the performance of various methods and recent solutions to mitigate this problem. Additionally, a detailed review of existing datasets useful for the development and evaluation of methods for COVID-19 related applications is also provided. Finally, to help advance the field further, a discussion on the main open challenges and future research direction is given.
translated by 谷歌翻译
Large training data and expensive model tweaking are standard features of deep learning for images. As a result, data owners often utilize cloud resources to develop large-scale complex models, which raises privacy concerns. Existing solutions are either too expensive to be practical or do not sufficiently protect the confidentiality of data and models. In this paper, we study and compare novel \emph{image disguising} mechanisms, DisguisedNets and InstaHide, aiming to achieve a better trade-off among the level of protection for outsourced DNN model training, the expenses, and the utility of data. DisguisedNets are novel combinations of image blocktization, block-level random permutation, and two block-level secure transformations: random multidimensional projection (RMT) and AES pixel-level encryption (AES). InstaHide is an image mixup and random pixel flipping technique \cite{huang20}. We have analyzed and evaluated them under a multi-level threat model. RMT provides a better security guarantee than InstaHide, under the Level-1 adversarial knowledge with well-preserved model quality. In contrast, AES provides a security guarantee under the Level-2 adversarial knowledge, but it may affect model quality more. The unique features of image disguising also help us to protect models from model-targeted attacks. We have done an extensive experimental evaluation to understand how these methods work in different settings for different datasets.
translated by 谷歌翻译
团体公平确保基于机器学习的结果(ML)决策系统的结果不会偏向于某些由性别或种族等敏感属性定义的人。在联合学习(FL)中实现群体公平性是具有挑战性的,因为缓解偏差固有地需要使用所有客户的敏感属性值,而FL则旨在通过不给客户数据访问来保护隐私。正如我们在本文中所显示的那样,可以通过将FL与安全的多方计算(MPC)和差异隐私(DP)相结合来解决FL中的公平与隐私之间的冲突。在此过程中,我们提出了一种在完整和正式的隐私保证下培训跨设备FL中的小组最大ML模型的方法,而无需客户披露其敏感属性值。
translated by 谷歌翻译