商业和政府部门中自动面部识别的扩散引起了个人的严重隐私问题。解决这些隐私问题的一种方法是采用逃避攻击针对启动面部识别系统的度量嵌入网络的攻击：面部混淆系统会产生不透彻的扰动图像，从而导致面部识别系统误解用户。受扰动的面孔是在公制嵌入网络上产生的，在面部识别的背景下，这是不公平的。人口公平的问题自然而然：面部混淆系统表现是否存在人口统计学差异？我们通过对最近的面部混淆系统的分析和经验探索来回答这个问题。指标嵌入网络在人口统计学上很有意识：面部嵌入由人口统计组群聚集。我们展示了这种聚类行为如何导致少数群体面孔的面部混淆实用性减少。直观的分析模型可以深入了解这些现象。

深度神经网络的面部识别模型已显示出容易受到对抗例子的影响。但是，过去的许多攻击都要求对手使用梯度下降来解决输入依赖性优化问题，这使该攻击实时不切实际。这些对抗性示例也与攻击模型紧密耦合，并且在转移到不同模型方面并不那么成功。在这项工作中，我们提出了Reface，这是对基于对抗性转换网络（ATN）的面部识别模型的实时，高度转移的攻击。 ATNS模型对抗性示例生成是馈送前向神经网络。我们发现，纯U-NET ATN的白盒攻击成功率大大低于基于梯度的攻击，例如大型面部识别数据集中的PGD。因此，我们为ATN提出了一个新的架构，该架构缩小了这一差距，同时维持PGD的10000倍加速。此外，我们发现在给定的扰动幅度下，与PGD相比，我们的ATN对抗扰动在转移到新的面部识别模型方面更有效。 Reface攻击可以在转移攻击环境中成功欺骗商业面部识别服务，并将面部识别精度从AWS SearchFaces API和Azure Face验证准确性从91％降低到50.1％，从而将面部识别精度从82％降低到16.4％。

已显示现有的面部分析系统对某些人口统计亚组产生偏见的结果。由于其对社会的影响，因此必须确保这些系统不会根据个人的性别，身份或肤色歧视。这导致了在AI系统中识别和减轻偏差的研究。在本文中，我们封装了面部分析的偏置检测/估计和缓解算法。我们的主要贡献包括对拟议理解偏见的算法的系统审查，以及分类和广泛概述现有的偏置缓解算法。我们还讨论了偏见面部分析领域的开放挑战。

当前用于面部识别的模型（FR）中存在人口偏见。我们在野外（BFW）数据集中平衡的面孔是衡量种族和性别亚组偏见的代理，使一个人可以表征每个亚组的FR表现。当单个分数阈值确定样本对是真实还是冒名顶替者时，我们显示的结果是非最佳选择的。在亚组中，性能通常与全球平均水平有很大差异。因此，仅适用于与验证数据相匹配的人群的特定错误率。我们使用新的域适应性学习方案来减轻性能不平衡，以使用最先进的神经网络提取的面部特征。该技术平衡了性能，但也可以提高整体性能。该建议的好处是在面部特征中保留身份信息，同时减少其所包含的人口统计信息。人口统计学知识的去除阻止了潜在的未来偏见被注入决策。由于对个人的可用信息或推断，因此此删除可改善隐私。我们定性地探索这一点；我们还定量地表明，亚组分类器不再从提出的域适应方案的特征中学习。有关源代码和数据描述，请参见https://github.com/visionjo/facerec-bias-bfw。

The authors thank Nicholas Carlini (UC Berkeley) and Dimitris Tsipras (MIT) for feedback to improve the survey quality. We also acknowledge X. Huang (Uni. Liverpool), K. R. Reddy (IISC), E. Valle (UNICAMP), Y. Yoo (CLAIR) and others for providing pointers to make the survey more comprehensive.

在本讨论文件中，我们调查了有关机器学习模型鲁棒性的最新研究。随着学习算法在数据驱动的控制系统中越来越流行，必须确保它们对数据不确定性的稳健性，以维持可靠的安全至关重要的操作。我们首先回顾了这种鲁棒性的共同形式主义，然后继续讨论训练健壮的机器学习模型的流行和最新技术，以及可证明这种鲁棒性的方法。从强大的机器学习的这种统一中，我们识别并讨论了该地区未来研究的迫切方向。

当模型向人们提供决定时，分销转移可能会造成不当差异。但是，由于模型及其训练集通常是专有的，因此外部实体很难检查分配变化。在本文中，我们介绍并研究了一种黑盒审计方法，以检测分配转移案例，从而导致跨人口组的模型差异。通过扩展在成员资格和属性推理攻击中使用的技术（旨在暴露于学习模型中的私人信息），我们证明了外部审核员可以仅通过查询模型来获取这些分配所需的信息，以识别这些分布的变化。我们对现实世界数据集的实验结果表明，这种方法是有效的，在检测培训集中人口统计组不足的转移方面达到了80--100％的AUC-ROC。研究人员和调查记者可以使用我们的工具对专有模型进行非授权审核，并在培训数据集中暴露出不足的案例。

Machine learning (ML) models, e.g., deep neural networks (DNNs), are vulnerable to adversarial examples: malicious inputs modified to yield erroneous model outputs, while appearing unmodified to human observers. Potential attacks include having malicious content like malware identified as legitimate or controlling vehicle behavior. Yet, all existing adversarial example attacks require knowledge of either the model internals or its training data. We introduce the first practical demonstration of an attacker controlling a remotely hosted DNN with no such knowledge. Indeed, the only capability of our black-box adversary is to observe labels given by the DNN to chosen inputs. Our attack strategy consists in training a local model to substitute for the target DNN, using inputs synthetically generated by an adversary and labeled by the target DNN. We use the local substitute to craft adversarial examples, and find that they are misclassified by the targeted DNN. To perform a real-world and properly-blinded evaluation, we attack a DNN hosted by MetaMind, an online deep learning API. We find that their DNN misclassifies 84.24% of the adversarial examples crafted with our substitute. We demonstrate the general applicability of our strategy to many ML techniques by conducting the same attack against models hosted by Amazon and Google, using logistic regression substitutes. They yield adversarial examples misclassified by Amazon and Google at rates of 96.19% and 88.94%. We also find that this black-box attack strategy is capable of evading defense strategies previously found to make adversarial example crafting harder.

机器学习（ML）在渲染影响社会各个群体的决策中起着越来越重要的作用。 ML模型为刑事司法的决定，银行业中的信贷延长以及公司的招聘做法提供了信息。这提出了模型公平性的要求，这表明自动化的决策对于受保护特征（例如，性别，种族或年龄）通常是公平的，这些特征通常在数据中代表性不足。我们假设这个代表性不足的问题是数据学习不平衡问题的必然性。此类不平衡通常反映在两个类别和受保护的功能中。例如，一个班级（那些获得信用的班级）对于另一个班级（未获得信用的人）可能会过分代表，而特定组（女性）（女性）的代表性可能与另一组（男性）有关。相对于受保护组的算法公平性的关键要素是同时减少了基础培训数据中的类和受保护的群体失衡，这促进了模型准确性和公平性的提高。我们通过展示这些领域中的关键概念如何重叠和相互补充，讨论弥合失衡学习和群体公平的重要性；并提出了一种新颖的过采样算法，即公平的过采样，该算法既解决偏斜的类别分布和受保护的特征。我们的方法：（i）可以用作标准ML算法的有效预处理算法，以共同解决不平衡和群体权益； （ii）可以与公平感知的学习算法结合使用，以提高其对不同水平不平衡水平的稳健性。此外，我们迈出了一步，将公平和不平衡学习之间的差距与新的公平实用程序之间的差距弥合，从而将平衡的准确性与公平性结合在一起。

Although deep neural networks (DNNs) have achieved great success in many tasks, they can often be fooled by adversarial examples that are generated by adding small but purposeful distortions to natural examples. Previous studies to defend against adversarial examples mostly focused on refining the DNN models, but have either shown limited success or required expensive computation. We propose a new strategy, feature squeezing, that can be used to harden DNN models by detecting adversarial examples. Feature squeezing reduces the search space available to an adversary by coalescing samples that correspond to many different feature vectors in the original space into a single sample. By comparing a DNN model's prediction on the original input with that on squeezed inputs, feature squeezing detects adversarial examples with high accuracy and few false positives.This paper explores two feature squeezing methods: reducing the color bit depth of each pixel and spatial smoothing. These simple strategies are inexpensive and complementary to other defenses, and can be combined in a joint detection framework to achieve high detection rates against state-of-the-art attacks.

Neural networks are known to be vulnerable to adversarial examples: inputs that are close to natural inputs but classified incorrectly. In order to better understand the space of adversarial examples, we survey ten recent proposals that are designed for detection and compare their efficacy. We show that all can be defeated by constructing new loss functions. We conclude that adversarial examples are significantly harder to detect than previously appreciated, and the properties believed to be intrinsic to adversarial examples are in fact not. Finally, we propose several simple guidelines for evaluating future proposed defenses.

Explainability has been widely stated as a cornerstone of the responsible and trustworthy use of machine learning models. With the ubiquitous use of Deep Neural Network (DNN) models expanding to risk-sensitive and safety-critical domains, many methods have been proposed to explain the decisions of these models. Recent years have also seen concerted efforts that have shown how such explanations can be distorted (attacked) by minor input perturbations. While there have been many surveys that review explainability methods themselves, there has been no effort hitherto to assimilate the different methods and metrics proposed to study the robustness of explanations of DNN models. In this work, we present a comprehensive survey of methods that study, understand, attack, and defend explanations of DNN models. We also present a detailed review of different metrics used to evaluate explanation methods, as well as describe attributional attack and defense methods. We conclude with lessons and take-aways for the community towards ensuring robust explanations of DNN model predictions.

媒体报道指责人们对“偏见”'，“”性别歧视“和”种族主义“的人士指责。研究文献中有共识，面部识别准确性为女性较低，妇女通常具有更高的假匹配率和更高的假非匹配率。然而，几乎没有出版的研究，旨在识别女性准确性较低的原因。例如，2019年的面部识别供应商测试将在广泛的算法和数据集中记录较低的女性准确性，并且数据集也列出了“分析原因和效果”在“我们没有做的东西”下''。我们介绍了第一个实验分析，以确定在去以前研究的数据集上对女性的较低人脸识别准确性的主要原因。在测试图像中控制相等的可见面部可见面积减轻了女性的表观更高的假非匹配率。其他分析表明，化妆平衡数据集进一步改善了女性以实现较低的虚假非匹配率。最后，聚类实验表明，两种不同女性的图像本质上比两种不同的男性更相似，潜在地占错误匹配速率的差异。

Recent studies demonstrate that machine learning algorithms can discriminate based on classes like race and gender. In this work, we present an approach to evaluate bias present in automated facial analysis algorithms and datasets with respect to phenotypic subgroups. Using the dermatologist approved Fitzpatrick Skin Type classification system, we characterize the gender and skin type distribution of two facial analysis benchmarks, IJB-A and Adience. We find that these datasets are overwhelmingly composed of lighter-skinned subjects (79.6% for IJB-A and 86.2% for Adience) and introduce a new facial analysis dataset which is balanced by gender and skin type. We evaluate 3 commercial gender classification systems using our dataset and show that darker-skinned females are the most misclassified group (with error rates of up to 34.7%). The maximum error rate for lighter-skinned males is 0.8%. The substantial disparities in the accuracy of classifying darker females, lighter females, darker males, and lighter males in gender classification systems require urgent attention if commercial companies are to build genuinely fair, transparent and accountable facial analysis algorithms.

Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS-and which illustrate a diverse set of defense strategies-can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result-showing that a defense was ineffective-this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. Some of our attack strategies are generalizable, but no single strategy would have been sufficient for all defenses. This underlines our key message that adaptive attacks cannot be automated and always require careful and appropriate tuning to a given defense. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.

机器学习算法已被证明通过系统修改（例如，图像识别）中的输入（例如，对抗性示例）的系统修改（例如，对抗性示例）容易受到对抗操作的影响。在默认威胁模型下，对手利用了图像的无约束性质。每个功能（像素）完全由对手控制。但是，尚不清楚这些攻击如何转化为限制对手可以修改的特征以及如何修改特征的约束域（例如，网络入侵检测）。在本文中，我们探讨了受约束的域是否比不受约束的域对对抗性示例生成算法不那么脆弱。我们创建了一种用于生成对抗草图的算法：针对性的通用扰动向量，该向量在域约束的信封内编码特征显着性。为了评估这些算法的性能，我们在受约束（例如网络入侵检测）和不受约束（例如图像识别）域中评估它们。结果表明，我们的方法在约束域中产生错误分类率，这些域与不受约束的域（大于95％）相当。我们的调查表明，受约束域暴露的狭窄攻击表面仍然足够大，可以制作成功的对抗性例子。因此，约束似乎并不能使域变得健壮。实际上，只有五个随机选择的功能，仍然可以生成对抗性示例。

自动面检测等计算机视觉应用用于各种目的，从解锁智能设备到跟踪监视的潜在感兴趣的人。这些申请的审计透露，他们倾向于对少数民族群体偏见，导致不公平和关于社会和政治结果。尽管随着时间的推移，但这些偏差尚未完全减轻，但实际上已经增加了年龄预测等任务。虽然这些系统审核了基准数据集，但有必要评估其对抗性投入的鲁棒性。在这项工作中，我们在多个系统和数据集上进行广泛的对手审核，并进行了许多关于观察 - 从以前的审计以来的一些任务对一些任务进行了准确性。虽然仍然对多个数据集的少数群体的个体仍然存在偏差，但更令人担忧的观察是这些偏差倾向于对少数群体的对抗意义进行过度发音。我们讨论了鉴于这些观察结果更广泛的社会影响以及关于如何共同应对这个问题的建议。

Speech-centric machine learning systems have revolutionized many leading domains ranging from transportation and healthcare to education and defense, profoundly changing how people live, work, and interact with each other. However, recent studies have demonstrated that many speech-centric ML systems may need to be considered more trustworthy for broader deployment. Specifically, concerns over privacy breaches, discriminating performance, and vulnerability to adversarial attacks have all been discovered in ML research fields. In order to address the above challenges and risks, a significant number of efforts have been made to ensure these ML systems are trustworthy, especially private, safe, and fair. In this paper, we conduct the first comprehensive survey on speech-centric trustworthy ML topics related to privacy, safety, and fairness. In addition to serving as a summary report for the research community, we point out several promising future research directions to inspire the researchers who wish to explore further in this area.

分发推断，有时称为财产推断，Infers关于从访问该数据训练的模型设置的训练的统计属性。分发推理攻击可能会在私人数据培训培训时构成严重风险，但难以从统计机器学习的内在目的区分 - 即生产捕获统计特性的模型。 yeom等人的推导框架的动机，我们提出了一般的主要定义，这足以描述区分可能训练分布的广泛攻击。我们展示了我们的定义如何捕获基于比率的属性推论攻击以及新类型的攻击，包括揭示训练图的平均节点度或聚类系数。为了理解分发推理风险，我们介绍了一种量化，通过将观察到的泄漏与泄漏直接提供给对手的样本来进行泄漏来介绍观察到的泄漏。我们在一系列不同的发行版中报告了一系列不同的分布，并使用全新的黑匣子攻击和最先进的白盒攻击版本。我们的研究结果表明，廉价的攻击往往与昂贵的元分类器攻击一样有效，并且攻击有效性令人惊讶的不对称。

Recent work has demonstrated that deep neural networks are vulnerable to adversarial examples-inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models. 1