深度神经网络容易受到称为对抗性攻击的小输入扰动。通过迭代最大限度地减少网络对真正阶级标签的信心来构建这些对手的事实，我们提出了旨在反对这种效果的反对派层。特别地，我们的层在对手1的相反方向上产生输入扰动，并馈送分类器的输入的扰动版本。我们的方法是无培训和理论上的支持。我们通过将我们的层与名义上和强大的培训模型组合来验证我们的方法的有效性，并从黑盒进行大规模实验到CIFAR10，CIFAR100和ImageNet的自适应攻击。我们的层显着提高了模型鲁棒性，同时在清洁准确性上没有成本。

在测试时间进行优化的自适应防御能力有望改善对抗性鲁棒性。我们对这种自适应测试时间防御措施进行分类，解释其潜在的好处和缺点，并评估图像分类的最新自适应防御能力的代表性。不幸的是，经过我们仔细的案例研究评估时，没有任何显着改善静态防御。有些甚至削弱了基本静态模型，同时增加了推理计算。尽管这些结果令人失望，但我们仍然认为自适应测试时间防御措施是一项有希望的研究途径，因此，我们为他们的彻底评估提供了建议。我们扩展了Carlini等人的清单。（2019年）通过提供针对自适应防御的具体步骤。

Adversarial examples are perturbed inputs designed to fool machine learning models. Adversarial training injects such examples into training data to increase robustness. To scale this technique to large datasets, perturbations are crafted using fast single-step methods that maximize a linear approximation of the model's loss. We show that this form of adversarial training converges to a degenerate global minimum, wherein small curvature artifacts near the data points obfuscate a linear approximation of the loss. The model thus learns to generate weak perturbations, rather than defend against strong ones. As a result, we find that adversarial training remains vulnerable to black-box attacks, where we transfer perturbations computed on undefended models, as well as to a powerful novel single-step attack that escapes the non-smooth vicinity of the input data via a small random step. We further introduce Ensemble Adversarial Training, a technique that augments training data with perturbations transferred from other models. On ImageNet, Ensemble Adversarial Training yields models with stronger robustness to blackbox attacks. In particular, our most robust model won the first round of the NIPS 2017 competition on Defenses against Adversarial Attacks (Kurakin et al., 2017c). However, subsequent work found that more elaborate black-box attacks could significantly enhance transferability and reduce the accuracy of our models.

本文研究了深度神经网络训练期间的语义对齐功能如何增加网络鲁棒性。最近的作品观察到对抗性训练导致强大的模型，其学众的特征似乎与人类感知相关。通过这种联系的启发，从鲁棒性到语义，我们研究了互补的连接：从语义到鲁棒性。为此，我们为基于距离的分类模型（基于群集的分类器）提供了一种稳健性证书。此外，我们表明该证书紧张，我们利用它提出植入攻击（鲁棒性培训），是一种基于集群和对抗的培训框架来学习强大的模型。有趣的是，\ Textit {Clustr}在强大的PGD攻击下优于普遍训练的网络，高达4 \％$ 4 \％。

This paper investigates recently proposed approaches for defending against adversarial examples and evaluating adversarial robustness. We motivate adversarial risk as an objective for achieving models robust to worst-case inputs. We then frame commonly used attacks and evaluation metrics as defining a tractable surrogate objective to the true adversarial risk. This suggests that models may optimize this surrogate rather than the true adversarial risk. We formalize this notion as obscurity to an adversary, and develop tools and heuristics for identifying obscured models and designing transparent models. We demonstrate that this is a significant problem in practice by repurposing gradient-free optimization techniques into adversarial attacks, which we use to decrease the accuracy of several recently proposed defenses to near zero. Our hope is that our formulations and results will help researchers to develop more powerful defenses.

The field of defense strategies against adversarial attacks has significantly grown over the last years, but progress is hampered as the evaluation of adversarial defenses is often insufficient and thus gives a wrong impression of robustness. Many promising defenses could be broken later on, making it difficult to identify the state-of-the-art. Frequent pitfalls in the evaluation are improper tuning of hyperparameters of the attacks, gradient obfuscation or masking. In this paper we first propose two extensions of the PGD-attack overcoming failures due to suboptimal step size and problems of the objective function. We then combine our novel attacks with two complementary existing ones to form a parameter-free, computationally affordable and user-independent ensemble of attacks to test adversarial robustness. We apply our ensemble to over 50 models from papers published at recent top machine learning and computer vision venues. In all except one of the cases we achieve lower robust test accuracy than reported in these papers, often by more than 10%, identifying several broken defenses.

深度卷积神经网络（CNN）很容易被输入图像的细微，不可察觉的变化所欺骗。为了解决此漏洞，对抗训练会创建扰动模式，并将其包括在培训设置中以鲁棒性化模型。与仅使用阶级有限信息的现有对抗训练方法（例如，使用交叉渗透损失）相反，我们建议利用功能空间中的其他信息来促进更强的对手，这些信息又用于学习强大的模型。具体来说，我们将使用另一类的目标样本的样式和内容信息以及其班级边界信息来创建对抗性扰动。我们以深入监督的方式应用了我们提出的多任务目标，从而提取了多尺度特征知识，以创建最大程度地分开对手。随后，我们提出了一种最大边缘对抗训练方法，该方法可最大程度地减少源图像与其对手之间的距离，并最大程度地提高对手和目标图像之间的距离。与最先进的防御能力相比，我们的对抗训练方法表明了强大的鲁棒性，可以很好地推广到自然发生的损坏和数据分配变化，并保留了清洁示例的模型准确性。

我们表明，当考虑到图像域$ [0,1] ^ D $时，已建立$ L_1 $ -Projected梯度下降（PGD）攻击是次优，因为它们不认为有效的威胁模型是交叉点$ l_1 $ -ball和$ [0,1] ^ d $。我们研究了这种有效威胁模型的最陡渐进步骤的预期稀疏性，并表明该组上的确切投影是计算可行的，并且产生更好的性能。此外，我们提出了一种自适应形式的PGD，即使具有小的迭代预算，这也是非常有效的。我们的结果$ l_1 $ -apgd是一个强大的白盒攻击，表明先前的作品高估了他们的$ l_1 $ -trobustness。使用$ l_1 $ -apgd for vercersarial培训，我们获得一个强大的分类器，具有sota $ l_1 $ -trobustness。最后，我们将$ l_1 $ -apgd和平方攻击的适应组合到$ l_1 $ to $ l_1 $ -autoattack，这是一个攻击的集合，可靠地评估$ l_1 $ -ball与$的威胁模型的对抗鲁棒性进行对抗[ 0,1] ^ d $。

深度神经网络已成为现代图像识别系统的驱动力。然而，神经网络对抗对抗性攻击的脆弱性对受这些系统影响的人构成严重威胁。在本文中，我们专注于一个真实的威胁模型，中间对手恶意拦截和erturbs网页用户上传在线。这种类型的攻击可以在简单的性能下降之上提高严重的道德问题。为了防止这种攻击，我们设计了一种新的双层优化算法，该算法在对抗对抗扰动的自然图像附近找到点。CiFar-10和Imagenet的实验表明我们的方法可以有效地强制在给定的修改预算范围内的自然图像。我们还显示所提出的方法可以在共同使用随机平滑时提高鲁棒性。

虽然深度神经网络在各种任务中表现出前所未有的性能，但对对抗性示例的脆弱性阻碍了他们在安全关键系统中的部署。许多研究表明，即使在黑盒设置中也可能攻击，其中攻击者无法访问目标模型的内部信息。大多数黑匣子攻击基于查询，每个都可以获得目标模型的输入输出，并且许多研究侧重于减少所需查询的数量。在本文中，我们注意了目标模型的输出完全对应于查询输入的隐含假设。如果将某些随机性引入模型中，它可以打破假设，因此，基于查询的攻击可能在梯度估计和本地搜索中具有巨大的困难，这是其攻击过程的核心。从这种动机来看，我们甚至观察到一个小的添加剂输入噪声可以中和大多数基于查询的攻击和名称这个简单但有效的方法小噪声防御（SND）。我们分析了SND如何防御基于查询的黑匣子攻击，并展示其与CIFAR-10和ImageNet数据集的八种最先进的攻击有效性。即使具有强大的防御能力，SND几乎保持了原始的分类准确性和计算速度。通过在推断下仅添加一行代码，SND很容易适用于预先训练的模型。

Recent work has demonstrated that deep neural networks are vulnerable to adversarial examples-inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models. 1

随机平滑是一种最近的技术，可以在训练中实现最先进的性能，从而确认强大的深度神经网络。虽然平滑的分布家族通常连接到用于认证的规范的选择，但这些分布的参数始终将其视为全局超级参数，独立于网络认证的输入数据。在这项工作中，我们重新访问高斯随机平滑，并表明可以在每个输入时优化高斯分布的方差，以最大程度地提高构建平滑分类器的认证半径。由于数据依赖性分类器未直接使用现有方法享受合理的认证，因此我们提出了一个可通过构造认证的记忆增强数据依赖的平滑分类器。这种新方法是通用，无参数且易于实现的。实际上，我们表明我们的数据依赖框架可以无缝地纳入3种随机平滑方法中，从而导致一致的提高认证准确性。当这些方法的训练例程中使用此框架，然后是数据依赖性认证时，我们比CIFAR10和Imagenet上0.5的最强基线的认证准确度提高了9％和6％。

在计算机视觉应用中广泛采用深度神经网络引起了对对抗性鲁棒性的重大兴趣。现有的研究表明，专门针对给定模型量身定制的恶意扰动输入（即，对抗性示例）可以成功地转移到另一个受过独立训练的模型中，以引起预测错误。此外，这种对抗性示例的属性归因于数据分布中的预测模式得出的特征。因此，我们有动力调查以下问题：对抗性防御，例如对抗性例子，可以成功地转移到其他受过独立训练的模型中？为此，我们提出了一种基于深度学习的预处理机制，我们将其称为可鲁棒的可转移功能提取器（RTFE）。在研究了理论动机和含义后，我们在实验上表明，我们的方法可以为多个独立训练的分类器提供对抗性的鲁棒性，这些分类器原本是对自适应白盒对手的无效性。此外，我们表明RTFE甚至可以为在不同数据集中独立训练的模型提供单发对手的鲁棒性。

尽管深层神经网络在各种任务中取得了巨大的成功，但它们对不可察觉的对抗性扰动的脆弱性阻碍了他们在现实世界中的部署。最近，与随机合奏的作品相对于经过最小的计算开销的标准对手训练（AT）模型，对对抗性训练（AT）模型的对抗性鲁棒性有了显着改善，这使它们成为安全临界资源限制应用程序的有前途解决方案。但是，这种令人印象深刻的表现提出了一个问题：这些稳健性是由随机合奏提供的吗？在这项工作中，我们从理论和经验上都解决了这个问题。从理论上讲，我们首先确定通常采用的鲁棒性评估方法（例如自适应PGD）在这种情况下提供了错误的安全感。随后，我们提出了一种理论上有效的对抗攻击算法（ARC），即使在自适应PGD无法做到这一点的情况下，也能妥协随机合奏。我们在各种网络体系结构，培训方案，数据集和规范上进行全面的实验，以支持我们的主张，并经验证明，随机合奏实际上比在模型上更容易受到$ \ ell_p $结合的对抗性扰动的影响。我们的代码可以在https://github.com/hsndbk4/arc上找到。

We identify obfuscated gradients, a kind of gradient masking, as a phenomenon that leads to a false sense of security in defenses against adversarial examples. While defenses that cause obfuscated gradients appear to defeat iterative optimizationbased attacks, we find defenses relying on this effect can be circumvented. We describe characteristic behaviors of defenses exhibiting the effect, and for each of the three types of obfuscated gradients we discover, we develop attack techniques to overcome it. In a case study, examining noncertified white-box-secure defenses at ICLR 2018, we find obfuscated gradients are a common occurrence, with 7 of 9 defenses relying on obfuscated gradients. Our new attacks successfully circumvent 6 completely, and 1 partially, in the original threat model each paper considers.

Adversarial training is an effective approach to make deep neural networks robust against adversarial attacks. Recently, different adversarial training defenses are proposed that not only maintain a high clean accuracy but also show significant robustness against popular and well studied adversarial attacks such as PGD. High adversarial robustness can also arise if an attack fails to find adversarial gradient directions, a phenomenon known as `gradient masking'. In this work, we analyse the effect of label smoothing on adversarial training as one of the potential causes of gradient masking. We then develop a guided mechanism to avoid local minima during attack optimization, leading to a novel attack dubbed Guided Projected Gradient Attack (G-PGA). Our attack approach is based on a `match and deceive' loss that finds optimal adversarial directions through guidance from a surrogate model. Our modified attack does not require random restarts, large number of attack iterations or search for an optimal step-size. Furthermore, our proposed G-PGA is generic, thus it can be combined with an ensemble attack strategy as we demonstrate for the case of Auto-Attack, leading to efficiency and convergence speed improvements. More than an effective attack, G-PGA can be used as a diagnostic tool to reveal elusive robustness due to gradient masking in adversarial defenses.

Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS-and which illustrate a diverse set of defense strategies-can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result-showing that a defense was ineffective-this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. Some of our attack strategies are generalizable, but no single strategy would have been sufficient for all defenses. This underlines our key message that adaptive attacks cannot be automated and always require careful and appropriate tuning to a given defense. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.

深度卷积神经网络可以准确地分类各种自然图像，但是在设计时可能很容易被欺骗，图像中嵌入了不可察觉的扰动。在本文中，我们设计了一种多管齐下的培训，输入转换和图像集成系统，该系统是攻击不可知论的，不容易估计。我们的系统结合了两个新型功能。第一个是一个转换层，该转换层从集体级训练数据示例中计算级别的多项式内核，并且迭代更新在推理时间上基于其特征内核差异的输入图像副本，以创建转换后的输入集合。第二个是一个分类系统，该系统将未防御网络的预测结合在一起，对被过滤图像的合奏进行了硬投票。我们在CIFAR10数据集上的评估显示，我们的系统提高了未防御性网络在不同距离指标下的各种有界和无限的白色盒子攻击的鲁棒性，同时牺牲了清洁图像的精度很小。反对自适应的全知攻击者创建端到端攻击，我们的系统成功地增强了对抗训练的网络的现有鲁棒性，为此，我们的方法最有效地应用了。

The problem of adversarial defenses for image classification, where the goal is to robustify a classifier against adversarial examples, is considered. Inspired by the hypothesis that these examples lie beyond the natural image manifold, a novel aDversarIal defenSe with local impliCit functiOns (DISCO) is proposed to remove adversarial perturbations by localized manifold projections. DISCO consumes an adversarial image and a query pixel location and outputs a clean RGB value at the location. It is implemented with an encoder and a local implicit module, where the former produces per-pixel deep features and the latter uses the features in the neighborhood of query pixel for predicting the clean RGB value. Extensive experiments demonstrate that both DISCO and its cascade version outperform prior defenses, regardless of whether the defense is known to the attacker. DISCO is also shown to be data and parameter efficient and to mount defenses that transfers across datasets, classifiers and attacks.

随着现实世界图像的大小不同，机器学习模型是包括上游图像缩放算法的较大系统的一部分。在本文中，我们研究了基于决策的黑框设置中图像缩放过程的漏洞与机器学习模型之间的相互作用。我们提出了一种新颖的采样策略，以端到端的方式使黑框攻击利用漏洞在缩放算法，缩放防御和最终的机器学习模型中。基于这种缩放感知的攻击，我们揭示了大多数现有的缩放防御能力在下游模型的威胁下无效。此外，我们从经验上观察到，标准的黑盒攻击可以通过利用脆弱的缩放程序来显着提高其性能。我们进一步在具有基于决策的黑盒攻击的商业图像分析API上证明了这个问题。