虽然近年来，深度神经网络（DNN）的采用率大幅增加，但尚未发现对对抗对抗例子的脆弱性的解决方案。因此，大量的研究工作致力于解决这种弱点，许多研究通常使用源图像的子集来生成对抗示例，将该子集中的每个图像视为相等。我们证明，实际上，不是每个来源图像都同样适用于这种评估。为此，我们设计了一个大规模的模型到模型转移性方案，我们通过利用三种最常用的攻击来精心分析来自想象成中的每个合适的源图像中的每个合适的源图像。在这种可转移性方案中，这涉及七种不同的DNN模型，包括最近提出的视觉变压器，我们揭示了在模型到模型转移性成功中获得高达12.5美元的差异，平均为1.01美元L_2 $扰动，平均每平均$ 0.03 $（$ 8/225 $），当所有合适的候选人中随机采样1000美元的源图像时，每次$ 0.03 $（$ 8/225 $）。然后，我们采取一个第一个步骤评估用于创造逆势示例的图像的稳健性，提出了许多简单但有效的方法来识别不合适的源图像，从而使得可以减轻实验中的极端情况并支持高质量的基准测试。

虽然ImageNet最初被提出为计算机愿景领域的性能基准的数据集，但它也支持各种其他研究工作。对抗机器学习是一种这样的研究努力，采用欺骗性输入来制作错误的预测。为了评估对抗机器学习领域的攻击和防御，Imagenet仍然是最常用的数据集之一。但是，尚待调查的主题是对抗性实例被错误分类的课程的性质。在本文中，我们对这些错误分类类进行了详细的分析，利用了想象群类层次结构并测量了对逆势示例的不受干扰的起源中上述类别的相对位置。我们发现71％的普遍例子，即实现模型 - 模型对抗性转移性的普遍例子被错误分类为对底层源图像预测的前5个类之一。我们还发现，实际上，大量未确定的错误分类子集实际上是分类到语义上类似的课程。根据这些调查结果，我们讨论在评估未确定的对抗性成功时需要考虑到Imageenet类层次结构。此外，我们倡导未来的研究努力，以合并分类信息。

Neural networks are vulnerable to adversarial examples, which poses a threat to their application in security sensitive systems. We propose high-level representation guided denoiser (HGD) as a defense for image classification. Standard denoiser suffers from the error amplification effect, in which small residual adversarial noise is progressively amplified and leads to wrong classifications. HGD overcomes this problem by using a loss function defined as the difference between the target model's outputs activated by the clean image and denoised image. Compared with ensemble adversarial training which is the state-of-the-art defending method on large images, HGD has three advantages. First, with HGD as a defense, the target model is more robust to either white-box or black-box adversarial attacks. Second, HGD can be trained on a small subset of the images and generalizes well to other images and unseen classes. Third, HGD can be transferred to defend models other than the one guiding it. In NIPS competition on defense against adversarial attacks, our HGD solution won the first place and outperformed other models by a large margin. 1 * Equal contribution.

The authors thank Nicholas Carlini (UC Berkeley) and Dimitris Tsipras (MIT) for feedback to improve the survey quality. We also acknowledge X. Huang (Uni. Liverpool), K. R. Reddy (IISC), E. Valle (UNICAMP), Y. Yoo (CLAIR) and others for providing pointers to make the survey more comprehensive.

We propose the Square Attack, a score-based black-box l2and l∞-adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking. Square Attack is based on a randomized search scheme which selects localized squareshaped updates at random positions so that at each iteration the perturbation is situated approximately at the boundary of the feasible set. Our method is significantly more query efficient and achieves a higher success rate compared to the state-of-the-art methods, especially in the untargeted setting. In particular, on ImageNet we improve the average query efficiency in the untargeted setting for various deep networks by a factor of at least 1.8 and up to 3 compared to the recent state-ofthe-art l∞-attack of Al-Dujaili & OReilly (2020). Moreover, although our attack is black-box, it can also outperform gradient-based white-box attacks on the standard benchmarks achieving a new state-of-the-art in terms of the success rate. The code of our attack is available at https://github.com/max-andr/square-attack.

Deep neural networks are vulnerable to adversarial examples, which poses security concerns on these algorithms due to the potentially severe consequences. Adversarial attacks serve as an important surrogate to evaluate the robustness of deep learning models before they are deployed. However, most of existing adversarial attacks can only fool a black-box model with a low success rate. To address this issue, we propose a broad class of momentum-based iterative algorithms to boost adversarial attacks. By integrating the momentum term into the iterative process for attacks, our methods can stabilize update directions and escape from poor local maxima during the iterations, resulting in more transferable adversarial examples. To further improve the success rates for black-box attacks, we apply momentum iterative algorithms to an ensemble of models, and show that the adversarially trained models with a strong defense ability are also vulnerable to our black-box attacks. We hope that the proposed methods will serve as a benchmark for evaluating the robustness of various deep models and defense methods. With this method, we won the first places in NIPS 2017 Non-targeted Adversarial Attack and Targeted Adversarial Attack competitions.

Although deep neural networks (DNNs) have achieved great success in many tasks, they can often be fooled by adversarial examples that are generated by adding small but purposeful distortions to natural examples. Previous studies to defend against adversarial examples mostly focused on refining the DNN models, but have either shown limited success or required expensive computation. We propose a new strategy, feature squeezing, that can be used to harden DNN models by detecting adversarial examples. Feature squeezing reduces the search space available to an adversary by coalescing samples that correspond to many different feature vectors in the original space into a single sample. By comparing a DNN model's prediction on the original input with that on squeezed inputs, feature squeezing detects adversarial examples with high accuracy and few false positives.This paper explores two feature squeezing methods: reducing the color bit depth of each pixel and spatial smoothing. These simple strategies are inexpensive and complementary to other defenses, and can be combined in a joint detection framework to achieve high detection rates against state-of-the-art attacks.

深度神经网络容易受到通过对输入对难以察觉的变化进行制作的对抗性示例。但是，这些对手示例在适用于模型及其参数的白盒设置中最成功。寻找可转移到其他模型或在黑匣子设置中开发的对抗性示例显着更加困难。在本文中，我们提出了可转移的对抗性实例的方向聚集的对抗性攻击。我们的方法在攻击过程中使用聚集方向，以避免产生的对抗性示例在白盒模型上过度拟合。关于Imagenet的广泛实验表明，我们的提出方法显着提高了对抗性实例的可转移性，优于最先进的攻击，特别是对抗对抗性稳健的模型。我们所提出的方法的最佳平均攻击成功率达到94.6 \％，针对三种对手训练模型和94.8％抵御五种防御方法。它还表明，目前的防御方法不会阻止可转移的对抗性攻击。

尖峰神经网络（SNN）因其高能量效率和分类性能的最新进展而引起了很多关注。但是，与传统的深度学习方法不同，对SNN对对抗性例子的鲁棒性的分析和研究仍然相对欠发达。在这项工作中，我们通过实验和分析三个重要的SNN安全属性来推进对抗机器学习的领域。首先，我们表明对SNN的成功白盒对抗性攻击高度依赖于潜在的替代梯度技术。其次，我们分析了SNN和其他最先进的体系结构（如视觉变压器和大型传输CNN）生成的对抗性示例的可传递性。我们证明，SNN并不经常被视觉变压器和某些类型的CNN产生的对抗典范所欺骗。最后，我们开发了一种新颖的白盒攻击，该攻击生成了能够同时欺骗SNN模型和非SNN模型的对抗性示例。我们的实验和分析是广泛而严格的，涵盖了两个数据集（CIFAR-10和CIFAR-100），五种不同的白色盒子攻击以及十二个不同的分类器模型。

虽然深度神经网络在各种任务中表现出前所未有的性能，但对对抗性示例的脆弱性阻碍了他们在安全关键系统中的部署。许多研究表明，即使在黑盒设置中也可能攻击，其中攻击者无法访问目标模型的内部信息。大多数黑匣子攻击基于查询，每个都可以获得目标模型的输入输出，并且许多研究侧重于减少所需查询的数量。在本文中，我们注意了目标模型的输出完全对应于查询输入的隐含假设。如果将某些随机性引入模型中，它可以打破假设，因此，基于查询的攻击可能在梯度估计和本地搜索中具有巨大的困难，这是其攻击过程的核心。从这种动机来看，我们甚至观察到一个小的添加剂输入噪声可以中和大多数基于查询的攻击和名称这个简单但有效的方法小噪声防御（SND）。我们分析了SND如何防御基于查询的黑匣子攻击，并展示其与CIFAR-10和ImageNet数据集的八种最先进的攻击有效性。即使具有强大的防御能力，SND几乎保持了原始的分类准确性和计算速度。通过在推断下仅添加一行代码，SND很容易适用于预先训练的模型。

With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks have been recently found vulnerable to well-designed input samples, called adversarial examples. Adversarial perturbations are imperceptible to human but can easily fool deep neural networks in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying deep neural networks in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for deep neural networks, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples. In addition, three major challenges in adversarial examples and the potential solutions are discussed.

There has been a concurrent significant improvement in the medical images used to facilitate diagnosis and the performance of machine learning techniques to perform tasks such as classification, detection, and segmentation in recent years. As a result, a rapid increase in the usage of such systems can be observed in the healthcare industry, for instance in the form of medical image classification systems, where these models have achieved diagnostic parity with human physicians. One such application where this can be observed is in computer vision tasks such as the classification of skin lesions in dermatoscopic images. However, as stakeholders in the healthcare industry, such as insurance companies, continue to invest extensively in machine learning infrastructure, it becomes increasingly important to understand the vulnerabilities in such systems. Due to the highly critical nature of the tasks being carried out by these machine learning models, it is necessary to analyze techniques that could be used to take advantage of these vulnerabilities and methods to defend against them. This paper explores common adversarial attack techniques. The Fast Sign Gradient Method and Projected Descent Gradient are used against a Convolutional Neural Network trained to classify dermatoscopic images of skin lesions. Following that, it also discusses one of the most popular adversarial defense techniques, adversarial training. The performance of the model that has been trained on adversarial examples is then tested against the previously mentioned attacks, and recommendations to improve neural networks robustness are thus provided based on the results of the experiment.

对抗性示例的可转移性是应用这种攻击基于真实环境中的深度学习（DL）的多媒体取证（MMF）技术应用这种攻击的关键问题。事实上，对攻击者没有全面了解待攻击系统的情况，对侵犯柜台取证攻击的部署也会开辟道路。一些初步作品表明，对基于CNN的图像取证检测器的对抗示例通常是不可转移的，至少当采用最受欢迎的库中实现的攻击的基本版本时。在本文中，我们介绍了一般的策略，以提高攻击的强度，并在这种强度变化时评估其可转化性。我们通过实验表明，通过这种方式，攻击可转让性可以在很大程度上增加，以牺牲更大的变形。我们的研究证实了甚至在多媒体取证方案中存在对抗性示例所带来的安全威胁，因此要求新的防御策略来提高基于DL的MMF技术的安全性。

基于转移的对手示例是最重要的黑匣子攻击类别之一。然而，在对抗性扰动的可转移性和难以察觉之间存在权衡。在此方向上的事先工作经常需要固定但大量的$ \ ell_p $ -norm扰动预算，达到良好的转移成功率，导致可察觉的对抗扰动。另一方面，目前的大多数旨在产生语义保留扰动的难以限制的对抗攻击患有对目标模型的可转移性较弱。在这项工作中，我们提出了一个几何形象感知框架，以产生具有最小变化的可转移的对抗性示例。类似于在统计机器学习中的模型选择，我们利用验证模型为$ \ ell _ {\ infty} $ - norm和不受限制的威胁模型中选择每个图像的最佳扰动预算。广泛的实验验证了我们对平衡令人难以置信的难以察觉和可转移性的框架的有效性。方法论是我们进入CVPR'21安全性AI挑战者的基础：对想象成的不受限制的对抗攻击，其中我们将第1位排名第1,559支队伍，并在决赛方面超过了亚军提交的提交4.59％和23.91％分别和平均图像质量水平。代码可在https://github.com/equationliu/ga-attack获得。

已知深度神经网络（DNN）容易受到用不可察觉的扰动制作的对抗性示例的影响，即，输入图像的微小变化会引起错误的分类，从而威胁着基于深度学习的部署系统的可靠性。经常采用对抗训练（AT）来通过训练损坏和干净的数据的混合物来提高DNN的鲁棒性。但是，大多数基于AT的方法在处理\ textit {转移的对抗示例}方面是无效的，这些方法是生成以欺骗各种防御模型的生成的，因此无法满足现实情况下提出的概括要求。此外，对抗性训练一般的国防模型不能对具有扰动的输入产生可解释的预测，而不同的领域专家则需要一个高度可解释的强大模型才能了解DNN的行为。在这项工作中，我们提出了一种基于Jacobian规范和选择性输入梯度正则化（J-SIGR）的方法，该方法通过Jacobian归一化提出了线性化的鲁棒性，还将基于扰动的显着性图正规化，以模仿模型的可解释预测。因此，我们既可以提高DNN的防御能力和高解释性。最后，我们评估了跨不同体系结构的方法，以针对强大的对抗性攻击。实验表明，提出的J-Sigr赋予了针对转移的对抗攻击的鲁棒性，我们还表明，来自神经网络的预测易于解释。

Deep neural networks are vulnerable to adversarial examples, which can mislead classifiers by adding imperceptible perturbations. An intriguing property of adversarial examples is their good transferability, making black-box attacks feasible in real-world applications. Due to the threat of adversarial attacks, many methods have been proposed to improve the robustness. Several state-of-the-art defenses are shown to be robust against transferable adversarial examples. In this paper, we propose a translation-invariant attack method to generate more transferable adversarial examples against the defense models. By optimizing a perturbation over an ensemble of translated images, the generated adversarial example is less sensitive to the white-box model being attacked and has better transferability. To improve the efficiency of attacks, we further show that our method can be implemented by convolving the gradient at the untranslated image with a pre-defined kernel. Our method is generally applicable to any gradient-based attack method. Extensive experiments on the ImageNet dataset validate the effectiveness of the proposed method. Our best attack fools eight state-of-the-art defenses at an 82% success rate on average based only on the transferability, demonstrating the insecurity of the current defense techniques.

Recent work has demonstrated that deep neural networks are vulnerable to adversarial examples-inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models. 1

对抗性实例的有趣现象引起了机器学习中的显着关注，对社区可能更令人惊讶的是存在普遍对抗扰动（UAPS），即欺骗目标DNN的单一扰动。随着对深层分类器的关注，本调查总结了最近普遍对抗攻击的进展，讨论了攻击和防御方的挑战，以及uap存在的原因。我们的目标是将此工作扩展为动态调查，该调查将定期更新其内容，以遵循关于在广泛的域中的UAP或通用攻击的新作品，例如图像，音频，视频，文本等。将讨论相关更新：https://bit.ly/2sbqlgg。我们欢迎未来的作者在该领域的作品，联系我们，包括您的新发现。

深度学习的进步使得广泛的有希望的应用程序。然而，这些系统容易受到对抗机器学习（AML）攻击的影响;对他们的意见的离前事实制作的扰动可能导致他们错误分类。若干最先进的对抗性攻击已经证明他们可以可靠地欺骗分类器，使这些攻击成为一个重大威胁。对抗性攻击生成算法主要侧重于创建成功的例子，同时控制噪声幅度和分布，使检测更加困难。这些攻击的潜在假设是脱机产生的对抗噪声，使其执行时间是次要考虑因素。然而，最近，攻击者机会自由地产生对抗性示例的立即对抗攻击已经可能。本文介绍了一个新问题：我们如何在实时约束下产生对抗性噪音，以支持这种实时对抗攻击？了解这一问题提高了我们对这些攻击对实时系统构成的威胁的理解，并为未来防御提供安全评估基准。因此，我们首先进行对抗生成算法的运行时间分析。普遍攻击脱机产生一般攻击，没有在线开销，并且可以应用于任何输入;然而，由于其一般性，他们的成功率是有限的。相比之下，在特定输入上工作的在线算法是计算昂贵的，使它们不适合在时间约束下的操作。因此，我们提出房间，一种新型实时在线脱机攻击施工模型，其中离线组件用于预热在线算法，使得可以在时间限制下产生高度成功的攻击。

Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS-and which illustrate a diverse set of defense strategies-can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result-showing that a defense was ineffective-this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. Some of our attack strategies are generalizable, but no single strategy would have been sufficient for all defenses. This underlines our key message that adaptive attacks cannot be automated and always require careful and appropriate tuning to a given defense. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.