由于在量化网络上的按位操作产生的有效存储器消耗和更快的计算，神经网络量化已经变得越来越受欢迎。尽管它们表现出优异的泛化能力，但其鲁棒性属性并不是很好地理解。在这项工作中，我们系统地研究量化网络对基于梯度的对抗性攻击的鲁棒性，并证明这些量化模型遭受梯度消失问题并显示出虚假的鲁棒感。通过归因于培训的网络中的渐变消失到较差的前后信号传播，我们引入了一个简单的温度缩放方法来缓解此问题，同时保留决策边界。尽管对基于梯度的对抗攻击进行了简单的修改，但具有多个网络架构的多个图像分类数据集的实验表明，我们的温度缩放攻击在量化网络上获得了近乎完美的成功率，同时表现出对普遍培训的模型以及浮动的原始攻击以及浮动 - 点网络。代码可在https://github.com/kartikgupta-at-anu/Attack-bnn获得。

已知深度神经网络（DNN）容易受到用不可察觉的扰动制作的对抗性示例的影响，即，输入图像的微小变化会引起错误的分类，从而威胁着基于深度学习的部署系统的可靠性。经常采用对抗训练（AT）来通过训练损坏和干净的数据的混合物来提高DNN的鲁棒性。但是，大多数基于AT的方法在处理\ textit {转移的对抗示例}方面是无效的，这些方法是生成以欺骗各种防御模型的生成的，因此无法满足现实情况下提出的概括要求。此外，对抗性训练一般的国防模型不能对具有扰动的输入产生可解释的预测，而不同的领域专家则需要一个高度可解释的强大模型才能了解DNN的行为。在这项工作中，我们提出了一种基于Jacobian规范和选择性输入梯度正则化（J-SIGR）的方法，该方法通过Jacobian归一化提出了线性化的鲁棒性，还将基于扰动的显着性图正规化，以模仿模型的可解释预测。因此，我们既可以提高DNN的防御能力和高解释性。最后，我们评估了跨不同体系结构的方法，以针对强大的对抗性攻击。实验表明，提出的J-Sigr赋予了针对转移的对抗攻击的鲁棒性，我们还表明，来自神经网络的预测易于解释。

The evaluation of robustness against adversarial manipulation of neural networks-based classifiers is mainly tested with empirical attacks as methods for the exact computation, even when available, do not scale to large networks. We propose in this paper a new white-box adversarial attack wrt the l p -norms for p ∈ {1, 2, ∞} aiming at finding the minimal perturbation necessary to change the class of a given input. It has an intuitive geometric meaning, yields quickly high quality results, minimizes the size of the perturbation (so that it returns the robust accuracy at every threshold with a single run). It performs better or similar to stateof-the-art attacks which are partially specialized to one l p -norm, and is robust to the phenomenon of gradient masking.

The field of defense strategies against adversarial attacks has significantly grown over the last years, but progress is hampered as the evaluation of adversarial defenses is often insufficient and thus gives a wrong impression of robustness. Many promising defenses could be broken later on, making it difficult to identify the state-of-the-art. Frequent pitfalls in the evaluation are improper tuning of hyperparameters of the attacks, gradient obfuscation or masking. In this paper we first propose two extensions of the PGD-attack overcoming failures due to suboptimal step size and problems of the objective function. We then combine our novel attacks with two complementary existing ones to form a parameter-free, computationally affordable and user-independent ensemble of attacks to test adversarial robustness. We apply our ensemble to over 50 models from papers published at recent top machine learning and computer vision venues. In all except one of the cases we achieve lower robust test accuracy than reported in these papers, often by more than 10%, identifying several broken defenses.

Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS-and which illustrate a diverse set of defense strategies-can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result-showing that a defense was ineffective-this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. Some of our attack strategies are generalizable, but no single strategy would have been sufficient for all defenses. This underlines our key message that adaptive attacks cannot be automated and always require careful and appropriate tuning to a given defense. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.

Designing powerful adversarial attacks is of paramount importance for the evaluation of $\ell_p$-bounded adversarial defenses. Projected Gradient Descent (PGD) is one of the most effective and conceptually simple algorithms to generate such adversaries. The search space of PGD is dictated by the steepest ascent directions of an objective. Despite the plethora of objective function choices, there is no universally superior option and robustness overestimation may arise from ill-suited objective selection. Driven by this observation, we postulate that the combination of different objectives through a simple loss alternating scheme renders PGD more robust towards design choices. We experimentally verify this assertion on a synthetic-data example and by evaluating our proposed method across 25 different $\ell_{\infty}$-robust models and 3 datasets. The performance improvement is consistent, when compared to the single loss counterparts. In the CIFAR-10 dataset, our strongest adversarial attack outperforms all of the white-box components of AutoAttack (AA) ensemble, as well as the most powerful attacks existing on the literature, achieving state-of-the-art results in the computational budget of our study ($T=100$, no restarts).

对抗性的鲁棒性已经成为深度学习的核心目标，无论是在理论和实践中。然而，成功的方法来改善对抗的鲁棒性（如逆势训练）在不受干扰的数据上大大伤害了泛化性能。这可能会对对抗性鲁棒性如何影响现实世界系统的影响（即，如果它可以提高未受干扰的数据的准确性），许多人可能选择放弃鲁棒性）。我们提出内插对抗培训，该培训最近雇用了在对抗培训框架内基于插值的基于插值的培训方法。在CiFar -10上，对抗性训练增加了标准测试错误（当没有对手时）从4.43％到12.32％，而我们的内插对抗培训我们保留了对抗性的鲁棒性，同时实现了仅6.45％的标准测试误差。通过我们的技术，强大模型标准误差的相对增加从178.1％降至仅为45.5％。此外，我们提供内插对抗性培训的数学分析，以确认其效率，并在鲁棒性和泛化方面展示其优势。

We propose the Square Attack, a score-based black-box l2and l∞-adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking. Square Attack is based on a randomized search scheme which selects localized squareshaped updates at random positions so that at each iteration the perturbation is situated approximately at the boundary of the feasible set. Our method is significantly more query efficient and achieves a higher success rate compared to the state-of-the-art methods, especially in the untargeted setting. In particular, on ImageNet we improve the average query efficiency in the untargeted setting for various deep networks by a factor of at least 1.8 and up to 3 compared to the recent state-ofthe-art l∞-attack of Al-Dujaili & OReilly (2020). Moreover, although our attack is black-box, it can also outperform gradient-based white-box attacks on the standard benchmarks achieving a new state-of-the-art in terms of the success rate. The code of our attack is available at https://github.com/max-andr/square-attack.

Adversarial training, in which a network is trained on adversarial examples, is one of the few defenses against adversarial attacks that withstands strong attacks. Unfortunately, the high cost of generating strong adversarial examples makes standard adversarial training impractical on large-scale problems like ImageNet. We present an algorithm that eliminates the overhead cost of generating adversarial examples by recycling the gradient information computed when updating model parameters.Our "free" adversarial training algorithm achieves comparable robustness to PGD adversarial training on the CIFAR-10 and CIFAR-100 datasets at negligible additional cost compared to natural training, and can be 7 to 30 times faster than other strong adversarial training methods. Using a single workstation with 4 P100 GPUs and 2 days of runtime, we can train a robust model for the large-scale ImageNet classification task that maintains 40% accuracy against PGD attacks. The code is available at https://github.com/ashafahi/free_adv_train.

Recent work has demonstrated that deep neural networks are vulnerable to adversarial examples-inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models. 1

评估防御模型的稳健性是对抗对抗鲁棒性研究的具有挑战性的任务。僵化的渐变，先前已经发现了一种梯度掩蔽，以许多防御方法存在并导致鲁棒性的错误信号。在本文中，我们确定了一种更细微的情况，称为不平衡梯度，也可能导致过高的对抗性鲁棒性。当边缘损耗的一个术语的梯度主导并将攻击朝向次优化方向推动时，发生不平衡梯度的现象。为了利用不平衡的梯度，我们制定了分解利润率损失的边缘分解（MD）攻击，并通过两阶段过程分别探讨了这些术语的攻击性。我们还提出了一个Multared和Ensemble版本的MD攻击。通过调查自2018年以来提出的17个防御模型，我们发现6种型号易受不平衡梯度的影响，我们的MD攻击可以减少由最佳基线独立攻击评估的鲁棒性另外2％。我们还提供了对不平衡梯度的可能原因和有效对策的深入分析。

许多最先进的ML模型在各种任务中具有优于图像分类的人类。具有如此出色的性能，ML模型今天被广泛使用。然而，存在对抗性攻击和数据中毒攻击的真正符合ML模型的稳健性。例如，Engstrom等人。证明了最先进的图像分类器可以容易地被任意图像上的小旋转欺骗。由于ML系统越来越纳入安全性和安全敏感的应用，对抗攻击和数据中毒攻击构成了相当大的威胁。本章侧重于ML安全的两个广泛和重要的领域：对抗攻击和数据中毒攻击。

虽然多步逆势培训被广泛流行作为对抗强烈的对抗攻击的有效防御方法，但其计算成本与标准培训相比，其计算成本是众所周知的。已经提出了几种单步侵权培训方法来减轻上述开销费用;但是，根据优化设置，它们的性能并不能充分可靠。为了克服这些限制，我们偏离了现有的基于输入空间的对抗性培训制度，并提出了一种单步潜在培训方法（SLAT），其利用潜在的代表梯度作为潜在的对抗扰动。我们证明，与所采用的潜伏扰动，恢复局部线性度并确保与现有的单步逆势训练方法相比，恢复局部线性度并确保可靠性的特征梯度的L1规范。因为潜伏的扰动基于可以在输入梯度计算过程中免费获得的潜在表示的梯度，所以所提出的方法与快速梯度标志方法相当成本。实验结果表明，尽管其结构简单，但优于最先进的加速的对抗训练方法。

到目前为止对抗训练是抵御对抗例子的最有效的策略。然而，由于每个训练步骤中的迭代对抗性攻击，它遭受了高的计算成本。最近的研究表明，通过随机初始化执行单步攻击，可以实现快速的对抗训练。然而，这种方法仍然落后于稳定性和模型稳健性的最先进的对手训练算法。在这项工作中，我们通过观察随机平滑的随机初始化来更好地优化内部最大化问题，对快速对抗培训进行新的理解。在这种新的视角之后，我们还提出了一种新的初始化策略，向后平滑，进一步提高单步强大培训方法的稳定性和模型稳健性。多个基准测试的实验表明，我们的方法在使用更少的训练时间（使用相同的培训计划时，使用更少的培训时间（$ \ sim $ 3x改进）时，我们的方法达到了类似的模型稳健性。

Deep neural networks are vulnerable to adversarial examples, which poses security concerns on these algorithms due to the potentially severe consequences. Adversarial attacks serve as an important surrogate to evaluate the robustness of deep learning models before they are deployed. However, most of existing adversarial attacks can only fool a black-box model with a low success rate. To address this issue, we propose a broad class of momentum-based iterative algorithms to boost adversarial attacks. By integrating the momentum term into the iterative process for attacks, our methods can stabilize update directions and escape from poor local maxima during the iterations, resulting in more transferable adversarial examples. To further improve the success rates for black-box attacks, we apply momentum iterative algorithms to an ensemble of models, and show that the adversarially trained models with a strong defense ability are also vulnerable to our black-box attacks. We hope that the proposed methods will serve as a benchmark for evaluating the robustness of various deep models and defense methods. With this method, we won the first places in NIPS 2017 Non-targeted Adversarial Attack and Targeted Adversarial Attack competitions.

This paper investigates recently proposed approaches for defending against adversarial examples and evaluating adversarial robustness. We motivate adversarial risk as an objective for achieving models robust to worst-case inputs. We then frame commonly used attacks and evaluation metrics as defining a tractable surrogate objective to the true adversarial risk. This suggests that models may optimize this surrogate rather than the true adversarial risk. We formalize this notion as obscurity to an adversary, and develop tools and heuristics for identifying obscured models and designing transparent models. We demonstrate that this is a significant problem in practice by repurposing gradient-free optimization techniques into adversarial attacks, which we use to decrease the accuracy of several recently proposed defenses to near zero. Our hope is that our formulations and results will help researchers to develop more powerful defenses.

虽然深度神经网络（DNN）在许多真实的任务中实现了出色的性能，但它们非常容易受到对抗的攻击。对抗这种攻击的主要防御是对抗的，一种技术，通过将对抗噪声引入其输入来训练DNN培训以训练为对抗性攻击的技术。此程序是有效的，但必须在培训阶段进行。在这项工作中，我们提出了增强随机森林（ARF），这是一个简单易用的策略，用于在不修改其权重的情况下强化现有的预磨损DNN。对于每个图像，我们通过应用不同颜色，模糊，噪声和几何变换来生成随机测试时间增强。然后我们使用DNN的Logits输出来训练一个简单的随机林来预测真正的类标签。我们的方法在自然图像的分类上最小的妥协，实现了最先进的对抗鲁棒性对白和黑匣子攻击的多样性。我们也针对许多适应性的白盒攻击测试ARF，并在与对抗训练结合时显示出优异的结果。代码可在https://github.com/giladcohen/arf获得。

Adversarial training is an effective approach to make deep neural networks robust against adversarial attacks. Recently, different adversarial training defenses are proposed that not only maintain a high clean accuracy but also show significant robustness against popular and well studied adversarial attacks such as PGD. High adversarial robustness can also arise if an attack fails to find adversarial gradient directions, a phenomenon known as `gradient masking'. In this work, we analyse the effect of label smoothing on adversarial training as one of the potential causes of gradient masking. We then develop a guided mechanism to avoid local minima during attack optimization, leading to a novel attack dubbed Guided Projected Gradient Attack (G-PGA). Our attack approach is based on a `match and deceive' loss that finds optimal adversarial directions through guidance from a surrogate model. Our modified attack does not require random restarts, large number of attack iterations or search for an optimal step-size. Furthermore, our proposed G-PGA is generic, thus it can be combined with an ensemble attack strategy as we demonstrate for the case of Auto-Attack, leading to efficiency and convergence speed improvements. More than an effective attack, G-PGA can be used as a diagnostic tool to reveal elusive robustness due to gradient masking in adversarial defenses.

深度神经网络很容易被称为对抗攻击的小扰动都愚弄。对抗性培训（AT）是一种近似解决了稳健的优化问题，以最大限度地减少最坏情况损失，并且被广泛认为是对这种攻击的最有效的防御。由于产生了强大的对抗性示例的高计算时间，已经提出了单步方法来减少培训时间。然而，这些方法遭受灾难性的过度装备，在训练期间侵犯准确度下降。虽然提出了改进，但它们增加了培训时间和稳健性远非多步骤。我们为FW优化（FW-AT）开发了对抗的对抗培训的理论框架，揭示了损失景观与$ \ ell_2 $失真之间的几何连接。我们分析地表明FW攻击的高变形相当于沿攻击路径的小梯度变化。然后在各种深度神经网络架构上进行实验证明，$ \ ell \ infty $攻击对抗强大的模型实现近乎最大的$ \ ell_2 $失真，而标准网络具有较低的失真。此外，实验表明，灾难性的过度拟合与FW攻击的低变形强烈相关。为了展示我们理论框架的效用，我们开发FW-AT-Adap，这是一种新的逆势训练算法，它使用简单的失真度量来调整攻击步骤的数量，以提高效率而不会影响鲁棒性。 FW-AT-Adapt提供培训时间以单步快速分配方法，并改善了在白色盒子和黑匣子设置中的普发内精度的最小损失和多步PGD之间的差距。

对抗培训，培训具有对抗性数据的深层学习模型的过程，是深度学习模型中最成功的对抗性防御方法之一。我们发现，如果我们在推理阶段微调这一模型以适应对抗的输入，可以进一步提高对普遍训练模型的白箱攻击的鲁棒性，以适应对手输入，其中包含额外信息。我们介绍了一种算法，即“邮政列车”在原始输出类和“邻居”类之间的推断阶段的模型，具有现有培训数据。预训练的FAST-FGSM CIFAR10分类器基础模型对白盒预计梯度攻击（PGD）的准确性可以通过我们的算法显着提高46.8％至64.5％。