深度学习的进步使得广泛的有希望的应用程序。然而，这些系统容易受到对抗机器学习（AML）攻击的影响;对他们的意见的离前事实制作的扰动可能导致他们错误分类。若干最先进的对抗性攻击已经证明他们可以可靠地欺骗分类器，使这些攻击成为一个重大威胁。对抗性攻击生成算法主要侧重于创建成功的例子，同时控制噪声幅度和分布，使检测更加困难。这些攻击的潜在假设是脱机产生的对抗噪声，使其执行时间是次要考虑因素。然而，最近，攻击者机会自由地产生对抗性示例的立即对抗攻击已经可能。本文介绍了一个新问题：我们如何在实时约束下产生对抗性噪音，以支持这种实时对抗攻击？了解这一问题提高了我们对这些攻击对实时系统构成的威胁的理解，并为未来防御提供安全评估基准。因此，我们首先进行对抗生成算法的运行时间分析。普遍攻击脱机产生一般攻击，没有在线开销，并且可以应用于任何输入;然而，由于其一般性，他们的成功率是有限的。相比之下，在特定输入上工作的在线算法是计算昂贵的，使它们不适合在时间约束下的操作。因此，我们提出房间，一种新型实时在线脱机攻击施工模型，其中离线组件用于预热在线算法，使得可以在时间限制下产生高度成功的攻击。

许多最先进的ML模型在各种任务中具有优于图像分类的人类。具有如此出色的性能，ML模型今天被广泛使用。然而，存在对抗性攻击和数据中毒攻击的真正符合ML模型的稳健性。例如，Engstrom等人。证明了最先进的图像分类器可以容易地被任意图像上的小旋转欺骗。由于ML系统越来越纳入安全性和安全敏感的应用，对抗攻击和数据中毒攻击构成了相当大的威胁。本章侧重于ML安全的两个广泛和重要的领域：对抗攻击和数据中毒攻击。

基于深度神经网络（DNN）的智能信息（IOT）系统已被广泛部署在现实世界中。然而，发现DNNS易受对抗性示例的影响，这提高了人们对智能物联网系统的可靠性和安全性的担忧。测试和评估IOT系统的稳健性成为必要和必要。最近已经提出了各种攻击和策略，但效率问题仍未纠正。现有方法是计算地广泛或耗时，这在实践中不适用。在本文中，我们提出了一种称为攻击启发GaN（AI-GaN）的新框架，在有条件地产生对抗性实例。曾经接受过培训，可以有效地给予对抗扰动的输入图像和目标类。我们在白盒设置的不同数据集中应用AI-GaN，黑匣子设置和由最先进的防御保护的目标模型。通过广泛的实验，AI-GaN实现了高攻击成功率，优于现有方法，并显着降低了生成时间。此外，首次，AI-GaN成功地缩放到复杂的数据集。 Cifar-100和Imagenet，所有课程中的成功率约为90美元。

尽管机器学习系统的效率和可扩展性，但最近的研究表明，许多分类方法，尤其是深神经网络（DNN），易受对抗的例子;即，仔细制作欺骗训练有素的分类模型的例子，同时无法区分从自然数据到人类。这使得在安全关键区域中应用DNN或相关方法可能不安全。由于这个问题是由Biggio等人确定的。 （2013）和Szegedy等人。（2014年），在这一领域已经完成了很多工作，包括开发攻击方法，以产生对抗的例子和防御技术的构建防范这些例子。本文旨在向统计界介绍这一主题及其最新发展，主要关注对抗性示例的产生和保护。在数值实验中使用的计算代码（在Python和R）公开可用于读者探讨调查的方法。本文希望提交人们将鼓励更多统计学人员在这种重要的令人兴奋的领域的产生和捍卫对抗的例子。

With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks have been recently found vulnerable to well-designed input samples, called adversarial examples. Adversarial perturbations are imperceptible to human but can easily fool deep neural networks in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying deep neural networks in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for deep neural networks, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples. In addition, three major challenges in adversarial examples and the potential solutions are discussed.

Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to adversarial examples: given an input x and any target classification t, it is possible to find a new input x that is similar to x but classified as t. This makes it difficult to apply neural networks in security-critical areas. Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from 95% to 0.5%.In this paper, we demonstrate that defensive distillation does not significantly increase the robustness of neural networks by introducing three new attack algorithms that are successful on both distilled and undistilled neural networks with 100% probability. Our attacks are tailored to three distance metrics used previously in the literature, and when compared to previous adversarial example generation algorithms, our attacks are often much more effective (and never worse). Furthermore, we propose using high-confidence adversarial examples in a simple transferability test we show can also be used to break defensive distillation. We hope our attacks will be used as a benchmark in future defense attempts to create neural networks that resist adversarial examples.

Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS-and which illustrate a diverse set of defense strategies-can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result-showing that a defense was ineffective-this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. Some of our attack strategies are generalizable, but no single strategy would have been sufficient for all defenses. This underlines our key message that adaptive attacks cannot be automated and always require careful and appropriate tuning to a given defense. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.

对抗性实例的有趣现象引起了机器学习中的显着关注，对社区可能更令人惊讶的是存在普遍对抗扰动（UAPS），即欺骗目标DNN的单一扰动。随着对深层分类器的关注，本调查总结了最近普遍对抗攻击的进展，讨论了攻击和防御方的挑战，以及uap存在的原因。我们的目标是将此工作扩展为动态调查，该调查将定期更新其内容，以遵循关于在广泛的域中的UAP或通用攻击的新作品，例如图像，音频，视频，文本等。将讨论相关更新：https://bit.ly/2sbqlgg。我们欢迎未来的作者在该领域的作品，联系我们，包括您的新发现。

最近对机器学习（ML）模型的攻击，例如逃避攻击，具有对抗性示例，并通过提取攻击窃取了一些模型，构成了几种安全性和隐私威胁。先前的工作建议使用对抗性训练从对抗性示例中保护模型，以逃避模型的分类并恶化其性能。但是，这种保护技术会影响模型的决策边界及其预测概率，因此可能会增加模型隐私风险。实际上，仅使用对模型预测输出的查询访问的恶意用户可以提取它并获得高智能和高保真替代模型。为了更大的提取，这些攻击利用了受害者模型的预测概率。实际上，所有先前关于提取攻击的工作都没有考虑到出于安全目的的培训过程中的变化。在本文中，我们提出了一个框架，以评估具有视觉数据集对对抗训练的模型的提取攻击。据我们所知，我们的工作是第一个进行此类评估的工作。通过一项广泛的实证研究，我们证明了受对抗训练的模型比在自然训练情况下获得的模型更容易受到提取攻击的影响。他们可以达到高达$ \ times1.2 $更高的准确性和同意，而疑问低于$ \ times0.75 $。我们还发现，与从自然训练的（即标准）模型中提取的DNN相比，从鲁棒模型中提取的对抗性鲁棒性能力可通过提取攻击（即从鲁棒模型提取的深神经网络（DNN）提取的深神网络（DNN））传递。

时间序列数据在许多现实世界中（例如，移动健康）和深神经网络（DNNS）中产生，在解决它们方面已取得了巨大的成功。尽管他们成功了，但对他们对对抗性攻击的稳健性知之甚少。在本文中，我们提出了一个通过统计特征（TSA-STAT）}称为时间序列攻击的新型对抗框架}。为了解决时间序列域的独特挑战，TSA-STAT对时间序列数据的统计特征采取限制来构建对抗性示例。优化的多项式转换用于创建比基于加性扰动的攻击（就成功欺骗DNN而言）更有效的攻击。我们还提供有关构建对抗性示例的统计功能规范的认证界限。我们对各种现实世界基准数据集的实验表明，TSA-STAT在欺骗DNN的时间序列域和改善其稳健性方面的有效性。 TSA-STAT算法的源代码可在https://github.com/tahabelkhouja/time-series-series-attacks-via-statity-features上获得

Although deep neural networks (DNNs) have achieved great success in many tasks, they can often be fooled by adversarial examples that are generated by adding small but purposeful distortions to natural examples. Previous studies to defend against adversarial examples mostly focused on refining the DNN models, but have either shown limited success or required expensive computation. We propose a new strategy, feature squeezing, that can be used to harden DNN models by detecting adversarial examples. Feature squeezing reduces the search space available to an adversary by coalescing samples that correspond to many different feature vectors in the original space into a single sample. By comparing a DNN model's prediction on the original input with that on squeezed inputs, feature squeezing detects adversarial examples with high accuracy and few false positives.This paper explores two feature squeezing methods: reducing the color bit depth of each pixel and spatial smoothing. These simple strategies are inexpensive and complementary to other defenses, and can be combined in a joint detection framework to achieve high detection rates against state-of-the-art attacks.

深度学习（DL）系统的安全性是一个极为重要的研究领域，因为它们正在部署在多个应用程序中，因为它们不断改善，以解决具有挑战性的任务。尽管有压倒性的承诺，但深度学习系统容易受到制作的对抗性例子的影响，这可能是人眼无法察觉的，但可能会导致模型错误分类。对基于整体技术的对抗性扰动的保护已被证明很容易受到更强大的对手的影响，或者证明缺乏端到端评估。在本文中，我们试图开发一种新的基于整体的解决方案，该解决方案构建具有不同决策边界的防御者模型相对于原始模型。通过（1）通过一种称为拆分和剃须的方法转换输入的分类器的合奏，以及（2）通过一种称为对比度功能的方法限制重要特征，显示出相对于相对于不同的梯度对抗性攻击，这减少了将对抗性示例从原始示例转移到针对同一类的防御者模型的机会。我们使用标准图像分类数据集（即MNIST，CIFAR-10和CIFAR-100）进行了广泛的实验，以实现最新的对抗攻击，以证明基于合奏的防御的鲁棒性。我们还在存在更强大的对手的情况下评估稳健性，该对手同时靶向合奏中的所有模型。已经提供了整体假阳性和误报的结果，以估计提出的方法的总体性能。

The authors thank Nicholas Carlini (UC Berkeley) and Dimitris Tsipras (MIT) for feedback to improve the survey quality. We also acknowledge X. Huang (Uni. Liverpool), K. R. Reddy (IISC), E. Valle (UNICAMP), Y. Yoo (CLAIR) and others for providing pointers to make the survey more comprehensive.

Video classification systems are vulnerable to adversarial attacks, which can create severe security problems in video verification. Current black-box attacks need a large number of queries to succeed, resulting in high computational overhead in the process of attack. On the other hand, attacks with restricted perturbations are ineffective against defenses such as denoising or adversarial training. In this paper, we focus on unrestricted perturbations and propose StyleFool, a black-box video adversarial attack via style transfer to fool the video classification system. StyleFool first utilizes color theme proximity to select the best style image, which helps avoid unnatural details in the stylized videos. Meanwhile, the target class confidence is additionally considered in targeted attacks to influence the output distribution of the classifier by moving the stylized video closer to or even across the decision boundary. A gradient-free method is then employed to further optimize the adversarial perturbations. We carry out extensive experiments to evaluate StyleFool on two standard datasets, UCF-101 and HMDB-51. The experimental results demonstrate that StyleFool outperforms the state-of-the-art adversarial attacks in terms of both the number of queries and the robustness against existing defenses. Moreover, 50% of the stylized videos in untargeted attacks do not need any query since they can already fool the video classification model. Furthermore, we evaluate the indistinguishability through a user study to show that the adversarial samples of StyleFool look imperceptible to human eyes, despite unrestricted perturbations.

已知深度神经网络（DNN）容易受到用不可察觉的扰动制作的对抗性示例的影响，即，输入图像的微小变化会引起错误的分类，从而威胁着基于深度学习的部署系统的可靠性。经常采用对抗训练（AT）来通过训练损坏和干净的数据的混合物来提高DNN的鲁棒性。但是，大多数基于AT的方法在处理\ textit {转移的对抗示例}方面是无效的，这些方法是生成以欺骗各种防御模型的生成的，因此无法满足现实情况下提出的概括要求。此外，对抗性训练一般的国防模型不能对具有扰动的输入产生可解释的预测，而不同的领域专家则需要一个高度可解释的强大模型才能了解DNN的行为。在这项工作中，我们提出了一种基于Jacobian规范和选择性输入梯度正则化（J-SIGR）的方法，该方法通过Jacobian归一化提出了线性化的鲁棒性，还将基于扰动的显着性图正规化，以模仿模型的可解释预测。因此，我们既可以提高DNN的防御能力和高解释性。最后，我们评估了跨不同体系结构的方法，以针对强大的对抗性攻击。实验表明，提出的J-Sigr赋予了针对转移的对抗攻击的鲁棒性，我们还表明，来自神经网络的预测易于解释。

机器学习模型严重易于来自对抗性示例的逃避攻击。通常，对逆势示例的修改输入类似于原始输入的修改输入，在WhiteBox设置下由对手的WhiteBox设置构成，完全访问模型。然而，最近的攻击已经显示出使用BlackBox攻击的对逆势示例的查询号显着减少。特别是，警报是从越来越多的机器学习提供的经过培训的模型的访问界面中利用分类决定作为包括Google，Microsoft，IBM的服务提供商，并由包含这些模型的多种应用程序使用的服务提供商来利用培训的模型。对手仅利用来自模型的预测标签的能力被区别为基于决策的攻击。在我们的研究中，我们首先深入潜入最近的ICLR和SP的最先进的决策攻击，以突出发现低失真对抗采用梯度估计方法的昂贵性质。我们开发了一种强大的查询高效攻击，能够避免在梯度估计方法中看到的嘈杂渐变中的局部最小和误导中的截留。我们提出的攻击方法，ramboattack利用随机块坐标下降的概念来探索隐藏的分类器歧管，针对扰动来操纵局部输入功能以解决梯度估计方法的问题。重要的是，ramboattack对对对手和目标类别可用的不同样本输入更加强大。总的来说，对于给定的目标类，ramboattack被证明在实现给定查询预算的较低失真时更加强大。我们使用大规模的高分辨率ImageNet数据集来策划我们的广泛结果，并在GitHub上开源我们的攻击，测试样本和伪影。

Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model. In practice, the threat model for real-world systems is often more restrictive than the typical black-box model where the adversary can observe the full output of the network on arbitrarily many chosen inputs. We define three realistic threat models that more accurately characterize many real-world classifiers: the query-limited setting, the partialinformation setting, and the label-only setting. We develop new attacks that fool classifiers under these more restrictive threat models, where previous methods would be impractical or ineffective. We demonstrate that our methods are effective against an ImageNet classifier under our proposed threat models. We also demonstrate a targeted black-box attack against a commercial classifier, overcoming the challenges of limited query access, partial information, and other practical issues to break the Google Cloud Vision API.

深度神经网络容易受到来自对抗性投入的攻击，并且最近，特洛伊木马误解或劫持模型的决定。我们通过探索有界抗逆性示例空间和生成的对抗网络内的自然输入空间来揭示有界面的对抗性实例 - 通用自然主义侵害贴片的兴趣类 - 我们呼叫TNT。现在，一个对手可以用一个自然主义的补丁来手臂自己，不太恶意，身体上可实现，高效 - 实现高攻击成功率和普遍性。 TNT是普遍的，因为在场景中的TNT中捕获的任何输入图像都将：i）误导网络（未确定的攻击）;或ii）迫使网络进行恶意决定（有针对性的攻击）。现在，有趣的是，一个对抗性补丁攻击者有可能发挥更大的控制水平 - 选择一个独立，自然的贴片的能力，与被限制为嘈杂的扰动的触发器 - 到目前为止只有可能与特洛伊木马攻击方法有可能干扰模型建设过程，以嵌入风险发现的后门;但是，仍然意识到在物理世界中部署的补丁。通过对大型视觉分类任务的广泛实验，想象成在其整个验证集50,000张图像中进行评估，我们展示了TNT的现实威胁和攻击的稳健性。我们展示了攻击的概括，以创建比现有最先进的方法实现更高攻击成功率的补丁。我们的结果表明，攻击对不同的视觉分类任务（CIFAR-10，GTSRB，PUBFIG）和多个最先进的深神经网络，如WieredEnet50，Inception-V3和VGG-16。

Adversarial attacks hamper the decision-making ability of neural networks by perturbing the input signal. The addition of calculated small distortion to images, for instance, can deceive a well-trained image classification network. In this work, we propose a novel attack technique called Sparse Adversarial and Interpretable Attack Framework (SAIF). Specifically, we design imperceptible attacks that contain low-magnitude perturbations at a small number of pixels and leverage these sparse attacks to reveal the vulnerability of classifiers. We use the Frank-Wolfe (conditional gradient) algorithm to simultaneously optimize the attack perturbations for bounded magnitude and sparsity with $O(1/\sqrt{T})$ convergence. Empirical results show that SAIF computes highly imperceptible and interpretable adversarial examples, and outperforms state-of-the-art sparse attack methods on the ImageNet dataset.

Video compression plays a crucial role in video streaming and classification systems by maximizing the end-user quality of experience (QoE) at a given bandwidth budget. In this paper, we conduct the first systematic study for adversarial attacks on deep learning-based video compression and downstream classification systems. Our attack framework, dubbed RoVISQ, manipulates the Rate-Distortion ($\textit{R}$-$\textit{D}$) relationship of a video compression model to achieve one or both of the following goals: (1) increasing the network bandwidth, (2) degrading the video quality for end-users. We further devise new objectives for targeted and untargeted attacks to a downstream video classification service. Finally, we design an input-invariant perturbation that universally disrupts video compression and classification systems in real time. Unlike previously proposed attacks on video classification, our adversarial perturbations are the first to withstand compression. We empirically show the resilience of RoVISQ attacks against various defenses, i.e., adversarial training, video denoising, and JPEG compression. Our extensive experimental results on various video datasets show RoVISQ attacks deteriorate peak signal-to-noise ratio by up to 5.6dB and the bit-rate by up to $\sim$ 2.4$\times$ while achieving over 90$\%$ attack success rate on a downstream classifier. Our user study further demonstrates the effect of RoVISQ attacks on users' QoE.