自然语言视频本地化(NLVL)是视觉语言理解区域的重要任务,该方面还要求深入了解单独的计算机视觉和自然语言侧,但更重要的是两侧之间的相互作用。对抗性脆弱性得到了很好的认可,作为深度神经网络模型的关键安全问题,需要谨慎调查。尽管在视频和语言任务中进行了广泛但分开的研究,但目前对NLVL等愿景联合任务的对抗鲁棒性的理解较少。因此,本文旨在通过检查攻击和防御方面的三个脆弱性,全面调查NLVL模型的对抗性鲁棒性。为了实现攻击目标,我们提出了一种新的对抗攻击范式,称为同义句子感知对抗对抗攻击对逆向(潜行),这捕获了视觉和语言侧面之间的跨模式相互作用。
translated by 谷歌翻译
Video classification systems are vulnerable to adversarial attacks, which can create severe security problems in video verification. Current black-box attacks need a large number of queries to succeed, resulting in high computational overhead in the process of attack. On the other hand, attacks with restricted perturbations are ineffective against defenses such as denoising or adversarial training. In this paper, we focus on unrestricted perturbations and propose StyleFool, a black-box video adversarial attack via style transfer to fool the video classification system. StyleFool first utilizes color theme proximity to select the best style image, which helps avoid unnatural details in the stylized videos. Meanwhile, the target class confidence is additionally considered in targeted attacks to influence the output distribution of the classifier by moving the stylized video closer to or even across the decision boundary. A gradient-free method is then employed to further optimize the adversarial perturbations. We carry out extensive experiments to evaluate StyleFool on two standard datasets, UCF-101 and HMDB-51. The experimental results demonstrate that StyleFool outperforms the state-of-the-art adversarial attacks in terms of both the number of queries and the robustness against existing defenses. Moreover, 50% of the stylized videos in untargeted attacks do not need any query since they can already fool the video classification model. Furthermore, we evaluate the indistinguishability through a user study to show that the adversarial samples of StyleFool look imperceptible to human eyes, despite unrestricted perturbations.
translated by 谷歌翻译
对抗性训练(AT)是针对对抗分类系统的对抗性攻击的简单而有效的防御,这是基于增强训练设置的攻击,从而最大程度地提高了损失。但是,AT作为视频分类的辩护的有效性尚未得到彻底研究。我们的第一个贡献是表明,为视频生成最佳攻击需要仔细调整攻击参数,尤其是步骤大小。值得注意的是,我们证明最佳步长随攻击预算线性变化。我们的第二个贡献是表明,在训练时间使用较小(次优的)攻击预算会导致测试时的性能更加强大。根据这些发现,我们提出了三个防御攻击预算的攻击的防御。自适应AT的第一个技术是一种技术,该技术是从随着训练迭代进行的。第二个课程是一项技术,随着训练的迭代进行,攻击预算的增加。第三个生成的AT,与deno的生成对抗网络一起,以提高稳健的性能。 UCF101数据集上的实验表明,所提出的方法改善了针对多种攻击类型的对抗性鲁棒性。
translated by 谷歌翻译
积极调查深度神经网络的对抗鲁棒性。然而,大多数现有的防御方法限于特定类型的对抗扰动。具体而言,它们通常不能同时为多次攻击类型提供抵抗力,即,它们缺乏多扰动鲁棒性。此外,与图像识别问题相比,视频识别模型的对抗鲁棒性相对未开发。虽然有几项研究提出了如何产生对抗性视频,但在文献中只发表了关于防御策略的少数关于防御策略的方法。在本文中,我们提出了用于视频识别的多种抗逆视频的第一战略之一。所提出的方法称为Multibn,使用具有基于学习的BN选择模块的多个独立批量归一化(BN)层对多个对冲视频类型进行对抗性训练。利用多个BN结构,每个BN Brach负责学习单个扰动类型的分布,从而提供更精确的分布估计。这种机制有利于处理多种扰动类型。 BN选择模块检测输入视频的攻击类型,并将其发送到相应的BN分支,使MultiBN全自动并允许端接训练。与目前的对抗训练方法相比,所提出的Multibn对不同甚至不可预见的对抗性视频类型具有更强的多扰动稳健性,从LP界攻击和物理上可实现的攻击范围。在不同的数据集和目标模型上保持真实。此外,我们进行了广泛的分析,以研究多BN结构的性质。
translated by 谷歌翻译
假新闻的扩散及其严重的负面社会影响力推动了假新闻检测方法成为网络经理的必要工具。同时,社交媒体的多媒体性质使多模式的假新闻检测因其捕获更多模态特征的能力而受欢迎,而不是单模式检测方法。但是,当前有关多模式检测的文献更有可能追求检测准确性,但忽略了检测器的鲁棒性。为了解决这个问题,我们提出了对多模式假新闻探测器的全面鲁棒性评估。在这项工作中,我们模拟了恶意用户和开发人员的攻击方法,即发布假新闻并注入后门。具体而言,我们使用五种对抗和两种后门攻击方法评估了多模式探测器。实验结果暗示:(1)在对抗攻击下,最先进的检测器的检测性能显着降解,甚至比一般检测器更糟; (2)大多数多模式探测器受到视觉模态的攻击比文本模态更容易受到攻击; (3)当受欢迎的事件的图像在探测器遭受后门攻击时会导致探测器的重大降解; (4)在多模式攻击下这些检测器的性能比在单模式攻击下更糟糕; (5)防御方法将改善多模式探测器的鲁棒性。
translated by 谷歌翻译
尽管机器学习系统的效率和可扩展性,但最近的研究表明,许多分类方法,尤其是深神经网络(DNN),易受对抗的例子;即,仔细制作欺骗训练有素的分类模型的例子,同时无法区分从自然数据到人类。这使得在安全关键区域中应用DNN或相关方法可能不安全。由于这个问题是由Biggio等人确定的。 (2013)和Szegedy等人。(2014年),在这一领域已经完成了很多工作,包括开发攻击方法,以产生对抗的例子和防御技术的构建防范这些例子。本文旨在向统计界介绍这一主题及其最新发展,主要关注对抗性示例的产生和保护。在数值实验中使用的计算代码(在Python和R)公开可用于读者探讨调查的方法。本文希望提交人们将鼓励更多统计学人员在这种重要的令人兴奋的领域的产生和捍卫对抗的例子。
translated by 谷歌翻译
Robustness evaluation against adversarial examples has become increasingly important to unveil the trustworthiness of the prevailing deep models in natural language processing (NLP). However, in contrast to the computer vision domain where the first-order projected gradient descent (PGD) is used as the benchmark approach to generate adversarial examples for robustness evaluation, there lacks a principled first-order gradient-based robustness evaluation framework in NLP. The emerging optimization challenges lie in 1) the discrete nature of textual inputs together with the strong coupling between the perturbation location and the actual content, and 2) the additional constraint that the perturbed text should be fluent and achieve a low perplexity under a language model. These challenges make the development of PGD-like NLP attacks difficult. To bridge the gap, we propose TextGrad, a new attack generator using gradient-driven optimization, supporting high-accuracy and high-quality assessment of adversarial robustness in NLP. Specifically, we address the aforementioned challenges in a unified optimization framework. And we develop an effective convex relaxation method to co-optimize the continuously-relaxed site selection and perturbation variables and leverage an effective sampling method to establish an accurate mapping from the continuous optimization variables to the discrete textual perturbations. Moreover, as a first-order attack generation method, TextGrad can be baked into adversarial training to further improve the robustness of NLP models. Extensive experiments are provided to demonstrate the effectiveness of TextGrad not only in attack generation for robustness evaluation but also in adversarial defense.
translated by 谷歌翻译
Video compression plays a crucial role in video streaming and classification systems by maximizing the end-user quality of experience (QoE) at a given bandwidth budget. In this paper, we conduct the first systematic study for adversarial attacks on deep learning-based video compression and downstream classification systems. Our attack framework, dubbed RoVISQ, manipulates the Rate-Distortion ($\textit{R}$-$\textit{D}$) relationship of a video compression model to achieve one or both of the following goals: (1) increasing the network bandwidth, (2) degrading the video quality for end-users. We further devise new objectives for targeted and untargeted attacks to a downstream video classification service. Finally, we design an input-invariant perturbation that universally disrupts video compression and classification systems in real time. Unlike previously proposed attacks on video classification, our adversarial perturbations are the first to withstand compression. We empirically show the resilience of RoVISQ attacks against various defenses, i.e., adversarial training, video denoising, and JPEG compression. Our extensive experimental results on various video datasets show RoVISQ attacks deteriorate peak signal-to-noise ratio by up to 5.6dB and the bit-rate by up to $\sim$ 2.4$\times$ while achieving over 90$\%$ attack success rate on a downstream classifier. Our user study further demonstrates the effect of RoVISQ attacks on users' QoE.
translated by 谷歌翻译
尽管在许多机器学习任务方面取得了巨大成功,但深度神经网络仍然易于对抗对抗样本。虽然基于梯度的对抗攻击方法在计算机视野领域探索,但由于文本的离散性质,直接应用于自然语言处理中,这是不切实际的。为了弥合这一差距,我们提出了一般框架,以适应现有的基于梯度的方法来制作文本对抗性样本。在该框架中,将基于梯度的连续扰动添加到嵌入层中,并在前向传播过程中被放大。然后用掩模语言模型头解码最终的扰动潜在表示以获得潜在的对抗性样本。在本文中,我们将我们的框架与\ textbf {t} Extual \ TextBF {P} ROJECTED \ TextBF {G} Radient \ TextBF {D} excent(\ TextBF {TPGD})进行ronject \ textbf {p}。我们通过在三个基准数据集上执行转移黑匣子攻击来评估我们的框架来评估我们的框架。实验结果表明,与强基线方法相比,我们的方法达到了更好的性能,并产生更精细和语法的对抗性样本。所有代码和数据都将公开。
translated by 谷歌翻译
基于深度神经网络(DNN)的智能信息(IOT)系统已被广泛部署在现实世界中。然而,发现DNNS易受对抗性示例的影响,这提高了人们对智能物联网系统的可靠性和安全性的担忧。测试和评估IOT系统的稳健性成为必要和必要。最近已经提出了各种攻击和策略,但效率问题仍未纠正。现有方法是计算地广泛或耗时,这在实践中不适用。在本文中,我们提出了一种称为攻击启发GaN(AI-GaN)的新框架,在有条件地产生对抗性实例。曾经接受过培训,可以有效地给予对抗扰动的输入图像和目标类。我们在白盒设置的不同数据集中应用AI-GaN,黑匣子设置和由最先进的防御保护的目标模型。通过广泛的实验,AI-GaN实现了高攻击成功率,优于现有方法,并显着降低了生成时间。此外,首次,AI-GaN成功地缩放到复杂的数据集。 Cifar-100和Imagenet,所有课程中的成功率约为90美元。
translated by 谷歌翻译
Adversarial robustness assessment for video recognition models has raised concerns owing to their wide applications on safety-critical tasks. Compared with images, videos have much high dimension, which brings huge computational costs when generating adversarial videos. This is especially serious for the query-based black-box attacks where gradient estimation for the threat models is usually utilized, and high dimensions will lead to a large number of queries. To mitigate this issue, we propose to simultaneously eliminate the temporal and spatial redundancy within the video to achieve an effective and efficient gradient estimation on the reduced searching space, and thus query number could decrease. To implement this idea, we design the novel Adversarial spatial-temporal Focus (AstFocus) attack on videos, which performs attacks on the simultaneously focused key frames and key regions from the inter-frames and intra-frames in the video. AstFocus attack is based on the cooperative Multi-Agent Reinforcement Learning (MARL) framework. One agent is responsible for selecting key frames, and another agent is responsible for selecting key regions. These two agents are jointly trained by the common rewards received from the black-box threat models to perform a cooperative prediction. By continuously querying, the reduced searching space composed of key frames and key regions is becoming precise, and the whole query number becomes less than that on the original video. Extensive experiments on four mainstream video recognition models and three widely used action recognition datasets demonstrate that the proposed AstFocus attack outperforms the SOTA methods, which is prevenient in fooling rate, query number, time, and perturbation magnitude at the same.
translated by 谷歌翻译
深度神经网络的图像分类容易受到对抗性扰动的影响。图像分类可以通过在输入图像中添加人造小且不可察觉的扰动来轻松愚弄。作为最有效的防御策略之一,提出了对抗性训练,以解决分类模型的脆弱性,其中创建了对抗性示例并在培训期间注入培训数据中。在过去的几年中,对分类模型的攻击和防御进行了深入研究。语义细分作为分类的扩展,最近也受到了极大的关注。最近的工作表明,需要大量的攻击迭代来创建有效的对抗性示例来欺骗分割模型。该观察结果既可以使鲁棒性评估和对分割模型的对抗性培训具有挑战性。在这项工作中,我们提出了一种称为SEGPGD的有效有效的分割攻击方法。此外,我们提供了收敛分析,以表明在相同数量的攻击迭代下,提出的SEGPGD可以创建比PGD更有效的对抗示例。此外,我们建议将SEGPGD应用于分割对抗训练的基础攻击方法。由于SEGPGD可以创建更有效的对抗性示例,因此使用SEGPGD的对抗训练可以提高分割模型的鲁棒性。我们的建议还通过对流行分割模型体系结构和标准分段数据集进行了验证。
translated by 谷歌翻译
深度神经网络(DNNS)最近在许多分类任务中取得了巨大的成功。不幸的是,它们容易受到对抗性攻击的影响,这些攻击会产生对抗性示例,这些示例具有很小的扰动,以欺骗DNN模型,尤其是在模型共享方案中。事实证明,对抗性训练是最有效的策略,它将对抗性示例注入模型训练中,以提高DNN模型的稳健性,以对对抗性攻击。但是,基于现有的对抗性示例的对抗训练无法很好地推广到标准,不受干扰的测试数据。为了在标准准确性和对抗性鲁棒性之间取得更好的权衡,我们提出了一个新型的对抗训练框架,称为潜在边界引导的对抗训练(梯子),该训练(梯子)在潜在的边界引导的对抗性示例上对对手进行对手训练DNN模型。与大多数在输入空间中生成对抗示例的现有方法相反,梯子通过增加对潜在特征的扰动而产生了无数的高质量对抗示例。扰动是沿SVM构建的具有注意机制的决策边界的正常情况进行的。我们从边界场的角度和可视化视图分析了生成的边界引导的对抗示例的优点。与Vanilla DNN和竞争性底线相比,对MNIST,SVHN,CELEBA和CIFAR-10的广泛实验和详细分析验证了梯子在标准准确性和对抗性鲁棒性之间取得更好的权衡方面的有效性。
translated by 谷歌翻译
我们提出了一种新颖且有效的纯化基于纯化的普通防御方法,用于预处理盲目的白色和黑匣子攻击。我们的方法仅在一般图像上进行了自我监督学习,在计算上效率和培训,而不需要对分类模型的任何对抗训练或再培训。我们首先显示对原始图像与其对抗示例之间的残余的对抗噪声的实证分析,几乎均为对称分布。基于该观察,我们提出了一种非常简单的迭代高斯平滑(GS),其可以有效地平滑对抗性噪声并实现大大高的鲁棒精度。为了进一步改进它,我们提出了神经上下文迭代平滑(NCIS),其以自我监督的方式列举盲点网络(BSN)以重建GS也平滑的原始图像的辨别特征。从我们使用四种分类模型对大型想象成的广泛实验,我们表明我们的方法既竞争竞争标准精度和最先进的强大精度,则针对最强大的净化器 - 盲目的白色和黑匣子攻击。此外,我们提出了一种用于评估基于商业图像分类API的纯化方法的新基准,例如AWS,Azure,Clarifai和Google。我们通过基于集合转移的黑匣子攻击产生对抗性实例,这可以促进API的完全错误分类,并证明我们的方法可用于增加API的抗逆性鲁棒性。
translated by 谷歌翻译
Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS-and which illustrate a diverse set of defense strategies-can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result-showing that a defense was ineffective-this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. Some of our attack strategies are generalizable, but no single strategy would have been sufficient for all defenses. This underlines our key message that adaptive attacks cannot be automated and always require careful and appropriate tuning to a given defense. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.
translated by 谷歌翻译
许多最先进的ML模型在各种任务中具有优于图像分类的人类。具有如此出色的性能,ML模型今天被广泛使用。然而,存在对抗性攻击和数据中毒攻击的真正符合ML模型的稳健性。例如,Engstrom等人。证明了最先进的图像分类器可以容易地被任意图像上的小旋转欺骗。由于ML系统越来越纳入安全性和安全敏感的应用,对抗攻击和数据中毒攻击构成了相当大的威胁。本章侧重于ML安全的两个广泛和重要的领域:对抗攻击和数据中毒攻击。
translated by 谷歌翻译
深度学习的进步使得广泛的有希望的应用程序。然而,这些系统容易受到对抗机器学习(AML)攻击的影响;对他们的意见的离前事实制作的扰动可能导致他们错误分类。若干最先进的对抗性攻击已经证明他们可以可靠地欺骗分类器,使这些攻击成为一个重大威胁。对抗性攻击生成算法主要侧重于创建成功的例子,同时控制噪声幅度和分布,使检测更加困难。这些攻击的潜在假设是脱机产生的对抗噪声,使其执行时间是次要考虑因素。然而,最近,攻击者机会自由地产生对抗性示例的立即对抗攻击已经可能。本文介绍了一个新问题:我们如何在实时约束下产生对抗性噪音,以支持这种实时对抗攻击?了解这一问题提高了我们对这些攻击对实时系统构成的威胁的理解,并为未来防御提供安全评估基准。因此,我们首先进行对抗生成算法的运行时间分析。普遍攻击脱机产生一般攻击,没有在线开销,并且可以应用于任何输入;然而,由于其一般性,他们的成功率是有限的。相比之下,在特定输入上工作的在线算法是计算昂贵的,使它们不适合在时间约束下的操作。因此,我们提出房间,一种新型实时在线脱机攻击施工模型,其中离线组件用于预热在线算法,使得可以在时间限制下产生高度成功的攻击。
translated by 谷歌翻译
现有的研究表明,对抗性示例可以直接归因于具有高度预测性的非稳态特征的存在,但很容易被对手对愚弄NLP模型进行操纵。在这项研究中,我们探讨了捕获特定于任务的鲁棒特征的可行性,同时使用信息瓶颈理论消除了非舒适的特征。通过广泛的实验,我们表明,通过我们的信息基于瓶颈的方法训练的模型能够在稳健的精度上取得显着提高,超过了所有先前报道的防御方法的性能,而在SST-2上几乎没有遭受清洁准确性的表现下降,Agnews和IMDB数据集。
translated by 谷歌翻译
Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks that would trigger misclassification of DNNs but may be imperceptible to human perception. Adversarial defense has been important ways to improve the robustness of DNNs. Existing attack methods often construct adversarial examples relying on some metrics like the $\ell_p$ distance to perturb samples. However, these metrics can be insufficient to conduct adversarial attacks due to their limited perturbations. In this paper, we propose a new internal Wasserstein distance (IWD) to capture the semantic similarity of two samples, and thus it helps to obtain larger perturbations than currently used metrics such as the $\ell_p$ distance We then apply the internal Wasserstein distance to perform adversarial attack and defense. In particular, we develop a novel attack method relying on IWD to calculate the similarities between an image and its adversarial examples. In this way, we can generate diverse and semantically similar adversarial examples that are more difficult to defend by existing defense methods. Moreover, we devise a new defense method relying on IWD to learn robust models against unseen adversarial examples. We provide both thorough theoretical and empirical evidence to support our methods.
translated by 谷歌翻译
深度神经网络(DNN)容易受到对抗性示例的影响,其中DNN由于含有不可察觉的扰动而被误导为虚假输出。对抗性训练是一种可靠有效的防御方法,可能会大大减少神经网络的脆弱性,并成为强大学习的事实上的标准。尽管许多最近的作品实践了以数据为中心的理念,例如如何生成更好的对抗性示例或使用生成模型来产生额外的培训数据,但我们回顾了模型本身,并从深度特征分布的角度重新审视对抗性的鲁棒性有见地的互补性。在本文中,我们建议分支正交性对抗训练(BORT)获得最先进的性能,仅使用原始数据集用于对抗训练。为了练习我们整合多个正交解决方案空间的设计思想,我们利用一个简单明了的多分支神经网络,可消除对抗性攻击而不会增加推理时间。我们启发提出相应的损耗函数,分支 - 正交丢失,以使多支出模型正交的每个溶液空间。我们分别在CIFAR-10,CIFAR-100和SVHN上评估了我们的方法,分别针对\ ell _ {\ infty}的规范触发尺寸\ epsilon = 8/255。进行了详尽的实验,以表明我们的方法超出了所有最新方法,而无需任何技巧。与所有不使用其他数据进行培训的方法相比,我们的模型在CIFAR-10和CIFAR-100上实现了67.3%和41.5%的鲁棒精度(在最先进的ART上提高了 +7.23%和 +9.07% )。我们还使用比我们的训练组胜过比我们的方法的表现要大得多。我们所有的模型和代码均可在https://github.com/huangd1999/bort上在线获得。
translated by 谷歌翻译