Adversarial robustness assessment for video recognition models has raised concerns owing to their wide applications on safety-critical tasks. Compared with images, videos have much high dimension, which brings huge computational costs when generating adversarial videos. This is especially serious for the query-based black-box attacks where gradient estimation for the threat models is usually utilized, and high dimensions will lead to a large number of queries. To mitigate this issue, we propose to simultaneously eliminate the temporal and spatial redundancy within the video to achieve an effective and efficient gradient estimation on the reduced searching space, and thus query number could decrease. To implement this idea, we design the novel Adversarial spatial-temporal Focus (AstFocus) attack on videos, which performs attacks on the simultaneously focused key frames and key regions from the inter-frames and intra-frames in the video. AstFocus attack is based on the cooperative Multi-Agent Reinforcement Learning (MARL) framework. One agent is responsible for selecting key frames, and another agent is responsible for selecting key regions. These two agents are jointly trained by the common rewards received from the black-box threat models to perform a cooperative prediction. By continuously querying, the reduced searching space composed of key frames and key regions is becoming precise, and the whole query number becomes less than that on the original video. Extensive experiments on four mainstream video recognition models and three widely used action recognition datasets demonstrate that the proposed AstFocus attack outperforms the SOTA methods, which is prevenient in fooling rate, query number, time, and perturbation magnitude at the same.
translated by 谷歌翻译
Adversarial patch is an important form of real-world adversarial attack that brings serious risks to the robustness of deep neural networks. Previous methods generate adversarial patches by either optimizing their perturbation values while fixing the pasting position or manipulating the position while fixing the patch's content. This reveals that the positions and perturbations are both important to the adversarial attack. For that, in this paper, we propose a novel method to simultaneously optimize the position and perturbation for an adversarial patch, and thus obtain a high attack success rate in the black-box setting. Technically, we regard the patch's position, the pre-designed hyper-parameters to determine the patch's perturbations as the variables, and utilize the reinforcement learning framework to simultaneously solve for the optimal solution based on the rewards obtained from the target model with a small number of queries. Extensive experiments are conducted on the Face Recognition (FR) task, and results on four representative FR models show that our method can significantly improve the attack success rate and query efficiency. Besides, experiments on the commercial FR service and physical environments confirm its practical application value. We also extend our method to the traffic sign recognition task to verify its generalization ability.
translated by 谷歌翻译
最近的研究表明,深神经网络(DNN)易受对抗的对抗性斑块,这引入了对输入的可察觉而且局部化的变化。尽管如此,现有的方法都集中在图像上产生对抗性补丁,视频中的对应于视频的探索。与图像相比,攻击视频更具挑战性,因为它不仅需要考虑空间线索,而且需要考虑时间线索。为了缩短这种差距,我们在本文中介绍了一种新的对抗性攻击,子弹屏幕评论(BSC)攻击,攻击了BSC的视频识别模型。具体地,通过增强学习(RL)框架产生对抗性BSC,其中环境被设置为目标模型,并且代理商扮演选择每个BSC的位置和透明度的作用。通过不断查询目标模型和接收反馈,代理程序逐渐调整其选择策略,以实现具有非重叠BSC的高鬼速。由于BSC可以被视为一种有意义的补丁,将它添加到清洁视频不会影响人们对视频内容的理解,也不会引起人们的怀疑。我们进行广泛的实验,以验证所提出的方法的有效性。在UCF-101和HMDB-51数据集中,我们的BSC攻击方法可以在攻击三个主流视频识别模型时达到约90 \%的愚蠢速率,同时仅在视频中封闭\无文无线8 \%区域。我们的代码可在https://github.com/kay -ck/bsc-attack获得。
translated by 谷歌翻译
对抗性攻击可以迫使基于CNN的模型通过巧妙地操纵人类侵犯的输入来产生不正确的输出。探索这种扰动可以帮助我们更深入地了解神经网络的脆弱性,并为反对杂项对手提供深入学习的鲁棒性。尽管大量研究着重于图像,音频和NLP的鲁棒性,但仍缺乏视觉对象跟踪的对抗示例(尤其是以黑盒方式)的作品。在本文中,我们提出了一种新颖的对抗性攻击方法,以在黑色框设置下为单个对象跟踪产生噪音,其中仅在跟踪序列的初始框架上添加了扰动,从整个视频剪辑的角度来看,这很难注意到这一点。具体而言,我们将算法分为三个组件,并利用加固学习,以精确地定位重要的框架贴片,同时减少不必要的计算查询开销。与现有技术相比,我们的方法需要在视频的初始化框架上进行更少的查询,以操纵竞争性甚至更好的攻击性能。我们在长期和短期数据集中测试我们的算法,包括OTB100,DOCT2018,UAV123和LASOT。广泛的实验证明了我们方法对三种主流类型的跟踪器类型的有效性:歧视,基于暹罗和强化学习的跟踪器。
translated by 谷歌翻译
To assess the vulnerability of deep learning in the physical world, recent works introduce adversarial patches and apply them on different tasks. In this paper, we propose another kind of adversarial patch: the Meaningful Adversarial Sticker, a physically feasible and stealthy attack method by using real stickers existing in our life. Unlike the previous adversarial patches by designing perturbations, our method manipulates the sticker's pasting position and rotation angle on the objects to perform physical attacks. Because the position and rotation angle are less affected by the printing loss and color distortion, adversarial stickers can keep good attacking performance in the physical world. Besides, to make adversarial stickers more practical in real scenes, we conduct attacks in the black-box setting with the limited information rather than the white-box setting with all the details of threat models. To effectively solve for the sticker's parameters, we design the Region based Heuristic Differential Evolution Algorithm, which utilizes the new-found regional aggregation of effective solutions and the adaptive adjustment strategy of the evaluation criteria. Our method is comprehensively verified in the face recognition and then extended to the image retrieval and traffic sign recognition. Extensive experiments show the proposed method is effective and efficient in complex physical conditions and has a good generalization for different tasks.
translated by 谷歌翻译
Video classification systems are vulnerable to adversarial attacks, which can create severe security problems in video verification. Current black-box attacks need a large number of queries to succeed, resulting in high computational overhead in the process of attack. On the other hand, attacks with restricted perturbations are ineffective against defenses such as denoising or adversarial training. In this paper, we focus on unrestricted perturbations and propose StyleFool, a black-box video adversarial attack via style transfer to fool the video classification system. StyleFool first utilizes color theme proximity to select the best style image, which helps avoid unnatural details in the stylized videos. Meanwhile, the target class confidence is additionally considered in targeted attacks to influence the output distribution of the classifier by moving the stylized video closer to or even across the decision boundary. A gradient-free method is then employed to further optimize the adversarial perturbations. We carry out extensive experiments to evaluate StyleFool on two standard datasets, UCF-101 and HMDB-51. The experimental results demonstrate that StyleFool outperforms the state-of-the-art adversarial attacks in terms of both the number of queries and the robustness against existing defenses. Moreover, 50% of the stylized videos in untargeted attacks do not need any query since they can already fool the video classification model. Furthermore, we evaluate the indistinguishability through a user study to show that the adversarial samples of StyleFool look imperceptible to human eyes, despite unrestricted perturbations.
translated by 谷歌翻译
近年来,一项大量的研究努力集中在对抗图像上的对抗攻击,而对抗性视频攻击很少被探索。我们提出了对叫做Deepsava的竞争对手攻击战略。我们的模型包括通过统一优化框架的添加剂扰动和空间转换,其中采用结构相似性指数(SSIM)测量来测量对抗距离。我们设计一种有效和新的优化方案,可替代地利用贝叶斯优化来识别视频和随机梯度下降(SGD)优化中最有影响力的帧,以产生添加剂和空间变换的扰动。这样做使DeepSava能够对视频进行非常稀疏的攻击,以维持人类难以察觉,同时在攻击成功率和对抗转移性方面仍然实现最先进的性能。我们对各种类型的深神经网络和视频数据集的密集实验证实了Deepsava的优越性。
translated by 谷歌翻译
Reinforcement learning (RL) is one of the most important branches of AI. Due to its capacity for self-adaption and decision-making in dynamic environments, reinforcement learning has been widely applied in multiple areas, such as healthcare, data markets, autonomous driving, and robotics. However, some of these applications and systems have been shown to be vulnerable to security or privacy attacks, resulting in unreliable or unstable services. A large number of studies have focused on these security and privacy problems in reinforcement learning. However, few surveys have provided a systematic review and comparison of existing problems and state-of-the-art solutions to keep up with the pace of emerging threats. Accordingly, we herein present such a comprehensive review to explain and summarize the challenges associated with security and privacy in reinforcement learning from a new perspective, namely that of the Markov Decision Process (MDP). In this survey, we first introduce the key concepts related to this area. Next, we cover the security and privacy issues linked to the state, action, environment, and reward function of the MDP process, respectively. We further highlight the special characteristics of security and privacy methodologies related to reinforcement learning. Finally, we discuss the possible future research directions within this area.
translated by 谷歌翻译
积极调查深度神经网络的对抗鲁棒性。然而,大多数现有的防御方法限于特定类型的对抗扰动。具体而言,它们通常不能同时为多次攻击类型提供抵抗力,即,它们缺乏多扰动鲁棒性。此外,与图像识别问题相比,视频识别模型的对抗鲁棒性相对未开发。虽然有几项研究提出了如何产生对抗性视频,但在文献中只发表了关于防御策略的少数关于防御策略的方法。在本文中,我们提出了用于视频识别的多种抗逆视频的第一战略之一。所提出的方法称为Multibn,使用具有基于学习的BN选择模块的多个独立批量归一化(BN)层对多个对冲视频类型进行对抗性训练。利用多个BN结构,每个BN Brach负责学习单个扰动类型的分布,从而提供更精确的分布估计。这种机制有利于处理多种扰动类型。 BN选择模块检测输入视频的攻击类型,并将其发送到相应的BN分支,使MultiBN全自动并允许端接训练。与目前的对抗训练方法相比,所提出的Multibn对不同甚至不可预见的对抗性视频类型具有更强的多扰动稳健性,从LP界攻击和物理上可实现的攻击范围。在不同的数据集和目标模型上保持真实。此外,我们进行了广泛的分析,以研究多BN结构的性质。
translated by 谷歌翻译
为了跟踪视频中的目标,当前的视觉跟踪器通常采用贪婪搜索每个帧中目标对象定位,也就是说,将选择最大响应分数的候选区域作为每个帧的跟踪结果。但是,我们发现这可能不是一个最佳选择,尤其是在遇到挑战性的跟踪方案(例如重闭塞和快速运动)时。为了解决这个问题,我们建议维护多个跟踪轨迹并将光束搜索策略应用于视觉跟踪,以便可以识别出更少的累积错误的轨迹。因此,本文介绍了一种新型的基于梁搜索策略的新型多代理增强学习策略,称为横梁。它主要是受图像字幕任务的启发,该任务将图像作为输入,并使用Beam搜索算法生成多种描述。因此,我们通过多个并行决策过程来将跟踪提出作为样本选择问题,每个过程旨在将一个样本作为每个帧的跟踪结果选择。每个维护的轨迹都与代理商相关联,以执行决策并确定应采取哪些操作来更新相关信息。处理所有帧时,我们将最大累积分数作为跟踪结果选择轨迹。在七个流行的跟踪基准数据集上进行了广泛的实验证实了所提出的算法的有效性。
translated by 谷歌翻译
对抗性训练(AT)是针对对抗分类系统的对抗性攻击的简单而有效的防御,这是基于增强训练设置的攻击,从而最大程度地提高了损失。但是,AT作为视频分类的辩护的有效性尚未得到彻底研究。我们的第一个贡献是表明,为视频生成最佳攻击需要仔细调整攻击参数,尤其是步骤大小。值得注意的是,我们证明最佳步长随攻击预算线性变化。我们的第二个贡献是表明,在训练时间使用较小(次优的)攻击预算会导致测试时的性能更加强大。根据这些发现,我们提出了三个防御攻击预算的攻击的防御。自适应AT的第一个技术是一种技术,该技术是从随着训练迭代进行的。第二个课程是一项技术,随着训练的迭代进行,攻击预算的增加。第三个生成的AT,与deno的生成对抗网络一起,以提高稳健的性能。 UCF101数据集上的实验表明,所提出的方法改善了针对多种攻击类型的对抗性鲁棒性。
translated by 谷歌翻译
在本文中,我们提出了一个名为OcSampler的框架,以探索一个紧凑而有效的视频表示,其中一个短剪辑以获得高效的视频识别。最近的作品宁愿通过根据其重要性选择一个框架作为顺序决策任务的帧采样,而我们呈现了一个专用的学习实例的视频冷凝策略的新范式,以选择仅在单个视频中表示整个视频的信息帧步。我们的基本动机是高效的视频识别任务在于一次地处理整个序列而不是顺序拾取帧。因此,这些策略在一个步骤中与简单而有效的策略网络一起导出从光加权略微脱脂网络。此外,我们以帧编号预算扩展了所提出的方法,使框架能够以尽可能少的帧的高度置信度产生正确的预测。四个基准测试,即ActivityNet,Mini-Kinetics,FCVID,Mini-Sports1M的实验证明了我们在准确性,理论计算费用,实际推理速度方面对先前方法的效果。我们还在不同分类器,采样框架和搜索空间上评估其泛化电量。特别是,我们在ActivityNet上达到76.9%的地图和21.7 GFLOPS,具有令人印象深刻的吞吐量:123.9个视频/ s在单个Titan XP GPU上。
translated by 谷歌翻译
最近的研究表明,深层增强学习剂容易受到代理投入的小对抗扰动,这提出了对在现实世界中部署这些药剂的担忧。为了解决这个问题,我们提出了一个主要的框架,是培训加强学习代理的主要框架,以改善鲁棒性,以防止$ L_P $ -NORM偏见的对抗性攻击。我们的框架与流行的深度加强学习算法兼容,我们用深Q学习,A3C和PPO展示了其性能。我们在三个深度RL基准(Atari,Mujoco和Procgen)上进行实验,以展示我们稳健的培训算法的有效性。我们的径向-RL代理始终如一地占据了不同强度的攻击时的现有方法,并且培训更加计算效率。此外,我们提出了一种新的评估方法,称为贪婪最坏情况奖励(GWC)来衡量深度RL代理商的攻击不良鲁棒性。我们表明GWC可以有效地评估,并且对最糟糕的对抗攻击序列是对奖励的良好估计。用于我们实验的所有代码可在https://github.com/tuomaso/radial_rl_v2上获得。
translated by 谷歌翻译
Although deep neural networks (DNNs) have achieved great success in many tasks, they can often be fooled by adversarial examples that are generated by adding small but purposeful distortions to natural examples. Previous studies to defend against adversarial examples mostly focused on refining the DNN models, but have either shown limited success or required expensive computation. We propose a new strategy, feature squeezing, that can be used to harden DNN models by detecting adversarial examples. Feature squeezing reduces the search space available to an adversary by coalescing samples that correspond to many different feature vectors in the original space into a single sample. By comparing a DNN model's prediction on the original input with that on squeezed inputs, feature squeezing detects adversarial examples with high accuracy and few false positives.This paper explores two feature squeezing methods: reducing the color bit depth of each pixel and spatial smoothing. These simple strategies are inexpensive and complementary to other defenses, and can be combined in a joint detection framework to achieve high detection rates against state-of-the-art attacks.
translated by 谷歌翻译
自然语言视频本地化(NLVL)是视觉语言理解区域的重要任务,该方面还要求深入了解单独的计算机视觉和自然语言侧,但更重要的是两侧之间的相互作用。对抗性脆弱性得到了很好的认可,作为深度神经网络模型的关键安全问题,需要谨慎调查。尽管在视频和语言任务中进行了广泛但分开的研究,但目前对NLVL等愿景联合任务的对抗鲁棒性的理解较少。因此,本文旨在通过检查攻击和防御方面的三个脆弱性,全面调查NLVL模型的对抗性鲁棒性。为了实现攻击目标,我们提出了一种新的对抗攻击范式,称为同义句子感知对抗对抗攻击对逆向(潜行),这捕获了视觉和语言侧面之间的跨模式相互作用。
translated by 谷歌翻译
最近的工作表明,深增强学习(DRL)政策易受对抗扰动的影响。对手可以通过扰乱药剂观察到的环境来误导DRL代理商的政策。现有攻击原则上是可行的,但在实践中面临挑战,例如通过太慢,无法实时欺骗DRL政策。我们表明,使用通用的对冲扰动(UAP)方法来计算扰动,独立于应用它们的各个输入,可以有效地欺骗DRL策略。我们描述了三种这样的攻击变体。通过使用三个Atari 2600游戏的广泛评估,我们表明我们的攻击是有效的,因为它们完全降低了三种不同的DRL代理商的性能(高达100%,即使在扰乱的$ L_ infty $绑定时也很小为0.01)。与不同DRL策略的响应时间(平均0.6ms)相比,它比不同DRL策略的响应时间(0.6ms)更快,并且比使用对抗扰动的前攻击更快(平均1.8ms)。我们还表明,我们的攻击技术是高效的,平均地产生0.027ms的在线计算成本。使用涉及机器人运动的两个进一步任务,我们确认我们的结果概括了更复杂的DRL任务。此外,我们证明了已知防御的有效性降低了普遍扰动。我们提出了一种有效的技术,可检测针对DRL政策的所有已知的对抗性扰动,包括本文呈现的所有普遍扰动。
translated by 谷歌翻译
基于深度学习的面部识别模型容易受到对抗攻击的影响。为了遏制这些攻击,大多数防御方法旨在提高对抗性扰动的识别模型的鲁棒性。但是,这些方法的概括能力非常有限。实际上,它们仍然容易受到看不见的对抗攻击。深度学习模型对于一般的扰动(例如高斯噪音)相当强大。一种直接的方法是使对抗性扰动失活,以便可以轻松地将它们作为一般扰动处理。在本文中,提出了一种称为扰动失活(PIN)的插件对抗防御方法,以使对抗防御的对抗性扰动灭活。我们发现,不同子空间中的扰动对识别模型有不同的影响。应该有一个称为免疫空间的子空间,其中扰动对识别模型的不利影响要比其他子空间更少。因此,我们的方法估计了免疫空间,并通过将它们限制在此子空间中来使对抗性扰动失活。可以将所提出的方法推广到看不见的对抗扰动,因为它不依赖于特定类型的对抗攻击方法。这种方法不仅优于几种最先进的对抗防御方法,而且还通过详尽的实验证明了卓越的概括能力。此外,提出的方法可以成功地应用于四个商业API,而无需额外的培训,这表明可以轻松地将其推广到现有的面部识别系统。源代码可从https://github.com/renmin1991/perturbation in-inactivate获得
translated by 谷歌翻译
深度神经网络(DNN)已广泛采用健康风险预测,以提供医疗保健诊断和治疗。为了评估其稳健性,现有研究在型号参数可访问的白色/灰度箱设置中进行对抗性攻击。然而,即使大多数现实世界的型号训练私有数据并在云上作为黑匣子服务发布,也是更现实的黑盒对抗性攻击。为了填补这一差距,我们提出了针对Medattacker的健康风险预测模型的第一个黑匣子对抗攻击方法来调查他们的脆弱性。 MedAttacker通过两个步骤解决了EHR数据所带来的挑战:层次定位选择,它选择强化学习(RL)框架中的攻击位置并替换替代替代基于分数的原则。特别是,通过考虑EHR中的时间上下文,它通过使用每次访问的贡献分数和每个代码的显着分数来初始化其RL位置选择策略,这可以与决定性变化决定的确定性替代选择过程很好地集成。在实验中,Medattacker始终如一地实现了最高的平均成功率,并且在某些情况下攻击了在多次真实数据集中的黑匣子环境中的三个高级健康风险预测模型时,最近的白盒EHR对抗攻击技术甚至优于最近的白盒EHR对抗性攻击技术。此外,基于实验结果,我们包括讨论捍卫EHR对抗性攻击。
translated by 谷歌翻译
基于深的神经网络(DNNS)基于合成孔径雷达(SAR)自动靶标识别(ATR)系统已显示出非常容易受到故意设计但几乎无法察觉的对抗扰动的影响,但是当添加到靶向物体中时,DNN推断可能会偏差。在将DNN应用于高级SAR ATR应用时,这会导致严重的安全问题。因此,增强DNN的对抗性鲁棒性对于对现代现实世界中的SAR ATR系统实施DNN至关重要。本文旨在构建更健壮的DNN基于DNN的SAR ATR模型,探讨了SAR成像过程的领域知识,并提出了一种新型的散射模型引导的对抗攻击(SMGAA)算法,该算法可以以电磁散射响应的形式产生对抗性扰动(称为对抗散射器) )。提出的SMGAA由两个部分组成:1)参数散射模型和相应的成像方法以及2)基于自定义的基于梯度的优化算法。首先,我们介绍了有效的归因散射中心模型(ASCM)和一种通用成像方法,以描述SAR成像过程中典型几何结构的散射行为。通过进一步制定几种策略来考虑SAR目标图像的领域知识并放松贪婪的搜索程序,建议的方法不需要经过审慎的态度,但是可以有效地找到有效的ASCM参数来欺骗SAR分类器并促进SAR分类器并促进强大的模型训练。对MSTAR数据集的全面评估表明,SMGAA产生的对抗散射器对SAR处理链中的扰动和转换比当前研究的攻击更为强大,并且有效地构建了针对恶意散射器的防御模型。
translated by 谷歌翻译
机器学习模型严重易于来自对抗性示例的逃避攻击。通常,对逆势示例的修改输入类似于原始输入的修改输入,在WhiteBox设置下由对手的WhiteBox设置构成,完全访问模型。然而,最近的攻击已经显示出使用BlackBox攻击的对逆势示例的查询号显着减少。特别是,警报是从越来越多的机器学习提供的经过培训的模型的访问界面中利用分类决定作为包括Google,Microsoft,IBM的服务提供商,并由包含这些模型的多种应用程序使用的服务提供商来利用培训的模型。对手仅利用来自模型的预测标签的能力被区别为基于决策的攻击。在我们的研究中,我们首先深入潜入最近的ICLR和SP的最先进的决策攻击,以突出发现低失真对抗采用梯度估计方法的昂贵性质。我们开发了一种强大的查询高效攻击,能够避免在梯度估计方法中看到的嘈杂渐变中的局部最小和误导中的截留。我们提出的攻击方法,ramboattack利用随机块坐标下降的概念来探索隐藏的分类器歧管,针对扰动来操纵局部输入功能以解决梯度估计方法的问题。重要的是,ramboattack对对对手和目标类别可用的不同样本输入更加强大。总的来说,对于给定的目标类,ramboattack被证明在实现给定查询预算的较低失真时更加强大。我们使用大规模的高分辨率ImageNet数据集来策划我们的广泛结果,并在GitHub上开源我们的攻击,测试样本和伪影。
translated by 谷歌翻译