后门攻击在训练期间注入中毒样本，目的是迫使机器学习模型在测试时间呈现特定触发时输出攻击者所选的类。虽然在各种环境中展示了后门攻击和针对不同的模型，但影响其有效性的因素仍然不太了解。在这项工作中，我们提供了一个统一的框架，以研究增量学习和影响功能的镜头下的后门学习过程。我们表明，后门攻击的有效性取决于：（i）由普通参数控制的学习算法的复杂性; （ii）注入训练集的后门样品的一部分; （iii）后门触发的大小和可见性。这些因素会影响模型学会与目标类别相关联的速度触发器的存在的速度。我们的分析推出了封路计空间中的区域的有趣存在，其中清洁试验样品的准确性仍然很高，而后门攻击无效，从而提示改善现有防御的新标准。

计算能力和大型培训数据集的可用性增加，机器学习的成功助长了。假设它充分代表了在测试时遇到的数据，则使用培训数据来学习新模型或更新现有模型。这种假设受到中毒威胁的挑战，这种攻击会操纵训练数据，以损害模型在测试时的表现。尽管中毒已被认为是行业应用中的相关威胁，到目前为止，已经提出了各种不同的攻击和防御措施，但对该领域的完整系统化和批判性审查仍然缺失。在这项调查中，我们在机器学习中提供了中毒攻击和防御措施的全面系统化，审查了过去15年中该领域发表的100多篇论文。我们首先对当前的威胁模型和攻击进行分类，然后相应地组织现有防御。虽然我们主要关注计算机视觉应用程序，但我们认为我们的系统化还包括其他数据模式的最新攻击和防御。最后，我们讨论了中毒研究的现有资源，并阐明了当前的局限性和该研究领域的开放研究问题。

后门攻击误导机器学习模型以在测试时间呈现特定触发时输出攻击者指定的类。这些攻击需要毒害训练数据来损害学习算法，例如，通过将包含触发器的中毒样本注入训练集中，以及所需的类标签。尽管对后门攻击和防御的研究数量越来越多，但影响后门攻击成功的潜在因素以及它们对学习算法的影响尚未得到很好的理解。在这项工作中，我们的目标是通过揭幕揭示触发样本周围的更光滑的决策功能来阐明这一问题 - 这是我们称之为\ Textit {后门平滑}的现象。为了量化后门平滑，我们定义了一种评估与输入样本周围分类器的预测相关的不确定性的度量。我们的实验表明，当触发器添加到输入样本时，平滑度会增加，并且这种现象更加明显，以获得更成功的攻击。我们还提供了初步证据，后者触发器不是唯一的平滑诱导模式，而是可以通过我们的方法来检测其他人工图案，铺平了解当前防御和设计新颖的局限性。

后门是针对深神经网络（DNN）的强大攻击。通过中毒训练数据，攻击者可以将隐藏的规则（后门）注入DNN，该规则仅在包含攻击特异性触发器的输入上激活。尽管现有工作已经研究了各种DNN模型的后门攻击，但它们仅考虑静态模型，这些模型在初始部署后保持不变。在本文中，我们研究了后门攻击对时变DNN模型更现实的情况的影响，其中定期更新模型权重以处理数据分布的漂移。具体而言，我们从经验上量化了后门针对模型更新的“生存能力”，并检查攻击参数，数据漂移行为和模型更新策略如何影响后门生存能力。我们的结果表明，即使攻击者会积极增加触发器的大小和毒药比，即使在几个模型更新中，一次射击后门攻击（即一次仅中毒训练数据）也无法幸免。为了保持模型更新影响，攻击者必须不断将损坏的数据引入培训管道。这些结果共同表明，当模型更新以学习新数据时，它们也将后门“忘记”为隐藏的恶意功能。旧培训数据之间的分配变化越大，后门被遗忘了。利用这些见解，我们应用了智能学习率调度程序，以进一步加速模型更新期间的后门遗忘，这阻止了单发后门在单个模型更新中幸存下来。

与令人印象深刻的进步触动了我们社会的各个方面，基于深度神经网络（DNN）的AI技术正在带来越来越多的安全问题。虽然在考试时间运行的攻击垄断了研究人员的初始关注，但是通过干扰培训过程来利用破坏DNN模型的可能性，代表了破坏训练过程的可能性，这是破坏AI技术的可靠性的进一步严重威胁。在后门攻击中，攻击者损坏了培训数据，以便在测试时间诱导错误的行为。然而，测试时间误差仅在存在与正确制作的输入样本对应的触发事件的情况下被激活。通过这种方式，损坏的网络继续正常输入的预期工作，并且只有当攻击者决定激活网络内隐藏的后门时，才会发生恶意行为。在过去几年中，后门攻击一直是强烈的研究活动的主题，重点是新的攻击阶段的发展，以及可能对策的提议。此概述文件的目标是审查发表的作品，直到现在，分类到目前为止提出的不同类型的攻击和防御。指导分析的分类基于攻击者对培训过程的控制量，以及防御者验证用于培训的数据的完整性，并监控DNN在培训和测试中的操作时间。因此，拟议的分析特别适合于参考他们在运营的应用方案的攻击和防御的强度和弱点。

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

With the success of deep learning algorithms in various domains, studying adversarial attacks to secure deep models in real world applications has become an important research topic. Backdoor attacks are a form of adversarial attacks on deep networks where the attacker provides poisoned data to the victim to train the model with, and then activates the attack by showing a specific small trigger pattern at the test time. Most state-of-the-art backdoor attacks either provide mislabeled poisoning data that is possible to identify by visual inspection, reveal the trigger in the poisoned data, or use noise to hide the trigger. We propose a novel form of backdoor attack where poisoned data look natural with correct labels and also more importantly, the attacker hides the trigger in the poisoned data and keeps the trigger secret until the test time.We perform an extensive study on various image classification settings and show that our attack can fool the model by pasting the trigger at random locations on unseen images although the model performs well on clean data. We also show that our proposed attack cannot be easily defended using a state-of-the-art defense algorithm for backdoor attacks.

有针对性的训练集攻击将恶意实例注入训练集中，以导致训练有素的模型错误地标记一个或多个特定的测试实例。这项工作提出了目标识别的任务，该任务决定了特定的测试实例是否是训练集攻击的目标。目标识别可以与对抗性识别相结合，以查找（并删除）攻击实例，从而减轻对其他预测的影响，从而减轻攻击。我们没有专注于单个攻击方法或数据模式，而是基于影响力估计，这量化了每个培训实例对模型预测的贡献。我们表明，现有的影响估计量的不良实际表现通常来自于他们对训练实例和迭代次数的过度依赖。我们重新归一化的影响估计器解决了这一弱点。他们的表现远远超过了原始估计量，可以在对抗和非对抗环境中识别有影响力的训练示例群体，甚至发现多达100％的对抗训练实例，没有清洁数据误报。然后，目标识别简化以检测具有异常影响值的测试实例。我们证明了我们的方法对各种数据域的后门和中毒攻击的有效性，包括文本，视觉和语音，以及针对灰色盒子的自适应攻击者，该攻击者专门优化了逃避我们方法的对抗性实例。我们的源代码可在https://github.com/zaydh/target_indistification中找到。

后门攻击已被证明是对深度学习模型的严重安全威胁，并且检测给定模型是否已成为后门成为至关重要的任务。现有的防御措施主要建立在观察到后门触发器通常尺寸很小或仅影响几个神经元激活的观察结果。但是，在许多情况下，尤其是对于高级后门攻击，违反了上述观察结果，阻碍了现有防御的性能和适用性。在本文中，我们提出了基于新观察的后门防御范围。也就是说，有效的后门攻击通常需要对中毒训练样本的高预测置信度，以确保训练有素的模型具有很高的可能性。基于此观察结果，Dtinspector首先学习一个可以改变最高信心数据的预测的补丁，然后通过检查在低信心数据上应用学习补丁后检查预测变化的比率来决定后门的存在。对五次后门攻击，四个数据集和三种高级攻击类型的广泛评估证明了拟议防御的有效性。

视觉变压器（VIT）最近在各种视觉任务上表现出了典范的性能，并被用作CNN的替代方案。它们的设计基于一种自我发挥的机制，该机制将图像作为一系列斑块进行处理，与CNN相比，这是完全不同的。因此，研究VIT是否容易受到后门攻击的影响很有趣。当攻击者出于恶意目的，攻击者毒害培训数据的一小部分时，就会发生后门攻击。模型性能在干净的测试图像上很好，但是攻击者可以通过在测试时间显示触发器来操纵模型的决策。据我们所知，我们是第一个证明VIT容易受到后门攻击的人。我们还发现VIT和CNNS之间存在着有趣的差异 - 解释算法有效地突出了VIT的测试图像的触发因素，但没有针对CNN。基于此观察结果，我们提出了一个测试时间图像阻止VIT的防御，这将攻击成功率降低了很大。代码可在此处找到：https：//github.com/ucdvision/backdoor_transformer.git

Data poisoning is an attack on machine learning models wherein the attacker adds examples to the training set to manipulate the behavior of the model at test time. This paper explores poisoning attacks on neural nets. The proposed attacks use "clean-labels"; they don't require the attacker to have any control over the labeling of training data. They are also targeted; they control the behavior of the classifier on a specific test instance without degrading overall classifier performance. For example, an attacker could add a seemingly innocuous image (that is properly labeled) to a training set for a face recognition engine, and control the identity of a chosen person at test time. Because the attacker does not need to control the labeling function, poisons could be entered into the training set simply by leaving them on the web and waiting for them to be scraped by a data collection bot. We present an optimization-based method for crafting poisons, and show that just one single poison image can control classifier behavior when transfer learning is used. For full end-to-end training, we present a "watermarking" strategy that makes poisoning reliable using multiple (≈ 50) poisoned training instances. We demonstrate our method by generating poisoned frog images from the CIFAR dataset and using them to manipulate image classifiers.

在对抗机器学习中，防止对深度学习系统的攻击的新防御能力在释放更强大的攻击后不久就会破坏。在这种情况下，法医工具可以通过追溯成功的根本原因来为现有防御措施提供宝贵的补充，并为缓解措施提供前进的途径，以防止将来采取类似的攻击。在本文中，我们描述了我们为开发用于深度神经网络毒物攻击的法医追溯工具的努力。我们提出了一种新型的迭代聚类和修剪解决方案，该解决方案修剪了“无辜”训练样本，直到所有剩余的是一组造成攻击的中毒数据。我们的方法群群训练样本基于它们对模型参数的影响，然后使用有效的数据解读方法来修剪无辜簇。我们从经验上证明了系统对三种类型的肮脏标签（后门）毒物攻击和三种类型的清洁标签毒药攻击的功效，这些毒物跨越了计算机视觉和恶意软件分类。我们的系统在所有攻击中都达到了98.4％的精度和96.8％的召回。我们还表明，我们的系统与专门攻击它的四种抗纤维法措施相对强大。

Learning-based pattern classifiers, including deep networks, have shown impressive performance in several application domains, ranging from computer vision to cybersecurity. However, it has also been shown that adversarial input perturbations carefully crafted either at training or at test time can easily subvert their predictions. The vulnerability of machine learning to such wild patterns (also referred to as adversarial examples), along with the design of suitable countermeasures, have been investigated in the research field of adversarial machine learning. In this work, we provide a thorough overview of the evolution of this research area over the last ten years and beyond, starting from pioneering, earlier work on the security of non-deep learning algorithms up to more recent work aimed to understand the security properties of deep learning algorithms, in the context of computer vision and cybersecurity tasks. We report interesting connections between these apparently-different lines of work, highlighting common misconceptions related to the security evaluation of machine-learning algorithms. We review the main threat models and attacks defined to this end, and discuss the main limitations of current work, along with the corresponding future challenges towards the design of more secure learning algorithms.

最近的研究表明，深神经网络（DNN）易受对抗性攻击的影响，包括逃避和后门（中毒）攻击。在防守方面，有密集的努力，改善了对逃避袭击的经验和可怜的稳健性;然而，对后门攻击的可稳健性仍然很大程度上是未开发的。在本文中，我们专注于认证机器学习模型稳健性，反对一般威胁模型，尤其是后门攻击。我们首先通过随机平滑技术提供统一的框架，并展示如何实例化以证明对逃避和后门攻击的鲁棒性。然后，我们提出了第一个强大的培训过程Rab，以平滑训练有素的模型，并证明其稳健性对抗后门攻击。我们派生机学习模型的稳健性突出了培训的机器学习模型，并证明我们的鲁棒性受到紧张。此外，我们表明，可以有效地训练强大的平滑模型，以适用于诸如k最近邻分类器的简单模型，并提出了一种精确的平滑训练算法，该算法消除了从这种模型的噪声分布采样采样的需要。经验上，我们对MNIST，CIFAR-10和Imagenet数据集等DNN，差异私有DNN和K-NN模型等不同机器学习（ML）型号进行了全面的实验，并为反卧系攻击提供认证稳健性的第一个基准。此外，我们在SPAMBase表格数据集上评估K-NN模型，以展示所提出的精确算法的优点。对多元化模型和数据集的综合评价既有关于普通训练时间攻击的进一步强劲学习策略的多样化模型和数据集的综合评价。

Backdoor attacks have emerged as one of the major security threats to deep learning models as they can easily control the model's test-time predictions by pre-injecting a backdoor trigger into the model at training time. While backdoor attacks have been extensively studied on images, few works have investigated the threat of backdoor attacks on time series data. To fill this gap, in this paper we present a novel generative approach for time series backdoor attacks against deep learning based time series classifiers. Backdoor attacks have two main goals: high stealthiness and high attack success rate. We find that, compared to images, it can be more challenging to achieve the two goals on time series. This is because time series have fewer input dimensions and lower degrees of freedom, making it hard to achieve a high attack success rate without compromising stealthiness. Our generative approach addresses this challenge by generating trigger patterns that are as realistic as real-time series patterns while achieving a high attack success rate without causing a significant drop in clean accuracy. We also show that our proposed attack is resistant to potential backdoor defenses. Furthermore, we propose a novel universal generator that can poison any type of time series with a single generator that allows universal attacks without the need to fine-tune the generative model for new time series datasets.

A recent trojan attack on deep neural network (DNN) models is one insidious variant of data poisoning attacks. Trojan attacks exploit an effective backdoor created in a DNN model by leveraging the difficulty in interpretability of the learned model to misclassify any inputs signed with the attacker's chosen trojan trigger. Since the trojan trigger is a secret guarded and exploited by the attacker, detecting such trojan inputs is a challenge, especially at run-time when models are in active operation. This work builds STRong Intentional Perturbation (STRIP) based run-time trojan attack detection system and focuses on vision system. We intentionally perturb the incoming input, for instance by superimposing various image patterns, and observe the randomness of predicted classes for perturbed inputs from a given deployed model-malicious or benign. A low entropy in predicted classes violates the input-dependence property of a benign model and implies the presence of a malicious input-a characteristic of a trojaned input. The high efficacy of our method is validated through case studies on three popular and contrasting datasets: MNIST, CIFAR10 and GTSRB. We achieve an overall false acceptance rate (FAR) of less than 1%, given a preset false rejection rate (FRR) of 1%, for different types of triggers. Using CIFAR10 and GTSRB, we have empirically achieved result of 0% for both FRR and FAR. We have also evaluated STRIP robustness against a number of trojan attack variants and adaptive attacks.

Semi-supervised learning methods can train high-accuracy machine learning models with a fraction of the labeled training samples required for traditional supervised learning. Such methods do not typically involve close review of the unlabeled training samples, making them tempting targets for data poisoning attacks. In this paper we investigate the vulnerabilities of semi-supervised learning methods to backdoor data poisoning attacks on the unlabeled samples. We show that simple poisoning attacks that influence the distribution of the poisoned samples' predicted labels are highly effective - achieving an average attack success rate as high as 96.9%. We introduce a generalized attack framework targeting semi-supervised learning methods to better understand and exploit their limitations and to motivate future defense strategies.

Backdoor attacks represent one of the major threats to machine learning models. Various efforts have been made to mitigate backdoors. However, existing defenses have become increasingly complex and often require high computational resources or may also jeopardize models' utility. In this work, we show that fine-tuning, one of the most common and easy-to-adopt machine learning training operations, can effectively remove backdoors from machine learning models while maintaining high model utility. Extensive experiments over three machine learning paradigms show that fine-tuning and our newly proposed super-fine-tuning achieve strong defense performance. Furthermore, we coin a new term, namely backdoor sequela, to measure the changes in model vulnerabilities to other attacks before and after the backdoor has been removed. Empirical evaluation shows that, compared to other defense methods, super-fine-tuning leaves limited backdoor sequela. We hope our results can help machine learning model owners better protect their models from backdoor threats. Also, it calls for the design of more advanced attacks in order to comprehensively assess machine learning models' backdoor vulnerabilities.

从外界培训的机器学习模型可能会被数据中毒攻击损坏，将恶意指向到模型的培训集中。对这些攻击的常见防御是数据消毒：在培训模型之前首先过滤出异常培训点。在本文中，我们开发了三次攻击，可以绕过广泛的常见数据消毒防御，包括基于最近邻居，训练损失和奇异值分解的异常探测器。通过增加3％的中毒数据，我们的攻击成功地将Enron垃圾邮件检测数据集的测试错误从3％增加到24％，并且IMDB情绪分类数据集从12％到29％。相比之下，没有明确占据这些数据消毒防御的现有攻击被他们击败。我们的攻击基于两个想法：（i）我们协调我们的攻击将中毒点彼此放置在彼此附近，（ii）我们将每个攻击制定为受限制的优化问题，限制旨在确保中毒点逃避检测。随着这种优化涉及解决昂贵的Bilevel问题，我们的三个攻击对应于基于影响功能的近似近似这个问题的方式; minimax二元性;和karush-kuhn-tucker（kkt）条件。我们的结果强调了对数据中毒攻击产生更强大的防御的必要性。

后门攻击已成为深度神经网络（DNN）的主要安全威胁。虽然现有的防御方法在检测或擦除后以后展示了有希望的结果，但仍然尚不清楚是否可以设计强大的培训方法，以防止后门触发器首先注入训练的模型。在本文中，我们介绍了\ emph {反后门学习}的概念，旨在培训\ emph {Clean}模型给出了后门中毒数据。我们将整体学习过程框架作为学习\ emph {clean}和\ emph {backdoor}部分的双重任务。从这种观点来看，我们确定了两个后门攻击的固有特征，因为他们的弱点2）后门任务与特定类（后门目标类）相关联。根据这两个弱点，我们提出了一般学习计划，反后门学习（ABL），在培训期间自动防止后门攻击。 ABL引入了标准培训的两级\ EMPH {梯度上升}机制，帮助分离早期训练阶段的后台示例，2）在后续训练阶段中断后门示例和目标类之间的相关性。通过对多个基准数据集的广泛实验，针对10个最先进的攻击，我们经验证明，后卫中毒数据上的ABL培训模型实现了与纯净清洁数据训练的相同性能。代码可用于\ url {https:/github.com/boylyg/abl}。