在本文中，我们通过合成自己的攻击数据集来提出基于零数据的重复比特翻转攻击（ZEBRA），精确地破坏了深度神经网络（DNN）。许多先前的对抗性重量攻击的作品不仅需要重量参数，而且需要在搜索易受攻击的比特中进行攻击的训练或测试数据集。我们建议通过利用受害者DNN模型中的批量归一化层统计来综合名为Dizeted目标数据的攻击数据集。配备蒸馏的目标数据，我们的Zebra算法可以在模型中搜索易受攻击的位，而无需访问培训或测试数据集。因此，我们的方法使对抗性重量攻击更致命的DNN的安全性。我们的实验结果表明，与先前的攻击方法相比，平均需要2.0倍（CiFar-10）和1.6倍（想象成）的比特翻转数量少。我们的代码可在https：// github上获得。COM / PDH930105 / ZEBRA。

神经网络权重的对抗性比特翻转攻击（BFA）可以通过翻转非常少量的比特来导致灾难性的精度下降。先前比特翻转攻击技术的主要缺点是他们对测试数据的依赖。包含敏感或专有数据的应用程序通常是不可能的。在本文中，我们提出了盲目数据侵犯比特 - 翻转攻击（BDFA），一种新颖的技术，使BFA能够无任何访问训练或测试数据。这是通过优化合成数据集来实现的，该数据集被设计为匹配跨网络的不同层和目标标签的批量标准化的统计数据。实验结果表明，BDFA可以显着降低75.96 \％至13.94 \％的resnet50的准确性，只有4位翻转。

Although deep neural networks (DNNs) have achieved great success in many tasks, they can often be fooled by adversarial examples that are generated by adding small but purposeful distortions to natural examples. Previous studies to defend against adversarial examples mostly focused on refining the DNN models, but have either shown limited success or required expensive computation. We propose a new strategy, feature squeezing, that can be used to harden DNN models by detecting adversarial examples. Feature squeezing reduces the search space available to an adversary by coalescing samples that correspond to many different feature vectors in the original space into a single sample. By comparing a DNN model's prediction on the original input with that on squeezed inputs, feature squeezing detects adversarial examples with high accuracy and few false positives.This paper explores two feature squeezing methods: reducing the color bit depth of each pixel and spatial smoothing. These simple strategies are inexpensive and complementary to other defenses, and can be combined in a joint detection framework to achieve high detection rates against state-of-the-art attacks.

我们提出了HashTAG，这是一种在检测性能上具有可证实范围的深度神经网络（DNN）对故障注射攻击的高精度检测的第一个框架。故障注射攻击中最近的文献显示了尺寸翻转引起的严重DNN精度劣化。在这种情况下，攻击者通过篡改程序的DRAM存储器来在DNN执行期间改变几个权重位。要检测运行时位翻转，HashTag在部署之前从良性DNN中提取唯一签名。签名后来用于验证DNN的完整性，并验证推动输出在速度。我们提出了一种新颖的敏感性分析方案，可准确地将最脆弱的DNN层识别到故障注射攻击。然后通过使用低碰撞散列函数对易受攻击层中的基础重量进行编码来构建DNN签名。部署DNN时，在推理期间从目标层提取新的哈希，并与地面真相签名进行比较。 HASHTAG采用了一种轻量级方法，可确保嵌入式平台上的低开销和实时故障检测。对各种DNN的最先进的位翻转攻击的广泛评估在攻击检测和执行开销方面，展示了HashTAG的竞争优势。

Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to adversarial examples: given an input x and any target classification t, it is possible to find a new input x that is similar to x but classified as t. This makes it difficult to apply neural networks in security-critical areas. Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from 95% to 0.5%.In this paper, we demonstrate that defensive distillation does not significantly increase the robustness of neural networks by introducing three new attack algorithms that are successful on both distilled and undistilled neural networks with 100% probability. Our attacks are tailored to three distance metrics used previously in the literature, and when compared to previous adversarial example generation algorithms, our attacks are often much more effective (and never worse). Furthermore, we propose using high-confidence adversarial examples in a simple transferability test we show can also be used to break defensive distillation. We hope our attacks will be used as a benchmark in future defense attempts to create neural networks that resist adversarial examples.

普遍的对策扰动是图像不可思议的和模型 - 无关的噪声，当添加到任何图像时可以误导训练的深卷积神经网络进入错误的预测。由于这些普遍的对抗性扰动可以严重危害实践深度学习应用的安全性和完整性，因此现有技术使用额外的神经网络来检测输入图像源的这些噪声的存在。在本文中，我们展示了一种攻击策略，即通过流氓手段激活（例如，恶意软件，木马）可以通过增强AI硬件加速器级的对抗噪声来绕过这些现有对策。我们使用Conv2D功能软件内核的共同仿真和FuseSoC环境下的硬件的Verilog RTL模型的共同仿真，展示了关于几个深度学习模型的加速度普遍对抗噪声。

深度神经网络模型大量部署在各种硬件平台上。这导致出现新的攻击向量，这些攻击向量大大扩展了标准攻击表面，这是由对抗机器学习社区广泛研究的。旨在通过瞄准存储在内存中的参数（权重）的第一个旨在极大地降低模型性能的攻击之一是位翼攻击（BFA）。在这项工作中，我们指出了与BFA相关的一些评估挑战。首先，标准威胁模型中缺乏对手的预算是有问题的，尤其是在处理身体攻击时。此外，由于BFA提出了关键的可变性，因此我们讨论了某些培训参数的影响以及模型体系结构的重要性。这项工作是第一个介绍BFA对与卷积神经网络相比呈现不同行为的完全连接体系结构的影响的作品。这些结果突出了定义鲁棒和合理评估方法的重要性，以正确评估基于参数的攻击的危险，并衡量国防提供的实际鲁棒性水平。

深度神经网络（DNN）的最新进步已经看到多个安全敏感域中的广泛部署。需要资源密集型培训和使用有价值的域特定培训数据，使这些模型成为模型所有者的顶级知识产权（IP）。 DNN隐私的主要威胁之一是模型提取攻击，前提是在DNN模型中试图窃取敏感信息。最近的研究表明，基于硬件的侧信道攻击可以揭示关于DNN模型的内部知识（例如，模型架构）但到目前为止，现有攻击不能提取详细的模型参数（例如，权重/偏置）。在这项工作中，我们首次提出了一种先进的模型提取攻击框架，借助记忆侧通道攻击有效地窃取了DNN权重。我们建议的深度包括两个关键阶段。首先，我们通过采用基于Rowhammer的硬件故障技术作为信息泄漏向量，开发一种名为HammerLeak的新重量位信息提取方法。 Hammerleak利用了用于DNN应用的几种新的系统级技术，以实现快速高效的重量窃取。其次，我们提出了一种具有平均聚类重量惩罚的新型替代模型训练算法，其利用部分泄漏的位信息有效地利用了目标受害者模型的替代原型。我们在三个流行的图像数据集（例如，CiFar-10/100 / GTSRB）和四个DNN架构上评估该替代模型提取方法（例如，Reset-18/34 / Wide-Reset / Vgg-11）。提取的替代模型在CiFar-10数据集的深度剩余网络上成功实现了超过90％的测试精度。此外，我们提取的替代模型也可能产生有效的对抗性输入样本来欺骗受害者模型。

基于新兴的非易失性记忆（NVM）设备基于内存的计算（CIM）体系结构，由于其高能量效率，具有深度神经网络（DNN）加速的巨大潜力。但是，NVM设备遭受了各种非理想性，尤其是由于设备的随机行为而导致的制造缺陷和周期到周期变化引起的设备对设备变化。因此，实际上映射到NVM设备的DNN权重可能显着偏离预期值，从而导致大量性能降解。为了解决这个问题，大多数现有的作品都集中在设备变化下的平均性能最大化。这个目标对于通用场景非常有效。但是对于关键安全应用，还必须考虑最差的案例性能。不幸的是，文献中很少探索这一点。在这项工作中，我们制定了确定在设备变化影响下CIM DNN加速器最差的问题的问题。我们进一步提出了一种方法，可以有效地找到高维空间中设备变化的特定组合，从而导致最差的性能。我们发现，即使设备变化很小，DNN的准确性也会大幅度下降，在部署CIM加速器中在安全至关重要的应用中引起担忧。最后，我们表明，令人惊讶的是，在扩展时，没有一种用于提高CIM加速器中平均DNN性能的现有方法非常有效，以增强最差的性能，并且需要进一步的研究来解决此问题。

When training early-stage deep neural networks (DNNs), generating intermediate features via convolution or linear layers occupied most of the execution time. Accordingly, extensive research has been done to reduce the computational burden of the convolution or linear layers. In recent mobile-friendly DNNs, however, the relative number of operations involved in processing these layers has significantly reduced. As a result, the proportion of the execution time of other layers, such as batch normalization layers, has increased. Thus, in this work, we conduct a detailed analysis of the batch normalization layer to efficiently reduce the runtime overhead in the batch normalization process. Backed up by the thorough analysis, we present an extremely efficient batch normalization, named LightNorm, and its associated hardware module. In more detail, we fuse three approximation techniques that are i) low bit-precision, ii) range batch normalization, and iii) block floating point. All these approximate techniques are carefully utilized not only to maintain the statistics of intermediate feature maps, but also to minimize the off-chip memory accesses. By using the proposed LightNorm hardware, we can achieve significant area and energy savings during the DNN training without hurting the training accuracy. This makes the proposed hardware a great candidate for the on-device training.

深度神经网络（DNN）的安全性因其在各种应用中的广泛使用而引起了人们的关注。最近，已被部署的DNN被证明容易受到特洛伊木马攻击的影响，该攻击操纵模型参数，以钻头翻转以注入隐藏的行为并通过特定的触发模式激活它。但是，所有现有的特洛伊木马攻击都采用了明显的基于补丁的触发器（例如，正方形模式），使其对人类可感知，并且很容易被机器发现。在本文中，我们提出了一种新颖的攻击，即几乎不可感知的特洛伊木马攻击（HPT）。 HPT通过利用添加噪声和每个像素流场来分别调整原始图像的像素值和位置，几乎无法察觉到可感知的特洛伊木马图像。为了实现卓越的攻击性能，我们建议共同优化位挡板，加性噪声和流场。由于DNN的重量位是二进制的，因此很难解决此问题。我们通过等效替换处理二进制约束，并提供有效的优化算法。关于CIFAR-10，SVHN和Imagenet数据集的广泛实验表明，所提出的HPT可以生成几乎不可感知的特洛伊木马图像，同时与先进的方法相比实现了可比或更好的攻击性能。该代码可在以下网址获得：https：//github.com/jiawangbai/hpt。

学习综合数据已成为零拍量化（ZSQ）的有希望的方向，其代表低位整数而不访问任何实际数据的神经网络。在本文中，我们在实际数据中观察到阶级内异质性的有趣现象，并表明现有方法未能在其合成图像中保留此属性，这导致有限的性能增加。要解决此问题，我们提出了一种新颖的零射量量化方法，称为IntraQ。首先，我们提出了一种局部对象加强件，该局部对象加强能够以不同的尺度和合成图像的位置定位目标对象。其次，我们引入了边缘距离约束，以形成分布在粗糙区域中的类相关的特征。最后，我们设计了一种软的成立损失，该损耗注射了软的先前标签，以防止合成图像过度接近固定物体。我们的intraQ被证明是在合成图像中提供阶级内的异质性，并且还观察到执行最先进的。例如，与高级ZSQ相比，当MobileNetv1的所有层被量化为4位时，我们的IntraIS获取9.17 \％增加了Imagenet上的前1个精度。代码是https://github.com/viperit/interq。

Adversarial examples that fool machine learning models, particularly deep neural networks, have been a topic of intense research interest, with attacks and defenses being developed in a tight back-and-forth. Most past defenses are best effort and have been shown to be vulnerable to sophisticated attacks. Recently a set of certified defenses have been introduced, which provide guarantees of robustness to normbounded attacks. However these defenses either do not scale to large datasets or are limited in the types of models they can support. This paper presents the first certified defense that both scales to large networks and datasets (such as Google's Inception network for ImageNet) and applies broadly to arbitrary model types. Our defense, called PixelDP, is based on a novel connection between robustness against adversarial examples and differential privacy, a cryptographically-inspired privacy formalism, that provides a rigorous, generic, and flexible foundation for defense.

With rapid progress and significant successes in a wide spectrum of applications, deep learning is being applied in many safety-critical environments. However, deep neural networks have been recently found vulnerable to well-designed input samples, called adversarial examples. Adversarial perturbations are imperceptible to human but can easily fool deep neural networks in the testing/deploying stage. The vulnerability to adversarial examples becomes one of the major risks for applying deep neural networks in safety-critical environments. Therefore, attacks and defenses on adversarial examples draw great attention. In this paper, we review recent findings on adversarial examples for deep neural networks, summarize the methods for generating adversarial examples, and propose a taxonomy of these methods. Under the taxonomy, applications for adversarial examples are investigated. We further elaborate on countermeasures for adversarial examples. In addition, three major challenges in adversarial examples and the potential solutions are discussed.

深度神经网络容易受到来自对抗性投入的攻击，并且最近，特洛伊木马误解或劫持模型的决定。我们通过探索有界抗逆性示例空间和生成的对抗网络内的自然输入空间来揭示有界面的对抗性实例 - 通用自然主义侵害贴片的兴趣类 - 我们呼叫TNT。现在，一个对手可以用一个自然主义的补丁来手臂自己，不太恶意，身体上可实现，高效 - 实现高攻击成功率和普遍性。 TNT是普遍的，因为在场景中的TNT中捕获的任何输入图像都将：i）误导网络（未确定的攻击）;或ii）迫使网络进行恶意决定（有针对性的攻击）。现在，有趣的是，一个对抗性补丁攻击者有可能发挥更大的控制水平 - 选择一个独立，自然的贴片的能力，与被限制为嘈杂的扰动的触发器 - 到目前为止只有可能与特洛伊木马攻击方法有可能干扰模型建设过程，以嵌入风险发现的后门;但是，仍然意识到在物理世界中部署的补丁。通过对大型视觉分类任务的广泛实验，想象成在其整个验证集50,000张图像中进行评估，我们展示了TNT的现实威胁和攻击的稳健性。我们展示了攻击的概括，以创建比现有最先进的方法实现更高攻击成功率的补丁。我们的结果表明，攻击对不同的视觉分类任务（CIFAR-10，GTSRB，PUBFIG）和多个最先进的深神经网络，如WieredEnet50，Inception-V3和VGG-16。

We introduce camouflaged data poisoning attacks, a new attack vector that arises in the context of machine unlearning and other settings when model retraining may be induced. An adversary first adds a few carefully crafted points to the training dataset such that the impact on the model's predictions is minimal. The adversary subsequently triggers a request to remove a subset of the introduced points at which point the attack is unleashed and the model's predictions are negatively affected. In particular, we consider clean-label targeted attacks (in which the goal is to cause the model to misclassify a specific test point) on datasets including CIFAR-10, Imagenette, and Imagewoof. This attack is realized by constructing camouflage datapoints that mask the effect of a poisoned dataset.

Although weight and activation quantization is an effective approach for Deep Neural Network (DNN) compression and has a lot of potentials to increase inference speed leveraging bit-operations, there is still a noticeable gap in terms of prediction accuracy between the quantized model and the full-precision model. To address this gap, we propose to jointly train a quantized, bit-operation-compatible DNN and its associated quantizers, as opposed to using fixed, handcrafted quantization schemes such as uniform or logarithmic quantization. Our method for learning the quantizers applies to both network weights and activations with arbitrary-bit precision, and our quantizers are easy to train. The comprehensive experiments on CIFAR-10 and ImageNet datasets show that our method works consistently well for various network structures such as AlexNet, VGG-Net, GoogLeNet, ResNet, and DenseNet, surpassing previous quantization methods in terms of accuracy by an appreciable margin. Code available at https://github.com/Microsoft/LQ-Nets

量化已成为压缩和加速神经网络最普遍的方法之一。最近，无数据量化已被广泛研究作为实用和有前途的解决方案。它根据FP32批量归一化（BN）统计，合成校准量化模型的数据，并显着降低了传统量化方法中实际训练数据的沉重依赖性。不幸的是，我们发现在实践中，BN统计的合成数据在分配水平和样品水平上具有严重均匀化，并且进一步引起量化模型的显着性能下降。我们提出了各种样品生成（DSG）方案，以减轻均质化引起的不利影响。具体而言，我们松弛BN层中的特征统计的对准，以在分配水平处放宽约束，并设计一个层状增强，以加强针对不同的数据样本的特定层。我们的DSG方案是多功能的，甚至能够应用于现代训练后的训练后的量化方法，如亚马逊。我们评估大规模图像分类任务的DSG方案，并始终如一地获得各种网络架构和量化方法的显着改进，特别是当量化到较低位时（例如，在W4A4上的高达22％）。此外，从增强的多样性受益，综合数据校准的模型均接近通过实际数据校准的那些，甚至在W4A4上越优于它们。

量化是一种流行的技术，即$将神经网络的参数表示从浮点数转换为低精度（$ e.g. $，8位整数）。它会降低记忆占用和计算成本，推断，促进了资源饥饿的模型的部署。但是，在量化之前和之后，该转换引起的参数扰动导致模型之间的$行为$ $差异$。例如，量化模型可以错误分类正确分类的测试时间样本。尚不清楚这些差异是否导致新的安全漏洞。我们假设对手可以控制这种差异以引入在量化时激活的具体行为。为研究这一假设，我们武装量化感知培训并提出了一种新的培训框架来实施对抗性量化结果。在此框架之后，我们展示了三次攻击我们通过量化进行：（i）对显着的精度损失的不分青红皂白攻击; （ii）针对特定样本的目标攻击; （iii）使用输入触发来控制模型的后门攻击。我们进一步表明，单个受损模型击败多种量化方案，包括鲁棒量化技术。此外，在联合学习情景中，我们证明了一系列伴侣可以注入我们量化激活的后门的恶意参与者。最后，我们讨论了潜在的反措施，并表明只有重新训练始终如一地删除攻击伪影。我们的代码可以在https://github.com/secure-ai-systems-group/qu-antigization获得

Deep neural networks (DNNs) are one of the most prominent technologies of our time, as they achieve state-of-the-art performance in many machine learning tasks, including but not limited to image classification, text mining, and speech processing. However, recent research on DNNs has indicated ever-increasing concern on the robustness to adversarial examples, especially for security-critical tasks such as traffic sign identification for autonomous driving. Studies have unveiled the vulnerability of a well-trained DNN by demonstrating the ability of generating barely noticeable (to both human and machines) adversarial images that lead to misclassification. Furthermore, researchers have shown that these adversarial images are highly transferable by simply training and attacking a substitute model built upon the target model, known as a black-box attack to DNNs.Similar to the setting of training substitute models, in this paper we propose an effective black-box attack that also only has access to the input (images) and the output (confidence scores) of a targeted DNN. However, different from leveraging attack transferability from substitute models, we propose zeroth order optimization (ZOO) based attacks to directly estimate the gradients of the targeted DNN for generating adversarial examples. We use zeroth order stochastic coordinate descent along with dimension reduction, hierarchical attack and importance sampling techniques to * Pin-Yu Chen and Huan Zhang contribute equally to this work.