深度学习技术在图像压缩中显示出令人鼓舞的结果，并具有竞争性的比特率和图像重建质量。但是，尽管图像压缩已经朝着更高的峰值信噪比（PSNR）和每个像素（BPP）较少的位置发展，但它们对对抗图像的稳健性从未经过审议。在这项工作中，我们首次研究了图像压缩系统的鲁棒性，其中不可察觉的输入图像的扰动会导致其压缩潜在的比特率显着增加。为了表征最先进的图像压缩的鲁棒性，我们安装了白色框和黑框攻击。我们的白框攻击在比特斯流的熵估计中采用快速梯度标志方法作为比特率近似。我们提出了DCT-NET，以建筑简单性和轻量级训练为Black-Box攻击中的替代品，并实现快速的对抗性转移性，以模拟JPEG压缩。我们在六个图像压缩模型上的结果，每个模型具有六个不同的比特率质量（总共36个模型），表明它们令人惊讶地脆弱，其中白盒攻击可达到56.326X和Black-Box 1.947X BPP的变化。为了提高鲁棒性，我们提出了一种新型的压缩体系结构ractatn，它结合了注意模块和一个基本分解的熵模型，从而在对抗性攻击方面的速率延伸性能与鲁棒性之间的有希望的权衡，超过了现有的学术图像压缩机。

Video compression plays a crucial role in video streaming and classification systems by maximizing the end-user quality of experience (QoE) at a given bandwidth budget. In this paper, we conduct the first systematic study for adversarial attacks on deep learning-based video compression and downstream classification systems. Our attack framework, dubbed RoVISQ, manipulates the Rate-Distortion ($\textit{R}$-$\textit{D}$) relationship of a video compression model to achieve one or both of the following goals: (1) increasing the network bandwidth, (2) degrading the video quality for end-users. We further devise new objectives for targeted and untargeted attacks to a downstream video classification service. Finally, we design an input-invariant perturbation that universally disrupts video compression and classification systems in real time. Unlike previously proposed attacks on video classification, our adversarial perturbations are the first to withstand compression. We empirically show the resilience of RoVISQ attacks against various defenses, i.e., adversarial training, video denoising, and JPEG compression. Our extensive experimental results on various video datasets show RoVISQ attacks deteriorate peak signal-to-noise ratio by up to 5.6dB and the bit-rate by up to $\sim$ 2.4$\times$ while achieving over 90$\%$ attack success rate on a downstream classifier. Our user study further demonstrates the effect of RoVISQ attacks on users' QoE.

Image compression is a fundamental research field and many well-known compression standards have been developed for many decades. Recently, learned compression methods exhibit a fast development trend with promising results. However, there is still a performance gap between learned compression algorithms and reigning compression standards, especially in terms of widely used PSNR metric. In this paper, we explore the remaining redundancy of recent learned compression algorithms. We have found accurate entropy models for rate estimation largely affect the optimization of network parameters and thus affect the rate-distortion performance. Therefore, in this paper, we propose to use discretized Gaussian Mixture Likelihoods to parameterize the distributions of latent codes, which can achieve a more accurate and flexible entropy model. Besides, we take advantage of recent attention modules and incorporate them into network architecture to enhance the performance. Experimental results demonstrate our proposed method achieves a state-of-the-art performance compared to existing learned compression methods on both Kodak and high-resolution datasets. To our knowledge our approach is the first work to achieve comparable performance with latest compression standard Versatile Video Coding (VVC) regarding PSNR. More importantly, our approach generates more visually pleasant results when optimized by MS-SSIM. The project page is at https://github.com/ZhengxueCheng/ Learned-Image-Compression-with-GMM-and-Attention.

基于神经网络的图像压缩已经过度研究。模型稳健性很大程度上被忽视，但它对服务能够实现至关重要。我们通过向原始源图像注入少量噪声扰动来执行对抗攻击，然后使用主要学习的图像压缩模型来编码这些对抗示例。实验报告对逆势实例的重建中的严重扭曲，揭示了现有方法的一般漏洞，无论用于底层压缩模型（例如，网络架构，丢失功能，质量标准）和用于注射扰动的优化策略（例如，噪声阈值，信号距离测量）。后来，我们应用迭代对抗的FineTuning来细化掠夺模型。在每次迭代中，将随机源图像和对抗示例混合以更新底层模型。结果通过大大提高压缩模型稳健性来表明提出的FineTuning策略的有效性。总体而言，我们的方法是简单，有效和更广泛的，使其具有开发稳健的学习图像压缩解决方案的吸引力。所有材料都在HTTPS://njuvision.github.io/trobustn中公开访问，以便可重复研究。

Recent models for learned image compression are based on autoencoders, learning approximately invertible mappings from pixels to a quantized latent representation. These are combined with an entropy model, a prior on the latent representation that can be used with standard arithmetic coding algorithms to yield a compressed bitstream. Recently, hierarchical entropy models have been introduced as a way to exploit more structure in the latents than simple fully factorized priors, improving compression performance while maintaining end-to-end optimization. Inspired by the success of autoregressive priors in probabilistic generative models, we examine autoregressive, hierarchical, as well as combined priors as alternatives, weighing their costs and benefits in the context of image compression. While it is well known that autoregressive models come with a significant computational penalty, we find that in terms of compression performance, autoregressive and hierarchical priors are complementary and, together, exploit the probabilistic structure in the latents better than all previous learned models. The combined model yields state-of-the-art rate-distortion performance, providing a 15.8% average reduction in file size over the previous state-of-the-art method based on deep learning, which corresponds to a 59.8% size reduction over JPEG, more than 35% reduction compared to WebP and JPEG2000, and bitstreams 8.4% smaller than BPG, the current state-of-the-art image codec. To the best of our knowledge, our model is the first learning-based method to outperform BPG on both PSNR and MS-SSIM distortion metrics.32nd Conference on Neural Information Processing Systems (NIPS 2018),

We describe an end-to-end trainable model for image compression based on variational autoencoders. The model incorporates a hyperprior to effectively capture spatial dependencies in the latent representation. This hyperprior relates to side information, a concept universal to virtually all modern image codecs, but largely unexplored in image compression using artificial neural networks (ANNs). Unlike existing autoencoder compression methods, our model trains a complex prior jointly with the underlying autoencoder. We demonstrate that this model leads to state-of-the-art image compression when measuring visual quality using the popular MS-SSIM index, and yields rate-distortion performance surpassing published ANN-based methods when evaluated using a more traditional metric based on squared error (PSNR). Furthermore, we provide a qualitative comparison of models trained for different distortion metrics.

In recent years, deep neural network approaches have been widely adopted for machine learning tasks, including classification. However, they were shown to be vulnerable to adversarial perturbations: carefully crafted small perturbations can cause misclassification of legitimate images. We propose Defense-GAN, a new framework leveraging the expressive capability of generative models to defend deep neural networks against such attacks. Defense-GAN is trained to model the distribution of unperturbed images. At inference time, it finds a close output to a given image which does not contain the adversarial changes. This output is then fed to the classifier. Our proposed method can be used with any classification model and does not modify the classifier structure or training procedure. It can also be used as a defense against any attack as it does not assume knowledge of the process for generating the adversarial examples. We empirically show that Defense-GAN is consistently effective against different attack methods and improves on existing defense strategies. Our code has been made publicly available at https://github.com/kabkabm/defensegan.

Recent increases in the computational demands of deep neural networks (DNNs) have sparked interest in efficient deep learning mechanisms, e.g., quantization or pruning. These mechanisms enable the construction of a small, efficient version of commercial-scale models with comparable accuracy, accelerating their deployment to resource-constrained devices. In this paper, we study the security considerations of publishing on-device variants of large-scale models. We first show that an adversary can exploit on-device models to make attacking the large models easier. In evaluations across 19 DNNs, by exploiting the published on-device models as a transfer prior, the adversarial vulnerability of the original commercial-scale models increases by up to 100x. We then show that the vulnerability increases as the similarity between a full-scale and its efficient model increase. Based on the insights, we propose a defense, $similarity$-$unpairing$, that fine-tunes on-device models with the objective of reducing the similarity. We evaluated our defense on all the 19 DNNs and found that it reduces the transferability up to 90% and the number of queries required by a factor of 10-100x. Our results suggest that further research is needed on the security (or even privacy) threats caused by publishing those efficient siblings.

我们提出了一种新颖且有效的纯化基于纯化的普通防御方法，用于预处理盲目的白色和黑匣子攻击。我们的方法仅在一般图像上进行了自我监督学习，在计算上效率和培训，而不需要对分类模型的任何对抗训练或再培训。我们首先显示对原始图像与其对抗示例之间的残余的对抗噪声的实证分析，几乎均为对称分布。基于该观察，我们提出了一种非常简单的迭代高斯平滑（GS），其可以有效地平滑对抗性噪声并实现大大高的鲁棒精度。为了进一步改进它，我们提出了神经上下文迭代平滑（NCIS），其以自我监督的方式列举盲点网络（BSN）以重建GS也平滑的原始图像的辨别特征。从我们使用四种分类模型对大型想象成的广泛实验，我们表明我们的方法既竞争竞争标准精度和最先进的强大精度，则针对最强大的净化器 - 盲目的白色和黑匣子攻击。此外，我们提出了一种用于评估基于商业图像分类API的纯化方法的新基准，例如AWS，Azure，Clarifai和Google。我们通过基于集合转移的黑匣子攻击产生对抗性实例，这可以促进API的完全错误分类，并证明我们的方法可用于增加API的抗逆性鲁棒性。

上下文自适应熵模型的应用显着提高了速率 - 渗透率（R-D）的性能，在该表现中，超级培训和自回归模型被共同利用来有效捕获潜在表示的空间冗余。但是，潜在表示仍然包含一些空间相关性。此外，这些基于上下文自适应熵模型的方法在解码过程中无法通过并行计算设备，例如FPGA或GPU。为了减轻这些局限性，我们提出了一个学识渊博的多分辨率图像压缩框架，该框架利用了最近开发的八度卷积，以将潜在表示形式分配到高分辨率（HR）和低分辨率（LR）部分，类似于小波变换，这进一步改善了R-D性能。为了加快解码的速度，我们的方案不使用上下文自适应熵模型。取而代之的是，我们利用一个额外的超层，包括超级编码器和超级解码器，以进一步删除潜在表示的空间冗余。此外，将跨分辨率参数估计（CRPE）引入提出的框架中，以增强信息流并进一步改善速率延伸性能。提出了对总损耗函数提出的其他信息损失，以调整LR部分对最终位流的贡献。实验结果表明，与最先进的学术图像压缩方法相比，我们的方法分别将解码时间减少了约73.35％和93.44％，R-D性能仍然优于H.266/VVC（4：4：：4：： 2：0）以及对PSNR和MS-SSIM指标的一些基于学习的方法。

在近期深度图像压缩神经网络中，熵模型在估计深度图像编码的先前分配时起着重要作用。现有方法将HydupRior与熵估计功能中的本地上下文组合。由于没有全球愿景，这大大限制了他们的表现。在这项工作中，我们提出了一种新的全局参考模型，用于图像压缩，以有效地利用本地和全局上下文信息，导致增强的压缩率。所提出的方法扫描解码的潜伏，然后找到最相关的潜伏，以帮助分布估计当前潜伏。这项工作的副产品是一种平均转换GDN模块的创新，进一步提高了性能。实验结果表明，所提出的模型优于行业中大多数最先进方法的速率变形性能。

视力变压器由于其出色的性能而越来越多地嵌入工业系统中，但是它们的记忆力和力量要求使它们部署到边缘设备是一项艰巨的任务。因此，现在，模型压缩技术被广泛用于在边缘设备上部署模型，因为它们减少了资源需求并使模型推理非常快速有效。但是，从安全角度来看，它们的可靠性和鲁棒性是安全至关重要应用中的另一个主要问题。对抗性攻击就像ML算法的光学幻象一样，它们可能会严重影响模型的准确性和可靠性。在这项工作中，我们研究了对抗样品在SOTA视觉变压器模型上跨3个SOTA压缩版本的可传递性，并推断出不同压缩技术对对抗攻击的影响。

Recent work has demonstrated that deep neural networks are vulnerable to adversarial examples-inputs that are almost indistinguishable from natural data and yet classified incorrectly by the network. In fact, some of the latest findings suggest that the existence of adversarial attacks may be an inherent weakness of deep learning models. To address this problem, we study the adversarial robustness of neural networks through the lens of robust optimization. This approach provides us with a broad and unifying view on much of the prior work on this topic. Its principled nature also enables us to identify methods for both training and attacking neural networks that are reliable and, in a certain sense, universal. In particular, they specify a concrete security guarantee that would protect against any adversary. These methods let us train networks with significantly improved resistance to a wide range of adversarial attacks. They also suggest the notion of security against a first-order adversary as a natural and broad security guarantee. We believe that robustness against such well-defined classes of adversaries is an important stepping stone towards fully resistant deep learning models. 1

In recent years, neural image compression (NIC) algorithms have shown powerful coding performance. However, most of them are not adaptive to the image content. Although several content adaptive methods have been proposed by updating the encoder-side components, the adaptability of both latents and the decoder is not well exploited. In this work, we propose a new NIC framework that improves the content adaptability on both latents and the decoder. Specifically, to remove redundancy in the latents, our content adaptive channel dropping (CACD) method automatically selects the optimal quality levels for the latents spatially and drops the redundant channels. Additionally, we propose the content adaptive feature transformation (CAFT) method to improve decoder-side content adaptability by extracting the characteristic information of the image content, which is then used to transform the features in the decoder side. Experimental results demonstrate that our proposed methods with the encoder-side updating algorithm achieve the state-of-the-art performance.

Deep neural networks (DNNs) are one of the most prominent technologies of our time, as they achieve state-of-the-art performance in many machine learning tasks, including but not limited to image classification, text mining, and speech processing. However, recent research on DNNs has indicated ever-increasing concern on the robustness to adversarial examples, especially for security-critical tasks such as traffic sign identification for autonomous driving. Studies have unveiled the vulnerability of a well-trained DNN by demonstrating the ability of generating barely noticeable (to both human and machines) adversarial images that lead to misclassification. Furthermore, researchers have shown that these adversarial images are highly transferable by simply training and attacking a substitute model built upon the target model, known as a black-box attack to DNNs.Similar to the setting of training substitute models, in this paper we propose an effective black-box attack that also only has access to the input (images) and the output (confidence scores) of a targeted DNN. However, different from leveraging attack transferability from substitute models, we propose zeroth order optimization (ZOO) based attacks to directly estimate the gradients of the targeted DNN for generating adversarial examples. We use zeroth order stochastic coordinate descent along with dimension reduction, hierarchical attack and importance sampling techniques to * Pin-Yu Chen and Huan Zhang contribute equally to this work.

发言人识别系统（SRSS）最近被证明容易受到对抗攻击的影响，从而引发了重大的安全问题。在这项工作中，我们系统地研究了基于确保SRSS的基于对抗性训练的防御。根据SRSS的特征，我们提出了22种不同的转换，并使用扬声器识别的7种最新有前途的对抗攻击（4个白盒和3个Black-Box）对其进行了彻底评估。仔细考虑了国防评估中的最佳实践，我们分析了转换的强度以承受适应性攻击。我们还评估并理解它们与对抗训练相结合的自适应攻击的有效性。我们的研究提供了许多有用的见解和发现，其中许多与图像和语音识别域中的结论是新的或不一致的，例如，可变和恒定的比特率语音压缩具有不同的性能，并且某些不可差的转换仍然有效地抗衡。当前有希望的逃避技术通常在图像域中很好地工作。我们证明，与完整的白色盒子设置中的唯一对抗性训练相比，提出的新型功能级转换与对抗训练相比是相当有效的，例如，将准确性提高了13.62％，而攻击成本则达到了两个数量级，而其他攻击成本则增加了。转型不一定会提高整体防御能力。这项工作进一步阐明了该领域的研究方向。我们还发布了我们的评估平台SpeakerGuard，以促进进一步的研究。

深度学习（DL）在许多与人类相关的任务中表现出巨大的成功，这导致其在许多计算机视觉的基础应用中采用，例如安全监控系统，自治车辆和医疗保健。一旦他们拥有能力克服安全关键挑战，这种安全关键型应用程序必须绘制他们的成功部署之路。在这些挑战中，防止或/和检测对抗性实例（AES）。对手可以仔细制作小型，通常是难以察觉的，称为扰动的噪声被添加到清洁图像中以产生AE。 AE的目的是愚弄DL模型，使其成为DL应用的潜在风险。在文献中提出了许多测试时间逃避攻击和对策，即防御或检测方法。此外，还发布了很少的评论和调查，理论上展示了威胁的分类和对策方法，几乎​​没有焦点检测方法。在本文中，我们专注于图像分类任务，并试图为神经网络分类器进行测试时间逃避攻击检测方法的调查。对此类方法的详细讨论提供了在四个数据集的不同场景下的八个最先进的探测器的实验结果。我们还为这一研究方向提供了潜在的挑战和未来的观点。

在本文中，我们提出了一类新的高效的深源通道编码方法，可以在非线性变换下的源分布下，可以在名称非线性变换源通道编码（NTSCC）下收集。在所考虑的模型中，发射器首先了解非线性分析变换以将源数据映射到潜伏空间中，然后通过深关节源通道编码将潜在的表示发送到接收器。我们的模型在有效提取源语义特征并提供源通道编码的侧面信息之前，我们的模型包括强度。与现有的传统深度联合源通道编码方法不同，所提出的NTSCC基本上学习源潜像和熵模型，作为先前的潜在表示。因此，开发了新的自适应速率传输和高辅助辅助编解码器改进机制以升级深关节源通道编码。整个系统设计被制定为优化问题，其目标是最小化建立感知质量指标下的端到端传输率失真性能。在简单的示例源和测试图像源上，我们发现所提出的NTSCC传输方法通常优于使用标准的深关节源通道编码和基于经典分离的数字传输的模拟传输。值得注意的是，由于其剧烈的内容感知能力，所提出的NTSCC方法可能会支持未来的语义通信。

It has been witnessed that learned image compression has outperformed conventional image coding techniques and tends to be practical in industrial applications. One of the most critical issues that need to be considered is the non-deterministic calculation, which makes the probability prediction cross-platform inconsistent and frustrates successful decoding. We propose to solve this problem by introducing well-developed post-training quantization and making the model inference integer-arithmetic-only, which is much simpler than presently existing training and fine-tuning based approaches yet still keeps the superior rate-distortion performance of learned image compression. Based on that, we further improve the discretization of the entropy parameters and extend the deterministic inference to fit Gaussian mixture models. With our proposed methods, the current state-of-the-art image compression models can infer in a cross-platform consistent manner, which makes the further development and practice of learned image compression more promising.

对于许多技术领域的专业用户，例如医学，遥感，精密工程和科学研究，无损和近乎无情的图像压缩至关重要。但是，尽管在基于学习的图像压缩方面的研究兴趣迅速增长，但没有发表的方法提供无损和近乎无情的模式。在本文中，我们提出了一个统一而强大的深层损失加上残留（DLPR）编码框架，以实现无损和近乎无情的图像压缩。在无损模式下，DLPR编码系统首先执行有损压缩，然后执行残差的无损编码。我们在VAE的方法中解决了关节损失和残留压缩问题，并添加残差的自回归上下文模型以增强无损压缩性能。在近乎荒谬的模式下，我们量化了原始残差以满足给定的$ \ ell_ \ infty $错误绑定，并提出了可扩展的近乎无情的压缩方案，该方案适用于可变$ \ ell_ \ infty $ bunds而不是训练多个网络。为了加快DLPR编码，我们通过新颖的编码环境设计提高了算法并行化的程度，并以自适应残留间隔加速熵编码。实验结果表明，DLPR编码系统以竞争性的编码速度实现了最先进的无损和近乎无效的图像压缩性能。