Learning-based image compression has improved to a level where it can outperform traditional image codecs such as HEVC and VVC in terms of coding performance. In addition to good compression performance, device interoperability is essential for a compression codec to be deployed, i.e., encoding and decoding on different CPUs or GPUs should be error-free and with negligible performance reduction. In this paper, we present a method to solve the device interoperability problem of a state-of-the-art image compression network. We implement quantization to entropy networks which output entropy parameters. We suggest a simple method which can ensure cross-platform encoding and decoding, and can be implemented quickly with minor performance deviation, of 0.3% BD-rate, from floating point model results.
translated by 谷歌翻译
It has been witnessed that learned image compression has outperformed conventional image coding techniques and tends to be practical in industrial applications. One of the most critical issues that need to be considered is the non-deterministic calculation, which makes the probability prediction cross-platform inconsistent and frustrates successful decoding. We propose to solve this problem by introducing well-developed post-training quantization and making the model inference integer-arithmetic-only, which is much simpler than presently existing training and fine-tuning based approaches yet still keeps the superior rate-distortion performance of learned image compression. Based on that, we further improve the discretization of the entropy parameters and extend the deterministic inference to fit Gaussian mixture models. With our proposed methods, the current state-of-the-art image compression models can infer in a cross-platform consistent manner, which makes the further development and practice of learned image compression more promising.
translated by 谷歌翻译
最近,基于深度学习的图像压缩已取得了显着的进步,并且在主观度量和更具挑战性的客观指标中,与最新的传统方法H.266/vvc相比,取得了更好的评分(R-D)性能。但是,一个主要问题是,许多领先的学识渊博的方案无法保持绩效和复杂性之间的良好权衡。在本文中,我们提出了一个效率和有效的图像编码框架,该框架的复杂性比最高的状态具有相似的R-D性能。首先,我们开发了改进的多尺度残差块(MSRB),该块可以扩展容纳长石,并且更容易获得全球信息。它可以进一步捕获和减少潜在表示的空间相关性。其次,引入了更高级的重要性图网络,以自适应地分配位置到图像的不同区域。第三,我们应用2D定量后flter(PQF)来减少视频编码中样本自适应偏移量(SAO)flter的动机。此外,我们认为编码器和解码器的复杂性对图像压缩性能有不同的影响。基于这一观察结果,我们设计了一个不对称范式,其中编码器采用三个阶段的MSRB来提高学习能力,而解码器只需要一个srb的一个阶段就可以产生令人满意的重建,从而在不牺牲性能的情况下降低了解码的复杂性。实验结果表明,与最先进的方法相比,所提出方法的编码和解码时间速度约为17倍,而R-D性能仅在Kodak和Tecnick数据集中降低了1%,而R-D性能仅少于1%。它仍然比H.266/VVC(4:4:4)和其他基于学习的方法更好。我们的源代码可在https://github.com/fengyurenpingsheng上公开获得。
translated by 谷歌翻译
Image compression is a fundamental research field and many well-known compression standards have been developed for many decades. Recently, learned compression methods exhibit a fast development trend with promising results. However, there is still a performance gap between learned compression algorithms and reigning compression standards, especially in terms of widely used PSNR metric. In this paper, we explore the remaining redundancy of recent learned compression algorithms. We have found accurate entropy models for rate estimation largely affect the optimization of network parameters and thus affect the rate-distortion performance. Therefore, in this paper, we propose to use discretized Gaussian Mixture Likelihoods to parameterize the distributions of latent codes, which can achieve a more accurate and flexible entropy model. Besides, we take advantage of recent attention modules and incorporate them into network architecture to enhance the performance. Experimental results demonstrate our proposed method achieves a state-of-the-art performance compared to existing learned compression methods on both Kodak and high-resolution datasets. To our knowledge our approach is the first work to achieve comparable performance with latest compression standard Versatile Video Coding (VVC) regarding PSNR. More importantly, our approach generates more visually pleasant results when optimized by MS-SSIM. The project page is at https://github.com/ZhengxueCheng/ Learned-Image-Compression-with-GMM-and-Attention.
translated by 谷歌翻译
学习的图像压缩技术近年来取得了相当大的发展。在本文中,我们发现性能瓶颈位于使用单个高度解码器,在这种情况下,三元高斯模型折叠到二进制文件。为了解决这个问题,我们建议使用三个高度解码器来分离混合参数的解码过程,以分散的高斯混合似然性,实现更准确的参数估计。实验结果表明,与最先进的方法相比,MS-SSSIM优化的所提出的方法实现了3.36%的BD速率。所提出的方法对编码时间和拖鞋的贡献可以忽略不计。
translated by 谷歌翻译
端到端的深层训练模型将超过视频和图像上传统手工制作的压缩技术的性能。核心思想是学习一个非线性转换,以深度神经网络建模,将输入图像映射到潜在空间中,并与潜在分布的熵模型共同映射到潜在的空间中。解码器也被学习为可训练的深层网络,重建图像可以测量失真。这些方法强迫潜在遵循一些先前的分布。由于这些先验是通过在整个训练组中优化学习的,因此性能平均是最佳的。但是,它不能完全适合每个新实例,因此可以通过扩大位流损坏压缩性能。在本文中,我们提出了一种简单但有效的基于实例的参数化方法,以较小的成本减少此摊销差距。所提出的方法适用于任何端到端的压缩方法,将压缩比特率提高了1%,而不会对重建质量产生任何影响。
translated by 谷歌翻译
上下文自适应熵模型的应用显着提高了速率 - 渗透率(R-D)的性能,在该表现中,超级培训和自回归模型被共同利用来有效捕获潜在表示的空间冗余。但是,潜在表示仍然包含一些空间相关性。此外,这些基于上下文自适应熵模型的方法在解码过程中无法通过并行计算设备,例如FPGA或GPU。为了减轻这些局限性,我们提出了一个学识渊博的多分辨率图像压缩框架,该框架利用了最近开发的八度卷积,以将潜在表示形式分配到高分辨率(HR)和低分辨率(LR)部分,类似于小波变换,这进一步改善了R-D性能。为了加快解码的速度,我们的方案不使用上下文自适应熵模型。取而代之的是,我们利用一个额外的超层,包括超级编码器和超级解码器,以进一步删除潜在表示的空间冗余。此外,将跨分辨率参数估计(CRPE)引入提出的框架中,以增强信息流并进一步改善速率延伸性能。提出了对总损耗函数提出的其他信息损失,以调整LR部分对最终位流的贡献。实验结果表明,与最先进的学术图像压缩方法相比,我们的方法分别将解码时间减少了约73.35%和93.44%,R-D性能仍然优于H.266/VVC(4:4::4:: 2:0)以及对PSNR和MS-SSIM指标的一些基于学习的方法。
translated by 谷歌翻译
Recent models for learned image compression are based on autoencoders, learning approximately invertible mappings from pixels to a quantized latent representation. These are combined with an entropy model, a prior on the latent representation that can be used with standard arithmetic coding algorithms to yield a compressed bitstream. Recently, hierarchical entropy models have been introduced as a way to exploit more structure in the latents than simple fully factorized priors, improving compression performance while maintaining end-to-end optimization. Inspired by the success of autoregressive priors in probabilistic generative models, we examine autoregressive, hierarchical, as well as combined priors as alternatives, weighing their costs and benefits in the context of image compression. While it is well known that autoregressive models come with a significant computational penalty, we find that in terms of compression performance, autoregressive and hierarchical priors are complementary and, together, exploit the probabilistic structure in the latents better than all previous learned models. The combined model yields state-of-the-art rate-distortion performance, providing a 15.8% average reduction in file size over the previous state-of-the-art method based on deep learning, which corresponds to a 59.8% size reduction over JPEG, more than 35% reduction compared to WebP and JPEG2000, and bitstreams 8.4% smaller than BPG, the current state-of-the-art image codec. To the best of our knowledge, our model is the first learning-based method to outperform BPG on both PSNR and MS-SSIM distortion metrics.32nd Conference on Neural Information Processing Systems (NIPS 2018),
translated by 谷歌翻译
由于深层网络的计算复杂性和功率约束的移动硬件的计算复杂性,因此在移动设备上实现神经视频编解码器的潜力是一项巨大的技术挑战。我们通过利用高通公司的技术和创新来证明可行性,从而弥合了从基于神经网络的编解码器模拟在壁式工作站运行的差距,再到由Snapdragon技术供电的移动设备上的实时操作。我们显示有史以来第一个在商用手机上运行的框架间神经视频解码器,实时解码高清视频,同时保持低比特率和高视觉质量。
translated by 谷歌翻译
我们提出了一种压缩具有隐式神经表示的全分辨率视频序列的方法。每个帧表示为映射坐标位置到像素值的神经网络。我们使用单独的隐式网络来调制坐标输入,从而实现帧之间的有效运动补偿。与一个小的残余网络一起,这允许我们有效地相对于前一帧压缩p帧。通过使用学习的整数量化存储网络权重,我们进一步降低了比特率。我们呼叫隐式像素流(IPF)的方法,提供了几种超简化的既定神经视频编解码器:它不需要接收器可以访问预先磨普的神经网络,不使用昂贵的内插基翘曲操作,而不是需要单独的培训数据集。我们展示了神经隐式压缩对图像和视频数据的可行性。
translated by 谷歌翻译
在近期深度图像压缩神经网络中,熵模型在估计深度图像编码的先前分配时起着重要作用。现有方法将HydupRior与熵估计功能中的本地上下文组合。由于没有全球愿景,这大大限制了他们的表现。在这项工作中,我们提出了一种新的全局参考模型,用于图像压缩,以有效地利用本地和全局上下文信息,导致增强的压缩率。所提出的方法扫描解码的潜伏,然后找到最相关的潜伏,以帮助分布估计当前潜伏。这项工作的副产品是一种平均转换GDN模块的创新,进一步提高了性能。实验结果表明,所提出的模型优于行业中大多数最先进方法的速率变形性能。
translated by 谷歌翻译
In recent years, neural image compression (NIC) algorithms have shown powerful coding performance. However, most of them are not adaptive to the image content. Although several content adaptive methods have been proposed by updating the encoder-side components, the adaptability of both latents and the decoder is not well exploited. In this work, we propose a new NIC framework that improves the content adaptability on both latents and the decoder. Specifically, to remove redundancy in the latents, our content adaptive channel dropping (CACD) method automatically selects the optimal quality levels for the latents spatially and drops the redundant channels. Additionally, we propose the content adaptive feature transformation (CAFT) method to improve decoder-side content adaptability by extracting the characteristic information of the image content, which is then used to transform the features in the decoder side. Experimental results demonstrate that our proposed methods with the encoder-side updating algorithm achieve the state-of-the-art performance.
translated by 谷歌翻译
与传统方法相比,学到的图像压缩已在PSNR和MS-SSIM中取得了非凡的速率延伸性能。但是,它遭受了密集的计算,这对于现实世界的应用是无法忍受的,目前导致其工业应用有限。在本文中,我们将神经体系结构搜索(NAS)介绍到具有较低延迟的更有效网络,并利用量化以加速推理过程。同时,已经为提高效率而做出了工程努力。使用PSNR和MS-SSIM的混合损失以更好的视觉质量进行了优化,我们获得的MSSIM比JPEG,JPEG XL和AVIF在所有比特率上都高得多,而JPEG XL和AVIF之间的PSNR则获得了PSNR。与JPEG-Turbo相比,我们的LIC的软件实施实现了可比较甚至更快的推理速度,而多次比JPEG XL和AVIF快。此外,我们的LIC实施达到了145 fps的惊人吞吐量,用于编码为208 fps,用于在Tesla T4 GPU上解码1080p图像。在CPU上,我们实施的延迟与JPEG XL相当。
translated by 谷歌翻译
对于许多技术领域的专业用户,例如医学,遥感,精密工程和科学研究,无损和近乎无情的图像压缩至关重要。但是,尽管在基于学习的图像压缩方面的研究兴趣迅速增长,但没有发表的方法提供无损和近乎无情的模式。在本文中,我们提出了一个统一而强大的深层损失加上残留(DLPR)编码框架,以实现无损和近乎无情的图像压缩。在无损模式下,DLPR编码系统首先执行有损压缩,然后执行残差的无损编码。我们在VAE的方法中解决了关节损失和残留压缩问题,并添加残差的自回归上下文模型以增强无损压缩性能。在近乎荒谬的模式下,我们量化了原始残差以满足给定的$ \ ell_ \ infty $错误绑定,并提出了可扩展的近乎无情的压缩方案,该方案适用于可变$ \ ell_ \ infty $ bunds而不是训练多个网络。为了加快DLPR编码,我们通过新颖的编码环境设计提高了算法并行化的程度,并以自适应残留间隔加速熵编码。实验结果表明,DLPR编码系统以竞争性的编码速度实现了最先进的无损和近乎无效的图像压缩性能。
translated by 谷歌翻译
无损图像压缩是图像压缩中必不可少的研究领域。最近,与传统的无损方法(例如WebP,JPEG2000和FLIF)相比,基于学习的图像压缩方法具有令人印象深刻的性能。但是,仍然有许多令人印象深刻的有损压缩方法可应用于无损压缩。因此,在本文中,我们探讨了广泛用于有损压缩的方法,并将其应用于无损压缩。受损失压缩显示的高斯混合模型(GMM)的令人印象深刻的性能的启发,我们与GMM生成了无损网络体系结构。除了注意到注意模块和自回归模型的成功成就外,我们建议利用注意模块,并为我们的网络体系结构中的原始图像添加额外的自动回归模型,以提高性能。实验结果表明,我们的方法优于大多数经典的无损压缩方法和现有基于学习的方法。
translated by 谷歌翻译
可扩展的编码,可以适应通道带宽变化,在当今复杂的网络环境中表现良好。然而,现有的可扩展压缩方法面临两个挑战:降低压缩性能和可扩展性不足。在本文中,我们提出了第一所学习的细粒度可扩展图像压缩模型(DeepFGS)来克服上述两个缺点。具体地,我们介绍一个特征分离骨干,将图像信息划分为基本和可伸缩的功能,然后通过信息重新排列策略通过通道重新分配特征通道。以这种方式,我们可以通过一次通过编码来生成连续可扩展的比特流。此外,我们重复使用解码器以降低DeepFGS的参数和计算复杂性。实验表明,我们的DeePFGS优于PSNR和MS-SSIM度量中的所有基于学习的可伸缩图像压缩模型和传统可伸缩图像编解码器。据我们所知,我们的DeePFGS是对学习的细粒度可扩展编码的首次探索,与基于学习的方法相比,实现了最优质的可扩展性。
translated by 谷歌翻译
这项工作解决了基于深度神经网络的端到端学习图像压缩(LIC)的两个主要问题:可变速率学习,其中需要单独的网络以不同的质量生成压缩图像,以及可微分近似之间的列车测试不匹配量化和真正的硬量化。我们介绍了LIC的在线元学习(OML)设置,将Meta学习和在线学习中的思想结合在条件变分自动编码器(CVAE)框架中。通过将条件变量视为元参数并将生成的条件特征视为元前沿,可以通过元参数控制所需的重建以适应变量质量的压缩。在线学习框架用于更新元参数,以便为当前图像自适应地调整条件重建。通过OML机制,可以通过SGD有效更新元参数。条件重建基于解码器网络中的量化潜在表示,因此有助于弥合训练估计与真正量化的潜在分布之间的间隙。实验表明,我们的OML方法可以灵活地应用于不同的最先进的LIC方法,以实现具有很少的计算和传输开销的额外性能改进。
translated by 谷歌翻译
Recent neural compression methods have been based on the popular hyperprior framework. It relies on Scalar Quantization and offers a very strong compression performance. This contrasts from recent advances in image generation and representation learning, where Vector Quantization is more commonly employed. In this work, we attempt to bring these lines of research closer by revisiting vector quantization for image compression. We build upon the VQ-VAE framework and introduce several modifications. First, we replace the vanilla vector quantizer by a product quantizer. This intermediate solution between vector and scalar quantization allows for a much wider set of rate-distortion points: It implicitly defines high-quality quantizers that would otherwise require intractably large codebooks. Second, inspired by the success of Masked Image Modeling (MIM) in the context of self-supervised learning and generative image models, we propose a novel conditional entropy model which improves entropy coding by modelling the co-dependencies of the quantized latent codes. The resulting PQ-MIM model is surprisingly effective: its compression performance on par with recent hyperprior methods. It also outperforms HiFiC in terms of FID and KID metrics when optimized with perceptual losses (e.g. adversarial). Finally, since PQ-MIM is compatible with image generation frameworks, we show qualitatively that it can operate under a hybrid mode between compression and generation, with no further training or finetuning. As a result, we explore the extreme compression regime where an image is compressed into 200 bytes, i.e., less than a tweet.
translated by 谷歌翻译
随着深度学习技术的发展,深度学习与图像压缩的结合引起了很多关注。最近,学到的图像压缩方法在速率绩效方面超出了其经典对应物。但是,连续的速率适应仍然是一个悬而未决的问题。一些学到的图像压缩方法将多个网络用于多个速率,而另一些则使用一个模型,而牺牲了计算复杂性的增加和性能降解。在本文中,我们提出了一个不断的可调节率的学术图像压缩框架,不对称获得了变异自动编码器(AG-VAE)。 AG-VAE利用一对增益单元在一个单个模型中实现离散率适应,并具有可忽略的附加计算。然后,通过使用指数插值,可以在不损害性能的情况下实现连续速率适应。此外,我们提出了不对称的高斯熵模型,以进行更准确的熵估计。详尽的实验表明,与经典图像编解码器相比,我们的方法通过SOTA学习的图像压缩方法获得了可比的定量性能,并且定性性能更好。在消融研究中,我们证实了增益单元和不对称高斯熵模型的有用性和优势。
translated by 谷歌翻译
Recently, many neural network-based image compression methods have shown promising results superior to the existing tool-based conventional codecs. However, most of them are often trained as separate models for different target bit rates, thus increasing the model complexity. Therefore, several studies have been conducted for learned compression that supports variable rates with single models, but they require additional network modules, layers, or inputs that often lead to complexity overhead, or do not provide sufficient coding efficiency. In this paper, we firstly propose a selective compression method that partially encodes the latent representations in a fully generalized manner for deep learning-based variable-rate image compression. The proposed method adaptively determines essential representation elements for compression of different target quality levels. For this, we first generate a 3D importance map as the nature of input content to represent the underlying importance of the representation elements. The 3D importance map is then adjusted for different target quality levels using importance adjustment curves. The adjusted 3D importance map is finally converted into a 3D binary mask to determine the essential representation elements for compression. The proposed method can be easily integrated with the existing compression models with a negligible amount of overhead increase. Our method can also enable continuously variable-rate compression via simple interpolation of the importance adjustment curves among different quality levels. The extensive experimental results show that the proposed method can achieve comparable compression efficiency as those of the separately trained reference compression models and can reduce decoding time owing to the selective compression. The sample codes are publicly available at https://github.com/JooyoungLeeETRI/SCR.
translated by 谷歌翻译