使用具有固定尺度的图像超分辨率(SR)的深度学习技术,已经取得了巨大的成功。为了提高其现实世界的适用性,还提出了许多模型来恢复具有任意尺度因子的SR图像,包括不对称的图像,其中图像沿水平和垂直方向大小为不同的尺度。尽管大多数模型仅针对单向上升尺度任务进行了优化,同时假设针对低分辨率(LR)输入的预定义的缩小内核,但基于可逆神经网络(INN)的最新模型能够通过优化降低和降低尺度和降低范围的降低准确性来显着提高上升的准确性共同。但是,受创新体系结构的限制,它被限制在固定的整数尺度因素上,并且需要每个量表的一个模型。在不增加模型复杂性的情况下,提出了一个简单有效的可逆重新恢复网络(IARN),以通过在这项工作中仅训练一个模型来实现任意图像重新缩放。使用创新的组件,例如位置感知量表编码和先发制通道拆分,该网络被优化,以将不可固化的重新恢复周期转换为有效的可逆过程。证明它可以在双向任意重新缩放中实现最新的(SOTA)性能,而不会在LR输出中损害感知质量。还可以证明,使用相同的网络体系结构在不对称尺度的测试上表现良好。
translated by 谷歌翻译
通过将自然图像的复杂分布近似通过可逆神经网络(INN)近似于潜在空间中的简单拖延分布,已成功地用于生成图像超分辨率(SR)。这些模型可以使用潜在空间中的随机采样点从一个低分辨率(LR)输入中生成多个逼真的SR图像,从而模拟图像升级的不足的性质,其中多个高分辨率(HR)图像对应于同一LR。最近,INN中的可逆过程也通过双向图像重新缩放模型(如IRN和HCFLOW)成功使用,以优化降尺度和逆向上尺度的关节,从而显着改善了高尺度的图像质量。尽管它们也被优化用于图像降尺度,但图像降尺度的不良性质可以根据不同的插值内核和重新采样方法将一个HR图像缩小到多个LR图像。除了代表图像放大的不确定性的原始缩小潜在变量外,还引入了图像降压过程中的模型变化。这种双重可变变量增强功能适用于不同的图像重新缩放模型,并且在广泛的实验中显示,它可以始终如一地提高图像升级精度,而无需牺牲缩小的LR图像中的图像质量。它还显示可有效增强基于Inn的其他模型,用于图像恢复应用(例如图像隐藏)。
translated by 谷歌翻译
盲级超分辨率(SR)旨在从低分辨率(LR)图像中恢复高质量的视觉纹理,通常通过下采样模糊内核和添加剂噪声来降解。由于现实世界中复杂的图像降解的挑战,此任务非常困难。现有的SR方法要么假定预定义的模糊内核或固定噪声,这限制了这些方法在具有挑战性的情况下。在本文中,我们提出了一个用于盲目超级分辨率(DMSR)的降解引导的元修复网络,该网络促进了真实病例的图像恢复。 DMSR由降解提取器和元修复模块组成。萃取器估计LR输入中的降解,并指导元恢复模块以预测恢复参数的恢复参数。 DMSR通过新颖的降解一致性损失和重建损失共同优化。通过这样的优化,DMSR在三个广泛使用的基准上以很大的边距优于SOTA。一项包括16个受试者的用户研究进一步验证了现实世界中的盲目SR任务中DMSR的优势。
translated by 谷歌翻译
Learning continuous image representations is recently gaining popularity for image super-resolution (SR) because of its ability to reconstruct high-resolution images with arbitrary scales from low-resolution inputs. Existing methods mostly ensemble nearby features to predict the new pixel at any queried coordinate in the SR image. Such a local ensemble suffers from some limitations: i) it has no learnable parameters and it neglects the similarity of the visual features; ii) it has a limited receptive field and cannot ensemble relevant features in a large field which are important in an image; iii) it inherently has a gap with real camera imaging since it only depends on the coordinate. To address these issues, this paper proposes a continuous implicit attention-in-attention network, called CiaoSR. We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features. Furthermore, we embed a scale-aware attention in this implicit attention network to exploit additional non-local information. Extensive experiments on benchmark datasets demonstrate CiaoSR significantly outperforms the existing single image super resolution (SISR) methods with the same backbone. In addition, the proposed method also achieves the state-of-the-art performance on the arbitrary-scale SR task. The effectiveness of the method is also demonstrated on the real-world SR setting. More importantly, CiaoSR can be flexibly integrated into any backbone to improve the SR performance.
translated by 谷歌翻译
图像缩小和升级是两个基本的重新划分操作。一旦图像缩小,由于信息丢失,难以通过Upscaling重建。为了使这两个过程更加兼容并提高重建性能,一些努力将它们模拟为联合编码解码任务,其中约束是缩小(即编码)的低分辨率(LR)图像必须保留原始视觉外观。要实现此约束,大多数方法通过使用原始高分辨率(HR)图像的双向较低的LR版本监督缩减模块。然而,这种双向LR引导可以是随后的上升(即解码)的次优,并限制最终的重建性能。在本文中,不直接应用LR引导,我们提出了一种额外的可逆性流动指导模块(FGM),其可以在较次编制的情况下将次要表示转换为视觉上可粘合图像并在升级期间重新转换。从FGM的可逆性受益,较次要的代表可以摆脱LR指导,不会打扰较低的升级过程。它允许我们删除对缩小模块的限制,并以端到端的方式优化缩减和上升模块。以这种方式,这两个模块可以协作以最大限度地提高HR重建性能。广泛的实验表明,所提出的方法可以在缩小和重建图像上实现最先进的(SOTA)性能。
translated by 谷歌翻译
Convolutional Neural Network (CNN)-based image super-resolution (SR) has exhibited impressive success on known degraded low-resolution (LR) images. However, this type of approach is hard to hold its performance in practical scenarios when the degradation process is unknown. Despite existing blind SR methods proposed to solve this problem using blur kernel estimation, the perceptual quality and reconstruction accuracy are still unsatisfactory. In this paper, we analyze the degradation of a high-resolution (HR) image from image intrinsic components according to a degradation-based formulation model. We propose a components decomposition and co-optimization network (CDCN) for blind SR. Firstly, CDCN decomposes the input LR image into structure and detail components in feature space. Then, the mutual collaboration block (MCB) is presented to exploit the relationship between both two components. In this way, the detail component can provide informative features to enrich the structural context and the structure component can carry structural context for better detail revealing via a mutual complementary manner. After that, we present a degradation-driven learning strategy to jointly supervise the HR image detail and structure restoration process. Finally, a multi-scale fusion module followed by an upsampling layer is designed to fuse the structure and detail features and perform SR reconstruction. Empowered by such degradation-based components decomposition, collaboration, and mutual optimization, we can bridge the correlation between component learning and degradation modelling for blind SR, thereby producing SR results with more accurate textures. Extensive experiments on both synthetic SR datasets and real-world images show that the proposed method achieves the state-of-the-art performance compared to existing methods.
translated by 谷歌翻译
Recent research on super-resolution has progressed with the development of deep convolutional neural networks (DCNN). In particular, residual learning techniques exhibit improved performance. In this paper, we develop an enhanced deep super-resolution network (EDSR) with performance exceeding those of current state-of-the-art SR methods. The significant performance improvement of our model is due to optimization by removing unnecessary modules in conventional residual networks. The performance is further improved by expanding the model size while we stabilize the training procedure. We also propose a new multi-scale deep super-resolution system (MDSR) and training method, which can reconstruct high-resolution images of different upscaling factors in a single model. The proposed methods show superior performance over the state-of-the-art methods on benchmark datasets and prove its excellence by winning the NTIRE2017 Super-Resolution Challenge [26].
translated by 谷歌翻译
图像超分辨率(SR)是重要的图像处理方法之一,可改善计算机视野领域的图像分辨率。在过去的二十年中,在超级分辨率领域取得了重大进展,尤其是通过使用深度学习方法。这项调查是为了在深度学习的角度进行详细的调查,对单像超分辨率的最新进展进行详细的调查,同时还将告知图像超分辨率的初始经典方法。该调查将图像SR方法分类为四个类别,即经典方法,基于学习的方法,无监督学习的方法和特定领域的SR方法。我们还介绍了SR的问题,以提供有关图像质量指标,可用参考数据集和SR挑战的直觉。使用参考数据集评估基于深度学习的方法。一些审查的最先进的图像SR方法包括增强的深SR网络(EDSR),周期循环gan(Cincgan),多尺度残留网络(MSRN),Meta残留密度网络(META-RDN) ,反复反射网络(RBPN),二阶注意网络(SAN),SR反馈网络(SRFBN)和基于小波的残留注意网络(WRAN)。最后,这项调查以研究人员将解决SR的未来方向和趋势和开放问题的未来方向和趋势。
translated by 谷歌翻译
Real-world image super-resolution (RISR) has received increased focus for improving the quality of SR images under unknown complex degradation. Existing methods rely on the heavy SR models to enhance low-resolution (LR) images of different degradation levels, which significantly restricts their practical deployments on resource-limited devices. In this paper, we propose a novel Dynamic Channel Splitting scheme for efficient Real-world Image Super-Resolution, termed DCS-RISR. Specifically, we first introduce the light degradation prediction network to regress the degradation vector to simulate the real-world degradations, upon which the channel splitting vector is generated as the input for an efficient SR model. Then, a learnable octave convolution block is proposed to adaptively decide the channel splitting scale for low- and high-frequency features at each block, reducing computation overhead and memory cost by offering the large scale to low-frequency features and the small scale to the high ones. To further improve the RISR performance, Non-local regularization is employed to supplement the knowledge of patches from LR and HR subspace with free-computation inference. Extensive experiments demonstrate the effectiveness of DCS-RISR on different benchmark datasets. Our DCS-RISR not only achieves the best trade-off between computation/parameter and PSNR/SSIM metric, and also effectively handles real-world images with different degradation levels.
translated by 谷歌翻译
Existing convolutional neural networks (CNN) based image super-resolution (SR) methods have achieved impressive performance on bicubic kernel, which is not valid to handle unknown degradations in real-world applications. Recent blind SR methods suggest to reconstruct SR images relying on blur kernel estimation. However, their results still remain visible artifacts and detail distortion due to the estimation errors. To alleviate these problems, in this paper, we propose an effective and kernel-free network, namely DSSR, which enables recurrent detail-structure alternative optimization without blur kernel prior incorporation for blind SR. Specifically, in our DSSR, a detail-structure modulation module (DSMM) is built to exploit the interaction and collaboration of image details and structures. The DSMM consists of two components: a detail restoration unit (DRU) and a structure modulation unit (SMU). The former aims at regressing the intermediate HR detail reconstruction from LR structural contexts, and the latter performs structural contexts modulation conditioned on the learned detail maps at both HR and LR spaces. Besides, we use the output of DSMM as the hidden state and design our DSSR architecture from a recurrent convolutional neural network (RCNN) view. In this way, the network can alternatively optimize the image details and structural contexts, achieving co-optimization across time. Moreover, equipped with the recurrent connection, our DSSR allows low- and high-level feature representations complementary by observing previous HR details and contexts at every unrolling time. Extensive experiments on synthetic datasets and real-world images demonstrate that our method achieves the state-of-the-art against existing methods. The source code can be found at https://github.com/Arcananana/DSSR.
translated by 谷歌翻译
随着深度学习(DL)的出现,超分辨率(SR)也已成为一个蓬勃发展的研究领域。然而,尽管结果有希望,但该领域仍然面临需要进一步研究的挑战,例如,允许灵活地采样,更有效的损失功能和更好的评估指标。我们根据最近的进步来回顾SR的域,并检查最新模型,例如扩散(DDPM)和基于变压器的SR模型。我们对SR中使用的当代策略进行了批判性讨论,并确定了有前途但未开发的研究方向。我们通过纳入该领域的最新发展,例如不确定性驱动的损失,小波网络,神经体系结构搜索,新颖的归一化方法和最新评估技术来补充先前的调查。我们还为整章中的模型和方法提供了几种可视化,以促进对该领域趋势的全球理解。最终,这篇综述旨在帮助研究人员推动DL应用于SR的界限。
translated by 谷歌翻译
Image super-resolution (SR) is a technique to recover lost high-frequency information in low-resolution (LR) images. Spatial-domain information has been widely exploited to implement image SR, so a new trend is to involve frequency-domain information in SR tasks. Besides, image SR is typically application-oriented and various computer vision tasks call for image arbitrary magnification. Therefore, in this paper, we study image features in the frequency domain to design a novel scale-arbitrary image SR network. First, we statistically analyze LR-HR image pairs of several datasets under different scale factors and find that the high-frequency spectra of different images under different scale factors suffer from different degrees of degradation, but the valid low-frequency spectra tend to be retained within a certain distribution range. Then, based on this finding, we devise an adaptive scale-aware feature division mechanism using deep reinforcement learning, which can accurately and adaptively divide the frequency spectrum into the low-frequency part to be retained and the high-frequency one to be recovered. Finally, we design a scale-aware feature recovery module to capture and fuse multi-level features for reconstructing the high-frequency spectrum at arbitrary scale factors. Extensive experiments on public datasets show the superiority of our method compared with state-of-the-art methods.
translated by 谷歌翻译
具有窄光谱带的高光谱图像(HSI)可以捕获丰富的光谱信息,但它在该过程中牺牲其空间分辨率。最近提出了许多基于机器学习的HSI超分辨率(SR)算法。然而,这些方法的基本限制之一是它们高度依赖于图像和相机设置,并且只能学会用另一个特定设置用一个特定的设置映射输入的HSI。然而,由于HSI相机的多样性,不同的相机捕获具有不同光谱响应函数和频带编号的图像。因此,现有的基于机器学习的方法无法学习用于各种输入输出频带设置的超声波HSIS。我们提出了一种基于元学习的超分辨率(MLSR)模型,其可以在任意数量的输入频带'峰值波长下采用HSI图像,并产生具有任意数量的输出频带'峰值波长的SR HSIS。我们利用NTIRE2020和ICVL数据集训练并验证MLSR模型的性能。结果表明,单个提出的模型可以在任意输入 - 输出频带设置下成功生成超分辨的HSI频段。结果更好或至少与在特定输入输出频带设置上单独培训的基线相当。
translated by 谷歌翻译
图像恢复算法(如超分辨率(SR)都是用于在劣化图像中的对象检测的必不可少的预处理模块。然而,大多数这些算法假设劣化是固定的并且已知先验。当真实劣化未知或与假设不同时,预处理模块和随后的高级任务(如对象检测)将失败。在这里,我们提出了一种新颖的框架,重新定位,以检测降低的低分辨率图像中的对象。 Restoredet利用下采样的降级作为自我监督信号的一种转换,以探索针对各种分辨率和其他降级条件的等分性表示。具体地,我们通过从一对原始和随机降级的图像编码和解码劣化转换来学习这种内在视觉结构。该框架可以进一步利用先进的SR架构的优点,该架构具有任意分辨率还原解码器以重建来自劣化的输入图像的原始对应关系。代表学习和对象检测都以端到端的培训方式共同优化。 Restoredet是一个通用框架,可以在任何主流对象检测架构上实现。广泛的实验表明,与在面对变体退化情况时,我们基于Centernet的框架已经实现了卓越的性能。我们的代码即将发布。
translated by 谷歌翻译
本文回顾了AIM 2022上压缩图像和视频超级分辨率的挑战。这项挑战包括两条曲目。轨道1的目标是压缩图像的超分辨率,轨迹〜2靶向压缩视频的超分辨率。在轨道1中,我们使用流行的数据集DIV2K作为培训,验证和测试集。在轨道2中,我们提出了LDV 3.0数据集,其中包含365个视频,包括LDV 2.0数据集(335个视频)和30个其他视频。在这一挑战中,有12支球队和2支球队分别提交了赛道1和赛道2的最终结果。所提出的方法和解决方案衡量了压缩图像和视频上超分辨率的最先进。提出的LDV 3.0数据集可在https://github.com/renyang-home/ldv_dataset上找到。此挑战的首页是在https://github.com/renyang-home/aim22_compresssr。
translated by 谷歌翻译
自从Dong等人的第一个成功以来,基于深度学习的方法已在单像超分辨率领域中占主导地位。这取代了使用深神经网络的传统基于稀疏编码方法的所有手工图像处理步骤。与明确创建高/低分辨率词典的基于稀疏编码的方法相反,基于深度学习的方法中的词典被隐式地作为多种卷积的非线性组合被隐式获取。基于深度学习方法的缺点是,它们的性能因与训练数据集(室外图像)不同的图像而降低。我们提出了一个带有深层字典(SRDD)的端到端超分辨率网络,在该网络中,高分辨率词典在不牺牲深度学习优势的情况下明确学习。广泛的实验表明,高分辨率词典的显式学习使网络在维持内域测试图像的性能的同时更加强大。
translated by 谷歌翻译
极度依赖于从划痕的模型的降级或优化的降解或优化的迭代估计,现有的盲超分辨率(SR)方法通常是耗时和效率较低,因为退化的估计从盲初始化进行并且缺乏可解释降解前沿。为了解决它,本文提出了一种使用端到端网络的盲SR的过渡学习方法,没有任何额外的推断中的额外迭代,并探讨了未知降级的有效表示。首先,我们分析并证明降解的过渡性作为可解释的先前信息,以间接推断出未知的降解模型,包括广泛使用的添加剂和卷曲降解。然后,我们提出了一种新颖的过渡性学习方法,用于盲目超分辨率(TLSR),通过自适应地推断过渡转换功能来解决未知的降级而没有推断的任何迭代操作。具体地,端到端TLSR网络包括一定程度的过渡性(点)估计网络,同一性特征提取网络和过渡学习模块。对盲人SR任务的定量和定性评估表明,拟议的TLSR实现了优异的性能,并且对最先进的盲人SR方法的复杂性较少。该代码可在github.com/yuanfeihuang/tlsr获得。
translated by 谷歌翻译
单图超分辨率(SISR)的最新方法在从低分辨率(LR)图像产生高分辨率(HR)图像方面表现出了出色的性能。但是,这些方法中的大多数使用合成生成的LR图像显示出它们的优势,并且它们对现实世界图像的推广性通常并不令人满意。在本文中,我们注意针对可靠的超级分辨率(SR)开发的两种著名策略,即基于参考的SR(REFSR)和零摄影SR(ZSSR),并提出了一种综合解决方案,称为参考 - 基于零击SR(RZSR)。遵循ZSSR的原理,我们使用仅从输入图像本身提取的训练样本在测试时间训练特定于图像的SR网络。为了推进ZSSR,我们获得具有丰富纹理和高频细节的参考图像贴片,这些贴片也仅使用跨尺度匹配从输入图像中提取。为此,我们使用深度信息构建了一个内部参考数据集并从数据集中检索参考图像补丁。使用LR贴片及其相应的HR参考贴片,我们训练由非本地注意模块体现的REFSR网络。实验结果证明了与以前的ZSSR方法相比,与其他完全监督的SISR方法相比,所提出的RZSR的优越性与前所未有的图像相比。
translated by 谷歌翻译
中心位置是否完全能够代表像素?在离散的图像表示中表示具有它们的中心的像素的错误,但是在图像超分辨率(SR)上下文中的局域脉中的信号的聚合时,它更有意义地考虑每个像素。尽管任意级图像SR领域的基于坐标的隐式表示的能力很大,但该区域的像素的性质不完全考虑。为此,我们提出了集成的位置编码(IPE),通过聚合在像素区域上聚合频率信息来扩展传统的位置编码。我们将IPE应用于最先进的任意级图像超分辨率方法:本地隐式图像功能(LIIF),呈现IPE-LIIF。我们通过定量和定性评估显示IPE-LIIF的有效性,并进一步证明了IPE泛化能力与更大的图像尺度和基于多种隐式的方法。代码将被释放。
translated by 谷歌翻译
Image super-resolution (SR) serves as a fundamental tool for the processing and transmission of multimedia data. Recently, Transformer-based models have achieved competitive performances in image SR. They divide images into fixed-size patches and apply self-attention on these patches to model long-range dependencies among pixels. However, this architecture design is originated for high-level vision tasks, which lacks design guideline from SR knowledge. In this paper, we aim to design a new attention block whose insights are from the interpretation of Local Attribution Map (LAM) for SR networks. Specifically, LAM presents a hierarchical importance map where the most important pixels are located in a fine area of a patch and some less important pixels are spread in a coarse area of the whole image. To access pixels in the coarse area, instead of using a very large patch size, we propose a lightweight Global Pixel Access (GPA) module that applies cross-attention with the most similar patch in an image. In the fine area, we use an Intra-Patch Self-Attention (IPSA) module to model long-range pixel dependencies in a local patch, and then a $3\times3$ convolution is applied to process the finest details. In addition, a Cascaded Patch Division (CPD) strategy is proposed to enhance perceptual quality of recovered images. Extensive experiments suggest that our method outperforms state-of-the-art lightweight SR methods by a large margin. Code is available at https://github.com/passerer/HPINet.
translated by 谷歌翻译