Single Image Super-Resolution (SISR) tasks have achieved significant performance with deep neural networks. However, the large number of parameters in CNN-based met-hods for SISR tasks require heavy computations. Although several efficient SISR models have been recently proposed, most are handcrafted and thus lack flexibility. In this work, we propose a novel differentiable Neural Architecture Search (NAS) approach on both the cell-level and network-level to search for lightweight SISR models. Specifically, the cell-level search space is designed based on an information distillation mechanism, focusing on the combinations of lightweight operations and aiming to build a more lightweight and accurate SR structure. The network-level search space is designed to consider the feature connections among the cells and aims to find which information flow benefits the cell most to boost the performance. Unlike the existing Reinforcement Learning (RL) or Evolutionary Algorithm (EA) based NAS methods for SISR tasks, our search pipeline is fully differentiable, and the lightweight SISR models can be efficiently searched on both the cell-level and network-level jointly on a single GPU. Experiments show that our methods can achieve state-of-the-art performance on the benchmark datasets in terms of PSNR, SSIM, and model complexity with merely 68G Multi-Adds for $\times 2$ and 18G Multi-Adds for $\times 4$ SR tasks.

随着卷积神经网络最近的大规模发展，已经提出了用于边缘设备上实用部署的大量基于CNN的显着图像超分辨率方法。但是，大多数现有方法都集中在一个特定方面：网络或损失设计，这导致难以最大程度地减少模型大小。为了解决这个问题，我们得出结论，设计，架构搜索和损失设计，以获得更有效的SR结构。在本文中，我们提出了一个名为EFDN的边缘增强功能蒸馏网络，以保留在约束资源下的高频信息。详细说明，我们基于现有的重新处理方法构建了一个边缘增强卷积块。同时，我们提出了边缘增强的梯度损失，以校准重新分配的路径训练。实验结果表明，我们的边缘增强策略可以保持边缘并显着提高最终恢复质量。代码可在https://github.com/icandle/efdn上找到。

已经证明了深度卷积神经网络近年来对SISR有效。一方面，已经广泛使用了残余连接和密集连接，以便于前向信息和向后梯度流动以提高性能。然而，当前方法以次优的方式在大多数网络层中单独使用残留连接和密集连接。另一方面，虽然各种网络和方法旨在改善计算效率，节省参数或利用彼此的多种比例因子的训练数据来提升性能，但它可以在人力资源空间中进行超级分辨率来具有高计算成本或不能在不同尺度因子的模型之间共享参数以节省参数和推理时间。为了解决这些挑战，我们提出了一种使用双路径连接的高效单图像超分辨率网络，其多种规模学习命名为EMSRDPN。通过将双路径的双路径连接引入EMSRDPN，它在大多数网络层中以综合方式使用残差连接和密集连接。双路径连接具有重用残余连接的共同特征和探索密集连接的新功能，以了解SISR的良好代表。要利用多种比例因子的特征相关性，EMSRDPN在不同缩放因子之间共享LR空间中的所有网络单元，以学习共享功能，并且仅使用单独的重建单元进行每个比例因子，这可以利用多种规模因子的培训数据来帮助各个规模因素另外提高性能，同时可以节省参数并支持共享推理，以提高效率的多种规模因素。实验显示EMSRDPN通过SOTA方法实现更好的性能和比较或更好的参数和推理效率。

In recent years, deep learning methods have been successfully applied to single-image super-resolution tasks. Despite their great performances, deep learning methods cannot be easily applied to realworld applications due to the requirement of heavy computation. In this paper, we address this issue by proposing an accurate and lightweight deep network for image super-resolution. In detail, we design an architecture that implements a cascading mechanism upon a residual network. We also present variant models of the proposed cascading residual network to further improve efficiency. Our extensive experiments show that even with much fewer parameters and operations, our models achieve performance comparable to that of state-of-the-art methods.

随着深度学习（DL）的出现，超分辨率（SR）也已成为一个蓬勃发展的研究领域。然而，尽管结果有希望，但该领域仍然面临需要进一步研究的挑战，例如，允许灵活地采样，更有效的损失功能和更好的评估指标。我们根据最近的进步来回顾SR的域，并检查最新模型，例如扩散（DDPM）和基于变压器的SR模型。我们对SR中使用的当代策略进行了批判性讨论，并确定了有前途但未开发的研究方向。我们通过纳入该领域的最新发展，例如不确定性驱动的损失，小波网络，神经体系结构搜索，新颖的归一化方法和最新评估技术来补充先前的调查。我们还为整章中的模型和方法提供了几种可视化，以促进对该领域趋势的全球理解。最终，这篇综述旨在帮助研究人员推动DL应用于SR的界限。

Informative features play a crucial role in the single image super-resolution task. Channel attention has been demonstrated to be effective for preserving information-rich features in each layer. However, channel attention treats each convolution layer as a separate process that misses the correlation among different layers. To address this problem, we propose a new holistic attention network (HAN), which consists of a layer attention module (LAM) and a channel-spatial attention module (CSAM), to model the holistic interdependencies among layers, channels, and positions. Specifically, the proposed LAM adaptively emphasizes hierarchical features by considering correlations among layers. Meanwhile, CSAM learns the confidence at all the positions of each channel to selectively capture more informative features. Extensive experiments demonstrate that the proposed HAN performs favorably against the state-ofthe-art single image super-resolution approaches.

基于深度学习的单图像超分辨率（SISR）方法引起了人们的关注，并在现代高级GPU上取得了巨大的成功。但是，大多数最先进的方法都需要大量参数，记忆和计算资源，这些参数通常会显示在当前移动设备CPU/NPU上时显示出较低的推理时间。在本文中，我们提出了一个简单的普通卷积网络，该网络具有快速最近的卷积模块（NCNET），该模块对NPU友好，可以实时执行可靠的超级分辨率。提出的最近的卷积具有与最近的UP采样相同的性能，但更快，更适合Android NNAPI。我们的模型可以很容易地在具有8位量化的移动设备上部署，并且与所有主要的移动AI加速器完全兼容。此外，我们对移动设备上的不同张量操作进行了全面的实验，以说明网络体系结构的效率。我们的NCNET在DIV2K 3X数据集上进行了训练和验证，并且与其他有效的SR方法的比较表明，NCNET可以实现高保真SR结果，同时使用更少的推理时间。我们的代码和预估计的模型可在\ url {https://github.com/algolzw/ncnet}上公开获得。

单像超分辨率（SISR），作为传统的不良反对问题，通过最近的卷积神经网络（CNN）的发展得到了极大的振兴。这些基于CNN的方法通常将低分辨率图像映射到其相应的高分辨率版本，具有复杂的网络结构和损耗功能，显示出令人印象深刻的性能。本文对传统的SISR算法提供了新的洞察力，并提出了一种基本上不同的方法，依赖于迭代优化。提出了一种新颖的迭代超分辨率网络（ISRN），顶部是迭代优化。我们首先分析图像SR问题的观察模型，通过以更一般和有效的方式模仿和融合每次迭代来激发可行的解决方案。考虑到批量归一化的缺点，我们提出了一种特征归一化（F-NOM，FN）方法来调节网络中的功能。此外，开发了一种具有FN的新颖块以改善作为FNB称为FNB的网络表示。剩余剩余结构被提出形成一个非常深的网络，其中FNBS与长时间跳过连接，以获得更好的信息传递和稳定训练阶段。对BICUBIC（BI）降解的测试基准的广泛实验结果表明我们的ISRN不仅可以恢复更多的结构信息，而且还可以获得竞争或更好的PSNR / SSIM结果，与其他作品相比，参数更少。除BI之外，我们除了模拟模糊（BD）和低级噪声（DN）的实际降级。 ISRN及其延伸ISRN +两者都比使用BD和DN降级模型的其他产品更好。

Recently, great progress has been made in single-image super-resolution (SISR) based on deep learning technology. However, the existing methods usually require a large computational cost. Meanwhile, the activation function will cause some features of the intermediate layer to be lost. Therefore, it is a challenge to make the model lightweight while reducing the impact of intermediate feature loss on the reconstruction quality. In this paper, we propose a Feature Interaction Weighted Hybrid Network (FIWHN) to alleviate the above problem. Specifically, FIWHN consists of a series of novel Wide-residual Distillation Interaction Blocks (WDIB) as the backbone, where every third WDIBs form a Feature shuffle Weighted Group (FSWG) by mutual information mixing and fusion. In addition, to mitigate the adverse effects of intermediate feature loss on the reconstruction results, we introduced a well-designed Wide Convolutional Residual Weighting (WCRW) and Wide Identical Residual Weighting (WIRW) units in WDIB, and effectively cross-fused features of different finenesses through a Wide-residual Distillation Connection (WRDC) framework and a Self-Calibrating Fusion (SCF) unit. Finally, to complement the global features lacking in the CNN model, we introduced the Transformer into our model and explored a new way of combining the CNN and Transformer. Extensive quantitative and qualitative experiments on low-level and high-level tasks show that our proposed FIWHN can achieve a good balance between performance and efficiency, and is more conducive to downstream tasks to solve problems in low-pixel scenarios.

基于卷积神经网络（CNN）的现代单图像超分辨率（SISR）系统实现了花哨的性能，而需要巨大的计算成本。在视觉识别任务中对特征冗余的问题进行了很好的研究，但很少在SISR中进行讨论。基于这样的观察，SISR模型中的许多功能也彼此相似，我们建议使用Shift操作来生成冗余功能（即幽灵功能）。与在类似GPU的设备上耗时的深度卷积相比，Shift操作可以为CNN带来实用的推理加速度。我们分析了SISR操作对SISR任务的好处，并根据Gumbel-SoftMax技巧使Shift取向可学习。此外，基于预训练的模型探索了聚类过程，以识别用于生成内在特征的内在过滤器。幽灵功能将通过沿特定方向移动这些内在功能来得出。最后，完整的输出功能是通过将固有和幽灵特征串联在一起来构建的。在几个基准模型和数据集上进行的广泛实验表明，嵌入了所提出方法的非压缩和轻质SISR模型都可以实现与基准的可比性能，并大大降低了参数，拖台和GPU推荐延迟。例如，我们将参数降低46％，FLOPS掉落46％，而GPU推断潜伏期则减少了$ \ times2 $ EDSR网络的42％，基本上是无损的。

Convolutional neural network (CNN) depth is of crucial importance for image super-resolution (SR). However, we observe that deeper networks for image SR are more difficult to train. The lowresolution inputs and features contain abundant low-frequency information, which is treated equally across channels, hence hindering the representational ability of CNNs. To solve these problems, we propose the very deep residual channel attention networks (RCAN). Specifically, we propose a residual in residual (RIR) structure to form very deep network, which consists of several residual groups with long skip connections. Each residual group contains some residual blocks with short skip connections. Meanwhile, RIR allows abundant low-frequency information to be bypassed through multiple skip connections, making the main network focus on learning high-frequency information. Furthermore, we propose a channel attention mechanism to adaptively rescale channel-wise features by considering interdependencies among channels. Extensive experiments show that our RCAN achieves better accuracy and visual improvements against state-of-the-art methods.

Neural Architecture Search (NAS) for automatically finding the optimal network architecture has shown some success with competitive performances in various computer vision tasks. However, NAS in general requires a tremendous amount of computations. Thus reducing computational cost has emerged as an important issue. Most of the attempts so far has been based on manual approaches, and often the architectures developed from such efforts dwell in the balance of the network optimality and the search cost. Additionally, recent NAS methods for image restoration generally do not consider dynamic operations that may transform dimensions of feature maps because of the dimensionality mismatch in tensor calculations. This can greatly limit NAS in its search for optimal network structure. To address these issues, we re-frame the optimal search problem by focusing at component block level. From previous work, it's been shown that an effective denoising block can be connected in series to further improve the network performance. By focusing at block level, the search space of reinforcement learning becomes significantly smaller and evaluation process can be conducted more rapidly. In addition, we integrate an innovative dimension matching modules for dealing with spatial and channel-wise mismatch that may occur in the optimal design search. This allows much flexibility in optimal network search within the cell block. With these modules, then we employ reinforcement learning in search of an optimal image denoising network at a module level. Computational efficiency of our proposed Denoising Prior Neural Architecture Search (DPNAS) was demonstrated by having it complete an optimal architecture search for an image restoration task by just one day with a single GPU.

Real-world image super-resolution (RISR) has received increased focus for improving the quality of SR images under unknown complex degradation. Existing methods rely on the heavy SR models to enhance low-resolution (LR) images of different degradation levels, which significantly restricts their practical deployments on resource-limited devices. In this paper, we propose a novel Dynamic Channel Splitting scheme for efficient Real-world Image Super-Resolution, termed DCS-RISR. Specifically, we first introduce the light degradation prediction network to regress the degradation vector to simulate the real-world degradations, upon which the channel splitting vector is generated as the input for an efficient SR model. Then, a learnable octave convolution block is proposed to adaptively decide the channel splitting scale for low- and high-frequency features at each block, reducing computation overhead and memory cost by offering the large scale to low-frequency features and the small scale to the high ones. To further improve the RISR performance, Non-local regularization is employed to supplement the knowledge of patches from LR and HR subspace with free-computation inference. Extensive experiments demonstrate the effectiveness of DCS-RISR on different benchmark datasets. Our DCS-RISR not only achieves the best trade-off between computation/parameter and PSNR/SSIM metric, and also effectively handles real-world images with different degradation levels.

Recent research on super-resolution has progressed with the development of deep convolutional neural networks (DCNN). In particular, residual learning techniques exhibit improved performance. In this paper, we develop an enhanced deep super-resolution network (EDSR) with performance exceeding those of current state-of-the-art SR methods. The significant performance improvement of our model is due to optimization by removing unnecessary modules in conventional residual networks. The performance is further improved by expanding the model size while we stabilize the training procedure. We also propose a new multi-scale deep super-resolution system (MDSR) and training method, which can reconstruct high-resolution images of different upscaling factors in a single model. The proposed methods show superior performance over the state-of-the-art methods on benchmark datasets and prove its excellence by winning the NTIRE2017 Super-Resolution Challenge [26].

基于变压器的方法与基于CNN的方法相比，由于其对远程依赖性的模型，因此获得了令人印象深刻的图像恢复性能。但是，像Swinir这样的进步采用了基于窗口的和本地注意力的策略来平衡性能和计算开销，这限制了采用大型接收领域来捕获全球信息并在早期层中建立长期依赖性。为了进一步提高捕获全球信息的效率，在这项工作中，我们建议Swinfir通过更换具有整个图像范围的接收场的快速傅立叶卷积（FFC）组件来扩展Swinir。我们还重新访问其他先进技术，即数据增强，预训练和功能集合，以改善图像重建的效果。并且我们的功能合奏方法使模型的性能得以大大增强，而无需增加训练和测试时间。与现有方法相比，我们将算法应用于多个流行的大规模基准，并实现了最先进的性能。例如，我们的Swinfir在漫画109数据集上达到了32.83 dB的PSNR，该PSNR比最先进的Swinir方法高0.8 dB。

深度神经网络通过学习从低分辨率（LR）图像到高分辨率（HR）图像的映射，在图像超分辨率（SR）任务中表现出了显着的性能。但是，SR问题通常是一个不适的问题，现有方法将受到一些局限性。首先，由于可能存在许多不同的HR图像，因此SR的可能映射空间可能非常大，可以将其删除到相同的LR图像中。结果，很难直接从如此大的空间中学习有希望的SR映射。其次，通常不可避免地要开发具有极高计算成本的非常大型模型来产生有希望的SR性能。实际上，可以使用模型压缩技术通过降低模型冗余来获得紧凑的模型。然而，由于非常大的SR映射空间，现有模型压缩方法很难准确识别冗余组件。为了减轻第一个挑战，我们提出了一项双重回归学习计划，以减少可能的SR映射空间。具体而言，除了从LR到HR图像的映射外，我们还学习了一个附加的双回归映射，以估算下采样内核和重建LR图像。通过这种方式，双映射是减少可能映射空间的约束。为了应对第二项挑战，我们提出了一种轻巧的双回归压缩方法，以基于通道修剪来降低图层级别和通道级别的模型冗余。具体而言，我们首先开发了一种通道编号搜索方法，该方法将双重回归损耗最小化以确定每一层的冗余。鉴于搜索的通道编号，我们进一步利用双重回归方式来评估通道的重要性并修剪冗余。广泛的实验显示了我们方法在获得准确有效的SR模型方面的有效性。

单个图像超分辨率（SISR）是一个不良问题，旨在获得从低分辨率（LR）输入的高分辨率（HR）输出，在此期间应该添加额外的高频信息以改善感知质量。现有的SISR工作主要通过最小化平均平方重建误差来在空间域中运行。尽管高峰峰值信噪比（PSNR）结果，但难以确定模型是否正确地添加所需的高频细节。提出了一些基于基于残余的结构，以指导模型暗示高频率特征。然而，由于空间域度量的解释是有限的，如何验证这些人为细节的保真度仍然是一个问题。在本文中，我们提出了频率域视角来的直观管道，解决了这个问题。由现有频域的工作启发，我们将图像转换为离散余弦变换（DCT）块，然后改革它们以获取DCT功能映射，它用作我们模型的输入和目标。设计了专门的管道，我们进一步提出了符合频域任务的性质的频率损失功能。我们的SISR方法在频域中可以明确地学习高频信息，为SR图像提供保真度和良好的感知质量。我们进一步观察到我们的模型可以与其他空间超分辨率模型合并，以提高原始SR输出的质量。

卷积神经网络在过去十年中允许在单个图像超分辨率（SISR）中的显着进展。在SISR最近的进展中，关注机制对于高性能SR模型至关重要。但是，注意机制仍然不清楚为什么它在SISR中的工作原理。在这项工作中，我们试图量化和可视化SISR中的注意力机制，并表明并非所有关注模块都同样有益。然后，我们提出了关注网络（A $ ^ 2 $ n）的注意力，以获得更高效和准确的SISR。具体来说，$ ^ 2 $ n包括非关注分支和耦合注意力分支。提出了一种动态注意力模块，为这两个分支产生权重，以动态地抑制不需要的注意力调整，其中权重根据输入特征自适应地改变。这允许注意模块专门从事惩罚的有益实例，从而大大提高了注意力网络的能力，即几个参数开销。实验结果表明，我们的最终模型A $ ^ 2 $ n可以实现与类似尺寸的最先进网络相比的卓越的权衡性能。代码可以在https://github.com/haoyuc/a2n获得。

Image super-resolution (SR) serves as a fundamental tool for the processing and transmission of multimedia data. Recently, Transformer-based models have achieved competitive performances in image SR. They divide images into fixed-size patches and apply self-attention on these patches to model long-range dependencies among pixels. However, this architecture design is originated for high-level vision tasks, which lacks design guideline from SR knowledge. In this paper, we aim to design a new attention block whose insights are from the interpretation of Local Attribution Map (LAM) for SR networks. Specifically, LAM presents a hierarchical importance map where the most important pixels are located in a fine area of a patch and some less important pixels are spread in a coarse area of the whole image. To access pixels in the coarse area, instead of using a very large patch size, we propose a lightweight Global Pixel Access (GPA) module that applies cross-attention with the most similar patch in an image. In the fine area, we use an Intra-Patch Self-Attention (IPSA) module to model long-range pixel dependencies in a local patch, and then a $3\times3$ convolution is applied to process the finest details. In addition, a Cascaded Patch Division (CPD) strategy is proposed to enhance perceptual quality of recovered images. Extensive experiments suggest that our method outperforms state-of-the-art lightweight SR methods by a large margin. Code is available at https://github.com/passerer/HPINet.

具有强大学习能力的CNN被广泛选择以解决超分辨率问题。但是，CNN依靠更深的网络体系结构来提高图像超分辨率的性能，这可能会增加计算成本。在本文中，我们提出了一个增强的超分辨率组CNN（ESRGCNN），具有浅层架构，通过完全融合了深层和宽的通道特征，以在单图超级分辨率中的不同通道的相关性提取更准确的低频信息（ SISR）。同样，ESRGCNN中的信号增强操作对于继承更长途上下文信息以解决长期依赖性也很有用。将自适应上采样操作收集到CNN中，以获得具有不同大小的低分辨率图像的图像超分辨率模型。广泛的实验报告说，我们的ESRGCNN在SISR中的SISR性能，复杂性，执行速度，图像质量评估和SISR的视觉效果方面超过了最先进的实验。代码可在https://github.com/hellloxiaotian/esrgcnn上找到。