超级分辨率是一个不良问题，其中基本真理的高分辨率图像仅代表合理解决方案的空间中的一种可能性。然而，主导范式是采用像素 - 明智的损失，例如L_1，其驱动预测模糊的平均值。当与对抗性损失相结合时，这导致了根本相互矛盾的目标，这降低了最终质量。我们通过重新审视L_1丢失来解决此问题，并表明它对应于单层条件流程。灵感来自这一关系，我们探讨了一般流动作为L_1目标的忠诚替代品。我们证明，在与对抗性损失结合时，更深流量的灵活性导致更好的视觉质量和一致性。我们对三个数据集和比例因子进行广泛的用户研究，其中我们的方法被证明了为光逼真的超分辨率优于最先进的方法。代码和培训的型号可在：git.io/adflow

Despite the breakthroughs in accuracy and speed of single image super-resolution using faster and deeper convolutional neural networks, one central problem remains largely unsolved: how do we recover the finer texture details when we super-resolve at large upscaling factors? The behavior of optimization-based super-resolution methods is principally driven by the choice of the objective function. Recent work has largely focused on minimizing the mean squared reconstruction error. The resulting estimates have high peak signal-to-noise ratios, but they are often lacking high-frequency details and are perceptually unsatisfying in the sense that they fail to match the fidelity expected at the higher resolution. In this paper, we present SRGAN, a generative adversarial network (GAN) for image superresolution (SR). To our knowledge, it is the first framework capable of inferring photo-realistic natural images for 4× upscaling factors. To achieve this, we propose a perceptual loss function which consists of an adversarial loss and a content loss. The adversarial loss pushes our solution to the natural image manifold using a discriminator network that is trained to differentiate between the super-resolved images and original photo-realistic images. In addition, we use a content loss motivated by perceptual similarity instead of similarity in pixel space. Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks. An extensive mean-opinion-score (MOS) test shows hugely significant gains in perceptual quality using SRGAN. The MOS scores obtained with SRGAN are closer to those of the original high-resolution images than to those obtained with any state-of-the-art method.

成功地应用生成的对抗性网络（GaN）以研究感知单个图像超级度（SISR）。然而，GaN经常倾向于产生具有高频率细节的图像与真实的细节不一致。灵感来自传统细节增强算法，我们提出了一种新的先前知识，先前的细节，帮助GaN减轻这个问题并恢复更现实的细节。所提出的方法名为DSRAN，包括良好设计的详细提取算法，用于捕获图像中最重要的高频信息。然后，两种鉴别器分别用于在图像域和细节域修复上进行监督。 DSRGAN通过细节增强方式将恢复的细节合并到最终输出中。 DSRGAN的特殊设计从基于模型的常规算法和数据驱动的深度学习网络中获得了优势。实验结果表明，DSRGAN在感知度量上表现出最先进的SISR方法，并同时达到保真度量的可比结果。在DSRGAN之后，将其他传统的图像处理算法结合到深度学习网络中，以形成基于模型的深SISR。

Single image super-resolution is the task of inferring a high-resolution image from a single low-resolution input. Traditionally, the performance of algorithms for this task is measured using pixel-wise reconstruction measures such as peak signal-to-noise ratio (PSNR) which have been shown to correlate poorly with the human perception of image quality. As a result, algorithms minimizing these metrics tend to produce over-smoothed images that lack highfrequency textures and do not look natural despite yielding high PSNR values.We propose a novel application of automated texture synthesis in combination with a perceptual loss focusing on creating realistic textures rather than optimizing for a pixelaccurate reproduction of ground truth images during training. By using feed-forward fully convolutional neural networks in an adversarial training setting, we achieve a significant boost in image quality at high magnification ratios. Extensive experiments on a number of datasets show the effectiveness of our approach, yielding state-of-the-art results in both quantitative and qualitative benchmarks.

我们考虑单个图像超分辨率（SISR）问题，其中基于低分辨率（LR）输入产生高分辨率（HR）图像。最近，生成的对抗性网络（GANS）变得幻觉细节。大多数沿着这条线的方法依赖于预定义的单个LR-intle-hr映射，这对于SISR任务来说是足够灵活的。此外，GaN生成的假细节可能经常破坏整个图像的现实主义。我们通过为Rich-Detail SISR提出最好的伙伴GANS（Beby-GaN）来解决这些问题。放松不变的一对一的约束，我们允许估计的贴片在培训期间动态寻求最佳监督，这有利于产生更合理的细节。此外，我们提出了一种区域感知的对抗性学习策略，指导我们的模型专注于自适应地为纹理区域发电细节。广泛的实验证明了我们方法的有效性。还构建了超高分辨率4K数据集以促进未来的超分辨率研究。

最近的研究通过卷积神经网络（CNNS）显着提高了单图像超分辨率（SR）的性能。虽然可以有许多用于给定输入的高分辨率（HR）解决方案，但大多数现有的基于CNN的方法在推理期间不会探索替代解决方案。获得替代SR结果的典型方法是培训具有不同丢失权重的多个SR模型，并利用这些模型的组合。我们通过利用多任务学习，我们提出了一种更有效的方法来培训单个可调SR模型的单一可调SR模型。具体地，我们在训练期间优化具有条件目标的SR模型，其中目标是不同特征级别的多个感知损失的加权之和。权重根据给定条件而变化，并且该组重量被定义为样式控制器。此外，我们提出了一种适用于该训练方案的架构，该架构是配备有空间特征变换层的残留残余密集块。在推理阶段，我们培训的模型可以在样式控制地图上生成局部不同的输出。广泛的实验表明，所提出的SR模型在没有伪影的情况下产生各种所需的重建，并对最先进的SR方法产生相当的定量性能。

基于流量的生成超分辨率（SR）模型学会生产一组可行的SR解决方案，称为SR空间。 SR溶液的多样性随着潜在变量的温度（$ \ tau $）的增加而增加，这引入了样品溶液之间纹理的随机变化，从而导致视觉伪像和低忠诚度。在本文中，我们提出了一种简单但有效的图像结合/融合方法，以获得消除随机伪像的单个SR图像，并改善忠诚度，而不会显着损害感知质量。我们通过从流量模型跨越的SR空间中的一系列可行的光真实解决方案中受益，从而实现这一目标。我们提出了不同的图像结合和融合策略，这些策略提供了多种途径，可以根据手头任务的保真度与感知质量要求，以可控的方式将SR Slace样本解决方案移至感知延伸平面中更为理想的目的地。实验结果表明，与流量模型和经过对抗训练的模型所产生的样本SR图像相比，我们的图像结合/融合策略在定量指标和视觉质量方面实现了更有希望的感知依赖权衡。

Single-image super-resolution (SISR) networks trained with perceptual and adversarial losses provide high-contrast outputs compared to those of networks trained with distortion-oriented losses, such as L1 or L2. However, it has been shown that using a single perceptual loss is insufficient for accurately restoring locally varying diverse shapes in images, often generating undesirable artifacts or unnatural details. For this reason, combinations of various losses, such as perceptual, adversarial, and distortion losses, have been attempted, yet it remains challenging to find optimal combinations. Hence, in this paper, we propose a new SISR framework that applies optimal objectives for each region to generate plausible results in overall areas of high-resolution outputs. Specifically, the framework comprises two models: a predictive model that infers an optimal objective map for a given low-resolution (LR) input and a generative model that applies a target objective map to produce the corresponding SR output. The generative model is trained over our proposed objective trajectory representing a set of essential objectives, which enables the single network to learn various SR results corresponding to combined losses on the trajectory. The predictive model is trained using pairs of LR images and corresponding optimal objective maps searched from the objective trajectory. Experimental results on five benchmarks show that the proposed method outperforms state-of-the-art perception-driven SR methods in LPIPS, DISTS, PSNR, and SSIM metrics. The visual results also demonstrate the superiority of our method in perception-oriented reconstruction. The code and models are available at https://github.com/seungho-snu/SROOE.

The Super-Resolution Generative Adversarial Network (SR-GAN) [1] is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied with unpleasant artifacts. To further enhance the visual quality, we thoroughly study three key components of SRGANnetwork architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN). In particular, we introduce the Residual-in-Residual Dense Block (RRDB) without batch normalization as the basic network building unit. Moreover, we borrow the idea from relativistic GAN [2] to let the discriminator predict relative realness instead of the absolute value. Finally, we improve the perceptual loss by using the features before activation, which could provide stronger supervision for brightness consistency and texture recovery. Benefiting from these improvements, the proposed ESRGAN achieves consistently better visual quality with more realistic and natural textures than SRGAN and won the first place in the PIRM2018-SR Challenge 1 [3]. The code is available at https://github.com/xinntao/ESRGAN.

Despite that convolutional neural networks (CNN) have recently demonstrated high-quality reconstruction for single-image super-resolution (SR), recovering natural and realistic texture remains a challenging problem. In this paper, we show that it is possible to recover textures faithful to semantic classes. In particular, we only need to modulate features of a few intermediate layers in a single network conditioned on semantic segmentation probability maps. This is made possible through a novel Spatial Feature Transform (SFT) layer that generates affine transformation parameters for spatial-wise feature modulation. SFT layers can be trained end-to-end together with the SR network using the same loss function. During testing, it accepts an input image of arbitrary size and generates a high-resolution image with just a single forward pass conditioned on the categorical priors. Our final results show that an SR network equipped with SFT can generate more realistic and visually pleasing textures in comparison to state-of-the-art SRGAN [27] and EnhanceNet [38].

Deep Convolutional Neural Networks (DCNNs) have exhibited impressive performance on image super-resolution tasks. However, these deep learning-based super-resolution methods perform poorly in real-world super-resolution tasks, where the paired high-resolution and low-resolution images are unavailable and the low-resolution images are degraded by complicated and unknown kernels. To break these limitations, we propose the Unsupervised Bi-directional Cycle Domain Transfer Learning-based Generative Adversarial Network (UBCDTL-GAN), which consists of an Unsupervised Bi-directional Cycle Domain Transfer Network (UBCDTN) and the Semantic Encoder guided Super Resolution Network (SESRN). First, the UBCDTN is able to produce an approximated real-like LR image through transferring the LR image from an artificially degraded domain to the real-world LR image domain. Second, the SESRN has the ability to super-resolve the approximated real-like LR image to a photo-realistic HR image. Extensive experiments on unpaired real-world image benchmark datasets demonstrate that the proposed method achieves superior performance compared to state-of-the-art methods.

在本文中，我们提出了D2C-SR，这是一个新颖的框架，用于实现现实世界图像超级分辨率的任务。作为一个不适的问题，超分辨率相关任务的关键挑战是给定的低分辨率输入可能会有多个预测。大多数基于经典的深度学习方法都忽略了基本事实，缺乏对基础高频分布的明确建模，从而导致结果模糊。最近，一些基于GAN或学习的超分辨率空间的方法可以生成模拟纹理，但不能保证具有低定量性能的纹理的准确性。重新思考这两者，我们以离散形式了解了基本高频细节的分布，并提出了两阶段的管道：分歧阶段到收敛阶段。在发散阶段，我们提出了一个基于树的结构深网作为差异骨干。提出了发散损失，以鼓励基于树的网络产生的结果，以分解可能的高频表示，这是我们对基本高频分布进行离散建模的方式。在收敛阶段，我们分配空间权重以融合这些不同的预测，以获得更准确的细节，以获取最终输出。我们的方法为推理提供了方便的端到端方式。我们对几个现实世界基准进行评估，包括具有X8缩放系数的新提出的D2CrealSR数据集。我们的实验表明，D2C-SR针对最先进的方法实现了更好的准确性和视觉改进，参数编号明显较少，并且我们的D2C结构也可以作为广义结构应用于其他一些方法以获得改进。我们的代码和数据集可在https://github.com/megvii-research/d2c-sr上找到

The primary aim of single-image super-resolution is to construct a high-resolution (HR) image from a corresponding low-resolution (LR) input. In previous approaches, which have generally been supervised, the training objective typically measures a pixel-wise average distance between the super-resolved (SR) and HR images. Optimizing such metrics often leads to blurring, especially in high variance (detailed) regions. We propose an alternative formulation of the super-resolution problem based on creating realistic SR images that downscale correctly. We present a novel super-resolution algorithm addressing this problem, PULSE (Photo Upsampling via Latent Space Exploration), which generates high-resolution, realistic images at resolutions previously unseen in the literature. It accomplishes this in an entirely self-supervised fashion and is not confined to a specific degradation operator used during training, unlike previous methods (which require training on databases of LR-HR image pairs for supervised learning). Instead of starting with the LR image and slowly adding detail, PULSE traverses the high-resolution natural image manifold, searching for images that downscale to the original LR image. This is formalized through the "downscaling loss," which guides exploration through the latent space of a generative model. By leveraging properties of high-dimensional Gaussians, we restrict the search space to guarantee that our outputs are realistic. PULSE thereby generates super-resolved images that both are realistic and downscale correctly. We show extensive experimental results demonstrating the efficacy of our approach in the domain of face super-resolution (also known as face hallucination). We also present a discussion of the limitations and biases of the method as currently implemented with an accompanying model card with relevant metrics. Our method outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously pos-sible.

现实世界图像超分辨率（SR）的关键挑战是在低分辨率（LR）图像中恢复具有复杂未知降解（例如，下采样，噪声和压缩）的缺失细节。大多数以前的作品还原图像空间中的此类缺失细节。为了应对自然图像的高度多样性，他们要么依靠难以训练和容易训练和伪影的不稳定的甘体，要么诉诸于通常不可用的高分辨率（HR）图像中的明确参考。在这项工作中，我们提出了匹配SR（FEMASR）的功能，该功能在更紧凑的特征空间中恢复了现实的HR图像。与图像空间方法不同，我们的FEMASR通过将扭曲的LR图像{\ IT特征}与我们预读的HR先验中的无失真性HR对应物匹配来恢复HR图像，并解码匹配的功能以获得现实的HR图像。具体而言，我们的人力资源先验包含一个离散的特征代码簿及其相关的解码器，它们在使用量化的生成对抗网络（VQGAN）的HR图像上预估计。值得注意的是，我们在VQGAN中结合了一种新型的语义正则化，以提高重建图像的质量。对于功能匹配，我们首先提取由LR编码器组成的LR编码器的LR功能，然后遵循简单的最近邻居策略，将其与预读的代码簿匹配。特别是，我们为LR编码器配备了与解码器的残留快捷方式连接，这对于优化功能匹配损耗至关重要，还有助于补充可能的功能匹配错误。实验结果表明，我们的方法比以前的方法产生更现实的HR图像。代码以\ url {https://github.com/chaofengc/femasr}发布。

尽管深度学习使图像介绍方面取得了巨大的飞跃，但当前的方法通常无法综合现实的高频细节。在本文中，我们建议将超分辨率应用于粗糙的重建输出，以高分辨率进行精炼，然后将输出降低到原始分辨率。通过将高分辨率图像引入改进网络，我们的框架能够重建更多的细节，这些细节通常由于光谱偏置而被平滑 - 神经网络倾向于比高频更好地重建低频。为了协助培训大型高度孔洞的改进网络，我们提出了一种渐进的学习技术，其中缺失区域的大小随着培训的进行而增加。我们的缩放，完善和缩放策略，结合了高分辨率的监督和渐进学习，构成了一种框架 - 不合时宜的方法，用于增强高频细节，可应用于任何基于CNN的涂层方法。我们提供定性和定量评估以及消融分析，以显示我们方法的有效性。这种看似简单但功能强大的方法优于最先进的介绍方法。我们的代码可在https://github.com/google/zoom-to-inpaint中找到

图像缩小和升级是两个基本的重新划分操作。一旦图像缩小，由于信息丢失，难以通过Upscaling重建。为了使这两个过程更加兼容并提高重建性能，一些努力将它们模拟为联合编码解码任务，其中约束是缩小（即编码）的低分辨率（LR）图像必须保留原始视觉外观。要实现此约束，大多数方法通过使用原始高分辨率（HR）图像的双向较低的LR版本监督缩减模块。然而，这种双向LR引导可以是随后的上升（即解码）的次优，并限制最终的重建性能。在本文中，不直接应用LR引导，我们提出了一种额外的可逆性流动指导模块（FGM），其可以在较次编制的情况下将次要表示转换为视觉上可粘合图像并在升级期间重新转换。从FGM的可逆性受益，较次要的代表可以摆脱LR指导，不会打扰较低的升级过程。它允许我们删除对缩小模块的限制，并以端到端的方式优化缩减和上升模块。以这种方式，这两个模块可以协作以最大限度地提高HR重建性能。广泛的实验表明，所提出的方法可以在缩小和重建图像上实现最先进的（SOTA）性能。

Image super-resolution is a one-to-many problem, but most deep-learning based methods only provide one single solution to this problem. In this work, we tackle the problem of diverse super-resolution by reusing VD-VAE, a state-of-the art variational autoencoder (VAE). We find that the hierarchical latent representation learned by VD-VAE naturally separates the image low-frequency information, encoded in the latent groups at the top of the hierarchy, from the image high-frequency details, determined by the latent groups at the bottom of the latent hierarchy. Starting from this observation, we design a super-resolution model exploiting the specific structure of VD-VAE latent space. Specifically, we train an encoder to encode low-resolution images in the subset of VD-VAE latent space encoding the low-frequency information, and we combine this encoder with VD-VAE generative model to sample diverse super-resolved version of a low-resolution input. We demonstrate the ability of our method to generate diverse solutions to the super-resolution problem on face super-resolution with upsampling factors x4, x8, and x16.

尽管基准数据集的成功，但大多数先进的面部超分辨率模型在真实情况下表现不佳，因为真实图像与合成训练对之间的显着域间隙。为了解决这个问题，我们提出了一种用于野外面部超分辨率的新型域 - 自适应降级网络。该降级网络预测流场以及中间低分辨率图像。然后，通过翘曲中间图像来生成降级的对应物。利用捕获运动模糊的偏好，这种模型在保护原始图像和劣化之间保持身份一致性更好地执行。我们进一步提出了超分辨率网络的自我调节块。该块将输入图像作为条件术语，以有效地利用面部结构信息，从而消除了对显式前沿的依赖性，例如，面部地标或边界。我们的模型在Celeba和真实世界的面部数据集上实现了最先进的性能。前者展示了我们所提出的建筑的强大生成能力，而后者展示了现实世界中的良好的身份一致性和感知品质。

Reference-based image super-resolution (RefSR) is a promising SR branch and has shown great potential in overcoming the limitations of single image super-resolution. While previous state-of-the-art RefSR methods mainly focus on improving the efficacy and robustness of reference feature transfer, it is generally overlooked that a well reconstructed SR image should enable better SR reconstruction for its similar LR images when it is referred to as. Therefore, in this work, we propose a reciprocal learning framework that can appropriately leverage such a fact to reinforce the learning of a RefSR network. Besides, we deliberately design a progressive feature alignment and selection module for further improving the RefSR task. The newly proposed module aligns reference-input images at multi-scale feature spaces and performs reference-aware feature selection in a progressive manner, thus more precise reference features can be transferred into the input features and the network capability is enhanced. Our reciprocal learning paradigm is model-agnostic and it can be applied to arbitrary RefSR models. We empirically show that multiple recent state-of-the-art RefSR models can be consistently improved with our reciprocal learning paradigm. Furthermore, our proposed model together with the reciprocal learning strategy sets new state-of-the-art performances on multiple benchmarks.

通过将自然图像的复杂分布近似通过可逆神经网络（INN）近似于潜在空间中的简单拖延分布，已成功地用于生成图像超分辨率（SR）。这些模型可以使用潜在空间中的随机采样点从一个低分辨率（LR）输入中生成多个逼真的SR图像，从而模拟图像升级的不足的性质，其中多个高分辨率（HR）图像对应于同一LR。最近，INN中的可逆过程也通过双向图像重新缩放模型（如IRN和HCFLOW）成功使用，以优化降尺度和逆向上尺度的关节，从而显着改善了高尺度的图像质量。尽管它们也被优化用于图像降尺度，但图像降尺度的不良性质可以根据不同的插值内核和重新采样方法将一个HR图像缩小到多个LR图像。除了代表图像放大的不确定性的原始缩小潜在变量外，还引入了图像降压过程中的模型变化。这种双重可变变量增强功能适用于不同的图像重新缩放模型，并且在广泛的实验中显示，它可以始终如一地提高图像升级精度，而无需牺牲缩小的LR图像中的图像质量。它还显示可有效增强基于Inn的其他模型，用于图像恢复应用（例如图像隐藏）。