We present DeblurGAN, an end-to-end learned method for motion deblurring. The learning is based on a conditional GAN and the content loss . DeblurGAN achieves state-of-the art performance both in the structural similarity measure and visual appearance. The quality of the deblurring model is also evaluated in a novel way on a real-world problem -object detection on (de-)blurred images. The method is 5 times faster than the closest competitor -Deep-Deblur [25]. We also introduce a novel method for generating synthetic motion blurred images from sharp ones, allowing realistic dataset augmentation.The model, code and the dataset are available at https://github.com/KupynOrest/DeblurGAN

在本文中，我们介绍了一种快速运动脱棕色条件的生成对抗网络（FMD-CGAN），其有助于单个图像的盲运动去纹理。 FMD-CGAN在去修改图像后提供令人印象深刻的结构相似性和视觉外观。与其他深度神经网络架构一样，GAN也遭受大型模型大小（参数）和计算。在诸如移动设备和机器人等资源约束设备上部署模型并不容易。借助MobileNet基于MobileNet的架构，包括深度可分离卷积，我们降低了模型大小和推理时间，而不会丢失图像的质量。更具体地说，我们将模型大小与最近的竞争对手相比将3-60倍。由此产生的压缩去掩盖CGAN比其最接近的竞争对手更快，甚至定性和定量结果优于各种最近提出的最先进的盲运动去误紧模型。我们还可以使用我们的模型进行实时映像解擦干任务。标准数据集的当前实验显示了该方法的有效性。

We present a new end-to-end generative adversarial network (GAN) for single image motion deblurring, named DeblurGAN-v2, which considerably boosts state-of-the-art deblurring efficiency, quality, and flexibility. DeblurGAN-v2 is based on a relativistic conditional GAN with a doublescale discriminator. For the first time, we introduce the Feature Pyramid Network into deblurring, as a core building block in the generator of DeblurGAN-v2. It can flexibly work with a wide range of backbones, to navigate the balance between performance and efficiency. The plugin of sophisticated backbones (e.g., Inception-ResNet-v2) can lead to solid state-of-the-art deblurring. Meanwhile, with light-weight backbones (e.g., MobileNet and its variants), DeblurGAN-v2 reaches 10-100 times faster than the nearest competitors, while maintaining close to state-ofthe-art results, implying the option of real-time video deblurring. We demonstrate that DeblurGAN-v2 obtains very competitive performance on several popular benchmarks, in terms of deblurring quality (both objective and subjective), as well as efficiency. Besides, we show the architecture to be effective for general image restoration tasks too. Our codes, models and data are available at: https: //github.com/KupynOrest/DeblurGANv2.

Non-uniform blind deblurring for general dynamic scenes is a challenging computer vision problem as blurs arise not only from multiple object motions but also from camera shake, scene depth variation. To remove these complicated motion blurs, conventional energy optimization based methods rely on simple assumptions such that blur kernel is partially uniform or locally linear. Moreover, recent machine learning based methods also depend on synthetic blur datasets generated under these assumptions. This makes conventional deblurring methods fail to remove blurs where blur kernel is difficult to approximate or parameterize (e.g. object motion boundaries). In this work, we propose a multi-scale convolutional neural network that restores sharp images in an end-to-end manner where blur is caused by various sources. Together, we present multiscale loss function that mimics conventional coarse-to-fine approaches. Furthermore, we propose a new large-scale dataset that provides pairs of realistic blurry image and the corresponding ground truth sharp image that are obtained by a high-speed camera. With the proposed model trained on this dataset, we demonstrate empirically that our method achieves the state-of-the-art performance in dynamic scene deblurring not only qualitatively, but also quantitatively.

在本文中，我们介绍了基于稀疏学习的端到端生成的对抗网络（GAN），用于单幅图像盲运动去纹理，我们称为SL-Corpygan。在盲运运动去纹理中的第一次，我们提出了一种稀疏的Reset-块作为基于HTM（分层时间内存）的稀疏卷积层和可训练的空间池K-Winner的组合，以替换RESET中的非线性等非线性-Block的SL-Corpergan发电机。此外，与许多最先进的GaN的运动脱孔方法不同，将运动脱棕色作为线性端到端过程，我们从CompyGan的域名翻译能力中获取灵感，我们展示图像去孔可以是循环一致的，同时实现最佳定性结果。最后，我们在定性和定量上对流行的图像基准进行了广泛的实验，并在GoPro数据集上实现了38.087 dB的记录分布PSNR，比最新的去纹理方法优于5.377 dB。

Despite the breakthroughs in accuracy and speed of single image super-resolution using faster and deeper convolutional neural networks, one central problem remains largely unsolved: how do we recover the finer texture details when we super-resolve at large upscaling factors? The behavior of optimization-based super-resolution methods is principally driven by the choice of the objective function. Recent work has largely focused on minimizing the mean squared reconstruction error. The resulting estimates have high peak signal-to-noise ratios, but they are often lacking high-frequency details and are perceptually unsatisfying in the sense that they fail to match the fidelity expected at the higher resolution. In this paper, we present SRGAN, a generative adversarial network (GAN) for image superresolution (SR). To our knowledge, it is the first framework capable of inferring photo-realistic natural images for 4× upscaling factors. To achieve this, we propose a perceptual loss function which consists of an adversarial loss and a content loss. The adversarial loss pushes our solution to the natural image manifold using a discriminator network that is trained to differentiate between the super-resolved images and original photo-realistic images. In addition, we use a content loss motivated by perceptual similarity instead of similarity in pixel space. Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks. An extensive mean-opinion-score (MOS) test shows hugely significant gains in perceptual quality using SRGAN. The MOS scores obtained with SRGAN are closer to those of the original high-resolution images than to those obtained with any state-of-the-art method.

在文献中，粗细或缩放 - 重复性方法是从其低分辨率版本逐步恢复清洁图像，已成功用于单图像去孔。然而，现有方法的主要缺点是需要配对数据;即夏普尔图像对同一场景，这是一种复杂和繁琐的采集程序。此外，由于对损耗功能的强烈监督，此类网络的预先训练模型对训练期间的模糊强烈偏向，并且在推理时间内的新模糊内核面对时倾向于提供子最佳性能。为了解决上述问题，我们使用秤 - 自适应注意模块（Saam）提出了无监督的域特定的去孔。我们的网络不需要监督对进行训练，并且防夹机制主要由逆势丢失引导，从而使我们的网络适用于模糊功能的分布。给定模糊的输入图像，在训练期间我们的模型中使用相同图像的不同分辨率，Saam允许在整个分辨率上有效的信息流。对于特定规模的网络培训，Saam作为当前规模的函数参加较低的尺度功能。不同的消融研究表明，我们的粗细机制优于端到端无监督的模型，而Saam能够与文学中使用的注意力相比更好地参加。定性和定量比较（在无参考度量标准）表明我们的方法优于现有无监督的方法。

Single image super-resolution is the task of inferring a high-resolution image from a single low-resolution input. Traditionally, the performance of algorithms for this task is measured using pixel-wise reconstruction measures such as peak signal-to-noise ratio (PSNR) which have been shown to correlate poorly with the human perception of image quality. As a result, algorithms minimizing these metrics tend to produce over-smoothed images that lack highfrequency textures and do not look natural despite yielding high PSNR values.We propose a novel application of automated texture synthesis in combination with a perceptual loss focusing on creating realistic textures rather than optimizing for a pixelaccurate reproduction of ground truth images during training. By using feed-forward fully convolutional neural networks in an adversarial training setting, we achieve a significant boost in image quality at high magnification ratios. Extensive experiments on a number of datasets show the effectiveness of our approach, yielding state-of-the-art results in both quantitative and qualitative benchmarks.

在各种基于学习的图像恢复任务（例如图像降解和图像超分辨率）中，降解表示形式被广泛用于建模降解过程并处理复杂的降解模式。但是，在基于学习的图像deblurring中，它们的探索程度较低，因为在现实世界中挑战性的情况下，模糊内核估计不能很好地表现。我们认为，对于图像降低的降解表示形式是特别必要的，因为模糊模式通常显示出比噪声模式或高频纹理更大的变化。在本文中，我们提出了一个框架来学习模糊图像的空间自适应降解表示。提出了一种新颖的联合图像re毁和脱蓝色的学习过程，以提高降解表示的表现力。为了使学习的降解表示有效地启动和降解，我们提出了一个多尺度退化注入网络（MSDI-NET），以将它们集成到神经网络中。通过集成，MSDI-NET可以适应各种复杂的模糊模式。 GoPro和Realblur数据集上的实验表明，我们提出的具有学识渊博的退化表示形式的Deblurring框架优于最先进的方法，具有吸引人的改进。该代码在https://github.com/dasongli1/learning_degradation上发布。

盲目解构是一种在各种田地中产生的不良问题，从显微镜到天文学。问题的不良性质需要足够的前沿到达理想的解决方案。最近，已经表明，深度学习架构可以用作在无监督盲卷积优化期间的图像生成，然而甚至在单个图像上也呈现性能波动。我们建议使用Wiener-Deconvolulation在优化期间通过从高斯开始使用辅助内核估计来指导图像发生器在优化期间。我们观察到与低频特征相比，通过延迟再现去卷积的高频伪影。另外，图像发生器从模糊图像的速度再现解码图像的低频特征。我们在约束的优化框架中嵌入计算过程，并表明该方法在多个数据集中产生更高的稳定性和性能。此外，我们提供代码。

In single image deblurring, the "coarse-to-fine" scheme, i.e. gradually restoring the sharp image on different resolutions in a pyramid, is very successful in both traditional optimization-based methods and recent neural-networkbased approaches. In this paper, we investigate this strategy and propose a Scale-recurrent Network (SRN-DeblurNet) for this deblurring task. Compared with the many recent learning-based approaches in [25], it has a simpler network structure, a smaller number of parameters and is easier to train. We evaluate our method on large-scale deblurring datasets with complex motion. Results show that our method can produce better quality results than state-of-thearts, both quantitatively and qualitatively.

Generative adversarial networks (GANs) provide a way to learn deep representations without extensively annotated training data. They achieve this through deriving backpropagation signals through a competitive process involving a pair of networks. The representations that can be learned by GANs may be used in a variety of applications, including image synthesis, semantic image editing, style transfer, image super-resolution and classification. The aim of this review paper is to provide an overview of GANs for the signal processing community, drawing on familiar analogies and concepts where possible. In addition to identifying different methods for training and constructing GANs, we also point to remaining challenges in their theory and application.

Labels to Facade BW to Color Aerial to Map Labels to Street Scene Edges to Photo input output input input input input output output output output input output Day to Night Figure 1: Many problems in image processing, graphics, and vision involve translating an input image into a corresponding output image.These problems are often treated with application-specific algorithms, even though the setting is always the same: map pixels to pixels. Conditional adversarial nets are a general-purpose solution that appears to work well on a wide variety of these problems. Here we show results of the method on several. In each case we use the same architecture and objective, and simply train on different data.

随着深度学习（DL）的出现，超分辨率（SR）也已成为一个蓬勃发展的研究领域。然而，尽管结果有希望，但该领域仍然面临需要进一步研究的挑战，例如，允许灵活地采样，更有效的损失功能和更好的评估指标。我们根据最近的进步来回顾SR的域，并检查最新模型，例如扩散（DDPM）和基于变压器的SR模型。我们对SR中使用的当代策略进行了批判性讨论，并确定了有前途但未开发的研究方向。我们通过纳入该领域的最新发展，例如不确定性驱动的损失，小波网络，神经体系结构搜索，新颖的归一化方法和最新评估技术来补充先前的调查。我们还为整章中的模型和方法提供了几种可视化，以促进对该领域趋势的全球理解。最终，这篇综述旨在帮助研究人员推动DL应用于SR的界限。

在本文中，我们考虑了Defocus图像去缩合中的问题。以前的经典方法遵循两步方法，即首次散焦映射估计，然后是非盲目脱毛。在深度学习时代，一些研究人员试图解决CNN的这两个问题。但是，代表模糊级别的Defocus图的简单串联导致了次优性能。考虑到Defocus Blur的空间变体特性和Defocus Map中指示的模糊级别，我们采用Defocus Map作为条件指导来调整输入模糊图像而不是简单串联的特征。然后，我们提出了一个基于Defocus图的空间调制的简单但有效的网络。为了实现这一目标，我们设计了一个由三个子网络组成的网络，包括DeFocus Map估计网络，该网络将DeFocus Map编码为条件特征的条件网络以及根据条件功能执行空间动态调制的DeFocus Deblurring网络。此外，空间动态调制基于仿射变换函数，以调整输入模糊图像的特征。实验结果表明，与常用的公共测试数据集中的现有最新方法相比，我们的方法可以实现更好的定量和定性评估性能。

Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images, which is a domain-specific image restoration problem in the low-level computer vision area. The early face restoration methods mainly use statistic priors and degradation models, which are difficult to meet the requirements of real-world applications in practice. In recent years, face restoration has witnessed great progress after stepping into the deep learning era. However, there are few works to study deep learning-based face restoration methods systematically. Thus, this paper comprehensively surveys recent advances in deep learning techniques for face restoration. Specifically, we first summarize different problem formulations and analyze the characteristic of the face image. Second, we discuss the challenges of face restoration. Concerning these challenges, we present a comprehensive review of existing FR methods, including prior based methods and deep learning-based methods. Then, we explore developed techniques in the task of FR covering network architectures, loss functions, and benchmark datasets. We also conduct a systematic benchmark evaluation on representative methods. Finally, we discuss future directions, including network designs, metrics, benchmark datasets, applications,etc. We also provide an open-source repository for all the discussed methods, which is available at https://github.com/TaoWangzj/Awesome-Face-Restoration.

Multi-Scale and U-shaped Networks are widely used in various image restoration problems, including deblurring. Keeping in mind the wide range of applications, we present a comparison of these architectures and their effects on image deblurring. We also introduce a new block called as NFResblock. It consists of a Fast Fourier Transformation layer and a series of modified Non-Linear Activation Free Blocks. Based on these architectures and additions, we introduce NFResnet and NFResnet+, which are modified multi-scale and U-Net architectures, respectively. We also use three different loss functions to train these architectures: Charbonnier Loss, Edge Loss, and Frequency Reconstruction Loss. Extensive experiments on the Deep Video Deblurring dataset, along with ablation studies for each component, have been presented in this paper. The proposed architectures achieve a considerable increase in Peak Signal to Noise (PSNR) ratio and Structural Similarity Index (SSIM) value.

Convolutional Neural Network (CNN)-based image super-resolution (SR) has exhibited impressive success on known degraded low-resolution (LR) images. However, this type of approach is hard to hold its performance in practical scenarios when the degradation process is unknown. Despite existing blind SR methods proposed to solve this problem using blur kernel estimation, the perceptual quality and reconstruction accuracy are still unsatisfactory. In this paper, we analyze the degradation of a high-resolution (HR) image from image intrinsic components according to a degradation-based formulation model. We propose a components decomposition and co-optimization network (CDCN) for blind SR. Firstly, CDCN decomposes the input LR image into structure and detail components in feature space. Then, the mutual collaboration block (MCB) is presented to exploit the relationship between both two components. In this way, the detail component can provide informative features to enrich the structural context and the structure component can carry structural context for better detail revealing via a mutual complementary manner. After that, we present a degradation-driven learning strategy to jointly supervise the HR image detail and structure restoration process. Finally, a multi-scale fusion module followed by an upsampling layer is designed to fuse the structure and detail features and perform SR reconstruction. Empowered by such degradation-based components decomposition, collaboration, and mutual optimization, we can bridge the correlation between component learning and degradation modelling for blind SR, thereby producing SR results with more accurate textures. Extensive experiments on both synthetic SR datasets and real-world images show that the proposed method achieves the state-of-the-art performance compared to existing methods.

该论文根据近年来提出的基于经典定理和最先进的方法来分析图像过度的挑战性问题。通过光谱分析，我们从数学上显示了光谱正则化方法的有效性，并指出光谱滤波结果与正则化优化目标的解决方案之间的联系。对于诸如Image Deblurring之类的不适性问题，优化目标包含一个正则化项（也称为正则化功能），该项将我们的先验知识编码为解决方案。我们使用最大后验估计的想法来演示如何通过手工制作正规化术语。然后，我们指出了这种基于正则化方法的局限性，并介入基于神经网络的方法。基于Wasserstein生成对抗模型的想法，我们可以训练CNN学习正则化功能。这种数据驱动的方法能够捕获复杂性，这可能在分析上不可调节。此外，近年来，随着体系结构的改善，由于观察到模糊的观察，该网络已经能够近似于地面真相的图像。生成对抗网络（GAN）在此图像到图像翻译的想法上工作。我们分析了Orest Kupyn等人提出的DeBlurgan-V2方法。 [14] 2019年基于数值测试。并且，根据实验结果和我们的知识，我们提出了一些改进此方法的建议。

Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. The recently proposed Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only poor samples or fail to converge. We find that these problems are often due to the use of weight clipping in WGAN to enforce a Lipschitz constraint on the critic, which can lead to undesired behavior. We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models with continuous generators. We also achieve high quality generations on CIFAR-10 and LSUN bedrooms. † * Now at Google Brain † Code for our models is available at https://github.com/igul222/improved_wgan_training.