该论文根据近年来提出的基于经典定理和最先进的方法来分析图像过度的挑战性问题。通过光谱分析，我们从数学上显示了光谱正则化方法的有效性，并指出光谱滤波结果与正则化优化目标的解决方案之间的联系。对于诸如Image Deblurring之类的不适性问题，优化目标包含一个正则化项（也称为正则化功能），该项将我们的先验知识编码为解决方案。我们使用最大后验估计的想法来演示如何通过手工制作正规化术语。然后，我们指出了这种基于正则化方法的局限性，并介入基于神经网络的方法。基于Wasserstein生成对抗模型的想法，我们可以训练CNN学习正则化功能。这种数据驱动的方法能够捕获复杂性，这可能在分析上不可调节。此外，近年来，随着体系结构的改善，由于观察到模糊的观察，该网络已经能够近似于地面真相的图像。生成对抗网络（GAN）在此图像到图像翻译的想法上工作。我们分析了Orest Kupyn等人提出的DeBlurgan-V2方法。 [14] 2019年基于数值测试。并且，根据实验结果和我们的知识，我们提出了一些改进此方法的建议。

这本数字本书包含在物理模拟的背景下与深度学习相关的一切实际和全面的一切。尽可能多，所有主题都带有Jupyter笔记本的形式的动手代码示例，以便快速入门。除了标准的受监督学习的数据中，我们将看看物理丢失约束，更紧密耦合的学习算法，具有可微分的模拟，以及加强学习和不确定性建模。我们生活在令人兴奋的时期：这些方法具有从根本上改变计算机模拟可以实现的巨大潜力。

These notes were compiled as lecture notes for a course developed and taught at the University of the Southern California. They should be accessible to a typical engineering graduate student with a strong background in Applied Mathematics. The main objective of these notes is to introduce a student who is familiar with concepts in linear algebra and partial differential equations to select topics in deep learning. These lecture notes exploit the strong connections between deep learning algorithms and the more conventional techniques of computational physics to achieve two goals. First, they use concepts from computational physics to develop an understanding of deep learning algorithms. Not surprisingly, many concepts in deep learning can be connected to similar concepts in computational physics, and one can utilize this connection to better understand these algorithms. Second, several novel deep learning algorithms can be used to solve challenging problems in computational physics. Thus, they offer someone who is interested in modeling a physical phenomena with a complementary set of tools.

在这项工作中，我们研究了非盲目图像解卷积的问题，并提出了一种新的经常性网络架构，其导致高图像质量的竞争性恢复结果。通过现有大规模线性求解器的计算效率和稳健性的推动，我们设法将该问题的解决方案表达为一系列自适应非负数最小二乘问题的解决方案。这引发了我们提出的复发性最小二乘因解网络（RLSDN）架构，其包括在其输入和输出之间施加线性约束的隐式层。通过设计，我们的网络管理以同时服务两个重要的目的。首先，它隐含地模拟了可以充分表征这组自然图像的有效图像，而第二种是它恢复相应的最大后验（MAP）估计。近期最先进的方法的公开数据集的实验表明，我们提出的RLSDN方法可以实现所有测试方案的灰度和彩色图像的最佳报告性能。此外，我们介绍了一种新颖的培训策略，可以通过任何网络架构采用，这些架构涉及线性系统作为其管道的一部分的解决方案。我们的策略完全消除了线性求解器所需迭代的需要，因此，它在训练期间显着降低了内存占用。因此，这使得能够培训更深的网络架构，这可以进一步提高重建结果。

与CNN的分类，分割或对象检测相比，生成网络的目标和方法根本不同。最初，它们不是作为图像分析工具，而是生成自然看起来的图像。已经提出了对抗性训练范式来稳定生成方法，并已被证明是非常成功的 - 尽管绝不是第一次尝试。本章对生成对抗网络（GAN）的动机进行了基本介绍，并通​​过抽象基本任务和工作机制并得出了早期实用方法的困难来追溯其成功的道路。将显示进行更稳定的训练方法，也将显示出不良收敛及其原因的典型迹象。尽管本章侧重于用于图像生成和图像分析的gan，但对抗性训练范式本身并非特定于图像，并且在图像分析中也概括了任务。在将GAN与最近进入场景的进一步生成建模方法进行对比之前，将闻名图像语义分割和异常检测的架构示例。这将允许对限制的上下文化观点，但也可以对gans有好处。

这是一门专门针对STEM学生开发的介绍性机器学习课程。我们的目标是为有兴趣的读者提供基础知识，以在自己的项目中使用机器学习，并将自己熟悉术语作为进一步阅读相关文献的基础。在这些讲义中，我们讨论受监督，无监督和强化学习。注释从没有神经网络的机器学习方法的说明开始，例如原理分析，T-SNE，聚类以及线性回归和线性分类器。我们继续介绍基本和先进的神经网络结构，例如密集的进料和常规神经网络，经常性的神经网络，受限的玻尔兹曼机器，（变性）自动编码器，生成的对抗性网络。讨论了潜在空间表示的解释性问题，并使用梦和对抗性攻击的例子。最后一部分致力于加强学习，我们在其中介绍了价值功能和政策学习的基本概念。

生成的对抗网络由于研究人员的最新性能在生成新图像时仅使用目标分布的数据集，因此引起了研究人员的关注。已经表明，真实图像的频谱和假图像之间存在差异。由于傅立叶变换是一种徒图映射，因此说该模型在学习原始分布方面有一个重大问题是一个公平的结论。在这项工作中，我们研究了当前gan的架构和数学理论中提到的缺点的可能原因。然后，我们提出了一个新模型，以减少实际图像和假图像频谱之间的差异。为此，我们使用几何深度学习的蓝图为频域设计了一个全新的架构。然后，我们通过将原始数据的傅立叶域表示作为训练过程中的主要特征来表明生成图像的质量的有希望的改善。

随着深度学习（DL）的出现，超分辨率（SR）也已成为一个蓬勃发展的研究领域。然而，尽管结果有希望，但该领域仍然面临需要进一步研究的挑战，例如，允许灵活地采样，更有效的损失功能和更好的评估指标。我们根据最近的进步来回顾SR的域，并检查最新模型，例如扩散（DDPM）和基于变压器的SR模型。我们对SR中使用的当代策略进行了批判性讨论，并确定了有前途但未开发的研究方向。我们通过纳入该领域的最新发展，例如不确定性驱动的损失，小波网络，神经体系结构搜索，新颖的归一化方法和最新评估技术来补充先前的调查。我们还为整章中的模型和方法提供了几种可视化，以促进对该领域趋势的全球理解。最终，这篇综述旨在帮助研究人员推动DL应用于SR的界限。

Deep convolutional networks have become a popular tool for image generation and restoration. Generally, their excellent performance is imputed to their ability to learn realistic image priors from a large number of example images. In this paper, we show that, on the contrary, the structure of a generator network is sufficient to capture a great deal of low-level image statistics prior to any learning. In order to do so, we show that a randomly-initialized neural network can be used as a handcrafted prior with excellent results in standard inverse problems such as denoising, superresolution, and inpainting. Furthermore, the same prior can be used to invert deep neural representations to diagnose them, and to restore images based on flash-no flash input pairs.

这项正在进行的工作旨在为统计学习提供统一的介绍，从诸如GMM和HMM等经典模型到现代神经网络（如VAE和扩散模型）缓慢地构建。如今，有许多互联网资源可以孤立地解释这一点或新的机器学习算法，但是它们并没有（也不能在如此简短的空间中）将这些算法彼此连接起来，或者与统计模型的经典文献相连现代算法出现了。同样明显缺乏的是一个单一的符号系统，尽管对那些已经熟悉材料的人（如这些帖子的作者）不满意，但对新手的入境造成了重大障碍。同样，我的目的是将各种模型（尽可能）吸收到一个用于推理和学习的框架上，表明（以及为什么）如何以最小的变化将一个模型更改为另一个模型（其中一些是新颖的，另一些是文献中的）。某些背景当然是必要的。我以为读者熟悉基本的多变量计算，概率和统计以及线性代数。这本书的目标当然不是​​完整性，而是从基本知识到过去十年中极强大的新模型的直线路径或多或少。然后，目标是补充而不是替换，诸如Bishop的\ emph {模式识别和机器学习}之类的综合文本，该文本现在已经15岁了。

在计算机视觉和邻近字段中，已广泛研究了盲图片脱毛（BID）。投标的现代方法可以分为两类：使用统计推断和数值优化处理单个实例的单个实体方法，以及数据驱动的方法，这些方法可以直接训练深度学习模型来直接删除未来实例。数据驱动的方法可以摆脱得出准确的模型模型的困难，但从根本上受到培训数据的多样性和质量的限制 - 收集足够表达和现实的培训数据是一个坚定的挑战。在本文中，我们专注于保持竞争力和必不可少的单一稳定方法。但是，大多数此类方法没有规定如何处理未知内核大小和实质性噪音，从而排除了实际部署。实际上，我们表明，当核大小被明确指定时，几种最新的（SOTA）单位方法是不稳定的，并且/或噪声水平很高。从积极的一面来看，我们提出了一种实用的出价方法，该方法对这两者都是稳定的，这是同类的。我们的方法建立在最新的思想，即通过整合物理模型和结构深度神经网络而没有额外的培训数据来解决反问题。我们引入了几种关键修改以实现所需的稳定性。与SOTA单位结构以及数据驱动的方法相比，对标准合成数据集以及现实世界中的NTIRE2020和REALBLUR数据集进行了广泛的经验测试。我们方法的代码可在：\ url {https://github.com/sun-unm/blind-image-deblurring}中获得。

The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, auto-encoders, manifold learning, and deep networks. This motivates longer-term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation and manifold learning.

Deep neural networks provide unprecedented performance gains in many real world problems in signal and image processing. Despite these gains, future development and practical deployment of deep networks is hindered by their blackbox nature, i.e., lack of interpretability, and by the need for very large training sets. An emerging technique called algorithm unrolling or unfolding offers promise in eliminating these issues by providing a concrete and systematic connection between iterative algorithms that are used widely in signal processing and deep neural networks. Unrolling methods were first proposed to develop fast neural network approximations for sparse coding. More recently, this direction has attracted enormous attention and is rapidly growing both in theoretic investigations and practical applications. The growing popularity of unrolled deep networks is due in part to their potential in developing efficient, high-performance and yet interpretable network architectures from reasonable size training sets. In this article, we review algorithm unrolling for signal and image processing. We extensively cover popular techniques for algorithm unrolling in various domains of signal and image processing including imaging, vision and recognition, and speech processing. By reviewing previous works, we reveal the connections between iterative algorithms and neural networks and present recent theoretical results. Finally, we provide a discussion on current limitations of unrolling and suggest possible future research directions.

Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets.This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed-either explicitly or implicitly-to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, speed, and robustness. These claims are supported by extensive numerical experiments and a detailed error analysis.The specific benefits of randomized techniques depend on the computational environment. Consider the model problem of finding the k dominant components of the singular value decomposition of an m × n matrix. (i) For a dense input matrix, randomized algorithms require O(mn log(k)) floating-point operations (flops) in contrast with O(mnk) for classical algorithms. (ii) For a sparse input matrix, the flop count matches classical Krylov subspace methods, but the randomized approach is more robust and can easily be reorganized to exploit multi-processor architectures. (iii) For a matrix that is too large to fit in fast memory, the randomized techniques require only a constant number of passes over the data, as opposed to O(k) passes for classical algorithms. In fact, it is sometimes possible to perform matrix approximation with a single pass over the data.

We present a new end-to-end generative adversarial network (GAN) for single image motion deblurring, named DeblurGAN-v2, which considerably boosts state-of-the-art deblurring efficiency, quality, and flexibility. DeblurGAN-v2 is based on a relativistic conditional GAN with a doublescale discriminator. For the first time, we introduce the Feature Pyramid Network into deblurring, as a core building block in the generator of DeblurGAN-v2. It can flexibly work with a wide range of backbones, to navigate the balance between performance and efficiency. The plugin of sophisticated backbones (e.g., Inception-ResNet-v2) can lead to solid state-of-the-art deblurring. Meanwhile, with light-weight backbones (e.g., MobileNet and its variants), DeblurGAN-v2 reaches 10-100 times faster than the nearest competitors, while maintaining close to state-ofthe-art results, implying the option of real-time video deblurring. We demonstrate that DeblurGAN-v2 obtains very competitive performance on several popular benchmarks, in terms of deblurring quality (both objective and subjective), as well as efficiency. Besides, we show the architecture to be effective for general image restoration tasks too. Our codes, models and data are available at: https: //github.com/KupynOrest/DeblurGANv2.

在本文中，我们考虑使用Palentir在两个和三个维度中对分段常数对象的恢复和重建，这是相对于当前最新ART的显着增强的参数级别集（PALS）模型。本文的主要贡献是一种新的PALS公式，它仅需要一个单个级别的函数来恢复具有具有多个未知对比度的分段常数对象的场景。我们的模型比当前的多对抗性，多对象问题提供了明显的优势，所有这些问题都需要多个级别集并明确估计对比度大小。给定对比度上的上限和下限，我们的方法能够以任何对比度分布恢复对象，并消除需要知道给定场景中的对比度或其值的需求。我们提供了一个迭代过程，以找到这些空间变化的对比度限制。相对于使用径向基函数（RBF）的大多数PAL方法，我们的模型利用了非异型基函数，从而扩展了给定复杂性的PAL模型可以近似的形状类别。最后，Palentir改善了作为参数识别过程一部分所需的Jacobian矩阵的条件，因此通过控制PALS扩展系数的幅度来加速优化方法，固定基本函数的中心，以及参数映射到图像映射的唯一性，由新参数化提供。我们使用X射线计算机断层扫描，弥漫性光学断层扫描（DOT），Denoising，DeonConvolution问题的2D和3D变体证明了新方法的性能。应用于实验性稀疏CT数据和具有不同类型噪声的模拟数据，以进一步验证所提出的方法。

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics, and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than classical feed-forward neural networks, recurrent neural networks, and convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering, In this work, we review such methods and extend them for parametric studies as well as for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.

近年来，在诸如denoing，压缩感应，介入和超分辨率等反问题中使用深度学习方法的使用取得了重大进展。尽管这种作品主要是由实践算法和实验驱动的，但它也引起了各种有趣的理论问题。在本文中，我们调查了这一作品中一些突出的理论发展，尤其是生成先验，未经训练的神经网络先验和展开算法。除了总结这些主题中的现有结果外，我们还强调了一些持续的挑战和开放问题。

近年来，深度学习在图像重建方面取得了显着的经验成功。这已经促进了对关键用例中数据驱动方法的正确性和可靠性的精确表征的持续追求，例如在医学成像中。尽管基于深度学习的方法具有出色的性能和功效，但对其稳定性或缺乏稳定性的关注以及严重的实际含义。近年来，已经取得了重大进展，以揭示数据驱动的图像恢复方法的内部运作，从而挑战了其广泛认为的黑盒本质。在本文中，我们将为数据驱动的图像重建指定相关的融合概念，该概念将构成具有数学上严格重建保证的学习方法调查的基础。强调的一个例子是ICNN的作用，提供了将深度学习的力量与经典凸正则化理论相结合的可能性，用于设计被证明是融合的方法。这篇调查文章旨在通过提供对数据驱动的图像重建方法以及从业人员的理解，旨在通过提供可访问的融合概念的描述，并通过将一些现有的经验实践放在可靠的数学上，来推进我们对数据驱动图像重建方法的理解以及从业人员的了解。基础。