Recent advances in operator learning theory have improved our knowledge about learning maps between infinite dimensional spaces. However, for large-scale engineering problems such as concurrent multiscale simulation for mechanical properties, the training cost for the current operator learning methods is very high. The article presents a thorough analysis on the mathematical underpinnings of the operator learning paradigm and proposes a kernel learning method that maps between function spaces. We first provide a survey of modern kernel and operator learning theory, as well as discuss recent results and open problems. From there, the article presents an algorithm to how we can analytically approximate the piecewise constant functions on R for operator learning. This implies the potential feasibility of success of neural operators on clustered functions. Finally, a k-means clustered domain on the basis of a mechanistic response is considered and the Lippmann-Schwinger equation for micro-mechanical homogenization is solved. The article briefly discusses the mathematics of previous kernel learning methods and some preliminary results with those methods. The proposed kernel operator learning method uses graph kernel networks to come up with a mechanistic reduced order method for multiscale homogenization.

神经网络的经典发展主要集中在有限维欧基德空间或有限组之间的学习映射。我们提出了神经网络的概括，以学习映射无限尺寸函数空间之间的运算符。我们通过一类线性积分运算符和非线性激活函数的组成制定运营商的近似，使得组合的操作员可以近似复杂的非线性运算符。我们证明了我们建筑的普遍近似定理。此外，我们介绍了四类运算符参数化：基于图形的运算符，低秩运算符，基于多极图形的运算符和傅里叶运算符，并描述了每个用于用每个计算的高效算法。所提出的神经运营商是决议不变的：它们在底层函数空间的不同离散化之间共享相同的网络参数，并且可以用于零击超分辨率。在数值上，与现有的基于机器学习的方法，达西流程和Navier-Stokes方程相比，所提出的模型显示出卓越的性能，而与传统的PDE求解器相比，与现有的基于机器学习的方法有关的基于机器学习的方法。

Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics, and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than classical feed-forward neural networks, recurrent neural networks, and convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering, In this work, we review such methods and extend them for parametric studies as well as for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.

标准的神经网络可以近似一般的非线性操作员，要么通过数学运算符的组合（例如，在对流 - 扩散反应部分微分方程中）的组合，要么仅仅是黑匣子，例如黑匣子，例如一个系统系统。第一个神经操作员是基于严格的近似理论于2019年提出的深层操作员网络（DeepOnet）。从那时起，已经发布了其他一些较少的一般操作员，例如，基于图神经网络或傅立叶变换。对于黑匣子系统，对神经操作员的培训仅是数据驱动的，但是如果知道管理方程式可以在培训期间将其纳入损失功能，以开发物理知识的神经操作员。神经操作员可以用作设计问题，不确定性量化，自主系统以及几乎任何需要实时推断的应用程序中的代替代物。此外，通过将它们与相对轻的训练耦合，可以将独立的预训练deponets用作复杂多物理系统的组成部分。在这里，我们介绍了Deponet，傅立叶神经操作员和图神经操作员的评论，以及适当的扩展功能扩展，并突出显示它们在计算机械师中的各种应用中的实用性，包括多孔媒体，流体力学和固体机制， 。

神经运营商最近成为设计神经网络形式的功能空间之间的解决方案映射的流行工具。不同地，从经典的科学机器学习方法，以固定分辨率为输入参数的单个实例学习参数，神经运算符近似PDE系列的解决方案图。尽管他们取得了成功，但是神经运营商的用途迄今为止仅限于相对浅的神经网络，并限制了学习隐藏的管理法律。在这项工作中，我们提出了一种新颖的非局部神经运营商，我们将其称为非本体内核网络（NKN），即独立的分辨率，其特征在于深度神经网络，并且能够处理各种任务，例如学习管理方程和分类图片。我们的NKN源于神经网络的解释，作为离散的非局部扩散反应方程，在无限层的极限中，相当于抛物线非局部方程，其稳定性通过非本种载体微积分分析。与整体形式的神经运算符相似允许NKN捕获特征空间中的远程依赖性，而节点到节点交互的持续处理使NKNS分辨率独立于NKNS分辨率。与神经杂物中的相似性，在非本体意义上重新解释，并且层之间的稳定网络动态允许NKN的最佳参数从浅到深网络中的概括。这一事实使得能够使用浅层初始化技术。我们的测试表明，NKNS在学习管理方程和图像分类任务中占据基线方法，并概括到不同的分辨率和深度。

These notes were compiled as lecture notes for a course developed and taught at the University of the Southern California. They should be accessible to a typical engineering graduate student with a strong background in Applied Mathematics. The main objective of these notes is to introduce a student who is familiar with concepts in linear algebra and partial differential equations to select topics in deep learning. These lecture notes exploit the strong connections between deep learning algorithms and the more conventional techniques of computational physics to achieve two goals. First, they use concepts from computational physics to develop an understanding of deep learning algorithms. Not surprisingly, many concepts in deep learning can be connected to similar concepts in computational physics, and one can utilize this connection to better understand these algorithms. Second, several novel deep learning algorithms can be used to solve challenging problems in computational physics. Thus, they offer someone who is interested in modeling a physical phenomena with a complementary set of tools.

物理信息的神经网络（PINN）是神经网络（NNS），它们作为神经网络本身的组成部分编码模型方程，例如部分微分方程（PDE）。如今，PINN是用于求解PDE，分数方程，积分分化方程和随机PDE的。这种新颖的方法已成为一个多任务学习框架，在该框架中，NN必须在减少PDE残差的同时拟合观察到的数据。本文对PINNS的文献进行了全面的综述：虽然该研究的主要目标是表征这些网络及其相关的优势和缺点。该综述还试图将出版物纳入更广泛的基于搭配的物理知识的神经网络，这些神经网络构成了香草·皮恩（Vanilla Pinn）以及许多其他变体，例如物理受限的神经网络（PCNN），各种HP-VPINN，变量HP-VPINN，VPINN，VPINN，变体。和保守的Pinn（CPINN）。该研究表明，大多数研究都集中在通过不同的激活功能，梯度优化技术，神经网络结构和损耗功能结构来定制PINN。尽管使用PINN的应用范围广泛，但通过证明其在某些情况下比有限元方法（FEM）等经典数值技术更可行的能力，但仍有可能的进步，最著名的是尚未解决的理论问题。

Linear partial differential equations (PDEs) are an important, widely applied class of mechanistic models, describing physical processes such as heat transfer, electromagnetism, and wave propagation. In practice, specialized numerical methods based on discretization are used to solve PDEs. They generally use an estimate of the unknown model parameters and, if available, physical measurements for initialization. Such solvers are often embedded into larger scientific models or analyses with a downstream application such that error quantification plays a key role. However, by entirely ignoring parameter and measurement uncertainty, classical PDE solvers may fail to produce consistent estimates of their inherent approximation error. In this work, we approach this problem in a principled fashion by interpreting solving linear PDEs as physics-informed Gaussian process (GP) regression. Our framework is based on a key generalization of a widely-applied theorem for conditioning GPs on a finite number of direct observations to observations made via an arbitrary bounded linear operator. Crucially, this probabilistic viewpoint allows to (1) quantify the inherent discretization error; (2) propagate uncertainty about the model parameters to the solution; and (3) condition on noisy measurements. Demonstrating the strength of this formulation, we prove that it strictly generalizes methods of weighted residuals, a central class of PDE solvers including collocation, finite volume, pseudospectral, and (generalized) Galerkin methods such as finite element and spectral methods. This class can thus be directly equipped with a structured error estimate and the capability to incorporate uncertain model parameters and observations. In summary, our results enable the seamless integration of mechanistic models as modular building blocks into probabilistic models.

监督运营商学习是一种新兴机器学习范例，用于建模时空动态系统的演变和近似功能数据之间的一般黑盒关系的应用。我们提出了一种新颖的操作员学习方法，LOCA（学习操作员耦合注意力），激励了最近的注意机制的成功。在我们的体系结构中，输入函数被映射到有限的一组特征，然后按照依赖于输出查询位置的注意重量平均。通过将这些注意重量与积分变换一起耦合，LOCA能够明确地学习目标输出功能中的相关性，使我们能够近似非线性运算符，即使训练集测量中的输出功能的数量非常小。我们的配方伴随着拟议模型的普遍表现力的严格近似理论保证。经验上，我们在涉及普通和部分微分方程的系统管理的若干操作员学习场景中，评估LOCA的表现，以及黑盒气候预测问题。通过这些场景，我们展示了最先进的准确性，对噪声输入数据的鲁棒性以及在测试数据集上始终如一的错误传播，即使对于分发超出预测任务。

High-dimensional PDEs have been a longstanding computational challenge. We propose to solve highdimensional PDEs by approximating the solution with a deep neural network which is trained to satisfy the differential operator, initial condition, and boundary conditions. Our algorithm is meshfree, which is key since meshes become infeasible in higher dimensions. Instead of forming a mesh, the neural network is trained on batches of randomly sampled time and space points. The algorithm is tested on a class of high-dimensional free boundary PDEs, which we are able to accurately solve in up to 200 dimensions. The algorithm is also tested on a high-dimensional Hamilton-Jacobi-Bellman PDE and Burgers' equation. The deep learning algorithm approximates the general solution to the Burgers' equation for a continuum of different boundary conditions and physical conditions (which can be viewed as a high-dimensional space). We call the algorithm a "Deep Galerkin Method (DGM)" since it is similar in spirit to Galerkin methods, with the solution approximated by a neural network instead of a linear combination of basis functions. In addition, we prove a theorem regarding the approximation power of neural networks for a class of quasilinear parabolic PDEs.

矢量值随机变量的矩序列可以表征其定律。我们通过使用所谓的稳健签名矩来研究路径值随机变量（即随机过程）的类似问题。这使我们能够为随机过程定律得出最大平均差异类型的度量，并研究其在随机过程定律方面引起的拓扑。可以使用签名内核对该度量进行内核，从而有效地计算它。作为应用程序，我们为随机过程定律提供了非参数的两样本假设检验。

在本文中，我们在关注最先进的变压器中应用自我关注，这是第一次需要与部分微分方程相关的数据驱动的操作员学习问题。努力放在一起解释启发式，提高注意机制的功效。通过在希尔伯特空间中采用操作员近似理论，首次证明了Softmax归一化在缩放的点产品中的关注中足够但没有必要。在没有软墨中的情况下，可以证明线性化变换器变型的近似容量与Petrov-Galerkin投影层 - 明智相当，并且估计是相对于序列长度的独立性。提出了一种模仿Petrov-Galerkin投影的新层归一化方案，以允许缩放通过注意层传播，这有助于模型在具有非通信数据的操作员学习任务中实现显着准确性。最后，我们展示了三个操作员学习实验，包括粘虫汉堡方程，接口达西流程，以及逆接口系数识别问题。新提出的简单关注的算子学习者Galerkin变压器，在Softmax归一化的同行中，培训成本和评估准确性都显示出显着的改进。

Many applications, such as system identification, classification of time series, direct and inverse problems in partial differential equations, and uncertainty quantification lead to the question of approximation of a non-linear operator between metric spaces $\mathfrak{X}$ and $\mathfrak{Y}$. We study the problem of determining the degree of approximation of such operators on a compact subset $K_\mathfrak{X}\subset \mathfrak{X}$ using a finite amount of information. If $\mathcal{F}: K_\mathfrak{X}\to K_\mathfrak{Y}$, a well established strategy to approximate $\mathcal{F}(F)$ for some $F\in K_\mathfrak{X}$ is to encode $F$ (respectively, $\mathcal{F}(F)$) in terms of a finite number $d$ (repectively $m$) of real numbers. Together with appropriate reconstruction algorithms (decoders), the problem reduces to the approximation of $m$ functions on a compact subset of a high dimensional Euclidean space $\mathbb{R}^d$, equivalently, the unit sphere $\mathbb{S}^d$ embedded in $\mathbb{R}^{d+1}$. The problem is challenging because $d$, $m$, as well as the complexity of the approximation on $\mathbb{S}^d$ are all large, and it is necessary to estimate the accuracy keeping track of the inter-dependence of all the approximations involved. In this paper, we establish constructive methods to do this efficiently; i.e., with the constants involved in the estimates on the approximation on $\mathbb{S}^d$ being $\mathcal{O}(d^{1/6})$. We study different smoothness classes for the operators, and also propose a method for approximation of $\mathcal{F}(F)$ using only information in a small neighborhood of $F$, resulting in an effective reduction in the number of parameters involved.

我们为特殊神经网络架构，称为运营商复发性神经网络的理论分析，用于近似非线性函数，其输入是线性运算符。这些功能通常在解决方案算法中出现用于逆边值问题的问题。传统的神经网络将输入数据视为向量，因此它们没有有效地捕获与对应于这种逆问题中的数据的线性运算符相关联的乘法结构。因此，我们介绍一个类似标准的神经网络架构的新系列，但是输入数据在向量上乘法作用。由较小的算子出现在边界控制中的紧凑型操作员和波动方程的反边值问题分析，我们在网络中的选择权重矩阵中促进结构和稀疏性。在描述此架构后，我们研究其表示属性以及其近似属性。我们还表明，可以引入明确的正则化，其可以从所述逆问题的数学分析导出，并导致概括属性上的某些保证。我们观察到重量矩阵的稀疏性改善了概括估计。最后，我们讨论如何将运营商复发网络视为深度学习模拟，以确定诸如用于从边界测量的声波方程中重建所未知的WAVESTED的边界控制的算法算法。

众所周知，进食前馈神经网络的学习速度很慢，并且在深度学习应用中呈现了几十年的瓶颈。例如，广泛用于训练神经网络的基于梯度的学习算法在所有网络参数都必须迭代调整时往往会缓慢起作用。为了解决这个问题，研究人员和从业人员都尝试引入随机性来减少学习要求。基于Igelnik和Pao的原始结构，具有随机输入层的重量和偏见的单层神经网络在实践中取得了成功，但是缺乏必要的理论理由。在本文中，我们开始填补这一理论差距。我们提供了一个（校正的）严格证明，即Igelnik和PAO结构是连续函数在紧凑型域上连续函数的通用近似值，并且近似错误渐近地衰减，例如$ o（1/\ sqrt {n}）网络节点。然后，我们将此结果扩展到非反应设置，证明人们可以在$ n $的情况下实现任何理想的近似误差，而概率很大。我们进一步调整了这种随机神经网络结构，以近似欧几里得空间的平滑，紧凑的亚曼叶量的功能，从而在渐近和非催化形式的理论保证中提供了理论保证。最后，我们通过数值实验说明了我们在歧管上的结果。

在许多学科中，动态系统的数据信息预测模型的开发引起了广泛的兴趣。我们提出了一个统一的框架，用于混合机械和机器学习方法，以从嘈杂和部分观察到的数据中识别动态系统。我们将纯数据驱动的学习与混合模型进行比较，这些学习结合了不完善的域知识。我们的公式与所选的机器学习模型不可知，在连续和离散的时间设置中都呈现，并且与表现出很大的内存和错误的模型误差兼容。首先，我们从学习理论的角度研究无内存线性（W.R.T.参数依赖性）模型误差，从而定义了过多的风险和概括误差。对于沿阵行的连续时间系统，我们证明，多余的风险和泛化误差都通过与T的正方形介于T的术语（指定训练数据的时间间隔）的术语界定。其次，我们研究了通过记忆建模而受益的方案，证明了两类连续时间复发性神经网络（RNN）的通用近似定理：两者都可以学习与内存有关的模型误差。此外，我们将一类RNN连接到储层计算，从而将学习依赖性错误的学习与使用随机特征在Banach空间之间进行监督学习的最新工作联系起来。给出了数值结果（Lorenz '63，Lorenz '96多尺度系统），以比较纯粹的数据驱动和混合方法，发现混合方法较少，渴望数据较少，并且更有效。最后，我们从数值上证明了如何利用数据同化来从嘈杂，部分观察到的数据中学习隐藏的动态，并说明了通过这种方法和培训此类模型来表示记忆的挑战。

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

这项调查的目的是介绍对深神经网络的近似特性的解释性回顾。具体而言，我们旨在了解深神经网络如何以及为什么要优于其他经典线性和非线性近似方法。这项调查包括三章。在第1章中，我们回顾了深层网络及其组成非线性结构的关键思想和概念。我们通过在解决回归和分类问题时将其作为优化问题来形式化神经网络问题。我们简要讨论用于解决优化问题的随机梯度下降算法以及用于解决优化问题的后传播公式，并解决了与神经网络性能相关的一些问题，包括选择激活功能，成本功能，过度适应问题和正则化。在第2章中，我们将重点转移到神经网络的近似理论上。我们首先介绍多项式近似中的密度概念，尤其是研究实现连续函数的Stone-WeierStrass定理。然后，在线性近似的框架内，我们回顾了馈电网络的密度和收敛速率的一些经典结果，然后在近似Sobolev函数中进行有关深网络复杂性的最新发展。在第3章中，利用非线性近似理论，我们进一步详细介绍了深度和近似网络与其他经典非线性近似方法相比的近似优势。

在这项工作中，我们分析了不同程度的不同精度和分段多项式测试函数如何影响变异物理学知情神经网络（VPINN）的收敛速率，同时解决椭圆边界边界值问题，如何影响变异物理学知情神经网络（VPINN）的收敛速率。使用依靠INF-SUP条件的Petrov-Galerkin框架，我们在精确解决方案和合适的计算神经网络的合适的高阶分段插值之间得出了一个先验误差估计。数值实验证实了理论预测并突出了INF-SUP条件的重要性。我们的结果表明，以某种方式违反直觉，对于平滑解决方案，实现高衰减率的最佳策略在选择最低多项式程度的测试功能方面，同时使用适当高精度的正交公式。

Many scientific fields study data with an underlying structure that is a non-Euclidean space. Some examples include social networks in computational social sciences, sensor networks in communications, functional networks in brain imaging, regulatory networks in genetics, and meshed surfaces in computer graphics. In many applications, such geometric data are large and complex (in the case of social networks, on the scale of billions), and are natural targets for machine learning techniques. In particular, we would like to use deep neural networks, which have recently proven to be powerful tools for a broad range of problems from computer vision, natural language processing, and audio analysis. However, these tools have been most successful on data with an underlying Euclidean or grid-like structure, and in cases where the invariances of these structures are built into networks used to model them.Geometric deep learning is an umbrella term for emerging techniques attempting to generalize (structured) deep neural models to non-Euclidean domains such as graphs and manifolds. The purpose of this paper is to overview different examples of geometric deep learning problems and present available solutions, key difficulties, applications, and future research directions in this nascent field.