作为建模复杂关系的强大工具，HyperGraphs从图表学习社区中获得了流行。但是，深度刻画学习中的常用框架专注于具有边缘独立的顶点权重（EIVW）的超图，而无需考虑具有具有更多建模功率的边缘依赖性顶点权重（EDVWS）的超图。为了弥补这一点，我们提出了一般的超图光谱卷积（GHSC），这是一个通用学习框架，不仅可以处理EDVW和EIVW HyperGraphs，而且更重要的是，理论上可以明确地利用现有强大的图形卷积神经网络（GCNN）明确说明，从而很大程度上可以释放。超图神经网络的设计。在此框架中，给定的无向GCNN的图形拉普拉斯被统一的HyperGraph Laplacian替换，该统一的HyperGraph Laplacian通过将我们所定义的广义超透明牌与简单的无向图等同起来，从随机的步行角度将顶点权重信息替换。来自各个领域的广泛实验，包括社交网络分析，视觉目标分类和蛋白质学习，证明了拟议框架的最新性能。

图表是一个宇宙数据结构，广泛用于组织现实世界中的数据。像交通网络，社交和学术网络这样的各种实际网络网络可以由图表代表。近年来，目睹了在网络中代表顶点的快速发展，进入低维矢量空间，称为网络表示学习。表示学习可以促进图形数据上的新算法的设计。在本调查中，我们对网络代表学习的当前文献进行了全面审查。现有算法可以分为三组：浅埋模型，异构网络嵌入模型，图形神经网络的模型。我们为每个类别审查最先进的算法，并讨论这些算法之间的基本差异。调查的一个优点是，我们系统地研究了不同类别的算法底层的理论基础，这提供了深入的见解，以更好地了解网络表示学习领域的发展。

Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data. Recently, GCNs and subsequent variants have shown superior performance in various application areas on real-world datasets. Despite their success, most of the current GCN models are shallow, due to the over-smoothing problem.In this paper, we study the problem of designing and analyzing deep graph convolutional networks. We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques: Initial residual and Identity mapping. We provide theoretical and empirical evidence that the two techniques effectively relieves the problem of over-smoothing. Our experiments show that the deep GCNII model outperforms the state-of-the-art methods on various semi-and fullsupervised tasks. Code is available at https: //github.com/chennnM/GCNII.

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

图表神经网络（GNNS）在各种机器学习任务中获得了表示学习的提高。然而，应用邻域聚合的大多数现有GNN通常在图中的图表上执行不良，其中相邻的节点属于不同的类。在本文中，我们示出了在典型的异界图中，边缘可以被引导，以及是否像是处理边缘，也可以使它们过度地影响到GNN模型的性能。此外，由于异常的限制，节点对来自本地邻域之外的类似节点的消息非常有益。这些激励我们开发一个自适应地学习图表的方向性的模型，并利用潜在的长距离相关性节点之间。我们首先将图拉普拉斯概括为基于所提出的特征感知PageRank算法向数字化，该算法同时考虑节点之间的图形方向性和长距离特征相似性。然后，Digraph Laplacian定义了一个图形传播矩阵，导致一个名为{\ em diglaciangcn}的模型。基于此，我们进一步利用节点之间的通勤时间测量的节点接近度，以便在拓扑级别上保留节点的远距离相关性。具有不同级别的10个数据集的广泛实验，同意级别展示了我们在节点分类任务任务中对现有解决方案的有效性。

图形神经网络（GNNS）对图表上的半监督节点分类展示了卓越的性能，结果是它们能够同时利用节点特征和拓扑信息的能力。然而，大多数GNN隐含地假设曲线图中的节点和其邻居的标签是相同或一致的，其不包含在异质图中，其中链接节点的标签可能不同。因此，当拓扑是非信息性的标签预测时，普通的GNN可以显着更差，而不是在每个节点上施加多层Perceptrons（MLPS）。为了解决上述问题，我们提出了一种新的$ -laplacian基于GNN模型，称为$ ^ P $ GNN，其消息传递机制来自离散正则化框架，并且可以理论上解释为多项式图的近似值在$ p $ -laplacians的频谱域上定义过滤器。光谱分析表明，新的消息传递机制同时用作低通和高通滤波器，从而使$ ^ P $ GNNS对同性恋和异化图有效。关于现实世界和合成数据集的实证研究验证了我们的调查结果，并证明了$ ^ P $ GNN明显优于异交基准的几个最先进的GNN架构，同时在同性恋基准上实现竞争性能。此外，$ ^ p $ gnns可以自适应地学习聚合权重，并且对嘈杂的边缘具有强大。

图表表示学习有许多现实世界应用，从超级分辨率的成像，3D计算机视觉到药物重新扫描，蛋白质分类，社会网络分析。图表数据的足够表示对于图形结构数据的统计或机器学习模型的学习性能至关重要。在本文中，我们提出了一种用于图形数据的新型多尺度表示系统，称为抽取帧的图形数据，其在图表上形成了本地化的紧密框架。抽取的帧系统允许在粗粒链上存储图形数据表示，并在每个比例的多个尺度处处理图形数据，数据存储在子图中。基于此，我们通过建设性数据驱动滤波器组建立用于在多分辨率下分解和重建图数据的抽取G-Framewelet变换。图形帧构建基于基于链的正交基础，支持快速图傅里叶变换。由此，我们为抽取的G-Frameword变换或FGT提供了一种快速算法，该算法具有线性计算复杂度O（n），用于尺寸N的图表。用数值示例验证抽取的帧谱和FGT的理论，用于随机图形。现实世界应用的效果是展示的，包括用于交通网络的多分辨率分析，以及图形分类任务的图形神经网络。

在过去十年中，图形内核引起了很多关注，并在结构化数据上发展成为一种快速发展的学习分支。在过去的20年中，该领域发生的相当大的研究活动导致开发数十个图形内核，每个图形内核都对焦于图形的特定结构性质。图形内核已成功地成功地在广泛的域中，从社交网络到生物信息学。本调查的目标是提供图形内核的文献的统一视图。特别是，我们概述了各种图形内核。此外，我们对公共数据集的几个内核进行了实验评估，并提供了比较研究。最后，我们讨论图形内核的关键应用，并概述了一些仍有待解决的挑战。

Graph Neural Networks (graph NNs) are a promising deep learning approach for analyzing graph-structured data. However, it is known that they do not improve (or sometimes worsen) their predictive performance as we pile up many layers and add non-lineality. To tackle this problem, we investigate the expressive power of graph NNs via their asymptotic behaviors as the layer size tends to infinity. Our strategy is to generalize the forward propagation of a Graph Convolutional Network (GCN), which is a popular graph NN variant, as a specific dynamical system. In the case of a GCN, we show that when its weights satisfy the conditions determined by the spectra of the (augmented) normalized Laplacian, its output exponentially approaches the set of signals that carry information of the connected components and node degrees only for distinguishing nodes. Our theory enables us to relate the expressive power of GCNs with the topological information of the underlying graphs inherent in the graph spectra. To demonstrate this, we characterize the asymptotic behavior of GCNs on the Erdős -Rényi graph. We show that when the Erdős -Rényi graph is sufficiently dense and large, a broad range of GCNs on it suffers from the "information loss" in the limit of infinite layers with high probability. Based on the theory, we provide a principled guideline for weight normalization of graph NNs. We experimentally confirm that the proposed weight scaling enhances the predictive performance of GCNs in real data 1 .

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

Graph Neural Networks (GNNs) have been successfully applied in many applications in computer sciences. Despite the success of deep learning architectures in other domains, deep GNNs still underperform their shallow counterparts. There are many open questions about deep GNNs, but over-smoothing and over-squashing are perhaps the most intriguing issues. When stacking multiple graph convolutional layers, the over-smoothing and over-squashing problems arise and have been defined as the inability of GNNs to learn deep representations and propagate information from distant nodes, respectively. Even though the widespread definitions of both problems are similar, these phenomena have been studied independently. This work strives to understand the underlying relationship between over-smoothing and over-squashing from a topological perspective. We show that both problems are intrinsically related to the spectral gap of the Laplacian of the graph. Therefore, there is a trade-off between these two problems, i.e., we cannot simultaneously alleviate both over-smoothing and over-squashing. We also propose a Stochastic Jost and Liu curvature Rewiring (SJLR) algorithm based on a bound of the Ollivier's Ricci curvature. SJLR is less expensive than previous curvature-based rewiring methods while retaining fundamental properties. Finally, we perform a thorough comparison of SJLR with previous techniques to alleviate over-smoothing or over-squashing, seeking to gain a better understanding of both problems.

Many interesting problems in machine learning are being revisited with new deep learning tools. For graph-based semisupervised learning, a recent important development is graph convolutional networks (GCNs), which nicely integrate local vertex features and graph topology in the convolutional layers. Although the GCN model compares favorably with other state-of-the-art methods, its mechanisms are not clear and it still requires considerable amount of labeled data for validation and model selection. In this paper, we develop deeper insights into the GCN model and address its fundamental limits. First, we show that the graph convolution of the GCN model is actually a special form of Laplacian smoothing, which is the key reason why GCNs work, but it also brings potential concerns of oversmoothing with many convolutional layers. Second, to overcome the limits of the GCN model with shallow architectures, we propose both co-training and self-training approaches to train GCNs. Our approaches significantly improve GCNs in learning with very few labels, and exempt them from requiring additional labels for validation. Extensive experiments on benchmarks have verified our theory and proposals.

图形内核是历史上最广泛使用的图形分类任务的技术。然而，由于图的手工制作的组合特征，这些方法具有有限的性能。近年来，由于其性能卓越，图形神经网络（GNNS）已成为与下游图形相关任务的最先进的方法。大多数GNN基于消息传递神经网络（MPNN）框架。然而，最近的研究表明，MPNN不能超过Weisfeiler-Lehman（WL）算法在图形同构术中的力量。为了解决现有图形内核和GNN方法的限制，在本文中，我们提出了一种新的GNN框架，称为\ Texit {内核图形神经网络}（Kernnns），该框架将图形内核集成到GNN的消息传递过程中。通过卷积神经网络（CNNS）中的卷积滤波器的启发，KERGNNS采用可训练的隐藏图作为绘图过滤器，该绘图过滤器与子图组合以使用图形内核更新节点嵌入式。此外，我们表明MPNN可以被视为Kergnns的特殊情况。我们将Kergnns应用于多个与图形相关的任务，并使用交叉验证来与基准进行公平比较。我们表明，与现有的现有方法相比，我们的方法达到了竞争性能，证明了增加GNN的表现能力的可能性。我们还表明，KERGNNS中的训练有素的图形过滤器可以揭示数据集的本地图形结构，与传统GNN模型相比，显着提高了模型解释性。

图形卷积网络（GCN）类似于卷积神经网络（CNN），通常基于两个主要操作 - 空间和点的卷积。在GCN的背景下，与CNN不同，通常选择基于图形laplacian的预定的​​空间操作员，通常只允许学习点的操作。但是，学习有意义的空间操作员对于开发更具表现力的GCN以提高性能至关重要。在本文中，我们提出了PathGCN，这是一种从图上的随机路径学习空间操作员的新方法。我们分析方法的收敛及其与现有GCN的差异。此外，我们讨论了将我们所学的空间操作员与点卷积相结合的几种选择。我们在众多数据集上进行的广泛实验表明，通过适当地学习空间和角度的卷积，可以固有地避免诸如过度光滑的现象，并实现新的最先进的性能。

Graph clustering is a fundamental problem in unsupervised learning, with numerous applications in computer science and in analysing real-world data. In many real-world applications, we find that the clusters have a significant high-level structure. This is often overlooked in the design and analysis of graph clustering algorithms which make strong simplifying assumptions about the structure of the graph. This thesis addresses the natural question of whether the structure of clusters can be learned efficiently and describes four new algorithmic results for learning such structure in graphs and hypergraphs. All of the presented theoretical results are extensively evaluated on both synthetic and real-word datasets of different domains, including image classification and segmentation, migration networks, co-authorship networks, and natural language processing. These experimental results demonstrate that the newly developed algorithms are practical, effective, and immediately applicable for learning the structure of clusters in real-world data.

Graph convolution is the core of most Graph Neural Networks (GNNs) and usually approximated by message passing between direct (one-hop) neighbors. In this work, we remove the restriction of using only the direct neighbors by introducing a powerful, yet spatially localized graph convolution: Graph diffusion convolution (GDC). GDC leverages generalized graph diffusion, examples of which are the heat kernel and personalized PageRank. It alleviates the problem of noisy and often arbitrarily defined edges in real graphs. We show that GDC is closely related to spectral-based models and thus combines the strengths of both spatial (message passing) and spectral methods. We demonstrate that replacing message passing with graph diffusion convolution consistently leads to significant performance improvements across a wide range of models on both supervised and unsupervised tasks and a variety of datasets. Furthermore, GDC is not limited to GNNs but can trivially be combined with any graph-based model or algorithm (e.g. spectral clustering) without requiring any changes to the latter or affecting its computational complexity. Our implementation is available online. 1

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. We theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear classifier. Notably, our experimental evaluation demonstrates that these simplifications do not negatively impact accuracy in many downstream applications. Moreover, the resulting model scales to larger datasets, is naturally interpretable, and yields up to two orders of magnitude speedup over FastGCN.

最新提出的基于变压器的图形模型的作品证明了香草变压器用于图形表示学习的不足。要了解这种不足，需要研究变压器的光谱分析是否会揭示其对其表现力的见解。类似的研究已经确定，图神经网络（GNN）的光谱分析为其表现力提供了额外的观点。在这项工作中，我们系统地研究并建立了变压器领域中的空间和光谱域之间的联系。我们进一步提供了理论分析，并证明了变压器中的空间注意机制无法有效捕获所需的频率响应，因此，固有地限制了其在光谱空间中的表现力。因此，我们提出了feta，该框架旨在在整个图形频谱（即图形的实际频率成分）上进行注意力类似于空间空间中的注意力。经验结果表明，FETA在标准基准的所有任务中为香草变压器提供均匀的性能增益，并且可以轻松地扩展到具有低通特性的基于GNN的模型（例如GAT）。

尽管近期图形神经网络（GNN）成功，但常见的架构通常表现出显着的限制，包括对过天飞机，远程依赖性和杂散边缘的敏感性，例如，由于图形异常或对抗性攻击。至少部分地解决了一个简单的透明框架内的这些问题，我们考虑了一个新的GNN层系列，旨在模仿和整合两个经典迭代算法的更新规则，即近端梯度下降和迭代重复最小二乘（IRLS）。前者定义了一个可扩展的基础GNN架构，其免受过性的，而仍然可以通过允许任意传播步骤捕获远程依赖性。相反，后者产生了一种新颖的注意机制，该注意机制被明确地锚定到底层端到端能量函数，以及相对于边缘不确定性的稳定性。当结合时，我们获得了一个非常简单而强大的模型，我们在包括标准化基准，与异常扰动的图形，具有异化的图形和涉及远程依赖性的图形的不同方案的极其简单而强大的模型。在此过程中，我们与已明确为各个任务设计的SOTA GNN方法进行比较，实现竞争或卓越的节点分类准确性。我们的代码可以在https://github.com/fftyyy/twirls获得。

图表可以模拟实体之间的复杂交互，它在许多重要的应用程序中自然出现。这些应用程序通常可以投入到标准图形学习任务中，其中关键步骤是学习低维图表示。图形神经网络（GNN）目前是嵌入方法中最受欢迎的模型。然而，邻域聚合范例中的标准GNN患有区分\ EMPH {高阶}图形结构的有限辨别力，而不是\ EMPH {低位}结构。为了捕获高阶结构，研究人员求助于主题和开发的基于主题的GNN。然而，现有的基于主基的GNN仍然仍然遭受较少的辨别力的高阶结构。为了克服上述局限性，我们提出了一个新颖的框架，以更好地捕获高阶结构的新框架，铰接于我们所提出的主题冗余最小化操作员和注射主题组合的新颖框架。首先，MGNN生成一组节点表示W.R.T.每个主题。下一阶段是我们在图案中提出的冗余最小化，该主题在彼此相互比较并蒸馏出每个主题的特征。最后，MGNN通过组合来自不同图案的多个表示来执行节点表示的更新。特别地，为了增强鉴别的功率，MGNN利用重新注射功能来组合表示的函数w.r.t.不同的主题。我们进一步表明，我们的拟议体系结构增加了GNN的表现力，具有理论分析。我们展示了MGNN在节点分类和图形分类任务上的七个公共基准上表现出最先进的方法。