尽管在深度学习的其他应用领域中取得了非常深的架构，但流行的图神经网络是浅层模型。这降低了建模能力，并使模型无法捕获远程关系。浅设计的主要原因是过度平滑的，这导致节点状态随着深度的增加而变得更加相似。我们建立在GNNS和Pagerank之间的紧密联系的基础上，为此，个性化的Pagerank介绍了对个性化向量的考虑。通过这个想法，我们提出了个性化的Pagerank图神经网络（PPRGNN），该神经网络将图形卷积网络扩展到无限深度模型，该模型有机会将邻居聚集重置回每个迭代中的初始状态。我们引入了一个很好的解释调整，以重置重置并证明我们的方法与独特解决方案的收敛性，而无需放置任何限制，即使无限地进行了许多邻居聚集。与个性化的Pagerank一样，我们的结果不会过度光滑。在这样做的同时，在我们保持内存复杂性恒定的同时，时间复杂性保持线性，而与网络的深度无关，使其比较大图。我们从经验上展示了方法对各种节点和图形分类任务的有效性。在几乎所有情况下，PPRGNN优于可比较的方法。

Neural message passing algorithms for semi-supervised classification on graphs have recently achieved great success. However, for classifying a node these methods only consider nodes that are a few propagation steps away and the size of this utilized neighborhood is hard to extend. In this paper, we use the relationship between graph convolutional networks (GCN) and PageRank to derive an improved propagation scheme based on personalized PageRank. We utilize this propagation procedure to construct a simple model, personalized propagation of neural predictions (PPNP), and its fast approximation, APPNP. Our model's training time is on par or faster and its number of parameters on par or lower than previous models. It leverages a large, adjustable neighborhood for classification and can be easily combined with any neural network. We show that this model outperforms several recently proposed methods for semi-supervised classification in the most thorough study done so far for GCN-like models. Our implementation is available online. 1

Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data. Recently, GCNs and subsequent variants have shown superior performance in various application areas on real-world datasets. Despite their success, most of the current GCN models are shallow, due to the over-smoothing problem.In this paper, we study the problem of designing and analyzing deep graph convolutional networks. We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques: Initial residual and Identity mapping. We provide theoretical and empirical evidence that the two techniques effectively relieves the problem of over-smoothing. Our experiments show that the deep GCNII model outperforms the state-of-the-art methods on various semi-and fullsupervised tasks. Code is available at https: //github.com/chennnM/GCNII.

由于问题过度问题，大多数现有的图形神经网络只能使用其固有有限的聚合层捕获有限的依赖性。为了克服这一限制，我们提出了一种新型的图形卷积，称为图形隐式非线性扩散（GIND），该卷积隐含地可以访问邻居的无限啤酒花，同时具有非线性扩散的自适应聚集特征，以防止过度张开。值得注意的是，我们表明，学到的表示形式可以正式化为显式凸优化目标的最小化器。有了这个属性，我们可以从优化的角度从理论上表征GIND的平衡。更有趣的是，我们可以通过修改相应的优化目标来诱导新的结构变体。具体而言，我们可以将先前的特性嵌入到平衡中，并引入跳过连接以促进训练稳定性。广泛的实验表明，GIND擅长捕获长期依赖性，并且在具有非线性扩散的同粒细胞和异性图上表现良好。此外，我们表明，我们模型的优化引起的变体可以提高性能并提高训练稳定性和效率。结果，我们的GIND在节点级别和图形级任务上都获得了重大改进。

图形神经网络（GNNS）依赖于图形结构来定义聚合策略，其中每个节点通过与邻居的信息组合来更新其表示。已知GNN的限制是，随着层数的增加，信息被平滑，压扁并且节点嵌入式变得无法区分，对性能产生负面影响。因此，实用的GNN模型雇用了几层，只能在每个节点周围的有限邻域利用图形结构。不可避免地，实际的GNN不会根据图的全局结构捕获信息。虽然有几种研究GNNS的局限性和表达性，但是关于图形结构数据的实际应用的问题需要全局结构知识，仍然没有答案。在这项工作中，我们通过向几个GNN模型提供全球信息并观察其对下游性能的影响来认证解决这个问题。我们的研究结果表明，全球信息实际上可以为共同的图形相关任务提供显着的好处。我们进一步确定了一项新的正规化策略，导致所有考虑的任务的平均准确性提高超过5％。

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

图形神经网络（GNNS）对图表上的半监督节点分类展示了卓越的性能，结果是它们能够同时利用节点特征和拓扑信息的能力。然而，大多数GNN隐含地假设曲线图中的节点和其邻居的标签是相同或一致的，其不包含在异质图中，其中链接节点的标签可能不同。因此，当拓扑是非信息性的标签预测时，普通的GNN可以显着更差，而不是在每个节点上施加多层Perceptrons（MLPS）。为了解决上述问题，我们提出了一种新的$ -laplacian基于GNN模型，称为$ ^ P $ GNN，其消息传递机制来自离散正则化框架，并且可以理论上解释为多项式图的近似值在$ p $ -laplacians的频谱域上定义过滤器。光谱分析表明，新的消息传递机制同时用作低通和高通滤波器，从而使$ ^ P $ GNNS对同性恋和异化图有效。关于现实世界和合成数据集的实证研究验证了我们的调查结果，并证明了$ ^ P $ GNN明显优于异交基准的几个最先进的GNN架构，同时在同性恋基准上实现竞争性能。此外，$ ^ p $ gnns可以自适应地学习聚合权重，并且对嘈杂的边缘具有强大。

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

图表神经网络（GNNS）在各种机器学习任务中获得了表示学习的提高。然而，应用邻域聚合的大多数现有GNN通常在图中的图表上执行不良，其中相邻的节点属于不同的类。在本文中，我们示出了在典型的异界图中，边缘可以被引导，以及是否像是处理边缘，也可以使它们过度地影响到GNN模型的性能。此外，由于异常的限制，节点对来自本地邻域之外的类似节点的消息非常有益。这些激励我们开发一个自适应地学习图表的方向性的模型，并利用潜在的长距离相关性节点之间。我们首先将图拉普拉斯概括为基于所提出的特征感知PageRank算法向数字化，该算法同时考虑节点之间的图形方向性和长距离特征相似性。然后，Digraph Laplacian定义了一个图形传播矩阵，导致一个名为{\ em diglaciangcn}的模型。基于此，我们进一步利用节点之间的通勤时间测量的节点接近度，以便在拓扑级别上保留节点的远距离相关性。具有不同级别的10个数据集的广泛实验，同意级别展示了我们在节点分类任务任务中对现有解决方案的有效性。

作为建模复杂关系的强大工具，HyperGraphs从图表学习社区中获得了流行。但是，深度刻画学习中的常用框架专注于具有边缘独立的顶点权重（EIVW）的超图，而无需考虑具有具有更多建模功率的边缘依赖性顶点权重（EDVWS）的超图。为了弥补这一点，我们提出了一般的超图光谱卷积（GHSC），这是一个通用学习框架，不仅可以处理EDVW和EIVW HyperGraphs，而且更重要的是，理论上可以明确地利用现有强大的图形卷积神经网络（GCNN）明确说明，从而很大程度上可以释放。超图神经网络的设计。在此框架中，给定的无向GCNN的图形拉普拉斯被统一的HyperGraph Laplacian替换，该统一的HyperGraph Laplacian通过将我们所定义的广义超透明牌与简单的无向图等同起来，从随机的步行角度将顶点权重信息替换。来自各个领域的广泛实验，包括社交网络分析，视觉目标分类和蛋白质学习，证明了拟议框架的最新性能。

Graph neural networks have shown significant success in the field of graph representation learning. Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Nevertheless, one layer of these neighborhood aggregation methods only consider immediate neighbors, and the performance decreases when going deeper to enable larger receptive fields. Several recent studies attribute this performance deterioration to the over-smoothing issue, which states that repeated propagation makes node representations of different classes indistinguishable. In this work, we study this observation systematically and develop new insights towards deeper graph neural networks. First, we provide a systematical analysis on this issue and argue that the key factor compromising the performance significantly is the entanglement of representation transformation and propagation in current graph convolution operations. After decoupling these two operations, deeper graph neural networks can be used to learn graph node representations from larger receptive fields. We further provide a theoretical analysis of the above observation when building very deep models, which can serve as a rigorous and gentle description of the over-smoothing issue. Based on our theoretical and empirical analysis, we propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields. A set of experiments on citation, coauthorship, and co-purchase datasets have confirmed our analysis and insights and demonstrated the superiority of our proposed methods. CCS CONCEPTS• Mathematics of computing → Graph algorithms; • Computing methodologies → Artificial intelligence; Neural networks.

图形神经网络（GNNS）在各种基于图形的应用中显示了优势。大多数现有的GNNS假设图形结构的强大奇妙并应用邻居的置换不变本地聚合以学习每个节点的表示。然而，它们未能概括到异质图，其中大多数相邻节点具有不同的标签或特征，并且相关节点远处。最近的几项研究通过组合中央节点的隐藏表示（即，基于多跳的方法）的多个跳数来解决这个问题，或者基于注意力分数对相邻节点进行排序（即，基于排名的方法）来解决这个问题。结果，这些方法具有一些明显的限制。一方面，基于多跳的方法没有明确区分相关节点的大量多跳社区，导致严重的过平滑问题。另一方面，基于排名的模型不与结束任务进行联合优化节点排名，并导致次优溶液。在这项工作中，我们呈现图表指针神经网络（GPNN）来解决上述挑战。我们利用指针网络从大量的多跳邻域选择最相关的节点，这根据与中央节点的关系来构造有序序列。然后应用1D卷积以从节点序列中提取高级功能。 GPNN中的基于指针网络的Ranker是以端到端的方式与其他部件进行联合优化的。在具有异质图的六个公共节点分类数据集上进行了广泛的实验。结果表明，GPNN显着提高了最先进方法的分类性能。此外，分析还揭示了拟议的GPNN在过滤出无关邻居并减少过平滑的特权。

深度学习技术的普及更新了能够处理可以使用图形的复杂结构的神经结构的兴趣，由图形神经网络（GNN）的启发。我们将注意力集中在最初提出的Scarselli等人的GNN模型上。 2009，通过迭代扩散过程编码图表的节点的状态，即在学习阶段，必须在每个时期计算，直到达到学习状态转换功能的固定点，传播信息邻近节点。基于拉格朗日框架的约束优化，我们提出了一种在GNNS中学习的新方法。学习转换功能和节点状态是联合过程的结果，其中通过约束满足机制隐含地表达了状态会聚过程，避免了迭代巨头程序和网络展开。我们的计算结构在由权重组成的伴随空间中搜索拉格朗日的马鞍点，节点状态变量和拉格朗日乘法器。通过加速扩散过程的多个约束层进一步增强了该过程。实验分析表明，该方法在几个基准上的流行模型有利地比较。

提高GCN的深度（预计将允许更多表达性）显示出损害性能，尤其是在节点分类上。原因的主要原因在于过度平滑。过度平滑的问题将GCN的输出驱动到一个在节点之间包含有限的区别信息的空间，从而导致表现不佳。已经提出了一些有关完善GCN架构的作品，但理论上仍然未知这些改进是否能够缓解过度平衡。在本文中，我们首先从理论上分析了通用GCN如何与深度增加的作用，包括通用GCN，GCN，具有偏见，RESGCN和APPNP。我们发现所有这些模型都以通用过程为特征：所有节点融合到Cuboid。在该定理下，我们建议通过在每个训练时期随机去除一定数量的边缘来减轻过度光滑的状态。从理论上讲，Dropedge可以降低过度平滑的收敛速度，或者可以减轻尺寸崩溃引起的信息损失。对模拟数据集的实验评估已可视化不同GCN之间过度平滑的差异。此外，对几个真正的基准支持的广泛实验，这些实验始终如一地改善各种浅GCN和深度GCN的性能。

Over-fitting and over-smoothing are two main obstacles of developing deep Graph Convolutional Networks (GCNs) for node classification. In particular, over-fitting weakens the generalization ability on small dataset, while over-smoothing impedes model training by isolating output representations from the input features with the increase in network depth. This paper proposes DropEdge, a novel and flexible technique to alleviate both issues. At its core, DropEdge randomly removes a certain number of edges from the input graph at each training epoch, acting like a data augmenter and also a message passing reducer. Furthermore, we theoretically demonstrate that DropEdge either reduces the convergence speed of over-smoothing or relieves the information loss caused by it. More importantly, our DropEdge is a general skill that can be equipped with many other backbone models (e.g. GCN, ResGCN, GraphSAGE, and JKNet) for enhanced performance. Extensive experiments on several benchmarks verify that DropEdge consistently improves the performance on a variety of both shallow and deep GCNs. The effect of DropEdge on preventing over-smoothing is empirically visualized and validated as well. Codes are released on https://github.com/DropEdge/DropEdge.

Graph Neural Networks (GNNs) have been predominant for graph learning tasks; however, recent studies showed that a well-known graph algorithm, Label Propagation (LP), combined with a shallow neural network can achieve comparable performance to GNNs in semi-supervised node classification on graphs with high homophily. In this paper, we show that this approach falls short on graphs with low homophily, where nodes often connect to the nodes of the opposite classes. To overcome this, we carefully design a combination of a base predictor with LP algorithm that enjoys a closed-form solution as well as convergence guarantees. Our algorithm first learns the class compatibility matrix and then aggregates label predictions using LP algorithm weighted by class compatibilities. On a wide variety of benchmarks, we show that our approach achieves the leading performance on graphs with various levels of homophily. Meanwhile, it has orders of magnitude fewer parameters and requires less execution time. Empirical evaluations demonstrate that simple adaptations of LP can be competitive in semi-supervised node classification in both homophily and heterophily regimes.

图形神经网络（GNNS）由于图形数据的规模和模型参数的数量呈指数增长，因此限制了它们在实际应用中的效用，因此往往会遭受高计算成本。为此，最近的一些作品着重于用彩票假设（LTH）稀疏GNN，以降低推理成本，同时保持绩效水平。但是，基于LTH的方法具有两个主要缺点：1）它们需要对密集模型进行详尽且迭代的训练，从而产生了极大的训练计算成本，2）它们仅修剪图形结构和模型参数，但忽略了节点功能维度，存在大量冗余。为了克服上述局限性，我们提出了一个综合的图形渐进修剪框架，称为CGP。这是通过在一个训练过程中设计在训练图周期修剪范式上进行动态修剪GNN来实现的。与基于LTH的方法不同，提出的CGP方法不需要重新训练，这大大降低了计算成本。此外，我们设计了一个共同策略，以全面地修剪GNN的所有三个核心元素：图形结构，节点特征和模型参数。同时，旨在完善修剪操作，我们将重生过程引入我们的CGP框架，以重新建立修剪但重要的连接。提出的CGP通过在6个GNN体系结构中使用节点分类任务进行评估，包括浅层模型（GCN和GAT），浅但深度散发模型（SGC和APPNP）以及Deep Models（GCNII和RESGCN），总共有14个真实图形数据集，包括来自挑战性开放图基准的大规模图数据集。实验表明，我们提出的策略在匹配时大大提高了训练和推理效率，甚至超过了现有方法的准确性。

尽管近期图形神经网络（GNN）成功，但常见的架构通常表现出显着的限制，包括对过天飞机，远程依赖性和杂散边缘的敏感性，例如，由于图形异常或对抗性攻击。至少部分地解决了一个简单的透明框架内的这些问题，我们考虑了一个新的GNN层系列，旨在模仿和整合两个经典迭代算法的更新规则，即近端梯度下降和迭代重复最小二乘（IRLS）。前者定义了一个可扩展的基础GNN架构，其免受过性的，而仍然可以通过允许任意传播步骤捕获远程依赖性。相反，后者产生了一种新颖的注意机制，该注意机制被明确地锚定到底层端到端能量函数，以及相对于边缘不确定性的稳定性。当结合时，我们获得了一个非常简单而强大的模型，我们在包括标准化基准，与异常扰动的图形，具有异化的图形和涉及远程依赖性的图形的不同方案的极其简单而强大的模型。在此过程中，我们与已明确为各个任务设计的SOTA GNN方法进行比较，实现竞争或卓越的节点分类准确性。我们的代码可以在https://github.com/fftyyy/twirls获得。

Graph neural networks (GNNs), as the de-facto model class for representation learning on graphs, are built upon the multi-layer perceptrons (MLP) architecture with additional message passing layers to allow features to flow across nodes. While conventional wisdom largely attributes the success of GNNs to their advanced expressivity for learning desired functions on nodes' ego-graphs, we conjecture that this is \emph{not} the main cause of GNNs' superiority in node prediction tasks. This paper pinpoints the major source of GNNs' performance gain to their intrinsic generalization capabilities, by introducing an intermediate model class dubbed as P(ropagational)MLP, which is identical to standard MLP in training, and then adopt GNN's architecture in testing. Intriguingly, we observe that PMLPs consistently perform on par with (or even exceed) their GNN counterparts across ten benchmarks and different experimental settings, despite the fact that PMLPs share the same (trained) weights with poorly-performed MLP. This critical finding opens a door to a brand new perspective for understanding the power of GNNs, and allow bridging GNNs and MLPs for dissecting their generalization behaviors. As an initial step to analyze PMLP, we show its essential difference with MLP at infinite-width limit lies in the NTK feature map in the post-training stage. Moreover, though MLP and PMLP cannot extrapolate non-linear functions for extreme OOD data, PMLP has more freedom to generalize near the training support.

Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In this paper, we reduce this excess complexity through successively removing nonlinearities and collapsing weight matrices between consecutive layers. We theoretically analyze the resulting linear model and show that it corresponds to a fixed low-pass filter followed by a linear classifier. Notably, our experimental evaluation demonstrates that these simplifications do not negatively impact accuracy in many downstream applications. Moreover, the resulting model scales to larger datasets, is naturally interpretable, and yields up to two orders of magnitude speedup over FastGCN.