图表神经网络（GNNS）在各种机器学习任务中获得了表示学习的提高。然而，应用邻域聚合的大多数现有GNN通常在图中的图表上执行不良，其中相邻的节点属于不同的类。在本文中，我们示出了在典型的异界图中，边缘可以被引导，以及是否像是处理边缘，也可以使它们过度地影响到GNN模型的性能。此外，由于异常的限制，节点对来自本地邻域之外的类似节点的消息非常有益。这些激励我们开发一个自适应地学习图表的方向性的模型，并利用潜在的长距离相关性节点之间。我们首先将图拉普拉斯概括为基于所提出的特征感知PageRank算法向数字化，该算法同时考虑节点之间的图形方向性和长距离特征相似性。然后，Digraph Laplacian定义了一个图形传播矩阵，导致一个名为{\ em diglaciangcn}的模型。基于此，我们进一步利用节点之间的通勤时间测量的节点接近度，以便在拓扑级别上保留节点的远距离相关性。具有不同级别的10个数据集的广泛实验，同意级别展示了我们在节点分类任务任务中对现有解决方案的有效性。

Graph convolution is the core of most Graph Neural Networks (GNNs) and usually approximated by message passing between direct (one-hop) neighbors. In this work, we remove the restriction of using only the direct neighbors by introducing a powerful, yet spatially localized graph convolution: Graph diffusion convolution (GDC). GDC leverages generalized graph diffusion, examples of which are the heat kernel and personalized PageRank. It alleviates the problem of noisy and often arbitrarily defined edges in real graphs. We show that GDC is closely related to spectral-based models and thus combines the strengths of both spatial (message passing) and spectral methods. We demonstrate that replacing message passing with graph diffusion convolution consistently leads to significant performance improvements across a wide range of models on both supervised and unsupervised tasks and a variety of datasets. Furthermore, GDC is not limited to GNNs but can trivially be combined with any graph-based model or algorithm (e.g. spectral clustering) without requiring any changes to the latter or affecting its computational complexity. Our implementation is available online. 1

图表学习目的旨在将节点内容与图形结构集成以学习节点/图表示。然而，发现许多现有的图形学习方法在具有高异性级别的数据上不能很好地工作，这是不同类标签之间很大比例的边缘。解决这个问题的最新努力集中在改善消息传递机制上。但是，尚不清楚异质性是否确实会损害图神经网络（GNNS）的性能。关键是要展现一个节点与其直接邻居之间的关系，例如它们是异性还是同质性？从这个角度来看，我们在这里研究了杂质表示在披露连接节点之间的关系之前/之后的杂音表示的作用。特别是，我们提出了一个端到端框架，该框架既学习边缘的类型（即异性/同质性），并利用边缘类型的信息来提高图形神经网络的表现力。我们以两种不同的方式实施此框架。具体而言，为了避免通过异质边缘传递的消息，我们可以通过删除边缘分类器鉴定的异性边缘来优化图形结构。另外，可以利用有关异性邻居的存在的信息进行特征学习，因此，设计了一种混合消息传递方法来汇总同质性邻居，并根据边缘分类使异性邻居多样化。广泛的实验表明，在整个同质级别的多个数据集上，通过在多个数据集上提出的框架对GNN的绩效提高了显着提高。

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

作为建模复杂关系的强大工具，HyperGraphs从图表学习社区中获得了流行。但是，深度刻画学习中的常用框架专注于具有边缘独立的顶点权重（EIVW）的超图，而无需考虑具有具有更多建模功率的边缘依赖性顶点权重（EDVWS）的超图。为了弥补这一点，我们提出了一般的超图光谱卷积（GHSC），这是一个通用学习框架，不仅可以处理EDVW和EIVW HyperGraphs，而且更重要的是，理论上可以明确地利用现有强大的图形卷积神经网络（GCNN）明确说明，从而很大程度上可以释放。超图神经网络的设计。在此框架中，给定的无向GCNN的图形拉普拉斯被统一的HyperGraph Laplacian替换，该统一的HyperGraph Laplacian通过将我们所定义的广义超透明牌与简单的无向图等同起来，从随机的步行角度将顶点权重信息替换。来自各个领域的广泛实验，包括社交网络分析，视觉目标分类和蛋白质学习，证明了拟议框架的最新性能。

图形神经网络（GNNS）对图表上的半监督节点分类展示了卓越的性能，结果是它们能够同时利用节点特征和拓扑信息的能力。然而，大多数GNN隐含地假设曲线图中的节点和其邻居的标签是相同或一致的，其不包含在异质图中，其中链接节点的标签可能不同。因此，当拓扑是非信息性的标签预测时，普通的GNN可以显着更差，而不是在每个节点上施加多层Perceptrons（MLPS）。为了解决上述问题，我们提出了一种新的$ -laplacian基于GNN模型，称为$ ^ P $ GNN，其消息传递机制来自离散正则化框架，并且可以理论上解释为多项式图的近似值在$ p $ -laplacians的频谱域上定义过滤器。光谱分析表明，新的消息传递机制同时用作低通和高通滤波器，从而使$ ^ P $ GNNS对同性恋和异化图有效。关于现实世界和合成数据集的实证研究验证了我们的调查结果，并证明了$ ^ P $ GNN明显优于异交基准的几个最先进的GNN架构，同时在同性恋基准上实现竞争性能。此外，$ ^ p $ gnns可以自适应地学习聚合权重，并且对嘈杂的边缘具有强大。

图表可以模拟实体之间的复杂交互，它在许多重要的应用程序中自然出现。这些应用程序通常可以投入到标准图形学习任务中，其中关键步骤是学习低维图表示。图形神经网络（GNN）目前是嵌入方法中最受欢迎的模型。然而，邻域聚合范例中的标准GNN患有区分\ EMPH {高阶}图形结构的有限辨别力，而不是\ EMPH {低位}结构。为了捕获高阶结构，研究人员求助于主题和开发的基于主题的GNN。然而，现有的基于主基的GNN仍然仍然遭受较少的辨别力的高阶结构。为了克服上述局限性，我们提出了一个新颖的框架，以更好地捕获高阶结构的新框架，铰接于我们所提出的主题冗余最小化操作员和注射主题组合的新颖框架。首先，MGNN生成一组节点表示W.R.T.每个主题。下一阶段是我们在图案中提出的冗余最小化，该主题在彼此相互比较并蒸馏出每个主题的特征。最后，MGNN通过组合来自不同图案的多个表示来执行节点表示的更新。特别地，为了增强鉴别的功率，MGNN利用重新注射功能来组合表示的函数w.r.t.不同的主题。我们进一步表明，我们的拟议体系结构增加了GNN的表现力，具有理论分析。我们展示了MGNN在节点分类和图形分类任务上的七个公共基准上表现出最先进的方法。

Deep learning has been shown to be successful in a number of domains, ranging from acoustics, images, to natural language processing. However, applying deep learning to the ubiquitous graph data is non-trivial because of the unique characteristics of graphs. Recently, substantial research efforts have been devoted to applying deep learning methods to graphs, resulting in beneficial advances in graph analysis techniques. In this survey, we comprehensively review the different types of deep learning methods on graphs. We divide the existing methods into five categories based on their model architectures and training strategies: graph recurrent neural networks, graph convolutional networks, graph autoencoders, graph reinforcement learning, and graph adversarial methods. We then provide a comprehensive overview of these methods in a systematic manner mainly by following their development history. We also analyze the differences and compositions of different methods. Finally, we briefly outline the applications in which they have been used and discuss potential future research directions.

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

Graph neural networks have shown significant success in the field of graph representation learning. Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations. Nevertheless, one layer of these neighborhood aggregation methods only consider immediate neighbors, and the performance decreases when going deeper to enable larger receptive fields. Several recent studies attribute this performance deterioration to the over-smoothing issue, which states that repeated propagation makes node representations of different classes indistinguishable. In this work, we study this observation systematically and develop new insights towards deeper graph neural networks. First, we provide a systematical analysis on this issue and argue that the key factor compromising the performance significantly is the entanglement of representation transformation and propagation in current graph convolution operations. After decoupling these two operations, deeper graph neural networks can be used to learn graph node representations from larger receptive fields. We further provide a theoretical analysis of the above observation when building very deep models, which can serve as a rigorous and gentle description of the over-smoothing issue. Based on our theoretical and empirical analysis, we propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields. A set of experiments on citation, coauthorship, and co-purchase datasets have confirmed our analysis and insights and demonstrated the superiority of our proposed methods. CCS CONCEPTS• Mathematics of computing → Graph algorithms; • Computing methodologies → Artificial intelligence; Neural networks.

Graph neural networks (GNNs) have been increasingly deployed in various applications that involve learning on non-Euclidean data. However, recent studies show that GNNs are vulnerable to graph adversarial attacks. Although there are several defense methods to improve GNN robustness by eliminating adversarial components, they may also impair the underlying clean graph structure that contributes to GNN training. In addition, few of those defense models can scale to large graphs due to their high computational complexity and memory usage. In this paper, we propose GARNET, a scalable spectral method to boost the adversarial robustness of GNN models. GARNET first leverages weighted spectral embedding to construct a base graph, which is not only resistant to adversarial attacks but also contains critical (clean) graph structure for GNN training. Next, GARNET further refines the base graph by pruning additional uncritical edges based on probabilistic graphical model. GARNET has been evaluated on various datasets, including a large graph with millions of nodes. Our extensive experiment results show that GARNET achieves adversarial accuracy improvement and runtime speedup over state-of-the-art GNN (defense) models by up to 13.27% and 14.7x, respectively.

Graph classification is an important area in both modern research and industry. Multiple applications, especially in chemistry and novel drug discovery, encourage rapid development of machine learning models in this area. To keep up with the pace of new research, proper experimental design, fair evaluation, and independent benchmarks are essential. Design of strong baselines is an indispensable element of such works. In this thesis, we explore multiple approaches to graph classification. We focus on Graph Neural Networks (GNNs), which emerged as a de facto standard deep learning technique for graph representation learning. Classical approaches, such as graph descriptors and molecular fingerprints, are also addressed. We design fair evaluation experimental protocol and choose proper datasets collection. This allows us to perform numerous experiments and rigorously analyze modern approaches. We arrive to many conclusions, which shed new light on performance and quality of novel algorithms. We investigate application of Jumping Knowledge GNN architecture to graph classification, which proves to be an efficient tool for improving base graph neural network architectures. Multiple improvements to baseline models are also proposed and experimentally verified, which constitutes an important contribution to the field of fair model comparison.

图形神经网络（GNN）是用于建模图数据的流行机器学习方法。许多GNN在同质图上表现良好，同时在异质图上表现不佳。最近，一些研究人员将注意力转移到设计GNN，以通过调整消息传递机制或扩大消息传递的接收场来设计GNN。与从模型设计的角度来减轻异性疾病问题的现有作品不同，我们建议通过重新布线结构来从正交角度研究异质图，以减少异质性并使传统GNN的表现更好。通过全面的经验研究和分析，我们验证了重新布线方法的潜力。为了充分利用其潜力，我们提出了一种名为Deep Hertophilly Graph Rewiring（DHGR）的方法，以通过添加同粒子边缘和修剪异质边缘来重新线图。通过比较节点邻居的标签/特征 - 分布的相似性来确定重新布线的详细方法。此外，我们为DHGR设计了可扩展的实现，以确保高效率。 DHRG可以轻松地用作任何GNN的插件模块，即图形预处理步骤，包括同型和异性的GNN，以提高其在节点分类任务上的性能。据我们所知，这是研究图形的第一部重新绘图图形的作品。在11个公共图数据集上进行的广泛实验证明了我们提出的方法的优势。

图表是一个宇宙数据结构，广泛用于组织现实世界中的数据。像交通网络，社交和学术网络这样的各种实际网络网络可以由图表代表。近年来，目睹了在网络中代表顶点的快速发展，进入低维矢量空间，称为网络表示学习。表示学习可以促进图形数据上的新算法的设计。在本调查中，我们对网络代表学习的当前文献进行了全面审查。现有算法可以分为三组：浅埋模型，异构网络嵌入模型，图形神经网络的模型。我们为每个类别审查最先进的算法，并讨论这些算法之间的基本差异。调查的一个优点是，我们系统地研究了不同类别的算法底层的理论基础，这提供了深入的见解，以更好地了解网络表示学习领域的发展。

The core operation of current Graph Neural Networks (GNNs) is the aggregation enabled by the graph Laplacian or message passing, which filters the neighborhood information of nodes. Though effective for various tasks, in this paper, we show that they are potentially a problematic factor underlying all GNN models for learning on certain datasets, as they force the node representations similar, making the nodes gradually lose their identity and become indistinguishable. Hence, we augment the aggregation operations with their dual, i.e. diversification operators that make the node more distinct and preserve the identity. Such augmentation replaces the aggregation with a two-channel filtering process that, in theory, is beneficial for enriching the node representations. In practice, the proposed two-channel filters can be easily patched on existing GNN methods with diverse training strategies, including spectral and spatial (message passing) methods. In the experiments, we observe desired characteristics of the models and significant performance boost upon the baselines on 9 node classification tasks.

图形神经网络（GNN）已被证明可以实现竞争结果，以解决与图形相关的任务，例如节点和图形分类，链接预测和节点以及各种域中的图形群集。大多数GNN使用消息传递框架，因此称为MPNN。尽管有很有希望的结果，但据报道，MPNN会遭受过度平滑，过度阵型和不足的影响。文献中已经提出了图形重新布线和图形池作为解决这些局限性的解决方案。但是，大多数最先进的图形重新布线方法无法保留该图的全局拓扑，因此没有可区分（电感），并且需要调整超参数。在本文中，我们提出了Diffwire，这是一个在MPNN中进行图形重新布线的新型框架，它通过利用LOV \'ASZ绑定来原理，完全可区分且无参数。我们的方法通过提出两个新的，mpnns中的新的互补层来提供统一的图形重新布线：首先，ctlayer，一个学习通勤时间并将其用作边缘重新加权的相关函数；其次，Gaplayer是优化光谱差距的图层，具体取决于网络的性质和手头的任务。我们从经验上验证了我们提出的方法的价值，并使用基准数据集分别验证了这些层的每个层以进行图形分类。 Diffwire将通勤时间的可学习性汇集到相关的曲率定义，为发展更具表现力的MPNN的发展打开了大门。

Graph neural networks (GNNs) have shown remarkable performance on homophilic graph data while being far less impressive when handling non-homophilic graph data due to the inherent low-pass filtering property of GNNs. In general, since the real-world graphs are often a complex mixture of diverse subgraph patterns, learning a universal spectral filter on the graph from the global perspective as in most current works may still suffer from great difficulty in adapting to the variation of local patterns. On the basis of the theoretical analysis on local patterns, we rethink the existing spectral filtering methods and propose the \textbf{\underline{N}}ode-oriented spectral \textbf{\underline{F}}iltering for \textbf{\underline{G}}raph \textbf{\underline{N}}eural \textbf{\underline{N}}etwork (namely NFGNN). By estimating the node-oriented spectral filter for each node, NFGNN is provided with the capability of precise local node positioning via the generalized translated operator, thus discriminating the variations of local homophily patterns adaptively. Meanwhile, the utilization of re-parameterization brings a good trade-off between global consistency and local sensibility for learning the node-oriented spectral filters. Furthermore, we theoretically analyze the localization property of NFGNN, demonstrating that the signal after adaptive filtering is still positioned around the corresponding node. Extensive experimental results demonstrate that the proposed NFGNN achieves more favorable performance.

Graph Neural Networks (GNNs) have attracted increasing attention in recent years and have achieved excellent performance in semi-supervised node classification tasks. The success of most GNNs relies on one fundamental assumption, i.e., the original graph structure data is available. However, recent studies have shown that GNNs are vulnerable to the complex underlying structure of the graph, making it necessary to learn comprehensive and robust graph structures for downstream tasks, rather than relying only on the raw graph structure. In light of this, we seek to learn optimal graph structures for downstream tasks and propose a novel framework for semi-supervised classification. Specifically, based on the structural context information of graph and node representations, we encode the complex interactions in semantics and generate semantic graphs to preserve the global structure. Moreover, we develop a novel multi-measure attention layer to optimize the similarity rather than prescribing it a priori, so that the similarity can be adaptively evaluated by integrating measures. These graphs are fused and optimized together with GNN towards semi-supervised classification objective. Extensive experiments and ablation studies on six real-world datasets clearly demonstrate the effectiveness of our proposed model and the contribution of each component.

图形神经网络已成为从图形结构数据学习的不可缺少的工具之一，并且它们的实用性已在各种各样的任务中显示。近年来，建筑设计的巨大改进，导致各种预测任务的性能更好。通常，这些神经架构在同一层中使用可知的权重矩阵组合节点特征聚合和特征转换。这使得分析从各种跳过的节点特征和神经网络层的富有效力来挑战。由于不同的图形数据集显示在特征和类标签分布中的不同级别和异常级别，因此必须了解哪些特征对于没有任何先前信息的预测任务是重要的。在这项工作中，我们将节点特征聚合步骤和深度与图形神经网络分离，并经验分析了不同的聚合特征在预测性能中发挥作用。我们表明，并非通过聚合步骤生成的所有功能都很有用，并且通常使用这些较少的信息特征可能对GNN模型的性能有害。通过我们的实验，我们表明学习这些功能的某些子集可能会导致各种数据集的性能更好。我们建议使用Softmax作为常规器，并从不同跳距的邻居聚合的功能的“软选择器”;和L2 - GNN层的标准化。结合这些技术，我们呈现了一个简单浅的模型，特征选择图神经网络（FSGNN），并经验展示所提出的模型比九个基准数据集中的最先进的GNN模型实现了可比或甚至更高的准确性节点分类任务，具有显着的改进，可达51.1％。

最近，图形神经网络（GNN）通过利用图形结构和节点特征的知识来表现出图表表示的显着性能。但是，他们中的大多数都有两个主要限制。首先，GNN可以通过堆叠更多的层来学习高阶结构信息，但由于过度光滑的问题，无法处理较大的深度。其次，由于昂贵的计算成本和高内存使用情况，在大图上应用这些方法并不容易。在本文中，我们提出了节点自适应特征平滑（NAFS），这是一种简单的非参数方法，该方法构建了没有参数学习的节点表示。 NAFS首先通过特征平滑提取每个节点及其不同啤酒花的邻居的特征，然后自适应地结合了平滑的特征。此外，通过不同的平滑策略提取的平滑特征的合奏可以进一步增强构建的节点表示形式。我们在两个不同的应用程序方案上对四个基准数据集进行实验：节点群集和链接预测。值得注意的是，具有功能合奏的NAFS优于这些任务上最先进的GNN，并减轻上述大多数基于学习的GNN对应物的两个限制。