图表神经网络（GNN）已被广泛用于学习图形结构数据的矢量表示，并实现比传统方法更好的任务性能。 GNN的基础是消息传递过程，它将节点中的信息传播到其邻居。由于该过程每层进行一个步骤，因此节点之间的信息传播的范围在下层中很小，并且它朝向更高的层扩展。因此，GNN模型必须深入地捕获图中的全局结构信息。另一方面，众所周知，深入的GNN模型遭受性能下降，因为它们丢失了节点的本地信息，这对于良好的模型性能至关重要，通过许多消息传递步骤。在本研究中，我们提出了用于图形级分类任务的多级注意汇总（MLAP），这可以适应图表中的本地和全局结构信息。对于每个消息传递步骤，它具有注意池层，通过统一层方格图表示来计算最终图表示。 MLAP架构允许模型利用具有多个级别的本地图形的结构信息，因为它在由于过度的过天气丢失时保留了层面信息。我们的实验结果表明，与基线架构相比，MLAP架构提高了图形分类性能。此外，图表表示的分析表明，来自多个级别的地方的聚合信息确实具有提高学习图表表示的可怜的潜力。

Graph classification is an important area in both modern research and industry. Multiple applications, especially in chemistry and novel drug discovery, encourage rapid development of machine learning models in this area. To keep up with the pace of new research, proper experimental design, fair evaluation, and independent benchmarks are essential. Design of strong baselines is an indispensable element of such works. In this thesis, we explore multiple approaches to graph classification. We focus on Graph Neural Networks (GNNs), which emerged as a de facto standard deep learning technique for graph representation learning. Classical approaches, such as graph descriptors and molecular fingerprints, are also addressed. We design fair evaluation experimental protocol and choose proper datasets collection. This allows us to perform numerous experiments and rigorously analyze modern approaches. We arrive to many conclusions, which shed new light on performance and quality of novel algorithms. We investigate application of Jumping Knowledge GNN architecture to graph classification, which proves to be an efficient tool for improving base graph neural network architectures. Multiple improvements to baseline models are also proposed and experimentally verified, which constitutes an important contribution to the field of fair model comparison.

Advanced methods of applying deep learning to structured data such as graphs have been proposed in recent years. In particular, studies have focused on generalizing convolutional neural networks to graph data, which includes redefining the convolution and the downsampling (pooling) operations for graphs. The method of generalizing the convolution operation to graphs has been proven to improve performance and is widely used. However, the method of applying downsampling to graphs is still difficult to perform and has room for improvement. In this paper, we propose a graph pooling method based on selfattention. Self-attention using graph convolution allows our pooling method to consider both node features and graph topology. To ensure a fair comparison, the same training procedures and model architectures were used for the existing pooling methods and our method. The experimental results demonstrate that our method achieves superior graph classification performance on the benchmark datasets using a reasonable number of parameters.

随着各个领域的深度学习的巨大成功，图形神经网络（GNNS）也成为图形分类的主要方法。通过全局读出操作，只会聚合所有节点（或节点群集）表示，现有的GNN分类器获得输入图的图级表示，并使用表示来预测其类标签。但是，这种全局聚合不考虑每个节点的结构信息，这导致全局结构的信息丢失。特别地，它通过对所有节点表示来强制执行分类器的相同权重参数来限制辨别力;在实践中，他们中的每一个都有助于不同于其结构语义的目标类别。在这项工作中，我们提出了结构性语义读数（SSREAD）来总结位置级节点表示，这允许为分类模拟特定位置的权重参数，以及有效地捕获与全局结构相关的图形语义。给定输入图，SSREAD旨在通过使用其节点与结构原型之间的语义对齐来识别结构上有意义的位置，该结构原型编码每个位置的原型特征。结构原型经过优化，以最小化所有训练图的对准成本，而其他GNN参数训练以预测类标签。我们的实验结果表明，SSREAD显着提高了GNN分类器的分类性能和可解释性，同时兼容各种聚合函数，GNN架构和学习框架。

Many applications of machine learning require a model to make accurate predictions on test examples that are distributionally different from training ones, while task-specific labels are scarce during training. An effective approach to this challenge is to pre-train a model on related tasks where data is abundant, and then fine-tune it on a downstream task of interest. While pre-training has been effective in many language and vision domains, it remains an open question how to effectively use pre-training on graph datasets. In this paper, we develop a new strategy and self-supervised methods for pre-training Graph Neural Networks (GNNs). The key to the success of our strategy is to pre-train an expressive GNN at the level of individual nodes as well as entire graphs so that the GNN can learn useful local and global representations simultaneously. We systematically study pre-training on multiple graph classification datasets. We find that naïve strategies, which pre-train GNNs at the level of either entire graphs or individual nodes, give limited improvement and can even lead to negative transfer on many downstream tasks. In contrast, our strategy avoids negative transfer and improves generalization significantly across downstream tasks, leading up to 9.4% absolute improvements in ROC-AUC over non-pre-trained models and achieving state-of-the-art performance for molecular property prediction and protein function prediction.However, pre-training on graph datasets remains a hard challenge. Several key studies (

阐明并准确预测分子的吸毒性和生物活性在药物设计和发现中起关键作用，并且仍然是一个开放的挑战。最近，图神经网络（GNN）在基于图的分子属性预测方面取得了显着进步。但是，当前基于图的深度学习方法忽略了分子的分层信息以及特征通道之间的关系。在这项研究中，我们提出了一个精心设计的分层信息图神经网络框架（称为hignn），用于通过利用分子图和化学合成的可见的无限元素片段来预测分子特性。此外，首先在Hignn体系结构中设计了一个插件功能的注意块，以适应消息传递阶段后自适应重新校准原子特征。广泛的实验表明，Hignn在许多具有挑战性的药物发现相关基准数据集上实现了最先进的预测性能。此外，我们设计了一种分子碎片的相似性机制，以全面研究Hignn模型在子图水平上的解释性，表明Hignn作为强大的深度学习工具可以帮助化学家和药剂师识别出设计更好分子的关键分子，以设计更好的分子，以设计出所需的更好分子。属性或功能。源代码可在https://github.com/idruglab/hignn上公开获得。

Recently, graph neural networks (GNNs) have revolutionized the field of graph representation learning through effectively learned node embeddings, and achieved state-of-the-art results in tasks such as node classification and link prediction. However, current GNN methods are inherently flat and do not learn hierarchical representations of graphs-a limitation that is especially problematic for the task of graph classification, where the goal is to predict the label associated with an entire graph. Here we propose DIFFPOOL, a differentiable graph pooling module that can generate hierarchical representations of graphs and can be combined with various graph neural network architectures in an end-to-end fashion. DIFFPOOL learns a differentiable soft cluster assignment for nodes at each layer of a deep GNN, mapping nodes to a set of clusters, which then form the coarsened input for the next GNN layer. Our experimental results show that combining existing GNN methods with DIFFPOOL yields an average improvement of 5-10% accuracy on graph classification benchmarks, compared to all existing pooling approaches, achieving a new state-of-the-art on four out of five benchmark data sets.

增强图在正规化图形神经网络（GNNS）方面起着至关重要的作用，该图形以信息传递的形式利用沿图的边缘进行信息交换。由于其有效性，简单的边缘和节点操作（例如，添加和删除）已被广泛用于图表增强中。然而，这种常见的增强技术可以显着改变原始图的语义，从而导致过度侵略性增强，从而在GNN学习中拟合不足。为了解决掉落或添加图形边缘和节点引起的此问题，我们提出了SoftEdge，将随机权重分配给给定图的一部分以进行增强。 SoftEdge生成的合成图保持与原始图相同的节点及其连接性，从而减轻原始图的语义变化。我们从经验上表明，这种简单的方法获得了与流行节点和边缘操纵方法的卓越精度，并且具有明显的弹性，可抵御GNN深度的准确性降解。

近年来，图表表示学习越来越多地引起了越来越长的关注，特别是为了在节点和图表水平上学习对分类和建议任务的低维嵌入。为了能够在现实世界中的大规模图形数据上学习表示，许多研究专注于开发不同的抽样策略，以方便培训过程。这里，我们提出了一种自适应图策略驱动的采样模型（GPS），其中通过自适应相关计算实现了本地邻域中每个节点的影响。具体地，邻居的选择是由自适应策略算法指导的，直接贡献到消息聚合，节点嵌入更新和图级读出步骤。然后，我们从各种角度对图表分类任务进行全面的实验。我们所提出的模型在几个重要的基准测试中优于现有的3％-8％，实现了现实世界数据集的最先进的性能。

图表可以模拟实体之间的复杂交互，它在许多重要的应用程序中自然出现。这些应用程序通常可以投入到标准图形学习任务中，其中关键步骤是学习低维图表示。图形神经网络（GNN）目前是嵌入方法中最受欢迎的模型。然而，邻域聚合范例中的标准GNN患有区分\ EMPH {高阶}图形结构的有限辨别力，而不是\ EMPH {低位}结构。为了捕获高阶结构，研究人员求助于主题和开发的基于主题的GNN。然而，现有的基于主基的GNN仍然仍然遭受较少的辨别力的高阶结构。为了克服上述局限性，我们提出了一个新颖的框架，以更好地捕获高阶结构的新框架，铰接于我们所提出的主题冗余最小化操作员和注射主题组合的新颖框架。首先，MGNN生成一组节点表示W.R.T.每个主题。下一阶段是我们在图案中提出的冗余最小化，该主题在彼此相互比较并蒸馏出每个主题的特征。最后，MGNN通过组合来自不同图案的多个表示来执行节点表示的更新。特别地，为了增强鉴别的功率，MGNN利用重新注射功能来组合表示的函数w.r.t.不同的主题。我们进一步表明，我们的拟议体系结构增加了GNN的表现力，具有理论分析。我们展示了MGNN在节点分类和图形分类任务上的七个公共基准上表现出最先进的方法。

建议图表神经网络（GNNS）在不考虑训练和测试图之间的不可知分布的情况下，诱导GNN的泛化能力退化在分布外（OOD）设置。这种退化的根本原因是大多数GNN是基于I.I.D假设开发的。在这种设置中，GNN倾向于利用在培训中存在的微妙统计相关性用于预测，即使它是杂散的相关性。然而，这种杂散的相关性可能在测试环境中改变，导致GNN的失败。因此，消除了杂散相关的影响对于稳定的GNN来说是至关重要的。为此，我们提出了一个普遍的因果代表框架，称为稳定凝球。主要思想是首先从图数据中提取高级表示，并诉诸因因果推理的显着能力，以帮助模型摆脱虚假相关性。特别是，我们利用图形池化层以提取基于子图的表示作为高级表示。此外，我们提出了一种因果变量区别，以纠正偏置训练分布。因此，GNN将更多地集中在稳定的相关性上。对合成和现实世界ood图数据集的广泛实验良好地验证了所提出的框架的有效性，灵活性和可解释性。

近年来，基于Weisfeiler-Leman算法的算法和神经架构，是一个众所周知的Graph同构问题的启发式问题，它成为具有图形和关系数据的机器学习的强大工具。在这里，我们全面概述了机器学习设置中的算法的使用，专注于监督的制度。我们讨论了理论背景，展示了如何将其用于监督的图形和节点表示学习，讨论最近的扩展，并概述算法的连接（置换 - ）方面的神经结构。此外，我们概述了当前的应用和未来方向，以刺激进一步的研究。

Deep learning has been shown to be successful in a number of domains, ranging from acoustics, images, to natural language processing. However, applying deep learning to the ubiquitous graph data is non-trivial because of the unique characteristics of graphs. Recently, substantial research efforts have been devoted to applying deep learning methods to graphs, resulting in beneficial advances in graph analysis techniques. In this survey, we comprehensively review the different types of deep learning methods on graphs. We divide the existing methods into five categories based on their model architectures and training strategies: graph recurrent neural networks, graph convolutional networks, graph autoencoders, graph reinforcement learning, and graph adversarial methods. We then provide a comprehensive overview of these methods in a systematic manner mainly by following their development history. We also analyze the differences and compositions of different methods. Finally, we briefly outline the applications in which they have been used and discuss potential future research directions.

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

消息传递已作为设计图形神经网络（GNN）的有效工具的发展。但是，消息传递的大多数现有方法简单地简单或平均所有相邻的功能更新节点表示。它们受到两个问题的限制，即（i）缺乏可解释性来识别对GNN的预测重要的节点特征，以及（ii）特征过度混合，导致捕获长期依赖和无能为力的过度平滑问题在异质或低同质的下方处理图。在本文中，我们提出了一个节点级胶囊图神经网络（NCGNN），以通过改进的消息传递方案来解决这些问题。具体而言，NCGNN表示节点为节点级胶囊组，其中每个胶囊都提取其相应节点的独特特征。对于每个节点级胶囊，开发了一个新颖的动态路由过程，以适应适当的胶囊，以从设计的图形滤波器确定的子图中聚集。 NCGNN聚集仅有利的胶囊并限制无关的消息，以避免交互节点的过度混合特征。因此，它可以缓解过度平滑的问题，并通过同粒或异质的图表学习有效的节点表示。此外，我们提出的消息传递方案本质上是可解释的，并免于复杂的事后解释，因为图形过滤器和动态路由过程确定了节点特征的子集，这对于从提取的子分类中的模型预测最为重要。关于合成和现实图形的广泛实验表明，NCGNN可以很好地解决过度光滑的问题，并为半监视的节点分类产生更好的节点表示。它的表现优于同质和异质的艺术状态。

学习表达性分子表示对于促进分子特性的准确预测至关重要。尽管图形神经网络（GNNS）在分子表示学习中取得了显着进步，但它们通常面临诸如邻居探索，不足，过度光滑和过度阵列之类的局限性。同样，由于参数数量大，GNN通常具有较高的计算复杂性。通常，当面对相对大尺寸的图形或使用更深的GNN模型体系结构时，这种限制会出现或增加。克服这些问题的一个想法是将分子图简化为小型，丰富且有益的信息，这更有效，更具挑战性的培训GNN。为此，我们提出了一个新颖的分子图粗化框架，名为FUNQG利用函数组，作为分子的有影响力的构件来确定其性质，基于称为商图的图理论概念。通过实验，我们表明所产生的信息图比分子图小得多，因此是训练GNN的良好候选者。我们将FUNQG应用于流行的分子属性预测基准，然后比较所获得的数据集上的GNN体系结构的性能与原始数据集上的几个最先进的基线。通过实验，除了其参数数量和低计算复杂性的急剧减少之外，该方法除了其急剧减少之外，在各种数据集上的表现显着优于先前的基准。因此，FUNQG可以用作解决分子表示学习问题的简单，成本效益且可靠的方法。

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

变压器架构最近在图表表示学习中引起了人们的注意，因为它自然地克服了图神经网络（GNN）的几个局限性，避免了它们严格的结构电感偏置，而仅通过位置编码来编码图形结构。在这里，我们表明，具有位置编码的变压器生成的节点表示不一定捕获它们之间的结构相似性。为了解决这个问题，我们提出了结构感知的变压器，这是一类简单而灵活的图形变压器，建立在新的自我发项机制的基础上。这一新的自我注意力通过在计算注意力之前提取植根于每个节点的子图表来结合结构信息。我们提出了几种自动生成子图表表示的方法，并从理论上说明结果表示至少与子图表一样表现力。从经验上讲，我们的方法在五个图预测基准上实现了最先进的性能。我们的结构感知框架可以利用任何现有的GNN提取子图表表示，我们表明它系统地改善了相对于基本GNN模型的性能，成功地结合了GNN和变形金刚的优势。我们的代码可在https://github.com/borgwardtlab/sat上找到。

Recent deep learning approaches for representation learning on graphs follow a neighborhood aggregation procedure. We analyze some important properties of these models, and propose a strategy to overcome those. In particular, the range of "neighboring" nodes that a node's representation draws from strongly depends on the graph structure, analogous to the spread of a random walk. To adapt to local neighborhood properties and tasks, we explore an architecture -jumping knowledge (JK) networks -that flexibly leverages, for each node, different neighborhood ranges to enable better structure-aware representation. In a number of experiments on social, bioinformatics and citation networks, we demonstrate that our model achieves state-of-the-art performance. Furthermore, combining the JK framework with models like Graph Convolutional Networks, GraphSAGE and Graph Attention Networks consistently improves those models' performance.

Graphs are ubiquitous in nature and can therefore serve as models for many practical but also theoretical problems. For this purpose, they can be defined as many different types which suitably reflect the individual contexts of the represented problem. To address cutting-edge problems based on graph data, the research field of Graph Neural Networks (GNNs) has emerged. Despite the field's youth and the speed at which new models are developed, many recent surveys have been published to keep track of them. Nevertheless, it has not yet been gathered which GNN can process what kind of graph types. In this survey, we give a detailed overview of already existing GNNs and, unlike previous surveys, categorize them according to their ability to handle different graph types and properties. We consider GNNs operating on static and dynamic graphs of different structural constitutions, with or without node or edge attributes. Moreover, we distinguish between GNN models for discrete-time or continuous-time dynamic graphs and group the models according to their architecture. We find that there are still graph types that are not or only rarely covered by existing GNN models. We point out where models are missing and give potential reasons for their absence.