在处理表格数据时，基于回归和决策树的模型是一个流行的选择，因为与其他模型类别相比，它们在此类任务上提供了高精度及其易于应用。但是，在图形结构数据方面，当前的树学习算法不提供管理数据结构的工具，而不是依靠功能工程。在这项工作中，我们解决了上述差距，并引入了图形树（GTA），这是一个新的基于树的学习算法，旨在在图形上操作。 GTA既利用图形结构又利用了顶点的特征，并采用了一种注意机制，该机制允许决策专注于图形的子结构。我们分析了GTA模型，并表明它们比平原决策树更具表现力。我们还在多个图和节点预测基准上证明了GTA的好处。在这些实验中，GTA始终优于其他基于树的模型，并且通常优于其他类型的图形学习算法，例如图形神经网络（GNNS）和图核。最后，我们还为GTA提供了一种解释性机制，并证明它可以提供直观的解释。

Graph classification is an important area in both modern research and industry. Multiple applications, especially in chemistry and novel drug discovery, encourage rapid development of machine learning models in this area. To keep up with the pace of new research, proper experimental design, fair evaluation, and independent benchmarks are essential. Design of strong baselines is an indispensable element of such works. In this thesis, we explore multiple approaches to graph classification. We focus on Graph Neural Networks (GNNs), which emerged as a de facto standard deep learning technique for graph representation learning. Classical approaches, such as graph descriptors and molecular fingerprints, are also addressed. We design fair evaluation experimental protocol and choose proper datasets collection. This allows us to perform numerous experiments and rigorously analyze modern approaches. We arrive to many conclusions, which shed new light on performance and quality of novel algorithms. We investigate application of Jumping Knowledge GNN architecture to graph classification, which proves to be an efficient tool for improving base graph neural network architectures. Multiple improvements to baseline models are also proposed and experimentally verified, which constitutes an important contribution to the field of fair model comparison.

在过去十年中，图形内核引起了很多关注，并在结构化数据上发展成为一种快速发展的学习分支。在过去的20年中，该领域发生的相当大的研究活动导致开发数十个图形内核，每个图形内核都对焦于图形的特定结构性质。图形内核已成功地成功地在广泛的域中，从社交网络到生物信息学。本调查的目标是提供图形内核的文献的统一视图。特别是，我们概述了各种图形内核。此外，我们对公共数据集的几个内核进行了实验评估，并提供了比较研究。最后，我们讨论图形内核的关键应用，并概述了一些仍有待解决的挑战。

消息传递神经网络（MPNNS）是由于其简单性和可扩展性而大部分地进行图形结构数据的深度学习的领先架构。不幸的是，有人认为这些架构的表现力有限。本文提出了一种名为Comifariant Subgraph聚合网络（ESAN）的新颖框架来解决这个问题。我们的主要观察是，虽然两个图可能无法通过MPNN可区分，但它们通常包含可区分的子图。因此，我们建议将每个图形作为由某些预定义策略导出的一组子图，并使用合适的等分性架构来处理它。我们为图同构同构同构造的1立维Weisfeiler-Leman（1-WL）测试的新型变体，并在这些新的WL变体方面证明了ESAN的表达性下限。我们进一步证明，我们的方法增加了MPNNS和更具表现力的架构的表现力。此外，我们提供了理论结果，描述了设计选择诸如子图选择政策和等效性神经结构的设计方式如何影响我们的架构的表现力。要处理增加的计算成本，我们提出了一种子图采样方案，可以将其视为我们框架的随机版本。关于真实和合成数据集的一套全面的实验表明，我们的框架提高了流行的GNN架构的表现力和整体性能。

近年来，基于Weisfeiler-Leman算法的算法和神经架构，是一个众所周知的Graph同构问题的启发式问题，它成为具有图形和关系数据的机器学习的强大工具。在这里，我们全面概述了机器学习设置中的算法的使用，专注于监督的制度。我们讨论了理论背景，展示了如何将其用于监督的图形和节点表示学习，讨论最近的扩展，并概述算法的连接（置换 - ）方面的神经结构。此外，我们概述了当前的应用和未来方向，以刺激进一步的研究。

Graph神经网络（GNN）最近已成为使用图的机器学习的主要范式。对GNNS的研究主要集中于消息传递神经网络（MPNNS）的家族。与同构的Weisfeiler-Leman（WL）测试类似，这些模型遵循迭代的邻域聚合过程以更新顶点表示，并通过汇总顶点表示来更新顶点图表。尽管非常成功，但在过去的几年中，对MPNN进行了深入的研究。因此，需要新颖的体系结构，这将使该领域的研究能够脱离MPNN。在本文中，我们提出了一个新的图形神经网络模型，即所谓的$ \ pi $ -gnn，该模型学习了每个图的“软”排列（即双随机）矩阵，从而将所有图形投影到一个共同的矢量空间中。学到的矩阵在输入图的顶点上强加了“软”顺序，并基于此顺序，将邻接矩阵映射到向量中。这些向量可以被送入完全连接或卷积的层，以应对监督的学习任务。在大图的情况下，为了使模型在运行时间和记忆方面更有效，我们进一步放松了双随机矩阵，以使其排列随机矩阵。我们从经验上评估了图形分类和图形回归数据集的模型，并表明它与最新模型达到了性能竞争。

图形神经网络（GNN）已成为一种学习关系数据的强大技术。由于他们执行的消息传递步骤数量相对有限 - 因此一个较小的接收领域，人们对通过结合基础图的结构方面来提高其表现力引起了极大的兴趣。在本文中，我们探讨了亲和力措施作为图形神经网络中的特征，特别是由随机步行引起的措施，包括有效的阻力，击球和通勤时间。我们根据这些功能提出消息传递网络，并评估其在各种节点和图形属性预测任务上的性能。我们的体系结构具有较低的计算复杂性，而我们的功能对于基础图的排列不变。我们计算的措施使网络可以利用图表的连接性能，从而使我们能够超过相关的基准，用于各种任务，通常具有更少的消息传递步骤。在OGB-LSC-PCQM4MV1的最大公共图形回归数据集之一中，我们在编写时获得了最著名的单模验证MAE。

图形内核是历史上最广泛使用的图形分类任务的技术。然而，由于图的手工制作的组合特征，这些方法具有有限的性能。近年来，由于其性能卓越，图形神经网络（GNNS）已成为与下游图形相关任务的最先进的方法。大多数GNN基于消息传递神经网络（MPNN）框架。然而，最近的研究表明，MPNN不能超过Weisfeiler-Lehman（WL）算法在图形同构术中的力量。为了解决现有图形内核和GNN方法的限制，在本文中，我们提出了一种新的GNN框架，称为\ Texit {内核图形神经网络}（Kernnns），该框架将图形内核集成到GNN的消息传递过程中。通过卷积神经网络（CNNS）中的卷积滤波器的启发，KERGNNS采用可训练的隐藏图作为绘图过滤器，该绘图过滤器与子图组合以使用图形内核更新节点嵌入式。此外，我们表明MPNN可以被视为Kergnns的特殊情况。我们将Kergnns应用于多个与图形相关的任务，并使用交叉验证来与基准进行公平比较。我们表明，与现有的现有方法相比，我们的方法达到了竞争性能，证明了增加GNN的表现能力的可能性。我们还表明，KERGNNS中的训练有素的图形过滤器可以揭示数据集的本地图形结构，与传统GNN模型相比，显着提高了模型解释性。

尽管（消息通话）图形神经网络在图形或一般关系数据上近似置换量等函数方面具有明显的局限性，但更具表现力的高阶图神经网络不会扩展到大图。他们要么在$ k $ - 订单张量子上操作，要么考虑所有$ k $ - 节点子图，这意味着在内存需求中对$ k $的指数依赖，并且不适合图形的稀疏性。通过为图同构问题引入新的启发式方法，我们设计了一类通用的，置换式的图形网络，与以前的体系结构不同，该网络在表达性和可伸缩性之间提供了细粒度的控制，并适应了图的稀疏性。这些体系结构与监督节点和图形级别的标准高阶网络以及回归体系中的标准高阶图网络相比大大减少了计算时间，同时在预测性能方面显着改善了标准图神经网络和图形内核体系结构。

我们提出了一个新的图形神经网络，我们称为AgentNet，该网络专为图形级任务而设计。 AgentNet的灵感来自子宫性算法，具有独立于图形大小的计算复杂性。代理Net的体系结构从根本上与已知图神经网络的体系结构不同。在AgentNet中，一些受过训练的\ textit {神经代理}智能地行走图，然后共同决定输出。我们提供了对AgentNet的广泛理论分析：我们表明，代理可以学会系统地探索其邻居，并且AgentNet可以区分某些甚至3-WL无法区分的结构。此外，AgentNet能够将任何两个图形分开，这些图在子图方面完全不同。我们通过在难以辨认的图和现实图形分类任务上进行合成实验来确认这些理论结果。在这两种情况下，我们不仅与标准GNN相比，而且与计算更昂贵的GNN扩展相比。

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

图表可以模拟实体之间的复杂交互，它在许多重要的应用程序中自然出现。这些应用程序通常可以投入到标准图形学习任务中，其中关键步骤是学习低维图表示。图形神经网络（GNN）目前是嵌入方法中最受欢迎的模型。然而，邻域聚合范例中的标准GNN患有区分\ EMPH {高阶}图形结构的有限辨别力，而不是\ EMPH {低位}结构。为了捕获高阶结构，研究人员求助于主题和开发的基于主题的GNN。然而，现有的基于主基的GNN仍然仍然遭受较少的辨别力的高阶结构。为了克服上述局限性，我们提出了一个新颖的框架，以更好地捕获高阶结构的新框架，铰接于我们所提出的主题冗余最小化操作员和注射主题组合的新颖框架。首先，MGNN生成一组节点表示W.R.T.每个主题。下一阶段是我们在图案中提出的冗余最小化，该主题在彼此相互比较并蒸馏出每个主题的特征。最后，MGNN通过组合来自不同图案的多个表示来执行节点表示的更新。特别地，为了增强鉴别的功率，MGNN利用重新注射功能来组合表示的函数w.r.t.不同的主题。我们进一步表明，我们的拟议体系结构增加了GNN的表现力，具有理论分析。我们展示了MGNN在节点分类和图形分类任务上的七个公共基准上表现出最先进的方法。

Most graph neural network models rely on a particular message passing paradigm, where the idea is to iteratively propagate node representations of a graph to each node in the direct neighborhood. While very prominent, this paradigm leads to information propagation bottlenecks, as information is repeatedly compressed at intermediary node representations, which causes loss of information, making it practically impossible to gather meaningful signals from distant nodes. To address this issue, we propose shortest path message passing neural networks, where the node representations of a graph are propagated to each node in the shortest path neighborhoods. In this setting, nodes can directly communicate between each other even if they are not neighbors, breaking the information bottleneck and hence leading to more adequately learned representations. Theoretically, our framework generalizes message passing neural networks, resulting in provably more expressive models, and we show that some recent state-of-the-art models are special instances of this framework. Empirically, we verify the capacity of a basic model of this framework on dedicated synthetic experiments, and on real-world graph classification and regression benchmarks, and obtain state-of-the-art results.

Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs. GNNs follow a neighborhood aggregation scheme, where the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes. Many GNN variants have been proposed and have achieved state-of-the-art results on both node and graph classification tasks. However, despite GNNs revolutionizing graph representation learning, there is limited understanding of their representational properties and limitations. Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures. Our results characterize the discriminative power of popular GNN variants, such as Graph Convolutional Networks and GraphSAGE, and show that they cannot learn to distinguish certain simple graph structures. We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism test. We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance. * Equal contribution. † Work partially performed while in Tokyo, visiting Prof. Ken-ichi Kawarabayashi.

Graph neural networks (GNNs) are widely used for modeling complex interactions between entities represented as vertices of a graph. Despite recent efforts to theoretically analyze the expressive power of GNNs, a formal characterization of their ability to model interactions is lacking. The current paper aims to address this gap. Formalizing strength of interactions through an established measure known as separation rank, we quantify the ability of certain GNNs to model interaction between a given subset of vertices and its complement, i.e. between sides of a given partition of input vertices. Our results reveal that the ability to model interaction is primarily determined by the partition's walk index -- a graph-theoretical characteristic that we define by the number of walks originating from the boundary of the partition. Experiments with common GNN architectures corroborate this finding. As a practical application of our theory, we design an edge sparsification algorithm named Walk Index Sparsification (WIS), which preserves the ability of a GNN to model interactions when input edges are removed. WIS is simple, computationally efficient, and markedly outperforms alternative methods in terms of induced prediction accuracy. More broadly, it showcases the potential of improving GNNs by theoretically analyzing the interactions they can model.

在这项工作中，我们开发了一种新的方法，名为局部排列的图形神经网络，它为建立在本地节点邻域，通过子图形的构建图形神经网络的框架，同时使用置换等值更新功能。消息传递神经网络的消息被认为是有效应功率的限制，并且最近过度的方法缺乏可扩展性或需要将结构信息被编码为特征空间。这里呈现的一般框架克服了通过通过受限制表示在子图上操作的与全局排列等值相关的可扩展性问题。此外，我们证明了通过使用限制的陈述没有丧失表情。此外，所提出的框架仅需要选择$ k $-hops，用于创建用于为每层使用的子图和选择的表示空间，这使得该方法在一系列基于图形的域中可以容易地适用。我们通过实验验证了一系列图形基准分类任务的方法，在所有基准上展示了最先进的结果或非常竞争力的结果。此外，我们证明使用本地更新函数的使用在全球方法上提供了GPU存储器的显着改进。

自引入以来，图形注意力网络在图表表示任务中取得了出色的结果。但是，这些网络仅考虑节点之间的成对关系，然后它们无法完全利用许多现实世界数据集中存在的高阶交互。在本文中，我们介绍了细胞注意网络（CANS），这是一种在图表上定义的数据上运行的神经体系结构，将图表示为介绍的细胞复合物的1个骨骼，以捕获高阶相互作用。特别是，我们利用细胞复合物中的下层和上层社区来设计两种独立的掩盖自我发项机制，从而推广了常规的图形注意力策略。罐中使用的方法是层次结构的，并结合了以下步骤：i）从{\ it node demantion}中学习{\ it Edge功能}的提升算法}; ii）一种细胞注意机制，可以在下层和上邻居上找到边缘特征的最佳组合； iii）层次{\ it Edge Pooling}机制，以提取一组紧凑的有意义的功能集。实验结果表明，CAN是一种低复杂性策略，它与基于图的学​​习任务的最新结果相比。

Identifying similar network structures is key to capture graph isomorphisms and learn representations that exploit structural information encoded in graph data. This work shows that ego-networks can produce a structural encoding scheme for arbitrary graphs with greater expressivity than the Weisfeiler-Lehman (1-WL) test. We introduce IGEL, a preprocessing step to produce features that augment node representations by encoding ego-networks into sparse vectors that enrich Message Passing (MP) Graph Neural Networks (GNNs) beyond 1-WL expressivity. We describe formally the relation between IGEL and 1-WL, and characterize its expressive power and limitations. Experiments show that IGEL matches the empirical expressivity of state-of-the-art methods on isomorphism detection while improving performance on seven GNN architectures.

图形神经网络（GNNS）的表现力量受到限制，具有远程交互的斗争，缺乏模拟高阶结构的原则性方法。这些问题可以归因于计算图表和输入图结构之间的强耦合。最近提出的消息通过单独的网络通过执行图形的Clique复合物的消息来自然地解耦这些元素。然而，这些模型可能受到单纯复合物（SCS）的刚性组合结构的严重限制。在这项工作中，我们将最近的基于常规细胞复合物的理论结果扩展到常规细胞复合物，灵活地满满SCS和图表的拓扑物体。我们表明，该概括提供了一组强大的图表“提升”转换，每个图形是导致唯一的分层消息传递过程。我们集体呼叫CW Networks（CWNS）的结果方法比WL测试更强大，而不是比3 WL测试更强大。特别是，当应用于分子图问题时，我们证明了一种基于环的一个这样的方案的有效性。所提出的架构从可提供的较大的表达效益于常用的GNN，高阶信号的原则建模以及压缩节点之间的距离。我们展示了我们的模型在各种分子数据集上实现了最先进的结果。

In the last few years, graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. This emerging field has witnessed an extensive growth of promising techniques that have been applied with success to computer science, mathematics, biology, physics and chemistry. But for any successful field to become mainstream and reliable, benchmarks must be developed to quantify progress. This led us in March 2020 to release a benchmark framework that i) comprises of a diverse collection of mathematical and real-world graphs, ii) enables fair model comparison with the same parameter budget to identify key architectures, iii) has an open-source, easy-to-use and reproducible code infrastructure, and iv) is flexible for researchers to experiment with new theoretical ideas. As of December 2022, the GitHub repository has reached 2,000 stars and 380 forks, which demonstrates the utility of the proposed open-source framework through the wide usage by the GNN community. In this paper, we present an updated version of our benchmark with a concise presentation of the aforementioned framework characteristics, an additional medium-sized molecular dataset AQSOL, similar to the popular ZINC, but with a real-world measured chemical target, and discuss how this framework can be leveraged to explore new GNN designs and insights. As a proof of value of our benchmark, we study the case of graph positional encoding (PE) in GNNs, which was introduced with this benchmark and has since spurred interest of exploring more powerful PE for Transformers and GNNs in a robust experimental setting.