Recently, there has been an increasing interest in (supervised) learning with graph data, especially using graph neural networks. However, the development of meaningful benchmark datasets and standardized evaluation procedures is lagging, consequently hindering advancements in this area. To address this, we introduce the TUDATASET for graph classification and regression. The collection consists of over 120 datasets of varying sizes from a wide range of applications. We provide Python-based data loaders, kernel and graph neural network baseline implementations, and evaluation tools. Here, we give an overview of the datasets, standardized evaluation procedures, and provide baseline experiments. All datasets are available at www.graphlearning.io. The experiments are fully reproducible from the code available at www.github.com/chrsmrrs/tudataset.

尽管（消息通话）图形神经网络在图形或一般关系数据上近似置换量等函数方面具有明显的局限性，但更具表现力的高阶图神经网络不会扩展到大图。他们要么在$ k $ - 订单张量子上操作，要么考虑所有$ k $ - 节点子图，这意味着在内存需求中对$ k $的指数依赖，并且不适合图形的稀疏性。通过为图同构问题引入新的启发式方法，我们设计了一类通用的，置换式的图形网络，与以前的体系结构不同，该网络在表达性和可伸缩性之间提供了细粒度的控制，并适应了图的稀疏性。这些体系结构与监督节点和图形级别的标准高阶网络以及回归体系中的标准高阶图网络相比大大减少了计算时间，同时在预测性能方面显着改善了标准图神经网络和图形内核体系结构。

近年来，图形神经网络（GNNS）被出现为一个强大的神经结构，以学习在监督的端到端时尚中的节点和图表的矢量表示。到目前为止，只有经验评估GNNS - 显示有希望的结果。以下工作从理论的角度调查了GNN，并将它们与1美元 - 二维韦斯美犬 - Leman Graph同构Heuristic（1美元-WL）相关联。我们表明GNNS在区分非同义（子）图表中，GNN具有与1美元-WL相同的表现力。因此，这两种算法也具有相同的缺点。基于此，我们提出了GNN的概括，所谓的$ k $ -dimensional gnns（$ k $ -gnns），这可以考虑多个尺度的高阶图结构。这些高阶结构在社交网络和分子图的表征中起重要作用。我们的实验评估证实了我们的理论调查结果，并确认了更高阶信息在图形分类和回归的任务中有用。

近年来，基于Weisfeiler-Leman算法的算法和神经架构，是图形同构的着名启发式问题，它被成为具有图形和关系数据的（监督）机器学习的强大工具。在这里，我们全面概述了机器学习设置中的算法使用。我们讨论了理论背景，展示了如何将其用于监督的图形和节点分类，讨论最近的扩展，以及其与神经结构的连接。此外，我们概述了当前的应用和未来方向，以刺激研究。

在三维分子结构上运行的计算方法有可能解决生物学和化学的重要问题。特别地，深度神经网络的重视，但它们在生物分子结构域中的广泛采用受到缺乏系统性能基准或统一工具包的限制，用于与分子数据相互作用。为了解决这个问题，我们呈现Atom3D，这是一个新颖的和现有的基准数据集的集合，跨越几个密钥的生物分子。我们为这些任务中的每一个实施多种三维分子学习方法，并表明它们始终如一地提高了基于单维和二维表示的方法的性能。结构的具体选择对于性能至关重要，具有涉及复杂几何形状的任务的三维卷积网络，在需要详细位置信息的系统中表现出良好的图形网络，以及最近开发的设备越多的网络显示出显着承诺。我们的结果表明，许多分子问题符合三维分子学习的增益，并且有可能改善许多仍然过分曝光的任务。为了降低进入并促进现场进一步发展的障碍，我们还提供了一套全面的DataSet处理，模型培训和在我们的开源ATOM3D Python包中的评估工具套件。所有数据集都可以从https://www.atom3d.ai下载。

Graph神经网络（GNN）最近已成为使用图的机器学习的主要范式。对GNNS的研究主要集中于消息传递神经网络（MPNNS）的家族。与同构的Weisfeiler-Leman（WL）测试类似，这些模型遵循迭代的邻域聚合过程以更新顶点表示，并通过汇总顶点表示来更新顶点图表。尽管非常成功，但在过去的几年中，对MPNN进行了深入的研究。因此，需要新颖的体系结构，这将使该领域的研究能够脱离MPNN。在本文中，我们提出了一个新的图形神经网络模型，即所谓的$ \ pi $ -gnn，该模型学习了每个图的“软”排列（即双随机）矩阵，从而将所有图形投影到一个共同的矢量空间中。学到的矩阵在输入图的顶点上强加了“软”顺序，并基于此顺序，将邻接矩阵映射到向量中。这些向量可以被送入完全连接或卷积的层，以应对监督的学习任务。在大图的情况下，为了使模型在运行时间和记忆方面更有效，我们进一步放松了双随机矩阵，以使其排列随机矩阵。我们从经验上评估了图形分类和图形回归数据集的模型，并表明它与最新模型达到了性能竞争。

近年来，基于Weisfeiler-Leman算法的算法和神经架构，是一个众所周知的Graph同构问题的启发式问题，它成为具有图形和关系数据的机器学习的强大工具。在这里，我们全面概述了机器学习设置中的算法的使用，专注于监督的制度。我们讨论了理论背景，展示了如何将其用于监督的图形和节点表示学习，讨论最近的扩展，并概述算法的连接（置换 - ）方面的神经结构。此外，我们概述了当前的应用和未来方向，以刺激进一步的研究。

在过去十年中，图形内核引起了很多关注，并在结构化数据上发展成为一种快速发展的学习分支。在过去的20年中，该领域发生的相当大的研究活动导致开发数十个图形内核，每个图形内核都对焦于图形的特定结构性质。图形内核已成功地成功地在广泛的域中，从社交网络到生物信息学。本调查的目标是提供图形内核的文献的统一视图。特别是，我们概述了各种图形内核。此外，我们对公共数据集的几个内核进行了实验评估，并提供了比较研究。最后，我们讨论图形内核的关键应用，并概述了一些仍有待解决的挑战。

增强图在正规化图形神经网络（GNNS）方面起着至关重要的作用，该图形以信息传递的形式利用沿图的边缘进行信息交换。由于其有效性，简单的边缘和节点操作（例如，添加和删除）已被广泛用于图表增强中。然而，这种常见的增强技术可以显着改变原始图的语义，从而导致过度侵略性增强，从而在GNN学习中拟合不足。为了解决掉落或添加图形边缘和节点引起的此问题，我们提出了SoftEdge，将随机权重分配给给定图的一部分以进行增强。 SoftEdge生成的合成图保持与原始图相同的节点及其连接性，从而减轻原始图的语义变化。我们从经验上表明，这种简单的方法获得了与流行节点和边缘操纵方法的卓越精度，并且具有明显的弹性，可抵御GNN深度的准确性降解。

Many applications of machine learning require a model to make accurate predictions on test examples that are distributionally different from training ones, while task-specific labels are scarce during training. An effective approach to this challenge is to pre-train a model on related tasks where data is abundant, and then fine-tune it on a downstream task of interest. While pre-training has been effective in many language and vision domains, it remains an open question how to effectively use pre-training on graph datasets. In this paper, we develop a new strategy and self-supervised methods for pre-training Graph Neural Networks (GNNs). The key to the success of our strategy is to pre-train an expressive GNN at the level of individual nodes as well as entire graphs so that the GNN can learn useful local and global representations simultaneously. We systematically study pre-training on multiple graph classification datasets. We find that naïve strategies, which pre-train GNNs at the level of either entire graphs or individual nodes, give limited improvement and can even lead to negative transfer on many downstream tasks. In contrast, our strategy avoids negative transfer and improves generalization significantly across downstream tasks, leading up to 9.4% absolute improvements in ROC-AUC over non-pre-trained models and achieving state-of-the-art performance for molecular property prediction and protein function prediction.However, pre-training on graph datasets remains a hard challenge. Several key studies (

We introduce PyTorch Geometric, a library for deep learning on irregularly structured input data such as graphs, point clouds and manifolds, built upon PyTorch. In addition to general graph data structures and processing methods, it contains a variety of recently published methods from the domains of relational learning and 3D data processing. PyTorch Geometric achieves high data throughput by leveraging sparse GPU acceleration, by providing dedicated CUDA kernels and by introducing efficient mini-batch handling for input examples of different size. In this work, we present the library in detail and perform a comprehensive comparative study of the implemented methods in homogeneous evaluation scenarios.

图形神经网络（GNNS）通过考虑其内在的几何形状来扩展神经网络的成功到图形结构化数据。尽管根据图表学习基准的集合，已经对开发具有卓越性能的GNN模型进行了广泛的研究，但目前尚不清楚其探测给定模型的哪些方面。例如，他们在多大程度上测试模型利用图形结构与节点特征的能力？在这里，我们开发了一种原则性的方法来根据$ \ textit {敏感性配置文件} $进行基准测试数据集，该方法基于由于图形扰动的集合而导致的GNN性能变化了多少。我们的数据驱动分析提供了对GNN利用哪些基准测试数据特性的更深入的了解。因此，我们的分类法可以帮助选择和开发适当的图基准测试，并更好地评估未来的GNN方法。最后，我们在$ \ texttt {gtaxogym} $软件包中的方法和实现可扩展到多个图形预测任务类型和未来数据集。

许多现代神经架构的核心的卷积运算符可以有效地被视为在输入矩阵和滤波器之间执行点产品。虽然这很容易适用于诸如图像的数据，其可以在欧几里德空间中表示为常规网格，延伸卷积操作者以在图形上工作，而是由于它们的不规则结构而被证明更具有挑战性。在本文中，我们建议使用图形内部产品的图形内核，即在图形上计算内部产品，以将标准卷积运算符扩展到图形域。这使我们能够定义不需要计算输入图的嵌入的完全结构模型。我们的架构允许插入任何类型和数量的图形内核，并具有在培训过程中学到的结构面具方面提供一些可解释性的额外益处，类似于传统卷积神经网络中的卷积掩模发生的事情。我们执行广泛的消融研究，调查模型超参数的影响，我们表明我们的模型在标准图形分类数据集中实现了竞争性能。

In the last few years, graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. This emerging field has witnessed an extensive growth of promising techniques that have been applied with success to computer science, mathematics, biology, physics and chemistry. But for any successful field to become mainstream and reliable, benchmarks must be developed to quantify progress. This led us in March 2020 to release a benchmark framework that i) comprises of a diverse collection of mathematical and real-world graphs, ii) enables fair model comparison with the same parameter budget to identify key architectures, iii) has an open-source, easy-to-use and reproducible code infrastructure, and iv) is flexible for researchers to experiment with new theoretical ideas. As of December 2022, the GitHub repository has reached 2,000 stars and 380 forks, which demonstrates the utility of the proposed open-source framework through the wide usage by the GNN community. In this paper, we present an updated version of our benchmark with a concise presentation of the aforementioned framework characteristics, an additional medium-sized molecular dataset AQSOL, similar to the popular ZINC, but with a real-world measured chemical target, and discuss how this framework can be leveraged to explore new GNN designs and insights. As a proof of value of our benchmark, we study the case of graph positional encoding (PE) in GNNs, which was introduced with this benchmark and has since spurred interest of exploring more powerful PE for Transformers and GNNs in a robust experimental setting.

We present the OPEN GRAPH BENCHMARK (OGB), a diverse set of challenging and realistic benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning (ML) research. OGB datasets are large-scale, encompass multiple important graph ML tasks, and cover a diverse range of domains, ranging from social and information networks to biological networks, molecular graphs, source code ASTs, and knowledge graphs. For each dataset, we provide a unified evaluation protocol using meaningful application-specific data splits and evaluation metrics. In addition to building the datasets, we also perform extensive benchmark experiments for each dataset. Our experiments suggest that OGB datasets present significant challenges of scalability to large-scale graphs and out-of-distribution generalization under realistic data splits, indicating fruitful opportunities for future research. Finally, OGB provides an automated end-to-end graph ML pipeline that simplifies and standardizes the process of graph data loading, experimental setup, and model evaluation. OGB will be regularly updated and welcomes inputs from the community. OGB datasets as well as data loaders, evaluation scripts, baseline code, and leaderboards are publicly available at https://ogb.stanford.edu.

Graph classification is an important area in both modern research and industry. Multiple applications, especially in chemistry and novel drug discovery, encourage rapid development of machine learning models in this area. To keep up with the pace of new research, proper experimental design, fair evaluation, and independent benchmarks are essential. Design of strong baselines is an indispensable element of such works. In this thesis, we explore multiple approaches to graph classification. We focus on Graph Neural Networks (GNNs), which emerged as a de facto standard deep learning technique for graph representation learning. Classical approaches, such as graph descriptors and molecular fingerprints, are also addressed. We design fair evaluation experimental protocol and choose proper datasets collection. This allows us to perform numerous experiments and rigorously analyze modern approaches. We arrive to many conclusions, which shed new light on performance and quality of novel algorithms. We investigate application of Jumping Knowledge GNN architecture to graph classification, which proves to be an efficient tool for improving base graph neural network architectures. Multiple improvements to baseline models are also proposed and experimentally verified, which constitutes an important contribution to the field of fair model comparison.

In recent years, graph neural networks (GNNs) have emerged as a promising tool for solving machine learning problems on graphs. Most GNNs are members of the family of message passing neural networks (MPNNs). There is a close connection between these models and the Weisfeiler-Leman (WL) test of isomorphism, an algorithm that can successfully test isomorphism for a broad class of graphs. Recently, much research has focused on measuring the expressive power of GNNs. For instance, it has been shown that standard MPNNs are at most as powerful as WL in terms of distinguishing non-isomorphic graphs. However, these studies have largely ignored the distances between the representations of nodes/graphs which are of paramount importance for learning tasks. In this paper, we define a distance function between nodes which is based on the hierarchy produced by the WL algorithm, and propose a model that learns representations which preserve those distances between nodes. Since the emerging hierarchy corresponds to a tree, to learn these representations, we capitalize on recent advances in the field of hyperbolic neural networks. We empirically evaluate the proposed model on standard node and graph classification datasets where it achieves competitive performance with state-of-the-art models.

This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the representations of substructures of different scales (e.g., nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures. Furthermore, we further propose InfoGraph*, an extension of InfoGraph for semi-supervised scenarios. InfoGraph* maximizes the mutual information between unsupervised graph representations learned by InfoGraph and the representations learned by existing supervised methods. As a result, the supervised encoder learns from unlabeled data while preserving the latent semantic space favored by the current supervised task. Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state-of-the-art semi-supervised models.

图形神经网络（GNNS）的表现力量受到限制，具有远程交互的斗争，缺乏模拟高阶结构的原则性方法。这些问题可以归因于计算图表和输入图结构之间的强耦合。最近提出的消息通过单独的网络通过执行图形的Clique复合物的消息来自然地解耦这些元素。然而，这些模型可能受到单纯复合物（SCS）的刚性组合结构的严重限制。在这项工作中，我们将最近的基于常规细胞复合物的理论结果扩展到常规细胞复合物，灵活地满满SCS和图表的拓扑物体。我们表明，该概括提供了一组强大的图表“提升”转换，每个图形是导致唯一的分层消息传递过程。我们集体呼叫CW Networks（CWNS）的结果方法比WL测试更强大，而不是比3 WL测试更强大。特别是，当应用于分子图问题时，我们证明了一种基于环的一个这样的方案的有效性。所提出的架构从可提供的较大的表达效益于常用的GNN，高阶信号的原则建模以及压缩节点之间的距离。我们展示了我们的模型在各种分子数据集上实现了最先进的结果。

我们提出了一个新的图神经网络（GNN）模块，该模块基于最近提出的几何散射变换的松弛，该变换由图形小波滤波器组成。我们可学习的几何散射（腿）模块可以使小波的自适应调整能够鼓励乐队通道特征在学习的表示中出现。与许多流行的GNN相比，我们的腿部模块在GNN中的结合能够学习长期图形关系，这些GNN通常依赖于邻居之间的平滑度或相似性来编码图形结构。此外，与竞争性GNN相比，其小波先验会导致简化的架构，学到的参数明显少得多。我们证明了基于腿的网络在图形分类基准上的预测性能，以及在生化图数据探索任务中学到的功能的描述性质量。我们的结果表明，基于腿部的网络匹配或匹配流行的GNN，以及在许多数据集上，尤其是在生化域中的原始几何散射结构，同时保留了手工制作的（非学习）几何散射的某些数学特性。