Graph Neural Networks (GNNs) are deep learning models designed to process attributed graphs. GNNs can compute cluster assignments accounting both for the vertex features and for the graph topology. Existing GNNs for clustering are trained by optimizing an unsupervised minimum cut objective, which is approximated by a Spectral Clustering (SC) relaxation. SC offers a closed-form solution that, however, is not particularly useful for a GNN trained with gradient descent. Additionally, the SC relaxation is loose and yields overly smooth cluster assignments, which do not separate well the samples. We propose a GNN model that optimizes a tighter relaxation of the minimum cut based on graph total variation (GTV). Our model has two core components: i) a message-passing layer that minimizes the $\ell_1$ distance in the features of adjacent vertices, which is key to achieving sharp cluster transitions; ii) a loss function that minimizes the GTV in the cluster assignments while ensuring balanced partitions. By optimizing the proposed loss, our model can be self-trained to perform clustering. In addition, our clustering procedure can be used to implement graph pooling in deep GNN architectures for graph classification. Experiments show that our model outperforms other GNN-based approaches for clustering and graph pooling.

光谱群集中使用的目标函数通常由两个术语组成：i）一个术语最小化群集分配的局部二次变化，并且；ii）一个平衡聚类分区并有助于避免退化解决方案的术语。本文表明，配备合适消息传递层的图形神经网络可以通过仅优化平衡项来生成良好的集群分配。归因图数据集的结果显示了拟议方法在聚类性能和计算时间方面的有效性。

图形神经网络（GNN）已在许多图分析任务（例如节点分类和链接预测）上实现了最新结果。然而，事实证明，图形群集等图形上的重要无监督问题对GNN的进步具有更大的抵抗力。图群集的总体目标与GNN中的节点合并相同 - 这是否意味着GNN池方法在聚类图上做得很好？令人惊讶的是，答案是没有的 - 当前的GNN合并方法通常无法恢复群集结构，而在简单的基线（例如应用于学习的表示形式上的K-均值）良好工作的情况下。我们通过仔细设计一组实验来进一步研究，以研究图形结构和属性数据中的不同信噪比情景。为了解决这些方法在聚类中的性能不佳，我们引入了深层模块化网络（DMON），这是一种受群集质量模块化量度启发的无监督池方法，并显示了它如何解决现实世界图的挑战性聚类结构的恢复。同样，在现实世界中，我们表明DMON产生的高质量簇与地面真相标签密切相关，从而实现了最先进的结果，比不同指标的其他合并方法提高了40％以上。

Recently, graph neural networks (GNNs) have revolutionized the field of graph representation learning through effectively learned node embeddings, and achieved state-of-the-art results in tasks such as node classification and link prediction. However, current GNN methods are inherently flat and do not learn hierarchical representations of graphs-a limitation that is especially problematic for the task of graph classification, where the goal is to predict the label associated with an entire graph. Here we propose DIFFPOOL, a differentiable graph pooling module that can generate hierarchical representations of graphs and can be combined with various graph neural network architectures in an end-to-end fashion. DIFFPOOL learns a differentiable soft cluster assignment for nodes at each layer of a deep GNN, mapping nodes to a set of clusters, which then form the coarsened input for the next GNN layer. Our experimental results show that combining existing GNN methods with DIFFPOOL yields an average improvement of 5-10% accuracy on graph classification benchmarks, compared to all existing pooling approaches, achieving a new state-of-the-art on four out of five benchmark data sets.

Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.

随着各个领域的深度学习的巨大成功，图形神经网络（GNNS）也成为图形分类的主要方法。通过全局读出操作，只会聚合所有节点（或节点群集）表示，现有的GNN分类器获得输入图的图级表示，并使用表示来预测其类标签。但是，这种全局聚合不考虑每个节点的结构信息，这导致全局结构的信息丢失。特别地，它通过对所有节点表示来强制执行分类器的相同权重参数来限制辨别力;在实践中，他们中的每一个都有助于不同于其结构语义的目标类别。在这项工作中，我们提出了结构性语义读数（SSREAD）来总结位置级节点表示，这允许为分类模拟特定位置的权重参数，以及有效地捕获与全局结构相关的图形语义。给定输入图，SSREAD旨在通过使用其节点与结构原型之间的语义对齐来识别结构上有意义的位置，该结构原型编码每个位置的原型特征。结构原型经过优化，以最小化所有训练图的对准成本，而其他GNN参数训练以预测类标签。我们的实验结果表明，SSREAD显着提高了GNN分类器的分类性能和可解释性，同时兼容各种聚合函数，GNN架构和学习框架。

我们展示了如何使用图形神经网络来解决规范的图形着色问题。我们将颜色框架为多类节点分类问题，并基于统计物理Potts模型利用无监督的培训策略。对其他多级问题（例如社区检测，数据聚类和最低集团封面问题）的概括是简单的。我们提供数值基准结果，并通过端到端的应用程序说明了我们的方法，用于在全面的编码程序框架内实现现实世界调度案例。我们的优化方法在PAR或优于现有求解器上执行，并能够扩展到数百万变量的问题。

在用于图形结构数据的几台机器学习任务中，所考虑的图形可以由不同数量的节点组成。因此，需要设计汇集方法，该方法将不同大小的图形表示聚合到固定大小的表示，其可以用于下游任务，例如图形分类。现有的图形池池方法没有关于图形表示的相似性和其汇总版的保证。在这项工作中，我们通过提出流池来解决这些限制，通过最小化其Wassersein距离，通过最佳地将图形表示的统计数据统计到其汇集的对应物。这是通过对汇集的图形表示来执行Wasserstein梯度流来实现的。我们提出了我们的方法，可以通过任何基础成本考虑表示空间的几何形状。该实施依赖于与最近提出的隐式差异化方案的Wasserstein距离的计算。我们的汇集方法可用于自动分化，可以集成在端到端的深度学习架构中。此外，流量池是不变的，因此可以与GNN中的置换设备提取层组合，以便获得与节点的排序无关的预测。实验结果表明，与现有在图形分类任务中的现有汇集方法相比，我们的方法导致性能增加。

In the last few years, graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. This emerging field has witnessed an extensive growth of promising techniques that have been applied with success to computer science, mathematics, biology, physics and chemistry. But for any successful field to become mainstream and reliable, benchmarks must be developed to quantify progress. This led us in March 2020 to release a benchmark framework that i) comprises of a diverse collection of mathematical and real-world graphs, ii) enables fair model comparison with the same parameter budget to identify key architectures, iii) has an open-source, easy-to-use and reproducible code infrastructure, and iv) is flexible for researchers to experiment with new theoretical ideas. As of December 2022, the GitHub repository has reached 2,000 stars and 380 forks, which demonstrates the utility of the proposed open-source framework through the wide usage by the GNN community. In this paper, we present an updated version of our benchmark with a concise presentation of the aforementioned framework characteristics, an additional medium-sized molecular dataset AQSOL, similar to the popular ZINC, but with a real-world measured chemical target, and discuss how this framework can be leveraged to explore new GNN designs and insights. As a proof of value of our benchmark, we study the case of graph positional encoding (PE) in GNNs, which was introduced with this benchmark and has since spurred interest of exploring more powerful PE for Transformers and GNNs in a robust experimental setting.

由于其数值益处增加及其坚实的数学背景，光谱聚类方法的非线性重构近来的关注。我们在$ p $ -norm中提出了一种新的直接多道谱聚类算法，以$ p \ in（1,2] $。计算图表的多个特征向量的问题$ p $ -laplacian，标准的非线性概括Graph Laplacian，被重用作为Grassmann歧管的无约束最小化问题。$ P $的价值以伪连续的方式减少，促进对应于最佳图形的稀疏解决方案载体作为$ P $接近。监测单调减少平衡图削减了我们从$ P $ -Levels获得的最佳可用解决方案的保证。我们展示了我们算法在各种人工测试案件中的算法的有效性和准确性。我们的数值和比较结果具有各种状态-Art聚类方法表明，所提出的方法在均衡的图形剪切度量和标签分配的准确性方面取得高质量的集群。此外，我们进行S面部图像和手写字符分类的束缚，以展示现实数据集中的适用性。

图形神经网络（GNN）已被证明可以实现竞争结果，以解决与图形相关的任务，例如节点和图形分类，链接预测和节点以及各种域中的图形群集。大多数GNN使用消息传递框架，因此称为MPNN。尽管有很有希望的结果，但据报道，MPNN会遭受过度平滑，过度阵型和不足的影响。文献中已经提出了图形重新布线和图形池作为解决这些局限性的解决方案。但是，大多数最先进的图形重新布线方法无法保留该图的全局拓扑，因此没有可区分（电感），并且需要调整超参数。在本文中，我们提出了Diffwire，这是一个在MPNN中进行图形重新布线的新型框架，它通过利用LOV \'ASZ绑定来原理，完全可区分且无参数。我们的方法通过提出两个新的，mpnns中的新的互补层来提供统一的图形重新布线：首先，ctlayer，一个学习通勤时间并将其用作边缘重新加权的相关函数；其次，Gaplayer是优化光谱差距的图层，具体取决于网络的性质和手头的任务。我们从经验上验证了我们提出的方法的价值，并使用基准数据集分别验证了这些层的每个层以进行图形分类。 Diffwire将通勤时间的可学习性汇集到相关的曲率定义，为发展更具表现力的MPNN的发展打开了大门。

卷积神经网络（CNN）已通过卷积和汇总实现了图像分类的重大进展。特别是，图像池将连接的离散网格转换为具有相同连接性的还原网格，并允许还原功能考虑图像的所有像素。但是，对于图形而不存在满足此类属性的合并。实际上，某些方法基于一个顶点选择步骤，该步骤会导致重要信息丢失。其他方法学习了顶点集的模糊聚类，该聚类几乎诱导了几乎完全减少的图形。我们建议使用名为MivSpool的新合并方法克服这两个问题。该方法基于使用最大独立顶点集（MIV）和将其余顶点分配给幸存者的最大独立顶点集（MIV）的选择的顶点。因此，我们的方法不会丢弃任何顶点信息，也不会人为地增加图的密度。实验结果表明，各种标准数据集上的图形分类的精度有所提高。

我们提出了一种基于图形神经网络（GNN）的端到端框架，以平衡通用网格中的功率流。优化被帧为监督的顶点回归任务，其中GNN培训以预测每个网格分支的电流和功率注入，从而产生功率流量平衡。通过将电网表示为与顶点的分支的线图，我们可以培训一个更准确和强大的GNN来改变底层拓扑。此外，通过使用专门的GNN层，我们能够构建一个非常深的架构，该架构占图表上的大街区，同时仅实现本地化操作。我们执行三个不同的实验来评估：i）使用深入GNN模型时使用本地化而不是全球运营的好处和趋势; ii）图形拓扑中对扰动的弹性;和iii）能力同时在多个网格拓扑上同时培训模型以及新的看不见网格的概括性的改进。拟议的框架是有效的，而且与基于深度学习的其他求解器相比，不仅对网格组件上的物理量而且对拓扑的物理量具有鲁棒性。

图形神经网络（GNNS）对图表上的半监督节点分类展示了卓越的性能，结果是它们能够同时利用节点特征和拓扑信息的能力。然而，大多数GNN隐含地假设曲线图中的节点和其邻居的标签是相同或一致的，其不包含在异质图中，其中链接节点的标签可能不同。因此，当拓扑是非信息性的标签预测时，普通的GNN可以显着更差，而不是在每个节点上施加多层Perceptrons（MLPS）。为了解决上述问题，我们提出了一种新的$ -laplacian基于GNN模型，称为$ ^ P $ GNN，其消息传递机制来自离散正则化框架，并且可以理论上解释为多项式图的近似值在$ p $ -laplacians的频谱域上定义过滤器。光谱分析表明，新的消息传递机制同时用作低通和高通滤波器，从而使$ ^ P $ GNNS对同性恋和异化图有效。关于现实世界和合成数据集的实证研究验证了我们的调查结果，并证明了$ ^ P $ GNN明显优于异交基准的几个最先进的GNN架构，同时在同性恋基准上实现竞争性能。此外，$ ^ p $ gnns可以自适应地学习聚合权重，并且对嘈杂的边缘具有强大。

图形神经网络（GNNS）依赖于图形结构来定义聚合策略，其中每个节点通过与邻居的信息组合来更新其表示。已知GNN的限制是，随着层数的增加，信息被平滑，压扁并且节点嵌入式变得无法区分，对性能产生负面影响。因此，实用的GNN模型雇用了几层，只能在每个节点周围的有限邻域利用图形结构。不可避免地，实际的GNN不会根据图的全局结构捕获信息。虽然有几种研究GNNS的局限性和表达性，但是关于图形结构数据的实际应用的问题需要全局结构知识，仍然没有答案。在这项工作中，我们通过向几个GNN模型提供全球信息并观察其对下游性能的影响来认证解决这个问题。我们的研究结果表明，全球信息实际上可以为共同的图形相关任务提供显着的好处。我们进一步确定了一项新的正规化策略，导致所有考虑的任务的平均准确性提高超过5％。

在非欧几里得空间上卷积成功之后，在有关图形的各种任务上也验证了相应的合并方法。但是，由于固定的压缩配额和逐步合并设计，这些层次池方法仍然遭受局部结构损害和次优问题的困扰。在这项工作的启发下，我们提出了一种层次的合并方法，即SEP解决这两个问题。具体而言，在不分配特定层的压缩配额的情况下，全局优化算法旨在生成一次集群分配矩阵以一次汇总。然后，我们介绍了在环和网格合成图的重建中先前方法中局部结构损害的例证。除SEP外，我​​们还将分别设计两个分类模型，分别用于图形分类和节点分类。结果表明，SEP在图形分类基准上优于最先进的图形合并方法，并在节点分类上获得了卓越的性能。

Graph classification is an important area in both modern research and industry. Multiple applications, especially in chemistry and novel drug discovery, encourage rapid development of machine learning models in this area. To keep up with the pace of new research, proper experimental design, fair evaluation, and independent benchmarks are essential. Design of strong baselines is an indispensable element of such works. In this thesis, we explore multiple approaches to graph classification. We focus on Graph Neural Networks (GNNs), which emerged as a de facto standard deep learning technique for graph representation learning. Classical approaches, such as graph descriptors and molecular fingerprints, are also addressed. We design fair evaluation experimental protocol and choose proper datasets collection. This allows us to perform numerous experiments and rigorously analyze modern approaches. We arrive to many conclusions, which shed new light on performance and quality of novel algorithms. We investigate application of Jumping Knowledge GNN architecture to graph classification, which proves to be an efficient tool for improving base graph neural network architectures. Multiple improvements to baseline models are also proposed and experimentally verified, which constitutes an important contribution to the field of fair model comparison.

在过去的几年中，已经开发了图形绘图技术，目的是生成美学上令人愉悦的节点链接布局。最近，利用可区分损失功能的使用已为大量使用梯度下降和相关优化算法铺平了道路。在本文中，我们提出了一个用于开发图神经抽屉（GND）的新框架，即依靠神经计算来构建有效且复杂的图的机器。 GND是图形神经网络（GNN），其学习过程可以由任何提供的损失函数（例如图形图中通常使用的损失函数）驱动。此外，我们证明，该机制可以由通过前馈神经网络计算的损失函数来指导，并根据表达美容特性的监督提示，例如交叉边缘的最小化。在这种情况下，我们表明GNN可以通过位置功能很好地丰富与未标记的顶点处理。我们通过为边缘交叉构建损失函数来提供概念验证，并在提议的框架下工作的不同GNN模型之间提供定量和定性的比较。

Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this survey, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art graph neural networks into four categories, namely recurrent graph neural networks, convolutional graph neural networks, graph autoencoders, and spatial-temporal graph neural networks. We further discuss the applications of graph neural networks across various domains and summarize the open source codes, benchmark data sets, and model evaluation of graph neural networks. Finally, we propose potential research directions in this rapidly growing field.

Advanced methods of applying deep learning to structured data such as graphs have been proposed in recent years. In particular, studies have focused on generalizing convolutional neural networks to graph data, which includes redefining the convolution and the downsampling (pooling) operations for graphs. The method of generalizing the convolution operation to graphs has been proven to improve performance and is widely used. However, the method of applying downsampling to graphs is still difficult to perform and has room for improvement. In this paper, we propose a graph pooling method based on selfattention. Self-attention using graph convolution allows our pooling method to consider both node features and graph topology. To ensure a fair comparison, the same training procedures and model architectures were used for the existing pooling methods and our method. The experimental results demonstrate that our method achieves superior graph classification performance on the benchmark datasets using a reasonable number of parameters.