已经为图形生成模型提出了广泛的模型,需要采用有效的方法来评估其质量。到目前为止,大多数技术都使用基于子图计数的传统指标或随机初始化的图形神经网络(GNN)的表示。我们建议使用对比训练的GNN而不是随机GNN的表示形式,并表明这给出了更可靠的评估指标。但是,传统方法和基于GNN的方法都没有主导另一方:我们举例说明每种方法无法区分的示例。我们证明了图形子结构网络(GSN),以一种结合两种方法的方式,可以更好地区分图形数据集之间的距离。
translated by 谷歌翻译
Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.
translated by 谷歌翻译
图形神经网络是一种强大的深度学习工具,用于建模图形结构化数据,在众多图形学习任务上表现出了出色的性能。为了解决深图学习中的数据噪声和数据稀缺性问题,最近有关图形数据的研究已加剧。但是,常规数据增强方法几乎无法处理具有多模式性的非欧几里得空间中定义的图形结构化数据。在这项调查中,我们正式提出了图数据扩展的问题,并进一步审查了代表性技术及其在不同深度学习问题中的应用。具体而言,我们首先提出了图形数据扩展技术的分类法,然后通过根据增强信息方式对相关工作进行分类,从而提供结构化的审查。此外,我们总结了以数据为中心的深图学习中两个代表性问题中图数据扩展的应用:(1)可靠的图形学习,重点是增强输入图的实用性以及通过图数据增强的模型容量; (2)低资源图学习,其针对通过图数据扩大标记的训练数据量表的目标。对于每个问题,我们还提供层次结构问题分类法,并审查与图数据增强相关的现有文献。最后,我们指出了有希望的研究方向和未来研究的挑战。
translated by 谷歌翻译
Graph machine learning has been extensively studied in both academia and industry. Although booming with a vast number of emerging methods and techniques, most of the literature is built on the in-distribution hypothesis, i.e., testing and training graph data are identically distributed. However, this in-distribution hypothesis can hardly be satisfied in many real-world graph scenarios where the model performance substantially degrades when there exist distribution shifts between testing and training graph data. To solve this critical problem, out-of-distribution (OOD) generalization on graphs, which goes beyond the in-distribution hypothesis, has made great progress and attracted ever-increasing attention from the research community. In this paper, we comprehensively survey OOD generalization on graphs and present a detailed review of recent advances in this area. First, we provide a formal problem definition of OOD generalization on graphs. Second, we categorize existing methods into three classes from conceptually different perspectives, i.e., data, model, and learning strategy, based on their positions in the graph machine learning pipeline, followed by detailed discussions for each category. We also review the theories related to OOD generalization on graphs and introduce the commonly used graph datasets for thorough evaluations. Finally, we share our insights on future research directions. This paper is the first systematic and comprehensive review of OOD generalization on graphs, to the best of our knowledge.
translated by 谷歌翻译
Generalizable, transferrable, and robust representation learning on graph-structured data remains a challenge for current graph neural networks (GNNs). Unlike what has been developed for convolutional neural networks (CNNs) for image data, self-supervised learning and pre-training are less explored for GNNs. In this paper, we propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. We first design four types of graph augmentations to incorporate various priors. We then systematically study the impact of various combinations of graph augmentations on multiple datasets, in four different settings: semi-supervised, unsupervised, and transfer learning as well as adversarial attacks. The results show that, even without tuning augmentation extents nor using sophisticated GNN architectures, our GraphCL framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods. We also investigate the impact of parameterized graph augmentation extents and patterns, and observe further performance gains in preliminary experiments. Our codes are available at: https://github.com/Shen-Lab/GraphCL.
translated by 谷歌翻译
近年来,基于Weisfeiler-Leman算法的算法和神经架构,是一个众所周知的Graph同构问题的启发式问题,它成为具有图形和关系数据的机器学习的强大工具。在这里,我们全面概述了机器学习设置中的算法的使用,专注于监督的制度。我们讨论了理论背景,展示了如何将其用于监督的图形和节点表示学习,讨论最近的扩展,并概述算法的连接(置换 - )方面的神经结构。此外,我们概述了当前的应用和未来方向,以刺激进一步的研究。
translated by 谷歌翻译
图表是一个宇宙数据结构,广泛用于组织现实世界中的数据。像交通网络,社交和学术网络这样的各种实际网络网络可以由图表代表。近年来,目睹了在网络中代表顶点的快速发展,进入低维矢量空间,称为网络表示学习。表示学习可以促进图形数据上的新算法的设计。在本调查中,我们对网络代表学习的当前文献进行了全面审查。现有算法可以分为三组:浅埋模型,异构网络嵌入模型,图形神经网络的模型。我们为每个类别审查最先进的算法,并讨论这些算法之间的基本差异。调查的一个优点是,我们系统地研究了不同类别的算法底层的理论基础,这提供了深入的见解,以更好地了解网络表示学习领域的发展。
translated by 谷歌翻译
学习有效的蛋白质表示在生物学的各种任务中至关重要,例如预测蛋白质功能或结构。现有的方法通常在大量未标记的氨基酸序列上预先蛋白质语言模型,然后在下游任务中使用一些标记的数据来对模型进行修复。尽管基于序列的方法具有有效性,但尚未探索蛋白质性能预测的已知蛋白质结构的预处理功能,尽管蛋白质结构已知是蛋白质功能的决定因素,但尚未探索。在本文中,我们建议根据其3D结构预处理蛋白质。我们首先提出一个简单而有效的编码器,以学习蛋白质的几何特征。我们通过利用多视图对比学习和不同的自我预测任务来预先蛋白质图编码器。对功能预测和折叠分类任务的实验结果表明,我们提出的预处理方法表现优于或与最新的基于最新的序列方法相提并论,同时使用较少的数据。我们的实施可在https://github.com/deepgraphlearning/gearnet上获得。
translated by 谷歌翻译
消息传递神经网络(MPNNS)是由于其简单性和可扩展性而大部分地进行图形结构数据的深度学习的领先架构。不幸的是,有人认为这些架构的表现力有限。本文提出了一种名为Comifariant Subgraph聚合网络(ESAN)的新颖框架来解决这个问题。我们的主要观察是,虽然两个图可能无法通过MPNN可区分,但它们通常包含可区分的子图。因此,我们建议将每个图形作为由某些预定义策略导出的一组子图,并使用合适的等分性架构来处理它。我们为图同构同构同构造的1立维Weisfeiler-Leman(1-WL)测试的新型变体,并在这些新的WL变体方面证明了ESAN的表达性下限。我们进一步证明,我们的方法增加了MPNNS和更具表现力的架构的表现力。此外,我们提供了理论结果,描述了设计选择诸如子图选择政策和等效性神经结构的设计方式如何影响我们的架构的表现力。要处理增加的计算成本,我们提出了一种子图采样方案,可以将其视为我们框架的随机版本。关于真实和合成数据集的一套全面的实验表明,我们的框架提高了流行的GNN架构的表现力和整体性能。
translated by 谷歌翻译
图级表示在各种现实世界中至关重要,例如预测分子的特性。但是实际上,精确的图表注释通常非常昂贵且耗时。为了解决这个问题,图形对比学习构造实例歧视任务,将正面对(同一图的增强对)汇总在一起,并将负面对(不同图的增强对)推开,以进行无监督的表示。但是,由于为了查询,其负面因素是从所有图中均匀抽样的,因此现有方法遭受关键采样偏置问题的损失,即,否定物可能与查询具有相同的语义结构,从而导致性能降解。为了减轻这种采样偏见问题,在本文中,我们提出了一种典型的图形对比度学习(PGCL)方法。具体而言,PGCL通过将语义相似的图形群群归为同一组的群集数据的基础语义结构,并同时鼓励聚类的一致性,以实现同一图的不同增强。然后给出查询,它通过从与查询群集不同的群集中绘制图形进行负采样,从而确保查询及其阴性样本之间的语义差异。此外,对于查询,PGCL根据其原型(集群质心)和查询原型之间的距离进一步重新重新重新重新重新享受其负样本,从而使那些具有中等原型距离的负面因素具有相对较大的重量。事实证明,这种重新加权策略比统一抽样更有效。各种图基准的实验结果证明了我们的PGCL比最新方法的优势。代码可在https://github.com/ha-lins/pgcl上公开获取。
translated by 谷歌翻译
对比学习在图表学习领域表现出了巨大的希望。通过手动构建正/负样本,大多数图对比度学习方法依赖于基于矢量内部产品的相似性度量标准来区分图形表示样品。但是,手工制作的样品构建(例如,图表的节点或边缘的扰动)可能无法有效捕获图形的固有局部结构。同样,基于矢量内部产品的相似性度量标准无法完全利用图形的局部结构来表征图差。为此,在本文中,我们提出了一种基于自适应子图生成的新型对比度学习框架,以实现有效且强大的自我监督图表示学习,并且最佳传输距离被用作子绘图之间的相似性度量。它的目的是通过捕获图的固有结构来生成对比样品,并根据子图的特征和结构同时区分样品。具体而言,对于每个中心节点,通过自适应学习关系权重与相应邻域的节点,我们首先开发一个网络来生成插值子图。然后,我们分别构建来自相同和不同节点的子图的正和负对。最后,我们采用两种类型的最佳运输距离(即Wasserstein距离和Gromov-Wasserstein距离)来构建结构化的对比损失。基准数据集上的广泛节点分类实验验证了我们的图形对比学习方法的有效性。
translated by 谷歌翻译
We introduce a self-supervised approach for learning node and graph level representations by contrasting structural views of graphs. We show that unlike visual representation learning, increasing the number of views to more than two or contrasting multi-scale encodings do not improve performance, and the best performance is achieved by contrasting encodings from first-order neighbors and a graph diffusion. We achieve new state-ofthe-art results in self-supervised learning on 8 out of 8 node and graph classification benchmarks under the linear evaluation protocol. For example, on Cora (node) and Reddit-Binary (graph) classification benchmarks, we achieve 86.8% and 84.5% accuracy, which are 5.5% and 2.4% relative improvements over previous state-of-the-art. When compared to supervised baselines, our approach outperforms them in 4 out of 8 benchmarks.
translated by 谷歌翻译
我们从光谱的角度解决图形生成问题,首先生成图形laplacian光谱的主要部分,然后构建与这些特征值和特征向量相匹配的图。光谱调节允许直接建模全局和局部图结构,并有助于克服单发图生成器的表达性和模式崩溃问题。我们的新颖的甘(Spectre)称为Spectre,可以使用一声模型来产生比以前可能更大的图。Spectre的表现优于最先进的深度自动回归发电机在建模忠诚方面,同时还避免了昂贵的顺序产生和对节点排序的依赖。一个很好的例子,在相当大的合成和现实图形中,Specter的幽灵比最佳竞争对手的最佳竞争对手的改进是4到170倍,该竞争对手不合适,比自回旋发电机快23至30倍。
translated by 谷歌翻译
对比学习已被广​​泛应用于图形表示学习,其中观测发生器在产生有效的对比样本方面发挥着重要作用。大多数现有的对比学习方法采用预定义的视图生成方法,例如节点滴或边缘扰动,这通常不能适应输入数据或保持原始语义结构。为了解决这个问题,我们提出了一份名为自动化图形对比学习(AutoGCL)的小说框架。具体而言,AutoGCL采用一组由自动增强策略协调的一组学习图形视图生成器,其中每个图形视图生成器都会学习输入调节的图形的概率分布。虽然AutoGCL中的图形视图发生器在生成每个对比样本中保留原始图的最代表性结构,但自动增强学会在整个对比学习程序中介绍适当的增强差异的政策。此外,AutoGCL采用联合培训策略,以培训学习的视图发生器,图形编码器和分类器以端到端的方式,导致拓扑异质性,在产生对比样本时的语义相似性。关于半监督学习,无监督学习和转移学习的广泛实验展示了我们在图形对比学习中的最先进的自动支持者框架的优越性。此外,可视化结果进一步证实,与现有的视图生成方法相比,可学习的视图发生器可以提供更紧凑和语义有意义的对比样本。
translated by 谷歌翻译
图形神经网络(GNNS)通过考虑其内在的几何形状来扩展神经网络的成功到图形结构化数据。尽管根据图表学习基准的集合,已经对开发具有卓越性能的GNN模型进行了广泛的研究,但目前尚不清楚其探测给定模型的哪些方面。例如,他们在多大程度上测试模型利用图形结构与节点特征的能力?在这里,我们开发了一种原则性的方法来根据$ \ textit {敏感性配置文件} $进行基准测试数据集,该方法基于由于图形扰动的集合而导致的GNN性能变化了多少。我们的数据驱动分析提供了对GNN利用哪些基准测试数据特性的更深入的了解。因此,我们的分类法可以帮助选择和开发适当的图基准测试,并更好地评估未来的GNN方法。最后,我们在$ \ texttt {gtaxogym} $软件包中的方法和实现可扩展到多个图形预测任务类型和未来数据集。
translated by 谷歌翻译
Graph representation learning has emerged as a powerful technique for addressing real-world problems. Various downstream graph learning tasks have benefited from its recent developments, such as node classification, similarity search, and graph classification. However, prior arts on graph representation learning focus on domain specific problems and train a dedicated model for each graph dataset, which is usually non-transferable to out-of-domain data. Inspired by the recent advances in pre-training from natural language processing and computer vision, we design Graph Contrastive Coding (GCC) 1 -a self-supervised graph neural network pre-training framework-to capture the universal network topological properties across multiple networks. We design GCC's pre-training task as subgraph instance discrimination in and across networks and leverage contrastive learning to empower graph neural networks to learn the intrinsic and transferable structural representations. We conduct extensive experiments on three graph learning tasks and ten graph datasets. The results show that GCC pre-trained on a collection of diverse datasets can achieve competitive or better performance to its task-specific and trained-from-scratch counterparts. This suggests that the pre-training and fine-tuning paradigm presents great potential for graph representation learning.
translated by 谷歌翻译
In recent years, graph representation learning has achieved remarkable success while suffering from low-quality data problems. As a mature technology to improve data quality in computer vision, data augmentation has also attracted increasing attention in graph domain. For promoting the development of this emerging research direction, in this survey, we comprehensively review and summarize the existing graph data augmentation (GDAug) techniques. Specifically, we first summarize a variety of feasible taxonomies, and then classify existing GDAug studies based on fine-grained graph elements. Furthermore, for each type of GDAug technique, we formalize the general definition, discuss the technical details, and give schematic illustration. In addition, we also summarize common performance metrics and specific design metrics for constructing a GDAug evaluation system. Finally, we summarize the applications of GDAug from both data and model levels, as well as future directions.
translated by 谷歌翻译
对比度学习是图表学习中的有效无监督方法,对比度学习的关键组成部分在于构建正和负样本。以前的方法通常利用图中节点的接近度作为原理。最近,基于数据增强的对比度学习方法已进步以显示视觉域中的强大力量,一些作品将此方法从图像扩展到图形。但是,与图像上的数据扩展不同,图上的数据扩展远不那么直观,而且很难提供高质量的对比样品,这为改进留出了很大的空间。在这项工作中,通过引入一个对抗性图视图以进行数据增强,我们提出了一种简单但有效的方法,对抗图对比度学习(ARIEL),以在合理的约束中提取信息性的对比样本。我们开发了一种称为稳定训练的信息正则化的新技术,并使用子图抽样以进行可伸缩。我们通过将每个图形实例视为超级节点,从节点级对比度学习到图级。 Ariel始终优于在现实世界数据集上的节点级别和图形级分类任务的当前图对比度学习方法。我们进一步证明,面对对抗性攻击,Ariel更加强大。
translated by 谷歌翻译
Recently, contrastive learning (CL) has emerged as a successful method for unsupervised graph representation learning. Most graph CL methods first perform stochastic augmentation on the input graph to obtain two graph views and maximize the agreement of representations in the two views. Despite the prosperous development of graph CL methods, the design of graph augmentation schemes-a crucial component in CL-remains rarely explored. We argue that the data augmentation schemes should preserve intrinsic structures and attributes of graphs, which will force the model to learn representations that are insensitive to perturbation on unimportant nodes and edges. However, most existing methods adopt uniform data augmentation schemes, like uniformly dropping edges and uniformly shuffling features, leading to suboptimal performance. In this paper, we propose a novel graph contrastive representation learning method with adaptive augmentation that incorporates various priors for topological and semantic aspects of the graph. Specifically, on the topology level, we design augmentation schemes based on node centrality measures to highlight important connective structures. On the node attribute level, we corrupt node features by adding more noise to unimportant node features, to enforce the model to recognize underlying semantic information. We perform extensive experiments of node classification on a variety of real-world datasets. Experimental results demonstrate that our proposed method consistently outperforms existing state-of-the-art baselines and even surpasses some supervised counterparts, which validates the effectiveness of the proposed contrastive framework with adaptive augmentation. CCS CONCEPTS• Computing methodologies → Unsupervised learning; Neural networks; Learning latent representations.
translated by 谷歌翻译
Graph generative models have broad applications in biology, chemistry and social science. However, modelling and understanding the generative process of graphs is challenging due to the discrete and high-dimensional nature of graphs, as well as permutation invariance to node orderings in underlying graph distributions. Current leading autoregressive models fail to capture the permutation invariance nature of graphs for the reliance on generation ordering and have high time complexity. Here, we propose a continuous-time generative diffusion process for permutation invariant graph generation to mitigate these issues. Specifically, we first construct a forward diffusion process defined by a stochastic differential equation (SDE), which smoothly converts graphs within the complex distribution to random graphs that follow a known edge probability. Solving the corresponding reverse-time SDE, graphs can be generated from newly sampled random graphs. To facilitate the reverse-time SDE, we newly design a position-enhanced graph score network, capturing the evolving structure and position information from perturbed graphs for permutation equivariant score estimation. Under the evaluation of comprehensive metrics, our proposed generative diffusion process achieves competitive performance in graph distribution learning. Experimental results also show that GraphGDP can generate high-quality graphs in only 24 function evaluations, much faster than previous autoregressive models.
translated by 谷歌翻译