领先的图对比度学习(GCL)方法在两个时尚中执行图形增强:(1)随机损坏锚图,这可能会导致语义信息的丢失,或(2)使用域知识维护显着特征,这破坏了对概括的概括其他域。从不变性看GCL时,我们认为高性能的增强应保留有关实例歧视的锚图的显着语义。为此,我们将GCL与不变的理由发现联系起来,并提出了一个新的框架,即理由吸引图形对比度学习(RGCL)。具体而言,没有监督信号,RGCL使用基本原理生成器来揭示有关图形歧视的显着特征作为理由,然后为对比度学习创建理由吸引的视图。这种理由意识到的预训练方案赋予了骨干模型具有强大的表示能力,从而进一步促进了下游任务的微调。在MNIST-SUPERPIXEL和MUTAG数据集上,对发现的理由的视觉检查展示了基本原理生成器成功捕获了显着特征(即区分图中的语义节点)。在生化分子和社交网络基准数据集上,RGCL的最新性能证明了理由意识到对比度学习的有效性。我们的代码可在https://github.com/lsh0520/rgcl上找到。
translated by 谷歌翻译
对比学习已被广​​泛应用于图形表示学习,其中观测发生器在产生有效的对比样本方面发挥着重要作用。大多数现有的对比学习方法采用预定义的视图生成方法,例如节点滴或边缘扰动,这通常不能适应输入数据或保持原始语义结构。为了解决这个问题,我们提出了一份名为自动化图形对比学习(AutoGCL)的小说框架。具体而言,AutoGCL采用一组由自动增强策略协调的一组学习图形视图生成器,其中每个图形视图生成器都会学习输入调节的图形的概率分布。虽然AutoGCL中的图形视图发生器在生成每个对比样本中保留原始图的最代表性结构,但自动增强学会在整个对比学习程序中介绍适当的增强差异的政策。此外,AutoGCL采用联合培训策略,以培训学习的视图发生器,图形编码器和分类器以端到端的方式,导致拓扑异质性,在产生对比样本时的语义相似性。关于半监督学习,无监督学习和转移学习的广泛实验展示了我们在图形对比学习中的最先进的自动支持者框架的优越性。此外,可视化结果进一步证实,与现有的视图生成方法相比,可学习的视图发生器可以提供更紧凑和语义有意义的对比样本。
translated by 谷歌翻译
Generalizable, transferrable, and robust representation learning on graph-structured data remains a challenge for current graph neural networks (GNNs). Unlike what has been developed for convolutional neural networks (CNNs) for image data, self-supervised learning and pre-training are less explored for GNNs. In this paper, we propose a graph contrastive learning (GraphCL) framework for learning unsupervised representations of graph data. We first design four types of graph augmentations to incorporate various priors. We then systematically study the impact of various combinations of graph augmentations on multiple datasets, in four different settings: semi-supervised, unsupervised, and transfer learning as well as adversarial attacks. The results show that, even without tuning augmentation extents nor using sophisticated GNN architectures, our GraphCL framework can produce graph representations of similar or better generalizability, transferrability, and robustness compared to state-of-the-art methods. We also investigate the impact of parameterized graph augmentation extents and patterns, and observe further performance gains in preliminary experiments. Our codes are available at: https://github.com/Shen-Lab/GraphCL.
translated by 谷歌翻译
图形对比学习(GCL)已成为学习图形无监督表示的有效工具。关键思想是通过数据扩展最大化每个图的两个增强视图之间的一致性。现有的GCL模型主要集中在给定情况下的所有图表上应用\ textit {相同的增强策略}。但是,实际图通常不是单态,而是各种本质的抽象。即使在相同的情况下(例如,大分子和在线社区),不同的图形可能需要各种增强来执行有效的GCL。因此,盲目地增强所有图表而不考虑其个人特征可能会破坏GCL艺术的表现。 {a} u Mentigation(GPA),通过允许每个图选择自己的合适的增强操作来推进常规GCL。本质上,GPA根据其拓扑属性和节点属性通过可学习的增强选择器为每个图定制了量身定制的增强策略,该策略是插件模块,可以通过端到端的下游GCL型号有效地训练。来自不同类型和域的11个基准图的广泛实验证明了GPA与最先进的竞争对手的优势。此外,通过可视化不同类型的数据集中学习的增强分布,我们表明GPA可以有效地识别最合适的数据集每个图的增强基于其特征。
translated by 谷歌翻译
分子表示学习有助于多个下游任务,例如分子性质预测和药物设计。为了适当地代表分子,图形对比学习是一个有前途的范式,因为它利用自我监督信号并没有人类注释要求。但是,先前的作品未能将基本域名知识纳入图表语义,因此忽略了具有共同属性的原子之间的相关性,但不通过键连接连接。为了解决这些问题,我们构建化学元素知识图(KG),总结元素之间的微观关联,并提出了一种用于分子代表学习的新颖知识增强的对比学习(KCL)框架。 KCL框架由三个模块组成。第一个模块,知识引导的图形增强,基于化学元素kg增强原始分子图。第二模块,知识意识的图形表示,利用用于原始分子图的公共曲线图编码器和通过神经网络(KMPNN)的知识感知消息来提取分子表示来编码增强分子图中的复杂信息。最终模块是一种对比目标,在那里我们在分子图的这两个视图之间最大化协议。广泛的实验表明,KCL获得了八个分子数据集上的最先进基线的优异性能。可视化实验适当地解释了在增强分子图中从原子和属性中了解的KCL。我们的代码和数据可用于补充材料。
translated by 谷歌翻译
Graph Contrastive Learning (GCL) has recently drawn much research interest for learning generalizable node representations in a self-supervised manner. In general, the contrastive learning process in GCL is performed on top of the representations learned by a graph neural network (GNN) backbone, which transforms and propagates the node contextual information based on its local neighborhoods. However, nodes sharing similar characteristics may not always be geographically close, which poses a great challenge for unsupervised GCL efforts due to their inherent limitations in capturing such global graph knowledge. In this work, we address their inherent limitations by proposing a simple yet effective framework -- Simple Neural Networks with Structural and Semantic Contrastive Learning} (S^3-CL). Notably, by virtue of the proposed structural and semantic contrastive learning algorithms, even a simple neural network can learn expressive node representations that preserve valuable global structural and semantic patterns. Our experiments demonstrate that the node representations learned by S^3-CL achieve superior performance on different downstream tasks compared with the state-of-the-art unsupervised GCL methods. Implementation and more experimental details are publicly available at \url{https://github.com/kaize0409/S-3-CL.}
translated by 谷歌翻译
图级表示在各种现实世界中至关重要,例如预测分子的特性。但是实际上,精确的图表注释通常非常昂贵且耗时。为了解决这个问题,图形对比学习构造实例歧视任务,将正面对(同一图的增强对)汇总在一起,并将负面对(不同图的增强对)推开,以进行无监督的表示。但是,由于为了查询,其负面因素是从所有图中均匀抽样的,因此现有方法遭受关键采样偏置问题的损失,即,否定物可能与查询具有相同的语义结构,从而导致性能降解。为了减轻这种采样偏见问题,在本文中,我们提出了一种典型的图形对比度学习(PGCL)方法。具体而言,PGCL通过将语义相似的图形群群归为同一组的群集数据的基础语义结构,并同时鼓励聚类的一致性,以实现同一图的不同增强。然后给出查询,它通过从与查询群集不同的群集中绘制图形进行负采样,从而确保查询及其阴性样本之间的语义差异。此外,对于查询,PGCL根据其原型(集群质心)和查询原型之间的距离进一步重新重新重新重新重新享受其负样本,从而使那些具有中等原型距离的负面因素具有相对较大的重量。事实证明,这种重新加权策略比统一抽样更有效。各种图基准的实验结果证明了我们的PGCL比最新方法的优势。代码可在https://github.com/ha-lins/pgcl上公开获取。
translated by 谷歌翻译
由于现实世界图形/网络数据中的广泛标签稀缺问题,因此,自我监督的图形神经网络(GNN)非常需要。曲线图对比度学习(GCL),通过训练GNN以其不同的增强形式最大化相同图表之间的表示之间的对应关系,即使在不使用标签的情况下也可以产生稳健和可转移的GNN。然而,GNN由传统的GCL培训经常冒险捕获冗余图形特征,因此可能是脆弱的,并在下游任务中提供子对比。在这里,我们提出了一种新的原理,称为普通的普通GCL(AD-GCL),其使GNN能够通过优化GCL中使用的对抗性图形增强策略来避免在训练期间捕获冗余信息。我们将AD-GCL与理论解释和设计基于可训练的边缘滴加图的实际实例化。我们通过与最先进的GCL方法进行了实验验证了AD-GCL,并在无监督,6 \%$ 14 \%$ 6 \%$ 14 \%$ 6 \%$ 6 \%$ 3 \%$ 3 \%$达到半监督总体学习设置,具有18个不同的基准数据集,用于分子属性回归和分类和社交网络分类。
translated by 谷歌翻译
无监督的图形表示学习是图形数据的非琐碎主题。在结构化数据的无监督代表学习中对比学习和自我监督学习的成功激发了图表上的类似尝试。使用对比损耗的当前无监督的图形表示学习和预培训主要基于手工增强图数据之间的对比度。但是,由于不可预测的不变性,图数据增强仍然没有很好地探索。在本文中,我们提出了一种新颖的协作图形神经网络对比学习框架(CGCL),它使用多个图形编码器来观察图形。不同视图观察的特征充当了图形编码器之间对比学习的图表增强,避免了任何扰动以保证不变性。 CGCL能够处理图形级和节点级表示学习。广泛的实验表明CGCL在无监督的图表表示学习中的优势以及图形表示学习的手工数据增强组合的非必要性。
translated by 谷歌翻译
尽管有关超图的机器学习吸引了很大的关注,但大多数作品都集中在(半)监督的学习上,这可能会导致繁重的标签成本和不良的概括。最近,对比学习已成为一种成功的无监督表示学习方法。尽管其他领域中对比度学习的发展繁荣,但对超图的对比学习仍然很少探索。在本文中,我们提出了Tricon(三个方向对比度学习),这是对超图的对比度学习的一般框架。它的主要思想是三个方向对比度,具体来说,它旨在在两个增强视图中最大化同一节点之间的协议(a),(b)在同一节点之间以及(c)之间,每个组之间的成员及其成员之间的协议(b) 。加上简单但令人惊讶的有效数据增强和负抽样方案,这三种形式的对比使Tricon能够在节点嵌入中捕获显微镜和介观结构信息。我们使用13种基线方法,5个数据集和两个任务进行了广泛的实验,这证明了Tricon的有效性,最明显的是,Tricon始终优于无监督的竞争对手,而且(半)受监督的竞争对手,大多数是由大量的节点分类的大量差额。
translated by 谷歌翻译
关于图表的深度学习最近吸引了重要的兴趣。然而,大多数作品都侧重于(半)监督学习,导致缺点包括重标签依赖,普遍性差和弱势稳健性。为了解决这些问题,通过良好设计的借口任务在不依赖于手动标签的情况下提取信息知识的自我监督学习(SSL)已成为图形数据的有希望和趋势的学习范例。与计算机视觉和自然语言处理等其他域的SSL不同,图表上的SSL具有独家背景,设计理念和分类。在图表的伞下自我监督学习,我们对采用图表数据采用SSL技术的现有方法及时及全面的审查。我们构建一个统一的框架,数学上正式地规范图表SSL的范例。根据借口任务的目标,我们将这些方法分为四类:基于生成的,基于辅助性的,基于对比的和混合方法。我们进一步描述了曲线图SSL在各种研究领域的应用,并总结了绘图SSL的常用数据集,评估基准,性能比较和开源代码。最后,我们讨论了该研究领域的剩余挑战和潜在的未来方向。
translated by 谷歌翻译
Graph machine learning has been extensively studied in both academia and industry. Although booming with a vast number of emerging methods and techniques, most of the literature is built on the in-distribution hypothesis, i.e., testing and training graph data are identically distributed. However, this in-distribution hypothesis can hardly be satisfied in many real-world graph scenarios where the model performance substantially degrades when there exist distribution shifts between testing and training graph data. To solve this critical problem, out-of-distribution (OOD) generalization on graphs, which goes beyond the in-distribution hypothesis, has made great progress and attracted ever-increasing attention from the research community. In this paper, we comprehensively survey OOD generalization on graphs and present a detailed review of recent advances in this area. First, we provide a formal problem definition of OOD generalization on graphs. Second, we categorize existing methods into three classes from conceptually different perspectives, i.e., data, model, and learning strategy, based on their positions in the graph machine learning pipeline, followed by detailed discussions for each category. We also review the theories related to OOD generalization on graphs and introduce the commonly used graph datasets for thorough evaluations. Finally, we share our insights on future research directions. This paper is the first systematic and comprehensive review of OOD generalization on graphs, to the best of our knowledge.
translated by 谷歌翻译
最近的作品以自我监督的方式探索学习图表表示。在图形对比学习中,基准方法应用各种图形增强方法。但是,大多数增强方法都是不可学习的,这导致发出不束缚的增强图。这种增强可以缩短曲线图对比学学习方法的表现能力。因此,我们激励我们的方法通过可学习的图形增强器来生成增强图,称为元图形增强器(Mega)。然后,我们阐明了“良好”的图形增强必须在特征级别的实例级别和信息性上具有均匀性。为此,我们提出了一种新颖的方法来学习图形增强者,可以以统一和信息性产生增强。图表增强器的目的是促进我们的特征提取网络,以学习更辨别的特征表示,这激励我们提出元学范式。经验上,多个基准数据集的实验表明,Mega优于图形自我监督学习任务中的最先进的方法。进一步的实验研究证明了巨型术语的有效性。
translated by 谷歌翻译
Inspired by the impressive success of contrastive learning (CL), a variety of graph augmentation strategies have been employed to learn node representations in a self-supervised manner. Existing methods construct the contrastive samples by adding perturbations to the graph structure or node attributes. Although impressive results are achieved, it is rather blind to the wealth of prior information assumed: with the increase of the perturbation degree applied on the original graph, 1) the similarity between the original graph and the generated augmented graph gradually decreases; 2) the discrimination between all nodes within each augmented view gradually increases. In this paper, we argue that both such prior information can be incorporated (differently) into the contrastive learning paradigm following our general ranking framework. In particular, we first interpret CL as a special case of learning to rank (L2R), which inspires us to leverage the ranking order among positive augmented views. Meanwhile, we introduce a self-ranking paradigm to ensure that the discriminative information among different nodes can be maintained and also be less altered to the perturbations of different degrees. Experiment results on various benchmark datasets verify the effectiveness of our algorithm compared with the supervised and unsupervised models.
translated by 谷歌翻译
灵感来自最近应用于图像上的自我监督方法的成功,图形结构数据的自我监督学习已经看到迅速增长,特别是基于增强的对比方法。但是,我们认为没有精心设计的增强技术,图形上的增强可能是任意行为的,因为图形的底层语义可以急剧地改变。因此,现有增强的方法的性能高度依赖于增强方案的选择,即与增强相关联的超级参数。在本文中,我们提出了一种名为AFGRL的图表的一种新的增强自我监督学习框架。具体地,我们通过发现与图形共享本地结构信息和全局语义的节点来生成图表的替代视图。各种数据集的各种节点级任务,即节点分类,群集和相似性搜索的广泛实验证明了AFGRL的优越性。 AFGRL的源代码可在https://github.com/namkyeong/afgrl中获得。
translated by 谷歌翻译
Many applications of machine learning require a model to make accurate predictions on test examples that are distributionally different from training ones, while task-specific labels are scarce during training. An effective approach to this challenge is to pre-train a model on related tasks where data is abundant, and then fine-tune it on a downstream task of interest. While pre-training has been effective in many language and vision domains, it remains an open question how to effectively use pre-training on graph datasets. In this paper, we develop a new strategy and self-supervised methods for pre-training Graph Neural Networks (GNNs). The key to the success of our strategy is to pre-train an expressive GNN at the level of individual nodes as well as entire graphs so that the GNN can learn useful local and global representations simultaneously. We systematically study pre-training on multiple graph classification datasets. We find that naïve strategies, which pre-train GNNs at the level of either entire graphs or individual nodes, give limited improvement and can even lead to negative transfer on many downstream tasks. In contrast, our strategy avoids negative transfer and improves generalization significantly across downstream tasks, leading up to 9.4% absolute improvements in ROC-AUC over non-pre-trained models and achieving state-of-the-art performance for molecular property prediction and protein function prediction.However, pre-training on graph datasets remains a hard challenge. Several key studies (
translated by 谷歌翻译
Existing graph contrastive learning methods rely on augmentation techniques based on random perturbations (e.g., randomly adding or dropping edges and nodes). Nevertheless, altering certain edges or nodes can unexpectedly change the graph characteristics, and choosing the optimal perturbing ratio for each dataset requires onerous manual tuning. In this paper, we introduce Implicit Graph Contrastive Learning (iGCL), which utilizes augmentations in the latent space learned from a Variational Graph Auto-Encoder by reconstructing graph topological structure. Importantly, instead of explicitly sampling augmentations from latent distributions, we further propose an upper bound for the expected contrastive loss to improve the efficiency of our learning algorithm. Thus, graph semantics can be preserved within the augmentations in an intelligent way without arbitrary manual design or prior human knowledge. Experimental results on both graph-level and node-level tasks show that the proposed method achieves state-of-the-art performance compared to other benchmarks, where ablation studies in the end demonstrate the effectiveness of modules in iGCL.
translated by 谷歌翻译
最近,测试时间适应(TTA)由于其处理现实世界中的分销转移问题而引起了越来越多的关注。与用于图像数据的卷积神经网络(CNN)开发的内容不同,图形神经网络(GNN)的探索较少。仍然缺乏针对具有不规则结构的图的有效算法。在本文中,我们提出了一种新颖的测试时间适应策略,称为图形伪群体对比度(GAPGC),用于图神经网络TTA,以更好地适应非分布(OOD)测试数据。具体而言,GAPGC在TTA期间采用了对比度学习变体作为一项自制任务,配备了对抗性可学习的增强器和组伪阳性样本,以增强自我监督任务与主要任务之间的相关性,从而提高主要任务。此外,我们提供了理论上的证据,表明GAPGC可以从信息理论的角度提取主要任务的最小信息。关于分子支架OOD数据集的广泛实验表明,所提出的方法在GNN上实现了最先进的性能。
translated by 谷歌翻译
Inspired by the success of contrastive learning (CL) in computer vision and natural language processing, graph contrastive learning (GCL) has been developed to learn discriminative node representations on graph datasets. However, the development of GCL on Heterogeneous Information Networks (HINs) is still in the infant stage. For example, it is unclear how to augment the HINs without substantially altering the underlying semantics, and how to design the contrastive objective to fully capture the rich semantics. Moreover, early investigations demonstrate that CL suffers from sampling bias, whereas conventional debiasing techniques are empirically shown to be inadequate for GCL. How to mitigate the sampling bias for heterogeneous GCL is another important problem. To address the aforementioned challenges, we propose a novel Heterogeneous Graph Contrastive Multi-view Learning (HGCML) model. In particular, we use metapaths as the augmentation to generate multiple subgraphs as multi-views, and propose a contrastive objective to maximize the mutual information between any pairs of metapath-induced views. To alleviate the sampling bias, we further propose a positive sampling strategy to explicitly select positives for each node via jointly considering semantic and structural information preserved on each metapath view. Extensive experiments demonstrate HGCML consistently outperforms state-of-the-art baselines on five real-world benchmark datasets.
translated by 谷歌翻译
自我监督的学习逐渐被出现为一种强大的图形表示学习技术。然而,在图表数据上进行可转换,概括和强大的表示学习仍然是对预训练图形神经网络的挑战。在本文中,我们提出了一种简单有效的自我监督的自我监督的预训练策略,命名为成对半图歧视(PHD),明确地预先在图形级别进行了图形神经网络。 PHD被设计为简单的二进制分类任务,以辨别两个半图是否来自同一源。实验表明,博士学位是一种有效的预训练策略,与最先进的策略相比,在13图分类任务上提供了可比或优越的性能,并在与节点级策略结合时实现了显着的改进。此外,所学习代表的可视化透露,博士策略确实赋予了模型来学习像分子支架等图形级知识。这些结果已将博士学位作为图形级别代表学习中的强大有效的自我监督的学习策略。
translated by 谷歌翻译