学习强大的表示是图形神经网络(GNN)的一个中心主题。它需要从输入图中炼制关键信息,而不是琐碎的模式,以丰富表示。为此,图表注意力和汇集方法占上风。他们主要遵循“学会参加”的范式。它最大限度地提高了上述子图和地面真理标签之间的相互信息。然而,这种训练范例易于捕获微级子图和标签之间的虚假相关性。这种杂散的相关性对分布(ID)测试评估有益,但在分布外(OOD)测试数据中引起差的概括。在这项工作中,我们从因果角度重新审视GNN建模。在我们的因果假设之上,琐碎的信息是关键信息和标签之间的混淆,它在它们之间打开了一个后门路径,使它们保持虚拟相关。因此,我们提出了一个新的解压缩训练范式(DTP),更好地减轻了批评信息的混淆效果并锁存,以提高表示和泛化能力。具体而言,我们采用注意模块解开关键的子图和微不足道的子图。然后我们使每个关键的子图相当与不同的琐碎子图相互作用,以实现稳定的预测。它允许GNN捕获一个更可靠的子图,其与标签的关系跨越不同的分布。我们对综合和现实世界数据集进行了广泛的实验,以证明有效性。
translated by 谷歌翻译
Graph machine learning has been extensively studied in both academia and industry. Although booming with a vast number of emerging methods and techniques, most of the literature is built on the in-distribution hypothesis, i.e., testing and training graph data are identically distributed. However, this in-distribution hypothesis can hardly be satisfied in many real-world graph scenarios where the model performance substantially degrades when there exist distribution shifts between testing and training graph data. To solve this critical problem, out-of-distribution (OOD) generalization on graphs, which goes beyond the in-distribution hypothesis, has made great progress and attracted ever-increasing attention from the research community. In this paper, we comprehensively survey OOD generalization on graphs and present a detailed review of recent advances in this area. First, we provide a formal problem definition of OOD generalization on graphs. Second, we categorize existing methods into three classes from conceptually different perspectives, i.e., data, model, and learning strategy, based on their positions in the graph machine learning pipeline, followed by detailed discussions for each category. We also review the theories related to OOD generalization on graphs and introduce the commonly used graph datasets for thorough evaluations. Finally, we share our insights on future research directions. This paper is the first systematic and comprehensive review of OOD generalization on graphs, to the best of our knowledge.
translated by 谷歌翻译
建议图表神经网络(GNNS)在不考虑训练和测试图之间的不可知分布的情况下,诱导GNN的泛化能力退化在分布外(OOD)设置。这种退化的根本原因是大多数GNN是基于I.I.D假设开发的。在这种设置中,GNN倾向于利用在培训中存在的微妙统计相关性用于预测,即使它是杂散的相关性。然而,这种杂散的相关性可能在测试环境中改变,导致GNN的失败。因此,消除了杂散相关的影响对于稳定的GNN来说是至关重要的。为此,我们提出了一个普遍的因果代表框架,称为稳定凝球。主要思想是首先从图数据中提取高级表示,并诉诸因因果推理的显着能力,以帮助模型摆脱虚假相关性。特别是,我们利用图形池化层以提取基于子图的表示作为高级表示。此外,我们提出了一种因果变量区别,以纠正偏置训练分布。因此,GNN将更多地集中在稳定的相关性上。对合成和现实世界ood图数据集的广泛实验良好地验证了所提出的框架的有效性,灵活性和可解释性。
translated by 谷歌翻译
大多数图形神经网络(GNN)通过学习输入图和标签之间的相关性来预测看不见的图的标签。但是,通过对具有严重偏见的训练图进行图形分类调查,我们发现GNN始终倾向于探索伪造的相关性以做出决定,即使因果关系始终存在。这意味着在此类偏见的数据集中接受培训的现有GNN将遭受概括能力差。通过在因果观点中分析此问题,我们发现从偏见图中解开和去偏置因果和偏见的潜在变量对于偏见至关重要。在此鼓舞下,我们提出了一个普遍的分解GNN框架,分别学习因果子结构和偏见子结构。特别是,我们设计了一个参数化的边蒙版生成器,以将输入图明确分为因果和偏置子图。然后,分别由因果/偏见感知损失函数监督的两个GNN模块进行培训,以编码因果关系和偏置子图表中的相应表示。通过分离的表示,我们合成了反事实无偏的训练样本,以进一步脱离因果变量和偏见变量。此外,为了更好地基于严重的偏见问题,我们构建了三个新的图形数据集,这些数据集具有可控的偏置度,并且更容易可视化和解释。实验结果很好地表明,我们的方法比现有基线实现了优越的概括性能。此外,由于学习的边缘面膜,该拟议的模型具有吸引人的解释性和可转让性。代码和数据可在以下网址获得:https://github.com/googlebaba/disc。
translated by 谷歌翻译
Out-of-distribution (OOD) generalization on graphs is drawing widespread attention. However, existing efforts mainly focus on the OOD issue of correlation shift. While another type, covariate shift, remains largely unexplored but is the focus of this work. From a data generation view, causal features are stable substructures in data, which play key roles in OOD generalization. While their complementary parts, environments, are unstable features that often lead to various distribution shifts. Correlation shift establishes spurious statistical correlations between environments and labels. In contrast, covariate shift means that there exist unseen environmental features in test data. Existing strategies of graph invariant learning and data augmentation suffer from limited environments or unstable causal features, which greatly limits their generalization ability on covariate shift. In view of that, we propose a novel graph augmentation strategy: Adversarial Causal Augmentation (AdvCA), to alleviate the covariate shift. Specifically, it adversarially augments the data to explore diverse distributions of the environments. Meanwhile, it keeps the causal features invariant across diverse environments. It maintains the environmental diversity while ensuring the invariance of the causal features, thereby effectively alleviating the covariate shift. Extensive experimental results with in-depth analyses demonstrate that AdvCA can outperform 14 baselines on synthetic and real-world datasets with various covariate shifts.
translated by 谷歌翻译
图表神经网络(GNNS)在测试和训练图数据来自相同分布时取得了令人印象深刻的性能。然而,现有的GNN缺乏分发的泛化能力,使得它们的性能在测试和训练图数据之间存在分布时显着降低。为了解决这个问题,在这项工作中,我们提出了一个用于在具有训练图的不同分布的看不见的分布的看不见的令人满意的令人满意的令人满意的通用图形神经网络(OOD-GNN)。我们所提出的OOD-GNN采用新颖的非线性图形表示去序方法,利用随机傅里叶特征,这鼓励模型通过迭代优化样本图权重和图形编码器来消除相关和无关的图表表示之间的统计依赖性。我们进一步设计了一个全局重量估计器,以学习训练图的权重,使得图形表示中的变量被迫独立。学习权重有助于图形编码器摆脱虚假相关性,并且反过来,更集中学习鉴别图形表示与地面真理标签之间的真实连接。我们进行广泛的实验,以验证两个合成和12个现实世界数据集的分发外概括能力,分配换档。结果表明,我们所提出的OOD-GNN显着优于最先进的基线。
translated by 谷歌翻译
Uncovering rationales behind predictions of graph neural networks (GNNs) has received increasing attention over recent years. Instance-level GNN explanation aims to discover critical input elements, like nodes or edges, that the target GNN relies upon for making predictions. Though various algorithms are proposed, most of them formalize this task by searching the minimal subgraph which can preserve original predictions. However, an inductive bias is deep-rooted in this framework: several subgraphs can result in the same or similar outputs as the original graphs. Consequently, they have the danger of providing spurious explanations and fail to provide consistent explanations. Applying them to explain weakly-performed GNNs would further amplify these issues. To address this problem, we theoretically examine the predictions of GNNs from the causality perspective. Two typical reasons of spurious explanations are identified: confounding effect of latent variables like distribution shift, and causal factors distinct from the original input. Observing that both confounding effects and diverse causal rationales are encoded in internal representations, we propose a simple yet effective countermeasure by aligning embeddings. Concretely, concerning potential shifts in the high-dimensional space, we design a distribution-aware alignment algorithm based on anchors. This new objective is easy to compute and can be incorporated into existing techniques with no or little effort. Theoretical analysis shows that it is in effect optimizing a more faithful explanation objective in design, which further justifies the proposed approach.
translated by 谷歌翻译
流行的图神经网络模型在图表学习方面取得了重大进展。但是,在本文中,我们发现了一个不断被忽视的现象:用完整图测试的预训练的图表学习模型的表现不佳,该模型用良好的图表测试。该观察结果表明,图中存在混杂因素,这可能会干扰模型学习语义信息,而当前的图表表示方法并未消除其影响。为了解决这个问题,我们建议强大的因果图表示学习(RCGRL)学习可靠的图形表示,以防止混杂效应。 RCGRL引入了一种主动方法,可以在无条件的力矩限制下生成仪器变量,该方法使图表学习模型能够消除混杂因素,从而捕获与下游预测有因果关系的歧视性信息。我们提供定理和证明,以保证拟议方法的理论有效性。从经验上讲,我们对合成数据集和多个基准数据集进行了广泛的实验。结果表明,与最先进的方法相比,RCGRL实现了更好的预测性能和泛化能力。
translated by 谷歌翻译
需要解释的图表学习是需要的,因为许多科学应用都取决于学习模型来从图形结构数据中收集见解。先前的工作主要集中在使用事后方法来解释预训练的模型(尤其是图形神经网络模型)。他们反对固有的可解释模型,因为对这些模型的良好解释通常是以其预测准确性为代价。而且,广泛使用的固有解释的注意力机制通常无法在图形学习任务中提供忠实的解释。在这项工作中,我们通过提出图形随机关注(GSAT)来解决这两个问题,这是一种来自信息瓶颈原理的注意机制。 GSAT利用随机关注来阻止从任务 - 核定图组件中的信息,同时学习降低随机性的注意力以选择与任务相关的子图以进行解释。 GSAT也可以通过随机注意机制应用于微调和解释预训练的模型。八个数据集的广泛实验表明,GSAT在解释AUC中的最高最高为20%$ \ uparrow $,而预测准确性则高于最高的最高$ \ uparrow $。
translated by 谷歌翻译
尽管最近在欧几里得数据(例如图像)上使用不变性原理(OOD)概括(例如图像),但有关图数据的研究仍然受到限制。与图像不同,图形的复杂性质给采用不变性原理带来了独特的挑战。特别是,图表上的分布变化可以以多种形式出现,例如属性和结构,因此很难识别不变性。此外,在欧几里得数据上通常需要的域或环境分区通常需要的图形可能非常昂贵。为了弥合这一差距,我们提出了一个新的框架,以捕获图形的不变性,以在各种分配变化下进行保证的OOD概括。具体而言,我们表征了具有因果模型的图形上的潜在分布变化,得出结论,当模型仅关注包含有关标签原因最多信息的子图时,可以实现图形上的OOD概括。因此,我们提出了一个信息理论目标,以提取最大地保留不变的阶级信息的所需子图。用这些子图学习不受分配变化的影响。对合成和现实世界数据集进行的广泛实验,包括在AI ADED药物发现中充满挑战的环境,验证了我们方法的上等OOD概括能力。
translated by 谷歌翻译
Machine learning models rely on various assumptions to attain high accuracy. One of the preliminary assumptions of these models is the independent and identical distribution, which suggests that the train and test data are sampled from the same distribution. However, this assumption seldom holds in the real world due to distribution shifts. As a result models that rely on this assumption exhibit poor generalization capabilities. Over the recent years, dedicated efforts have been made to improve the generalization capabilities of these models collectively known as -- \textit{domain generalization methods}. The primary idea behind these methods is to identify stable features or mechanisms that remain invariant across the different distributions. Many generalization approaches employ causal theories to describe invariance since causality and invariance are inextricably intertwined. However, current surveys deal with the causality-aware domain generalization methods on a very high-level. Furthermore, we argue that it is possible to categorize the methods based on how causality is leveraged in that method and in which part of the model pipeline is it used. To this end, we categorize the causal domain generalization methods into three categories, namely, (i) Invariance via Causal Data Augmentation methods which are applied during the data pre-processing stage, (ii) Invariance via Causal representation learning methods that are utilized during the representation learning stage, and (iii) Invariance via Transferring Causal mechanisms methods that are applied during the classification stage of the pipeline. Furthermore, this survey includes in-depth insights into benchmark datasets and code repositories for domain generalization methods. We conclude the survey with insights and discussions on future directions.
translated by 谷歌翻译
分数(OOD)学习涉及培训和测试数据遵循不同分布的方案。尽管在机器学习中已经深入研究了一般的OOD问题,但图形OOD只是一个新兴领域。目前,缺少针对图形OOD方法评估的系统基准。在这项工作中,我们旨在为图表开发一个被称为GOOD的OOD基准。我们明确地在协变量和概念变化和设计数据拆分之间进行了区分,以准确反映不同的变化。我们考虑图形和节点预测任务,因为在设计变化时存在关键差异。总体而言,Good包含8个具有14个域选择的数据集。当与协变量,概念和无移位结合使用时,我们获得了42个不同的分裂。我们在7种常见的基线方法上提供了10种随机运行的性能结果。这总共导致294个数据集模型组合。我们的结果表明,分布和OOD设置之间的性能差距很大。我们的结果还阐明了通过不同方法的协变量和概念转移之间的不同性能趋势。我们的良好基准是一个不断增长的项目,并希望随着该地区的发展,数量和种类繁多。可以通过$ \ href {https://github.com/divelab/good/} {\ text {https://github.com/divelab/good/good/}} $访问良好基准。
translated by 谷歌翻译
Graph Neural Networks (GNNs) are a powerful tool for machine learning on graphs. GNNs combine node feature information with the graph structure by recursively passing neural messages along edges of the input graph. However, incorporating both graph structure and feature information leads to complex models and explaining predictions made by GNNs remains unsolved. Here we propose GNNEXPLAINER, the first general, model-agnostic approach for providing interpretable explanations for predictions of any GNN-based model on any graph-based machine learning task. Given an instance, GNNEXPLAINER identifies a compact subgraph structure and a small subset of node features that have a crucial role in GNN's prediction. Further, GNNEXPLAINER can generate consistent and concise explanations for an entire class of instances. We formulate GNNEXPLAINER as an optimization task that maximizes the mutual information between a GNN's prediction and distribution of possible subgraph structures. Experiments on synthetic and real-world graphs show that our approach can identify important graph structures as well as node features, and outperforms alternative baseline approaches by up to 43.0% in explanation accuracy. GNNEXPLAINER provides a variety of benefits, from the ability to visualize semantically relevant structures to interpretability, to giving insights into errors of faulty GNNs.
translated by 谷歌翻译
理由定义为最能解释或支持机器学习模型预测的输入功能的子集。基本原理识别改善了神经网络在视觉和语言数据上的普遍性和解释性。在诸如分子和聚合物属性预测之类的图应用中,识别称为图理由的代表性子图结构在图神经网络的性能中起着至关重要的作用。现有的图形合并和/或分发干预方法缺乏示例,无法学习确定最佳图理由。在这项工作中,我们介绍了一个名为“环境替代”的新的增强操作,该操作自动创建虚拟数据示例以改善基本原理识别。我们提出了一个有效的框架,该框架在潜在空间中对真实和增强的示例进行基本环境分离和表示学习,以避免显式图解码和编码的高复杂性。与最近的技术相比,对七个分子和四个聚合物实际数据集进行的实验证明了拟议的基于增强的图形合理化框架的有效性和效率。
translated by 谷歌翻译
Graph neural networks (GNNs) have received remarkable success in link prediction (GNNLP) tasks. Existing efforts first predefine the subgraph for the whole dataset and then apply GNNs to encode edge representations by leveraging the neighborhood structure induced by the fixed subgraph. The prominence of GNNLP methods significantly relies on the adhoc subgraph. Since node connectivity in real-world graphs is complex, one shared subgraph is limited for all edges. Thus, the choices of subgraphs should be personalized to different edges. However, performing personalized subgraph selection is nontrivial since the potential selection space grows exponentially to the scale of edges. Besides, the inference edges are not available during training in link prediction scenarios, so the selection process needs to be inductive. To bridge the gap, we introduce a Personalized Subgraph Selector (PS2) as a plug-and-play framework to automatically, personally, and inductively identify optimal subgraphs for different edges when performing GNNLP. PS2 is instantiated as a bi-level optimization problem that can be efficiently solved differently. Coupling GNNLP models with PS2, we suggest a brand-new angle towards GNNLP training: by first identifying the optimal subgraphs for edges; and then focusing on training the inference model by using the sampled subgraphs. Comprehensive experiments endorse the effectiveness of our proposed method across various GNNLP backbones (GCN, GraphSage, NGCF, LightGCN, and SEAL) and diverse benchmarks (Planetoid, OGB, and Recommendation datasets). Our code is publicly available at \url{https://github.com/qiaoyu-tan/PS2}
translated by 谷歌翻译
图形神经网络(GNNS)在许多图形挖掘任务中取得了巨大的成功,这些任务从消息传递策略中受益,该策略融合了局部结构和节点特征,从而为更好的图表表示学习。尽管GNN成功,并且与其他类型的深神经网络相似,但发现GNN容易受到图形结构和节点特征的不明显扰动。已经提出了许多对抗性攻击,以披露在不同的扰动策略下创建对抗性例子的GNN的脆弱性。但是,GNNS对成功后门攻击的脆弱性直到最近才显示。在本文中,我们披露了陷阱攻击,这是可转移的图形后门攻击。核心攻击原则是用基于扰动的触发器毒化训练数据集,这可以导致有效且可转移的后门攻击。图形的扰动触发是通过通过替代模型的基于梯度的得分矩阵在图形结构上执行扰动动作来生成的。与先前的作品相比,陷阱攻击在几种方面有所不同:i)利用替代图卷积网络(GCN)模型来生成基于黑盒的后门攻击的扰动触发器; ii)它产生了没有固定模式的样品特异性扰动触发器; iii)在使用锻造中毒训练数据集训练时,在GNN的背景下,攻击转移到了不同​​的GNN模型中。通过对四个现实世界数据集进行广泛的评估,我们证明了陷阱攻击使用四个现实世界数据集在四个不同流行的GNN中构建可转移的后门的有效性
translated by 谷歌翻译
本文着重于由于看不见的分布变化而导致性能下降的图表上的分布概括。以前的图形域概括始终诉诸于不同源域之间的不变预测因子。但是,他们假设在培训期间提供了足够的源域,为现实应用带来了巨大挑战。相比之下,我们通过从源域中构造多个种群来提出一个新的图形域概括框架,称为DPS。具体而言,DPS旨在发现多个\ textbf {d} iverse和\ textbf {p}可redictable \ textbf {s}带有一组发电机的ubgraphs,即,子图是彼此不同的,但它们彼此不同,但所有这些都与相同的语义共享输入图。这些生成的源域被利用以学习跨域的\ textIt {Equi-Prestivical}图神经网络(GNN),这有望很好地概括到看不见的目标域。通常,DPS是模型不合时宜的,可以与各种GNN骨架合并。节点级别和图形基准测试的广泛实验表明,所提出的DPS为各种图形域概括任务实现了令人印象深刻的性能。
translated by 谷歌翻译
近年来,自我监督学习(SSL)已广泛探索。特别是,生成的SSL在自然语言处理和其他AI领域(例如BERT和GPT的广泛采用)中获得了新的成功。尽管如此,对比度学习 - 严重依赖结构数据的增强和复杂的培训策略,这是图SSL的主要方法,而迄今为止,生成SSL在图形上的进度(尤其是GAES)尚未达到潜在的潜力。正如其他领域所承诺的。在本文中,我们确定并检查对GAE的发展产生负面影响的问题,包括其重建目标,训练鲁棒性和错误指标。我们提出了一个蒙版的图形自动编码器Graphmae,该图可以减轻这些问题,以预处理生成性自我监督图。我们建议没有重建图形结构,而是提议通过掩盖策略和缩放余弦误差将重点放在特征重建上,从而使GraphMae的强大训练受益。我们在21个公共数据集上进行了大量实验,以实现三个不同的图形学习任务。结果表明,Graphmae-A简单的图形自动编码器具有仔细的设计-CAN始终在对比度和生成性最新基准相比,始终产生优于性的表现。这项研究提供了对图自动编码器的理解,并证明了在图上的生成自我监督预训练的潜力。
translated by 谷歌翻译
Many applications of machine learning require a model to make accurate predictions on test examples that are distributionally different from training ones, while task-specific labels are scarce during training. An effective approach to this challenge is to pre-train a model on related tasks where data is abundant, and then fine-tune it on a downstream task of interest. While pre-training has been effective in many language and vision domains, it remains an open question how to effectively use pre-training on graph datasets. In this paper, we develop a new strategy and self-supervised methods for pre-training Graph Neural Networks (GNNs). The key to the success of our strategy is to pre-train an expressive GNN at the level of individual nodes as well as entire graphs so that the GNN can learn useful local and global representations simultaneously. We systematically study pre-training on multiple graph classification datasets. We find that naïve strategies, which pre-train GNNs at the level of either entire graphs or individual nodes, give limited improvement and can even lead to negative transfer on many downstream tasks. In contrast, our strategy avoids negative transfer and improves generalization significantly across downstream tasks, leading up to 9.4% absolute improvements in ROC-AUC over non-pre-trained models and achieving state-of-the-art performance for molecular property prediction and protein function prediction.However, pre-training on graph datasets remains a hard challenge. Several key studies (
translated by 谷歌翻译
无监督的图形表示学习是图形数据的非琐碎主题。在结构化数据的无监督代表学习中对比学习和自我监督学习的成功激发了图表上的类似尝试。使用对比损耗的当前无监督的图形表示学习和预培训主要基于手工增强图数据之间的对比度。但是,由于不可预测的不变性,图数据增强仍然没有很好地探索。在本文中,我们提出了一种新颖的协作图形神经网络对比学习框架(CGCL),它使用多个图形编码器来观察图形。不同视图观察的特征充当了图形编码器之间对比学习的图表增强,避免了任何扰动以保证不变性。 CGCL能够处理图形级和节点级表示学习。广泛的实验表明CGCL在无监督的图表表示学习中的优势以及图形表示学习的手工数据增强组合的非必要性。
translated by 谷歌翻译