基于分解的模型(FMS),例如Distmult,在知识图完成(KGC)任务中享有持久的成功,通常优于图形神经网络(GNNS)。但是,与GNN不同,FMS难以合并节点特征并概括在归纳环境中看不见的节点。我们的工作通过提出重构GNN来弥合FMS和GNN之间的差距。这种新的体系结构借鉴了两种建模范式,以前在很大程度上被认为是不结合的。具体地说,使用消息通讯的形式主义,我们通过将梯度下降程序重新定义为消息传播操作来展示如何将FMS施加为GNN,这构成了我们重构GNN的基础。在众多成熟的KGC基准测试中,我们的重构GNN可以实现与FMS相当的转导性能以及最先进的归纳性能,同时使用较少的参数阶数。
translated by 谷歌翻译
链接预测是图形上非常基本的任务。在本文中受到传统路径的方法的启发,我们提出了一种基于链路预测路径的一般和灵活的表示学习框架。具体地,我们将一对节点的表示定义为所有路径表示的广义和,每个路径表示为路径中的边缘表示的广义乘积。通过贝尔曼-Ford算法来解决最短路径问题,我们表明,所提出的路径配方可以通过广义的Bellman-Ford算法有效地解决。为了进一步提高路径制构的能力,我们提出了神经贝尔曼 - 福特网络(NBFNET),这是一种全图神经网络框架,其解决了通过广义Bellman-Ford算法中的学习运算符的路径制定。 NBFNET使用3个神经元件,即指示器,消息和聚合函数参数,即分别对应于边界条件,乘法运算符和求和运算符。 NBFNET非常一般,涵盖许多传统的基于路径的方法,并且可以应用于转导和归纳设置的同质图和多关系图(例如,知识图表)。两个均匀图表和知识图表的实验表明,所提出的NBFNET在转换和归纳设置中的大幅度优于现有方法,实现了新的最先进的结果。
translated by 谷歌翻译
In knowledge graph completion (KGC), predicting triples involving emerging entities and/or relations, which are unseen when the KG embeddings are learned, has become a critical challenge. Subgraph reasoning with message passing is a promising and popular solution. Some recent methods have achieved good performance, but they (i) usually can only predict triples involving unseen entities alone, failing to address more realistic fully inductive situations with both unseen entities and unseen relations, and (ii) often conduct message passing over the entities with the relation patterns not fully utilized. In this study, we propose a new method named RMPI which uses a novel Relational Message Passing network for fully Inductive KGC. It passes messages directly between relations to make full use of the relation patterns for subgraph reasoning with new techniques on graph transformation, graph pruning, relation-aware neighborhood attention, addressing empty subgraphs, etc., and can utilize the relation semantics defined in the ontological schema of KG. Extensive evaluation on multiple benchmarks has shown the effectiveness of techniques involved in RMPI and its better performance compared with the existing methods that support fully inductive KGC. RMPI is also comparable to the state-of-the-art partially inductive KGC methods with very promising results achieved. Our codes and data are available at https://github.com/zjukg/RMPI.
translated by 谷歌翻译
知识图(kgs)由于能够存储适用于许多领域的关系知识的能力,因此有助于多种应用。尽管在创造和维护方面进行了巨大的努力,但即使是最大的公斤也远非完整。因此,KG完成(KGC)已成为KG研究最关键的任务之一。最近,该领域的大量文献围绕着使用图神经网络(GNN)学习强大的嵌入,从而利用KGS中的拓扑结构。具体而言,已经做出了专门的努力,以扩展GNN,通常是为简单的同质和单一相关图设计的,以通过设计更复杂的聚合方案而不是相邻节点(关键的节点)(通过设计更复杂的聚合方案)(为GNN绩效)适当利用多关系信息。这些方法的成功自然归因于GNN在简单的多层感知器(MLP)模型上使用,这是由于它们的附加聚合功能。在这项工作中,我们发现简单的MLP模型能够达到与GNN的可比性能,这表明聚集可能并不像以前那样重要。通过进一步的探索,我们显示出仔细的评分功能和损失功能设计对KGC模型性能的影响要大得多,并且实际上不需要聚集。这表明了评分功能设计,损失功能设计和先前工作中的聚集结合,并有很有希望的见解当今最先进的KGC方法的可伸缩性,以及对KGC任务更合适的聚合设计的仔细注意明天。该实现可在线获得:https://github.com/juanhui28/are_mpnns_helpful。
translated by 谷歌翻译
Knowledge graph (KG) link prediction aims to infer new facts based on existing facts in the KG. Recent studies have shown that using the graph neighborhood of a node via graph neural networks (GNNs) provides more useful information compared to just using the query information. Conventional GNNs for KG link prediction follow the standard message-passing paradigm on the entire KG, which leads to over-smoothing of representations and also limits their scalability. On a large scale, it becomes computationally expensive to aggregate useful information from the entire KG for inference. To address the limitations of existing KG link prediction frameworks, we propose a novel retrieve-and-read framework, which first retrieves a relevant subgraph context for the query and then jointly reasons over the context and the query with a high-capacity reader. As part of our exemplar instantiation for the new framework, we propose a novel Transformer-based GNN as the reader, which incorporates graph-based attention structure and cross-attention between query and context for deep fusion. This design enables the model to focus on salient context information relevant to the query. Empirical results on two standard KG link prediction datasets demonstrate the competitive performance of the proposed method.
translated by 谷歌翻译
最近,图形神经网络(GNNS)在各种现实情景中获得了普及。尽管取得了巨大成功,但GNN的建筑设计严重依赖于体力劳动。因此,自动化图形神经网络(Autopmn)引起了研究界的兴趣和关注,近年来显着改善。然而,现有的autopnn工作主要采用隐式方式来模拟并利用图中的链接信息,这对图中的链路预测任务不充分规范化,并限制了自动启动的其他图表任务。在本文中,我们介绍了一个新的Autognn工作,该工作明确地模拟了缩写为autogel的链接信息。以这种方式,AutoGel可以处理链路预测任务并提高Autognns对节点分类和图形分类任务的性能。具体地,AutoGel提出了一种新的搜索空间,包括层内和层间设计中的各种设计尺寸,并采用更强大的可分辨率搜索算法,以进一步提高效率和有效性。基准数据集的实验结果展示了自动池上的优势在几个任务中。
translated by 谷歌翻译
在本文中,我们提供了一种使用图形神经网络(GNNS)的理论,用于多节点表示学习(我们有兴趣学习一组多个节点的表示)。我们知道GNN旨在学习单节点表示。当我们想学习涉及多个节点的节点集表示时,先前作品中的常见做法是直接将GNN学习的多节点表示与节点集的关节表示。在本文中,我们显示了这种方法的基本限制,即无法捕获节点集中节点之间的依赖性,并且认为直接聚合各个节点表示不会导致多个节点的有效关节表示。然后,我们注意到,以前的一些成功的工作作品用于多节点表示学习,包括密封,距离编码和ID-GNN,所有使用的节点标记。这些方法根据应用GNN之前的与目标节点集的关系,首先标记图中的节点。然后,在标记的图表中获得的节点表示被聚合到节点集表示中。通过调查其内部机制,我们将这些节点标记技术统一到单个和最基本的形式,即标记技巧。我们证明,通过标记技巧,可以获得足够富有表现力的GNN学习最具表现力的节点集表示,因此原则上可以解决节点集的任何联合学习任务。关于一个重要的双节点表示学习任务,链接预测,验证了我们理论的实验。我们的工作建立了使用GNN在节点集上使用GNN进行联合预测任务的理论基础。
translated by 谷歌翻译
开发用于训练图形的可扩展解决方案,用于链路预测任务的Neural网络(GNNS)由于具有高计算成本和巨大内存占用的高数据依赖性,因此由于高数据依赖性而具有挑战性。我们提出了一种新的方法,用于缩放知识图形嵌入模型的培训,以满足这些挑战。为此,我们提出了以下算法策略:自给自足的分区,基于约束的负采样和边缘迷你批量培训。两者都是分区策略和基于约束的负面采样,避免在训练期间交叉分区数据传输。在我们的实验评估中,我们表明,我们基于GNN的知识图形嵌入模型的缩放解决方案在基准数据集中实现了16倍的加速,同时将可比的模型性能作为标准度量的非分布式方法。
translated by 谷歌翻译
Learning node embeddings that capture a node's position within the broader graph structure is crucial for many prediction tasks on graphs. However, existing Graph Neural Network (GNN) architectures have limited power in capturing the position/location of a given node with respect to all other nodes of the graph. Here we propose Position-aware Graph Neural Networks (P-GNNs), a new class of GNNs for computing position-aware node embeddings. P-GNN first samples sets of anchor nodes, computes the distance of a given target node to each anchor-set, and then learns a non-linear distance-weighted aggregation scheme over the anchor-sets. This way P-GNNs can capture positions/locations of nodes with respect to the anchor nodes. P-GNNs have several advantages: they are inductive, scalable, and can incorporate node feature information. We apply P-GNNs to multiple prediction tasks including link prediction and community detection. We show that P-GNNs consistently outperform state of the art GNNs, with up to 66% improvement in terms of the ROC AUC score.Node embedding methods can be categorized into Graph Neural Networks (GNNs) approaches (Scarselli et al., 2009),
translated by 谷歌翻译
事实证明,信息提取方法可有效从结构化或非结构化数据中提取三重。以(头部实体,关系,尾部实体)形式组织这样的三元组的组织称为知识图(kgs)。当前的大多数知识图都是不完整的。为了在下游任务中使用kgs,希望预测kgs中缺少链接。最近,通过将实体和关系嵌入到低维的矢量空间中,旨在根据先前访问的三元组来预测三元组,从而对KGS表示不同的方法。根据如何独立或依赖对三元组进行处理,我们将知识图完成的任务分为传统和图形神经网络表示学习,并更详细地讨论它们。在传统的方法中,每个三重三倍将独立处理,并在基于GNN的方法中进行处理,三倍也考虑了他们的当地社区。查看全文
translated by 谷歌翻译
Formulating and answering logical queries is a standard communication interface for knowledge graphs (KGs). Alleviating the notorious incompleteness of real-world KGs, neural methods achieved impressive results in link prediction and complex query answering tasks by learning representations of entities, relations, and queries. Still, most existing query answering methods rely on transductive entity embeddings and cannot generalize to KGs containing new entities without retraining the entity embeddings. In this work, we study the inductive query answering task where inference is performed on a graph containing new entities with queries over both seen and unseen entities. To this end, we devise two mechanisms leveraging inductive node and relational structure representations powered by graph neural networks (GNNs). Experimentally, we show that inductive models are able to perform logical reasoning at inference time over unseen nodes generalizing to graphs up to 500% larger than training ones. Exploring the efficiency--effectiveness trade-off, we find the inductive relational structure representation method generally achieves higher performance, while the inductive node representation method is able to answer complex queries in the inference-only regime without any training on queries and scales to graphs of millions of nodes. Code is available at https://github.com/DeepGraphLearning/InductiveQE.
translated by 谷歌翻译
We present the OPEN GRAPH BENCHMARK (OGB), a diverse set of challenging and realistic benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning (ML) research. OGB datasets are large-scale, encompass multiple important graph ML tasks, and cover a diverse range of domains, ranging from social and information networks to biological networks, molecular graphs, source code ASTs, and knowledge graphs. For each dataset, we provide a unified evaluation protocol using meaningful application-specific data splits and evaluation metrics. In addition to building the datasets, we also perform extensive benchmark experiments for each dataset. Our experiments suggest that OGB datasets present significant challenges of scalability to large-scale graphs and out-of-distribution generalization under realistic data splits, indicating fruitful opportunities for future research. Finally, OGB provides an automated end-to-end graph ML pipeline that simplifies and standardizes the process of graph data loading, experimental setup, and model evaluation. OGB will be regularly updated and welcomes inputs from the community. OGB datasets as well as data loaders, evaluation scripts, baseline code, and leaderboards are publicly available at https://ogb.stanford.edu.
translated by 谷歌翻译
链接预测是图神经网络(GNN)的重要应用。链接预测的大多数现有GNN基于一维Weisfeiler-Lehman(1-WL)测试。 1-wl-gnn首先通过迭代的相邻节点特征来计算中心,然后通过汇总成对节点表示来获得链接表示。正如先前的作品所指出的那样,这两步过程会导致较低的区分功能,因为自然而然地学习节点级表示而不是链接级别。在本文中,我们研究了一种完全不同的方法,该方法可以基于\ textit {二维WEISFEILER-LEHMAN(2-WL)测试直接获得节点对(链接)表示。 2-WL测试直接使用链接(2个小说)作为消息传递单元而不是节点,因此可以直接获得链接表示。我们理论上分析了2-WL测试的表达能力以区分非晶状体链接,并证明其优越的链接与1-WL相比。基于不同的2-WL变体,我们提出了一系列用于链路预测的新型2-WL-GNN模型。在广泛的现实数据集上进行的实验证明了它们对最先进的基线的竞争性能以及优于普通1-WL-GNN的优势。
translated by 谷歌翻译
归纳链路预测(ILP)是考虑到新兴知识图(kgs)中未见实体的联系,考虑到KGS的发展性质。一个更具挑战性的场景是,新兴的kg仅由看不见的实体组成,被称为已断开新兴kgs(DEKGS)。 DEKGS的现有研究仅专注于预测封闭链接,即预测新兴KG内部的联系。到目前为止,先前的工作尚未对将进化信息从原始KG到DEKG进行进化信息。为了填补空白,我们提出了一个名为DEKG-ILP的新型模型(由以下两个组成部分组成的dekg-ilp(断开新兴知识图形的归纳链路预测)。 (1)模块CLRM(基于对比的关系特定特征特征建模)是为了提取基于全球关系的语义特征而开发的,它们在原始KGS和DEKGS之间以新颖的采样策略共享。 (2)提出了模块GSM(基于GNN的子图建模),以提取围绕KGS中每个链接的局部子图拓扑信息。在几个基准数据集上进行的广泛实验表明,与最新方法相比,DEKG-ILP具有明显的性能改进,用于封闭和桥接链路预测。源代码可在线获得。
translated by 谷歌翻译
Pre-publication draft of a book to be published byMorgan & Claypool publishers. Unedited version released with permission. All relevant copyrights held by the author and publisher extend to this pre-publication draft.
translated by 谷歌翻译
多跳跃逻辑推理是在知识图(KGS)上学习领域的一个已建立问题。它涵盖了单跳连接预测以及其他更复杂的逻辑查询类型。现有的算法仅在经典的三重基图上运行,而现代KG经常采用超相关的建模范式。在此范式中,键入的边缘可能具有几对键值对,称为限定符,可为事实提供细粒度的环境。在查询中,此上下文修改了关系的含义,通常会减少答案集。经常在现实世界中的应用程序中观察到超相关的查询,并且现有的近似查询答案方法无法使用预选赛对。在这项工作中,我们弥合了这一差距,并将多跳的推理问题扩展到了超级关系的KG,允许解决这一新类型的复杂查询。在图形神经网络和查询嵌入技术的最新进展之下,我们研究了如何嵌入和回答超相关的连词查询。除此之外,我们还提出了一种回答此类查询并在我们的实验中证明的方法,即预选赛可以改善对各种查询模式的查询回答。
translated by 谷歌翻译
Graph neural networks (GNNs) have received remarkable success in link prediction (GNNLP) tasks. Existing efforts first predefine the subgraph for the whole dataset and then apply GNNs to encode edge representations by leveraging the neighborhood structure induced by the fixed subgraph. The prominence of GNNLP methods significantly relies on the adhoc subgraph. Since node connectivity in real-world graphs is complex, one shared subgraph is limited for all edges. Thus, the choices of subgraphs should be personalized to different edges. However, performing personalized subgraph selection is nontrivial since the potential selection space grows exponentially to the scale of edges. Besides, the inference edges are not available during training in link prediction scenarios, so the selection process needs to be inductive. To bridge the gap, we introduce a Personalized Subgraph Selector (PS2) as a plug-and-play framework to automatically, personally, and inductively identify optimal subgraphs for different edges when performing GNNLP. PS2 is instantiated as a bi-level optimization problem that can be efficiently solved differently. Coupling GNNLP models with PS2, we suggest a brand-new angle towards GNNLP training: by first identifying the optimal subgraphs for edges; and then focusing on training the inference model by using the sampled subgraphs. Comprehensive experiments endorse the effectiveness of our proposed method across various GNNLP backbones (GCN, GraphSage, NGCF, LightGCN, and SEAL) and diverse benchmarks (Planetoid, OGB, and Recommendation datasets). Our code is publicly available at \url{https://github.com/qiaoyu-tan/PS2}
translated by 谷歌翻译
Recent years have witnessed the emerging success of graph neural networks (GNNs) for modeling structured data. However, most GNNs are designed for homogeneous graphs, in which all nodes and edges belong to the same types, making them infeasible to represent heterogeneous structures. In this paper, we present the Heterogeneous Graph Transformer (HGT) architecture for modeling Web-scale heterogeneous graphs. To model heterogeneity, we design node-and edge-type dependent parameters to characterize the heterogeneous attention over each edge, empowering HGT to maintain dedicated representations for different types of nodes and edges. To handle dynamic heterogeneous graphs, we introduce the relative temporal encoding technique into HGT, which is able to capture the dynamic structural dependency with arbitrary durations. To handle Web-scale graph data, we design the heterogeneous mini-batch graph sampling algorithm-HGSampling-for efficient and scalable training. Extensive experiments on the Open Academic Graph of 179 million nodes and 2 billion edges show that the proposed HGT model consistently outperforms all the state-of-the-art GNN baselines by 9%-21% on various downstream tasks. The dataset and source code of HGT are publicly available at https://github.com/acbull/pyHGT.
translated by 谷歌翻译
图表可以表示实体之间的关系信息,图形结构广泛用于许多智能任务,例如搜索,推荐和问题应答。然而,实际上大多数图形结构数据都遭受了不完整性,因此链路预测成为一个重要的研究问题。虽然提出了许多模型来用于链路预测,但以下两个问题仍然仍然较少:(1)大多数方法在不利用相关链路中使用丰富的信息,大多数方法都独立模型,并且(2)现有型号主要基于关联设计学习并没有考虑推理。通过这些问题,在本文中,我们提出了图表协作推理(GCR),它可以使用邻居与逻辑推理视角的关系中的关系推理。我们提供了一种简单的方法来将图形结构转换为逻辑表达式,以便链路预测任务可以转换为神经逻辑推理问题。我们应用逻辑受限的神经模块根据逻辑表达式构建网络架构,并使用反向传播以有效地学习模型参数,这在统一架构中桥接可分辨率的学习和象征性推理。为了展示我们工作的有效性,我们对图形相关任务进行实验,例如基于常用的基准数据集的链路预测和推荐,我们的图表合作推理方法实现了最先进的性能。
translated by 谷歌翻译
从结构化数据中学习是一项核心机器学习任务。通常,此类数据表示为图,通常仅考虑(键入)节点对之间的二进制关系。对于具有高度结构化数据的许多域而言,这是一个实质性的限制。一个重要的域是源代码,基于超图的表示可以更好地捕获代码的语义丰富和结构化的性质。在这项工作中,我们提出了热量,这是一种能够代表键入和合格的超图的神经模型,在该模型中,每个Hyperede都明确地符合参与节点的贡献。它可以看作是传递神经网络和变压器的消息的概括。我们使用新型程序代表程序来评估知识库完成和错误检测和维修的热量。在这两种情况下,它都优于强大的基线,表明其力量和通用性。
translated by 谷歌翻译