In this paper we show the surprising effectiveness of a simple observed features model in comparison to latent feature models on two benchmark knowledge base completion datasets, FB15K and WN18. We also compare latent and observed feature models on a more challenging dataset derived from FB15K, and additionally coupled with textual mentions from a web-scale corpus. We show that the observed features model is most effective at capturing the information present for entity pairs with textual relations, and a combination of the two combines the strengths of both model types.

Knowledge graphs enable a wide variety of applications, including question answering and information retrieval. Despite the great effort invested in their creation and maintenance, even the largest (e.g., Yago, DBPedia or Wikidata) remain incomplete. We introduce Relational Graph Convolutional Networks (R-GCNs) and apply them to two standard knowledge base completion tasks: Link prediction (recovery of missing facts, i.e. subject-predicate-object triples) and entity classification (recovery of missing entity attributes). R-GCNs are related to a recent class of neural networks operating on graphs, and are developed specifically to deal with the highly multi-relational data characteristic of realistic knowledge bases. We demonstrate the effectiveness of R-GCNs as a stand-alone model for entity classification. We further show that factorization models for link prediction such as DistMult can be significantly improved by enriching them with an encoder model to accumulate evidence over multiple inference steps in the relational graph, demonstrating a large improvement of 29.8% on FB15k-237 over a decoder-only baseline. * Equal contribution.

Link prediction for knowledge graphs is the task of predicting missing relationships between entities. Previous work on link prediction has focused on shallow, fast models which can scale to large knowledge graphs. However, these models learn less expressive features than deep, multi-layer modelswhich potentially limits performance. In this work we introduce ConvE, a multi-layer convolutional network model for link prediction, and report state-of-the-art results for several established datasets. We also show that the model is highly parameter efficient, yielding the same performance as DistMult and R-GCN with 8x and 17x fewer parameters. Analysis of our model suggests that it is particularly effective at modelling nodes with high indegree -which are common in highlyconnected, complex knowledge graphs such as Freebase and YAGO3. In addition, it has been noted that the WN18 and FB15k datasets suffer from test set leakage, due to inverse relations from the training set being present in the test sethowever, the extent of this issue has so far not been quantified. We find this problem to be severe: a simple rule-based model can achieve state-of-the-art results on both WN18 and FB15k. To ensure that models are evaluated on datasets where simply exploiting inverse relations cannot yield competitive results, we investigate and validate several commonly used datasets -deriving robust variants where necessary. We then perform experiments on these robust datasets for our own and several previously proposed models, and find that ConvE achieves state-of-the-art Mean Reciprocal Rank across most datasets.

In statistical relational learning, the link prediction problem is key to automatically understand the structure of large knowledge bases. As in previous studies, we propose to solve this problem through latent factorization. However, here we make use of complex valued embeddings. The composition of complex embeddings can handle a large variety of binary relations, among them symmetric and antisymmetric relations. Compared to state-of-the-art models such as Neural Tensor Network and Holographic Embeddings, our approach based on complex embeddings is arguably simpler, as it only uses the Hermitian dot product, the complex counterpart of the standard dot product between real vectors. Our approach is scalable to large datasets as it remains linear in both space and time, while consistently outperforming alternative approaches on standard link prediction benchmarks. 1

We address the challenge of building domain-specific knowledge models for industrial use cases, where labelled data and taxonomic information is initially scarce. Our focus is on inductive link prediction models as a basis for practical tools that support knowledge engineers with exploring text collections and discovering and linking new (so-called open-world) entities to the knowledge graph. We argue that - though neural approaches to text mining have yielded impressive results in the past years - current benchmarks do not reflect the typical challenges encountered in the industrial wild properly. Therefore, our first contribution is an open benchmark coined IRT2 (inductive reasoning with text) that (1) covers knowledge graphs of varying sizes (including very small ones), (2) comes with incidental, low-quality text mentions, and (3) includes not only triple completion but also ranking, which is relevant for supporting experts with discovery tasks. We investigate two neural models for inductive link prediction, one based on end-to-end learning and one that learns from the knowledge graph and text data in separate steps. These models compete with a strong bag-of-words baseline. The results show a significant advance in performance for the neural approaches as soon as the available graph data decreases for linking. For ranking, the results are promising, and the neural approaches outperform the sparse retriever by a wide margin.

最近公布的知识图形嵌入模型的实施，培训和评估的异质性已经公平和彻底的比较困难。为了评估先前公布的结果的再现性，我们在Pykeen软件包中重新实施和评估了21个交互模型。在这里，我们概述了哪些结果可以通过其报告的超参数再现，这只能以备用的超参数再现，并且无法再现，并且可以提供洞察力，以及为什么会有这种情况。然后，我们在四个数据集上进行了大规模的基准测试，其中数千个实验和24,804 GPU的计算时间。我们展示了最佳实践，每个模型的最佳配置以及可以通过先前发布的最佳配置进行改进的洞察。我们的结果强调了模型架构，训练方法，丢失功能和逆关系显式建模的组合对于模型的性能来说至关重要，而不仅由模型架构决定。我们提供了证据表明，在仔细配置时，若干架构可以获得对最先进的结果。我们制定了所有代码，实验配置，结果和分析，导致我们在https://github.com/pykeen/pykeen和https://github.com/pykeen/benchmarking中获得的解释

Relational machine learning studies methods for the statistical analysis of relational, or graph-structured, data. In this paper, we provide a review of how such statistical models can be "trained" on large knowledge graphs, and then used to predict new facts about the world (which is equivalent to predicting new edges in the graph). In particular, we discuss two fundamentally different kinds of statistical relational models, both of which can scale to massive datasets. The first is based on latent feature models such as tensor factorization and multiway neural networks. The second is based on mining observable patterns in the graph. We also show how to combine these latent and observable models to get improved modeling power at decreased computational cost. Finally, we discuss how such statistical models of graphs can be combined with text-based information extraction methods for automatically constructing knowledge graphs from the Web. To this end, we also discuss Google's Knowledge Vault project as an example of such combination.

我们研究了对知识图中链路预测任务的知识图形嵌入（KGE）模型产生数据中毒攻击的问题。为了毒害KGE模型，我们建议利用他们通过知识图中的对称性，反演和构图等关系模式捕获的归纳能力。具体而言，为了降低模型对目标事实的预测信心，建议改善模型对一系列诱饵事实的预测信心。因此，我们通过不同的推理模式来制作对逆势的添加能够改善模型对诱饵事实上的预测信心。我们的实验表明，拟议的中毒攻击在四个KGE模型上倾斜的最先进的基座，用于两个公共数据集。我们还发现基于对称模式的攻击遍历了所有模型 - 数据集合，指示KGE模型对此模式的灵敏度。

知识图（kgs）因其学习单一关系事实的表示能力而获得了突出。最近，研究重点是建模超级关系的事实，这些事实超出了单一关系事实的限制，使我们能够代表更复杂和现实的信息。但是，现有的超级关系中学习表征的方法主要集中于增强从预选赛到基础三元组的沟通，同时忽略了从基本三重限制者到资格赛的信息流。这可能会导致次级预选赛表示，尤其是在提出大量预选赛时。它促使我们设计一个利用多个聚合器来学习超级关系事实的表示框架：从基本三重的角度来看，一个框架从资格符的角度来看。实验证明了我们框架对多个数据集的超相关知识图完成的有效性。此外，我们进行了一项消融研究，以验证各个组件在我们的框架中的重要性。可以在\ url {https://github.com/harryshomer/quad}找到复制我们的结果的代码。

Knowledge graph embedding (KGE), which maps entities and relations in a knowledge graph into continuous vector spaces, has achieved great success in predicting missing links in knowledge graphs. However, knowledge graphs often contain incomplete triples that are difficult to inductively infer by KGEs. To address this challenge, we resort to analogical inference and propose a novel and general self-supervised framework AnKGE to enhance KGE models with analogical inference capability. We propose an analogical object retriever that retrieves appropriate analogical objects from entity-level, relation-level, and triple-level. And in AnKGE, we train an analogy function for each level of analogical inference with the original element embedding from a well-trained KGE model as input, which outputs the analogical object embedding. In order to combine inductive inference capability from the original KGE model and analogical inference capability enhanced by AnKGE, we interpolate the analogy score with the base model score and introduce the adaptive weights in the score function for prediction. Through extensive experiments on FB15k-237 and WN18RR datasets, we show that AnKGE achieves competitive results on link prediction task and well performs analogical inference.

The ability of knowledge graphs to represent complex relationships at scale has led to their adoption for various needs including knowledge representation, question-answering, fraud detection, and recommendation systems. Knowledge graphs are often incomplete in the information they represent, necessitating the need for knowledge graph completion tasks, such as link and relation prediction. Pre-trained and fine-tuned language models have shown promise in these tasks although these models ignore the intrinsic information encoded in the knowledge graph, namely the entity and relation types. In this work, we propose the Knowledge Graph Language Model (KGLM) architecture, where we introduce a new entity/relation embedding layer that learns to differentiate distinctive entity and relation types, therefore allowing the model to learn the structure of the knowledge graph. In this work, we show that further pre-training the language models with this additional embedding layer using the triples extracted from the knowledge graph, followed by the standard fine-tuning phase sets a new state-of-the-art performance for the link prediction task on the benchmark datasets.

学术知识图（KGS）提供了代表科学出版物编码的知识的丰富的结构化信息来源。随着出版的科学文学的庞大，包括描述科学概念的过多的非均匀实体和关系，这些公斤本质上是不完整的。我们呈现Exbert，一种利用预先训练的变压器语言模型来执行学术知识图形完成的方法。我们将知识图形的三元组模型为文本并执行三重分类（即，属于KG或不属于KG）。评估表明，在三重分类，链路预测和关系预测的任务中，Exbert在三个学术kg完成数据集中表现出其他基线。此外，我们将两个学术数据集作为研究界的资源，从公共公共公报和在线资源中收集。

学习知识图的嵌入对人工智能至关重要，可以使各种下游应用受益，例如推荐和问题回答。近年来，已经提出了许多研究努力，以嵌入知识图形。然而，最先前的知识图形嵌入方法忽略不同三元组中的相关实体和实体关系耦合之间的语义相似性，因为它们与评分函数分别优化每个三倍。为了解决这个问题，我们提出了一个简单但有效的对比学习框架，用于知识图形嵌入，可以缩短不同三元组中相关实体和实体关系耦合的语义距离，从而提高知识图形嵌入的表现力。我们在三个标准知识图形基准上评估我们提出的方法。值得注意的是，我们的方法可以产生一些新的最先进的结果，在WN18RR数据集中实现51.2％的MRR，46.8％HITS @ 1，59.1％的MRR，51.8％在YAGO3-10数据集中击打@ 1 。

实体类型预测是知识图中的一个重要问题（kg）研究。在这项工作中提出了一种新的KG实体类型预测方法，名为Core（复杂的空间回归和嵌入）。所提出的核心方法利用两个复杂空间嵌入模型的表现力;即，旋转和复杂的模型。它使用旋转或复杂地将实体和类型嵌入两个不同的复杂空间中。然后，我们推导了一个复杂的回归模型来链接这两个空格。最后，介绍了一种优化嵌入和回归参数的机制。实验表明，核心优于代表性KG实体型推理数据集的基准测试方法。分析了各种实体型预测方法的强度和弱点。

知识图（KG）通常不完整，我们经常希望推断出现有的新事实。这可以被认为是二进制分类问题;我们的目标是预测新事实是真或假的。不幸的是，我们通常只有积极的例子（已知事实），但我们也需要负面的例子来训练分类器。要解决此问题，通常使用负面采样策略生成否定示例。但是，这可以产生可能降低性能的假否定，是计算昂贵的，并且不会产生校准的分类概率。在本文中，我们提出了一种培训程序，通过向损失函数添加新的正则化术语来消除对负面采样的需要。我们的两个关系嵌入模型（DISTMULT和简单）的结果显示了我们的提案的优点。

事实证明，信息提取方法可有效从结构化或非结构化数据中提取三重。以（头部实体，关系，尾部实体）形式组织这样的三元组的组织称为知识图（kgs）。当前的大多数知识图都是不完整的。为了在下游任务中使用kgs，希望预测kgs中缺少链接。最近，通过将实体和关系嵌入到低维的矢量空间中，旨在根据先前访问的三元组来预测三元组，从而对KGS表示不同的方法。根据如何独立或依赖对三元组进行处理，我们将知识图完成的任务分为传统和图形神经网络表示学习，并更详细地讨论它们。在传统的方法中，每个三重三倍将独立处理，并在基于GNN的方法中进行处理，三倍也考虑了他们的当地社区。查看全文

In knowledge graph completion (KGC), predicting triples involving emerging entities and/or relations, which are unseen when the KG embeddings are learned, has become a critical challenge. Subgraph reasoning with message passing is a promising and popular solution. Some recent methods have achieved good performance, but they (i) usually can only predict triples involving unseen entities alone, failing to address more realistic fully inductive situations with both unseen entities and unseen relations, and (ii) often conduct message passing over the entities with the relation patterns not fully utilized. In this study, we propose a new method named RMPI which uses a novel Relational Message Passing network for fully Inductive KGC. It passes messages directly between relations to make full use of the relation patterns for subgraph reasoning with new techniques on graph transformation, graph pruning, relation-aware neighborhood attention, addressing empty subgraphs, etc., and can utilize the relation semantics defined in the ontological schema of KG. Extensive evaluation on multiple benchmarks has shown the effectiveness of techniques involved in RMPI and its better performance compared with the existing methods that support fully inductive KGC. RMPI is also comparable to the state-of-the-art partially inductive KGC methods with very promising results achieved. Our codes and data are available at https://github.com/zjukg/RMPI.

知识图嵌入（KGE）旨在将实体和关系映射到低维空间，并成为知识图完成的\ textit {de-facto}标准。大多数现有的KGE方法都受到稀疏挑战的困扰，在这种挑战中，很难预测在知识图中频繁的实体。在这项工作中，我们提出了一个新颖的框架KRACL，以减轻具有图表和对比度学习的KG中广泛的稀疏性。首先，我们建议知识关系网络（KRAT）通过同时将相邻的三元组投射到不同的潜在空间，并通过注意机制共同汇总信息来利用图形上下文。 KRAT能够捕获不同上下文三联的微妙的语义信息和重要性，并利用知识图中的多跳信息。其次，我们通过将对比度损失与跨熵损失相结合，提出知识对比损失，这引入了更多的负样本，从而丰富了对稀疏实体的反馈。我们的实验表明，KRACL在各种标准知识基准中取得了卓越的结果，尤其是在WN18RR和NELL-995上，具有大量低级内实体。广泛的实验还具有KRACL在处理稀疏知识图和鲁棒性三元组的鲁棒性方面的有效性。

Knowledge graph (KG) embedding is to embed components of a KG including entities and relations into continuous vector spaces, so as to simplify the manipulation while preserving the inherent structure of the KG. It can benefit a variety of downstream tasks such as KG completion and relation extraction, and hence has quickly gained massive attention. In this article, we provide a systematic review of existing techniques, including not only the state-of-the-arts but also those with latest trends. Particularly, we make the review based on the type of information used in the embedding task. Techniques that conduct embedding using only facts observed in the KG are first introduced. We describe the overall framework, specific model design, typical training procedures, as well as pros and cons of such techniques. After that, we discuss techniques that further incorporate additional information besides facts. We focus specifically on the use of entity types, relation paths, textual descriptions, and logical rules. Finally, we briefly introduce how KG embedding can be applied to and benefit a wide variety of downstream tasks such as KG completion, relation extraction, question answering, and so forth.

We study the problem of learning representations of entities and relations in knowledge graphs for predicting missing links. The success of such a task heavily relies on the ability of modeling and inferring the patterns of (or between) the relations. In this paper, we present a new approach for knowledge graph embedding called RotatE, which is able to model and infer various relation patterns including: symmetry/antisymmetry, inversion, and composition. Specifically, the RotatE model defines each relation as a rotation from the source entity to the target entity in the complex vector space. In addition, we propose a novel self-adversarial negative sampling technique for efficiently and effectively training the RotatE model. Experimental results on multiple benchmark knowledge graphs show that the proposed RotatE model is not only scalable, but also able to infer and model various relation patterns and significantly outperform existing state-of-the-art models for link prediction.