Analogical reasoning is fundamental to human cognition and holds an important place in various fields. However, previous studies mainly focus on single-modal analogical reasoning and ignore taking advantage of structure knowledge. Notably, the research in cognitive psychology has demonstrated that information from multimodal sources always brings more powerful cognitive transfer than single modality sources. To this end, we introduce the new task of multimodal analogical reasoning over knowledge graphs, which requires multimodal reasoning ability with the help of background knowledge. Specifically, we construct a Multimodal Analogical Reasoning dataSet (MARS) and a multimodal knowledge graph MarKG. We evaluate with multimodal knowledge graph embedding and pre-trained Transformer baselines, illustrating the potential challenges of the proposed task. We further propose a novel model-agnostic Multimodal analogical reasoning framework with Transformer (MarT) motivated by the structure mapping theory, which can obtain better performance.

知识图（kgs）将世界知识建模为结构三元组是不可避免的。多模式知识图（MMKGS）仍然存在此类问题。因此，知识图完成（KGC）对于预测现有KG中缺失的三元组至关重要。至于现有的KGC方法，基于嵌入的方法依靠手动设计来利用多模式信息，而基于芬太尼的方法在链接预​​测中并不优于基于嵌入的方法。为了解决这些问题，我们提出了一个Visualbert增强知识图完成模型（简称VBKGC）。 VBKGC可以为实体捕获深层融合的多模式信息，并将其集成到KGC模型中。此外，我们通过设计一种称为Twins Twins负抽样的新的负抽样策略来实现KGC模型的共同设计和负抽样。双胞胎阴性采样适用于多模式场景，可以对齐实体的不同嵌入。我们进行了广泛的实验，以显示VBKGC在链接预测任务上的出色表现，并进一步探索VBKGC。

学术知识图（KGS）提供了代表科学出版物编码的知识的丰富的结构化信息来源。随着出版的科学文学的庞大，包括描述科学概念的过多的非均匀实体和关系，这些公斤本质上是不完整的。我们呈现Exbert，一种利用预先训练的变压器语言模型来执行学术知识图形完成的方法。我们将知识图形的三元组模型为文本并执行三重分类（即，属于KG或不属于KG）。评估表明，在三重分类，链路预测和关系预测的任务中，Exbert在三个学术kg完成数据集中表现出其他基线。此外，我们将两个学术数据集作为研究界的资源，从公共公共公报和在线资源中收集。

Knowledge graphs (KG) have served as the key component of various natural language processing applications. Commonsense knowledge graphs (CKG) are a special type of KG, where entities and relations are composed of free-form text. However, previous works in KG completion and CKG completion suffer from long-tail relations and newly-added relations which do not have many know triples for training. In light of this, few-shot KG completion (FKGC), which requires the strengths of graph representation learning and few-shot learning, has been proposed to challenge the problem of limited annotated data. In this paper, we comprehensively survey previous attempts on such tasks in the form of a series of methods and applications. Specifically, we first introduce FKGC challenges, commonly used KGs, and CKGs. Then we systematically categorize and summarize existing works in terms of the type of KGs and the methods. Finally, we present applications of FKGC models on prediction tasks in different areas and share our thoughts on future research directions of FKGC.

由于知识图（kgs）的不完整，旨在预测kgs中未观察到的关系的零照片链接预测（ZSLP）引起了研究人员的最新兴趣。一个常见的解决方案是将关系的文本特征（例如表面名称或文本描述）用作辅助信息，以弥合所见关系和看不见的关系之间的差距。当前方法学习文本中每个单词令牌的嵌入。这些方法缺乏稳健性，因为它们遭受了量不足（OOV）的问题。同时，建立在字符n-grams上的模型具有为OOV单词生成表达式表示的能力。因此，在本文中，我们提出了一个为零链接预测（HNZSLP）的层次N-gram框架，该框架考虑了ZSLP的关系n-gram之间的依赖项。我们的方法通过首先在表面名称上构造层次n-gram图来进行起作用，以模拟导致表面名称的N-gram的组织结构。然后，将基于变压器的革兰amtransformer呈现，以建模层次n-gram图，以构建ZSLP的关系嵌入。实验结果表明，提出的HNZSLP在两个ZSLP数据集上实现了最先进的性能。

链路预测在知识图中起着重要作用，这是许多人工智能任务的重要资源，但它通常受不完整的限制。在本文中，我们提出了知识图表BERT for Link预测，名为LP-BERT，其中包含两个培训阶段：多任务预训练和知识图微调。预训练策略不仅使用掩码语言模型（MLM）来学习上下文语料库的知识，还引入掩模实体模型（MEM）和掩模关系模型（MRM），其可以通过预测语义来学习三元组的关系信息基于实体和关系元素。结构化三维关系信息可以转换为非结构化语义信息，可以将其与上下文语料库信息一起集成到培训模型中。在微调阶段，灵感来自对比学习，我们在样本批量中进行三样式的负面取样，这大大增加了负采样的比例，同时保持训练时间几乎不变。此外，我们提出了一种基于Triples的逆关系的数据增强方法，以进一步增加样本分集。我们在WN18RR和UMLS数据集上实现最先进的结果，特别是HITS @ 10指示器从WN18RR数据集上的先前最先进的结果提高了5 \％。

Multi-hop Question Answering over Knowledge Graph~(KGQA) aims to find the answer entities that are multiple hops away from the topic entities mentioned in a natural language question on a large-scale Knowledge Graph (KG). To cope with the vast search space, existing work usually adopts a two-stage approach: it firstly retrieves a relatively small subgraph related to the question and then performs the reasoning on the subgraph to accurately find the answer entities. Although these two stages are highly related, previous work employs very different technical solutions for developing the retrieval and reasoning models, neglecting their relatedness in task essence. In this paper, we propose UniKGQA, a novel approach for multi-hop KGQA task, by unifying retrieval and reasoning in both model architecture and parameter learning. For model architecture, UniKGQA consists of a semantic matching module based on a pre-trained language model~(PLM) for question-relation semantic matching, and a matching information propagation module to propagate the matching information along the edges on KGs. For parameter learning, we design a shared pre-training task based on question-relation matching for both retrieval and reasoning models, and then propose retrieval- and reasoning-oriented fine-tuning strategies. Compared with previous studies, our approach is more unified, tightly relating the retrieval and reasoning stages. Extensive experiments on three benchmark datasets have demonstrated the effectiveness of our method on the multi-hop KGQA task. Our codes and data are publicly available at https://github.com/RUCAIBox/UniKGQA.

知识图（KG）嵌入寻求学习实体和关系的向量表示。传统的模型理由是图形结构，但它们遭受了图形不完整和长尾实体的问题。最近的研究使用了预训练的语言模型根据实体和关系的文本信息来学习嵌入，但它们无法利用图形结构。在论文中，我们从经验上表明，这两种特征是KG嵌入的互补性。为此，我们提出了Cole，Cole是一种用于嵌入KG的共同介绍方法，可利用图形结构和文本信息的互补性。其图形嵌入模型使用变压器从其邻域子图中重建实体的表示。其文本嵌入模型使用预训练的语言模型来从其名称，描述和关系邻居的软提示中生成实体表示。为了让两个模型相互推广，我们提出了共同依据学习，使他们可以从彼此的预测逻辑中提取选择性知识。在我们的共同阶段学习中，每个模型既是老师又是学生。基准数据集上的实验表明，这两个模型的表现优于其相关基线，而与共同介绍学习的集合方法Cole可以推进KG嵌入的最先进。

知识图（kgs）在许多应用程序中越来越重要的基础架构，同时患有不完整问题。 KG完成任务（KGC）自动根据不完整的KG预测缺失的事实。但是，现有方法在现实情况下表现不佳。一方面，他们的性能将巨大的降解，而kg的稀疏性越来越大。另一方面，预测的推理过程是一个不可信的黑匣子。本文提出了一个稀疏kgc的新型可解释模型，将高阶推理组合到图形卷积网络中，即HOGRN。它不仅可以提高减轻信息不足问题的概括能力，而且还可以在保持模型的有效性和效率的同时提供可解释性。有两个主要组件无缝集成以进行关节优化。首先，高阶推理成分通过捕获关系之间的内源性相关性来学习高质量的关系表示。这可以反映逻辑规则，以证明更广泛的事实是合理的。其次，更新组件的实体利用无重量的图形卷积网络（GCN）有效地模拟具有可解释性的KG结构。与常规方法不同，我们在没有其他参数的情况下在关系空间中进行实体聚合和基于设计组成的注意。轻巧的设计使HOGRN更适合稀疏设置。为了进行评估，我们进行了广泛的实验 - HOGRN对几个稀疏KG的结果表现出了令人印象深刻的改善（平均为9％的MRR增益）。进一步的消融和案例研究证明了主要成分的有效性。我们的代码将在接受后发布。

Natural Language Processing (NLP) has been revolutionized by the use of Pre-trained Language Models (PLMs) such as BERT. Despite setting new records in nearly every NLP task, PLMs still face a number of challenges including poor interpretability, weak reasoning capability, and the need for a lot of expensive annotated data when applied to downstream tasks. By integrating external knowledge into PLMs, \textit{\underline{K}nowledge-\underline{E}nhanced \underline{P}re-trained \underline{L}anguage \underline{M}odels} (KEPLMs) have the potential to overcome the above-mentioned limitations. In this paper, we examine KEPLMs systematically through a series of studies. Specifically, we outline the common types and different formats of knowledge to be integrated into KEPLMs, detail the existing methods for building and evaluating KEPLMS, present the applications of KEPLMs in downstream tasks, and discuss the future research directions. Researchers will benefit from this survey by gaining a quick and comprehensive overview of the latest developments in this field.

The ability of knowledge graphs to represent complex relationships at scale has led to their adoption for various needs including knowledge representation, question-answering, fraud detection, and recommendation systems. Knowledge graphs are often incomplete in the information they represent, necessitating the need for knowledge graph completion tasks, such as link and relation prediction. Pre-trained and fine-tuned language models have shown promise in these tasks although these models ignore the intrinsic information encoded in the knowledge graph, namely the entity and relation types. In this work, we propose the Knowledge Graph Language Model (KGLM) architecture, where we introduce a new entity/relation embedding layer that learns to differentiate distinctive entity and relation types, therefore allowing the model to learn the structure of the knowledge graph. In this work, we show that further pre-training the language models with this additional embedding layer using the triples extracted from the knowledge graph, followed by the standard fine-tuning phase sets a new state-of-the-art performance for the link prediction task on the benchmark datasets.

Knowledge graph embedding (KGE), which maps entities and relations in a knowledge graph into continuous vector spaces, has achieved great success in predicting missing links in knowledge graphs. However, knowledge graphs often contain incomplete triples that are difficult to inductively infer by KGEs. To address this challenge, we resort to analogical inference and propose a novel and general self-supervised framework AnKGE to enhance KGE models with analogical inference capability. We propose an analogical object retriever that retrieves appropriate analogical objects from entity-level, relation-level, and triple-level. And in AnKGE, we train an analogy function for each level of analogical inference with the original element embedding from a well-trained KGE model as input, which outputs the analogical object embedding. In order to combine inductive inference capability from the original KGE model and analogical inference capability enhanced by AnKGE, we interpolate the analogy score with the base model score and introduce the adaptive weights in the score function for prediction. Through extensive experiments on FB15k-237 and WN18RR datasets, we show that AnKGE achieves competitive results on link prediction task and well performs analogical inference.

Recent years have witnessed the resurgence of knowledge engineering which is featured by the fast growth of knowledge graphs. However, most of existing knowledge graphs are represented with pure symbols, which hurts the machine's capability to understand the real world. The multi-modalization of knowledge graphs is an inevitable key step towards the realization of human-level machine intelligence. The results of this endeavor are Multi-modal Knowledge Graphs (MMKGs). In this survey on MMKGs constructed by texts and images, we first give definitions of MMKGs, followed with the preliminaries on multi-modal tasks and techniques. We then systematically review the challenges, progresses and opportunities on the construction and application of MMKGs respectively, with detailed analyses of the strength and weakness of different solutions. We finalize this survey with open research problems relevant to MMKGs.

知识嵌入（KE）通过将实体和关系嵌入连续的向量空间来表示知识图（kg）。现有方法主要基于结构或基于描述。基于结构的方法学习保留KGS固有结构的表示。它们不能很好地代表具有有限结构信息的现实世界中的丰富长尾实体。基于描述的方法利用文本信息和语言模型。朝这个方向迈出的先前方法几乎不能胜过基于结构的结构，并且遇到了昂贵的负面抽样和限制性描述需求等问题。在本文中，我们提出了LMKE，该LMKE采用语言模型来得出知识嵌入，旨在既富集了长尾实体的表示形式又旨在解决先前的基于描述的方法的问题。我们通过对比度学习框架制定基于描述的KE学习，以提高培训和评估的效率。实验结果表明，LMKE在链接预测和三重分类的KE基准上实现了最先进的性能，尤其是对于长尾实体。

知识库问题的最现有的方法接听（KBQA）关注特定的基础知识库，原因是该方法的固有假设，或者因为在不同的知识库上评估它需要非琐碎的变化。然而，许多流行知识库在其潜在模式中的相似性份额可以利用，以便于跨知识库的概括。为了实现这一概念化，我们基于2级架构介绍了一个KBQA框架，该架构明确地将语义解析与知识库交互分开，促进了数据集和知识图中的转移学习。我们表明，具有不同潜在知识库的数据集预先灌注可以提供显着的性能增益并降低样本复杂性。我们的方法可实现LC-Quad（DBPedia），WEDQSP（FreeBase），简单问话（Wikidata）和MetaQA（WikiMovies-KG）的可比性或最先进的性能。

在知识图上回答自然语言问题（KGQA）仍然是通过多跳推理理解复杂问题的巨大挑战。以前的努力通常利用与实体相关的文本语料库或知识图（kg）嵌入作为辅助信息来促进答案选择。但是，实体之间隐含的富裕语义远未得到很好的探索。本文提议通过利用关系路径的混合语义来改善多跳kgqa。具体而言，我们基于新颖的旋转和规模的实体链接链接预测框架，集成了关系路径的明确文本信息和隐式kg结构特征。在三个KGQA数据集上进行的广泛实验证明了我们方法的优势，尤其是在多跳场景中。进一步的调查证实了我们方法在问题和关系路径之间的系统协调，以识别答案实体。

机器学习方法尤其是深度神经网络取得了巨大的成功，但其中许多往往依赖于一些标记的样品进行训练。在真实世界的应用中，我们经常需要通过例如具有新兴预测目标和昂贵的样本注释的动态上下文来解决样本短缺。因此，低资源学习，旨在学习具有足够资源（特别是培训样本）的强大预测模型，现在正在被广泛调查。在所有低资源学习研究中，许多人更喜欢以知识图（kg）的形式利用一些辅助信息，这对于知识表示变得越来越受欢迎，以减少对标记样本的依赖。在这项调查中，我们非常全面地审查了90美元的报纸关于两个主要的低资源学习设置 - 零射击学习（ZSL）的预测，从未出现过训练，而且很少拍摄的学习（FSL）预测的新类仅具有可用的少量标记样本。我们首先介绍了ZSL和FSL研究中使用的KGS以及现有的和潜在的KG施工解决方案，然后系统地分类和总结了KG感知ZSL和FSL方法，将它们划分为不同的范例，例如基于映射的映射，数据增强，基于传播和基于优化的。我们接下来呈现了不同的应用程序，包括计算机视觉和自然语言处理中的kg增强预测任务，还包括kg完成的任务，以及每个任务的一些典型评估资源。我们最终讨论了一些关于新学习和推理范式的方面的一些挑战和未来方向，以及高质量的KGs的建设。

Multi-modal and multi-hop question answering aims to answer a question based on multiple input sources from different modalities. Previous methods retrieve the evidence separately and feed the retrieved evidence to a language model to generate the corresponding answer. However, these methods fail to build connections between candidates and thus cannot model the inter-dependent relation during retrieval. Moreover, the reasoning process over multi-modality candidates can be unbalanced without building alignments between different modalities. To address this limitation, we propose a Structured Knowledge and Unified Retrieval Generation based method (SKURG). We align the sources from different modalities via the shared entities and map them into a shared semantic space via structured knowledge. Then, we utilize a unified retrieval-generation decoder to integrate intermediate retrieval results for answer generation and adaptively determine the number of retrieval steps. We perform experiments on two multi-modal and multi-hop datasets: WebQA and MultimodalQA. The results demonstrate that SKURG achieves state-of-the-art performance on both retrieval and answer generation.

外部知识（A.K.A.侧面信息）在零拍摄学习（ZSL）中起着关键作用，该角色旨在预测从未出现在训练数据中的看不见的类。已被广泛调查了几种外部知识，例如文本和属性，但他们独自受到不完整的语义。因此，一些最近的研究提出了由于其高度富有效力和代表知识的兼容性而使用知识图表（千克）。但是，ZSL社区仍然缺乏用于学习和比较不同外部知识设置和基于不同的KG的ZSL方法的标准基准。在本文中，我们提出了六个资源，涵盖了三个任务，即零拍摄图像分类（ZS-IMGC），零拍摄关系提取（ZS-RE）和零拍KG完成（ZS-KGC）。每个资源都有一个正常的zsl基准标记和包含从文本到属性的kg的kg，从关系知识到逻辑表达式。我们已清楚地介绍了这些资源，包括其建设，统计数据格式和使用情况W.r.t.不同的ZSL方法。更重要的是，我们进行了一项全面的基准研究，具有两个通用和最先进的方法，两种特定方法和一种可解释方法。我们讨论并比较了不同的ZSL范式W.R.T.不同的外部知识设置，并发现我们的资源具有开发更高级ZSL方法的巨大潜力，并为应用KGS进行增强机学习的更多解决方案。所有资源都可以在https://github.com/china-uk-zsl/resources_for_kzsl上获得。

从头开始解决复杂问题通常是有挑战性的，但如果我们可以访问其解决方案的其他类似问题，则更容易 - 一种称为基于案例的推理（CBR）的范式。我们提出了一种神经象征性的CBR方法（CBR-KBQA），用于在大知识库上应答。 CBR-KBQA由非参数内存组成，该内存存储案例（问题和逻辑表单）和参数模型，该参数模型可以通过检索与其相关的案例来为新问题生成逻辑表单。在包含复杂问题的几个KBQA数据集上，CBR-KBQA实现了竞争性能。例如，在ComplexWebQuestions数据集上，CBR-KBQA以11 \％的准确度优于当前最新状态。此外，我们表明CBR-KBQA能够使用新案例\ EMPH {没有}任何进一步的培训：通过在案例存储器中纳入一些人类标记的示例，CBR-KBQA能够成功地生成包含未经看线KB实体的逻辑表格以及关系。