语言模型（LM）是否可以通过固有的关系推理能力在知识库中的地面问题解决方案（QA）任务？尽管以前仅使用LMS的模型在许多质量检查任务上都看到了一些成功，但最新的方法包括知识图（KG），以补充LMS的逻辑驱动的隐式知识。但是，有效从结构化数据（例如KGS）中提取信息，使LMS保持开放性问题，并且当前模型依靠图形技术来提取知识。在本文中，我们建议仅利用LMS将基于知识的问题的语言和知识与灵活性，覆盖范围和结构化推理相结合。具体而言，我们设计了一种知识构建方法，该方法可以通过动态跳跃来检索相关背景，该方法比传统的基于GNN的技术表达了更全面的。我们设计了一种深层融合机制，以进一步弥合语言和知识之间交换瓶颈的信息。广泛的实验表明，我们的模型始终证明了其对CommenSensenSENSENSESQA基准测试的最先进性能，从而展示了仅利用LMS将LMS稳健地质量质量质量质量质量固定到知识库的可能性。

现有的kg增强模型用于问题回答主要专注于设计精心图形神经网络（GNN）以模拟知识图（KG）。但是，它们忽略了（i）有效地融合和推理过问题上下文表示和kg表示，并且（ii）在推理期间自动从嘈杂的KG中选择相关节点。在本文中，我们提出了一种新颖的型号，其通过LMS和GNN的联合推理和动态KGS修剪机制解决了上述限制。具体而言，ConntLK通过新的密集双向注意模块在LMS和GNN之间执行联合推理，其中每个问题令牌参加KG节点，每个KG节点都会参加问题令牌，并且两个模态表示熔断和通过多次熔断和更新。步互动。然后，动态修剪模块使用通过联合推理产生的注意重量来递归修剪无关的kg节点。我们在CommanSENSEQA和OpenBookQA数据集上的结果表明，我们的模态融合和知识修剪方法可以更好地利用相关知识来推理。

使用从预先接受训练的语言模型（LMS）和知识图表（LMS）和知识图表（kgs）回答问题的问题提出了两个挑战：给定QA上下文（问答选择），方法需要（i）从大型千克识别相关知识，（ii）对QA上下文和kg进行联合推理。在这项工作中，我们提出了一种新的模型，QA-GNN，它通过两个关键创新解决了上述挑战：（i）相关评分，我们使用LMS来估计KG节点相对于给定的QA上下文的重要性，以及（ii）联合推理，我们将QA上下文和kg连接到联合图，并通过图形神经网络相互更新它们的表示。我们评估了QA基准的模型（CommanSeaseQA，OpenBookQA）和生物医学（MedQa-USMLE）域名。QA-GNN优于现有的LM和LM + kg模型，并表现出可解释和结构化推理的能力，例如，正确处理问题的否定。

Question Answering (QA) is a task that entails reasoning over natural language contexts, and many relevant works augment language models (LMs) with graph neural networks (GNNs) to encode the Knowledge Graph (KG) information. However, most existing GNN-based modules for QA do not take advantage of rich relational information of KGs and depend on limited information interaction between the LM and the KG. To address these issues, we propose Question Answering Transformer (QAT), which is designed to jointly reason over language and graphs with respect to entity relations in a unified manner. Specifically, QAT constructs Meta-Path tokens, which learn relation-centric embeddings based on diverse structural and semantic relations. Then, our Relation-Aware Self-Attention module comprehensively integrates different modalities via the Cross-Modal Relative Position Bias, which guides information exchange between relevant entities of different modalities. We validate the effectiveness of QAT on commonsense question answering datasets like CommonsenseQA and OpenBookQA, and on a medical question answering dataset, MedQA-USMLE. On all the datasets, our method achieves state-of-the-art performance. Our code is available at http://github.com/mlvlab/QAT.

我们研究了学习因果推理对程序文本的挑战，以回答“如果...”何时需要外常识知识。我们提出了一个新颖的多跳图推理模型，以1）有效地从大知识图中提取常识子图；2）通过推理从常识子图获得的表示以及问题与上下文之间的上下文相互作用来预测因果答案。我们评估了WIQA基准测试的模型，并与最近的模型相比实现了最先进的性能。

这项工作调查了以知识图（kg）形式的外部知识来源的理解问题的学习和推理的挑战。我们提出了一种新型的图形神经网络体系结构，称为动态相关图形网络（DRGN）。 DRGN根据问题和答案实体在给定的KG子图上运行，并使用节点之间的相关得分来动态建立新的边缘，以在图形网络中学习节点表示。相关性的这种显式用法作为图表具有以下优点，a）模型可以利用现有关系，重新缩放节点权重，并影响邻里节点的表示方式在kg子图中汇总的方式，b）恢复推理所需的千克中缺失的边缘。此外，作为副产品，由于考虑了问题节点与图形实体之间的相关性，我们的模型改善了处理负面问题。与最新发布的结果相比，我们提出的方法在两个质量检查基准CommonSenseQA和OpenBookQA上显示了竞争性能。

预训练的语言模型（PTLM）已显示出在自然语言任务上表现良好。许多先前的作品都以通过知识图（KGS）标记的关系链接的实体的形式利用结构性常识来协助PTLM。检索方法使用kg作为单独的静态模块，该模块限制了覆盖范围，因为kgs包含有限的知识。生成方法训练PTLMS kg三倍以提高获得知识的规模。但是，对符号KG实体的培训限制了其在涉及自然语言文本的任务中的适用性，在这些任务中，它们忽略了整体上下文。为了减轻这种情况，我们提出了一个以句子为条件的常识性上下文化器（COSE-CO）作为输入，以使其在生成与输入文本的整体上下文相关的任务中通常可用。为了训练Cose-Co，我们提出了一个新的数据集，其中包括句子和常识知识对。 COSE-CO推断出的知识是多种多样的，并且包含了基础KG中不存在的新实体。我们增强了在多选质量质量检查和开放式常识性推理任务中产生的知识，从而改善了CSQA，ARC，QASC和OBQA数据集的当前最佳方法。我们还展示了其在改善释义生成任务的基线模型方面的适用性。

Pre-trained Language Models (PLMs) which are trained on large text corpus through the self-supervised learning method, have yielded promising performance on various tasks in Natural Language Processing (NLP). However, though PLMs with huge parameters can effectively possess rich knowledge learned from massive training text and benefit downstream tasks at the fine-tuning stage, they still have some limitations such as poor reasoning ability due to the lack of external knowledge. Incorporating knowledge into PLMs has been tried to tackle these issues. In this paper, we present a comprehensive review of Knowledge-Enhanced Pre-trained Language Models (KE-PLMs) to provide a clear insight into this thriving field. We introduce appropriate taxonomies respectively for Natural Language Understanding (NLU) and Natural Language Generation (NLG) to highlight the focus of these two kinds of tasks. For NLU, we take several types of knowledge into account and divide them into four categories: linguistic knowledge, text knowledge, knowledge graph (KG), and rule knowledge. The KE-PLMs for NLG are categorized into KG-based and retrieval-based methods. Finally, we point out some promising future directions of KE-PLMs.

Natural Language Processing (NLP) has been revolutionized by the use of Pre-trained Language Models (PLMs) such as BERT. Despite setting new records in nearly every NLP task, PLMs still face a number of challenges including poor interpretability, weak reasoning capability, and the need for a lot of expensive annotated data when applied to downstream tasks. By integrating external knowledge into PLMs, \textit{\underline{K}nowledge-\underline{E}nhanced \underline{P}re-trained \underline{L}anguage \underline{M}odels} (KEPLMs) have the potential to overcome the above-mentioned limitations. In this paper, we examine KEPLMs systematically through a series of studies. Specifically, we outline the common types and different formats of knowledge to be integrated into KEPLMs, detail the existing methods for building and evaluating KEPLMS, present the applications of KEPLMs in downstream tasks, and discuss the future research directions. Researchers will benefit from this survey by gaining a quick and comprehensive overview of the latest developments in this field.

Knowledge graph (KG) link prediction aims to infer new facts based on existing facts in the KG. Recent studies have shown that using the graph neighborhood of a node via graph neural networks (GNNs) provides more useful information compared to just using the query information. Conventional GNNs for KG link prediction follow the standard message-passing paradigm on the entire KG, which leads to over-smoothing of representations and also limits their scalability. On a large scale, it becomes computationally expensive to aggregate useful information from the entire KG for inference. To address the limitations of existing KG link prediction frameworks, we propose a novel retrieve-and-read framework, which first retrieves a relevant subgraph context for the query and then jointly reasons over the context and the query with a high-capacity reader. As part of our exemplar instantiation for the new framework, we propose a novel Transformer-based GNN as the reader, which incorporates graph-based attention structure and cross-attention between query and context for deep fusion. This design enables the model to focus on salient context information relevant to the query. Empirical results on two standard KG link prediction datasets demonstrate the competitive performance of the proposed method.

Multi-modal and multi-hop question answering aims to answer a question based on multiple input sources from different modalities. Previous methods retrieve the evidence separately and feed the retrieved evidence to a language model to generate the corresponding answer. However, these methods fail to build connections between candidates and thus cannot model the inter-dependent relation during retrieval. Moreover, the reasoning process over multi-modality candidates can be unbalanced without building alignments between different modalities. To address this limitation, we propose a Structured Knowledge and Unified Retrieval Generation based method (SKURG). We align the sources from different modalities via the shared entities and map them into a shared semantic space via structured knowledge. Then, we utilize a unified retrieval-generation decoder to integrate intermediate retrieval results for answer generation and adaptively determine the number of retrieval steps. We perform experiments on two multi-modal and multi-hop datasets: WebQA and MultimodalQA. The results demonstrate that SKURG achieves state-of-the-art performance on both retrieval and answer generation.

知识图表问题基于信息检索旨在通过从大型知识图表中检索答案来回答问题来回答（即，kgqa）。大多数现有方法首先粗略地检索可能包含候选答案的知识子图（KSG），然后搜索子图中的确切答案。然而，粗略检索的KSG可以包含数千个候选节点，因为查询中涉及的知识图通常是大规模的。为了解决这个问题，我们首先建议通过新的子图分区算法将检索到的ksg分区为几个较小的子ksgs，然后呈现一个图形增强学习，以便测量模型以从中选择排名的子ksgs。我们所提出的模型结合了新的子图匹配网络，以捕获问题和子图中的全局交互以及增强的双边多视角匹配模型，以捕获局部交互。最后，我们分别在全KSG和排名级分ksg上应用答案选择模型，以验证我们提出的图形增强学习的效果。多个基准数据集的实验结果表明了我们方法的有效性。

今天的大部分AI系统都专注于使用自我关注机制和变压器架构在大量多样化的数据中实现令人印象深刻的性能收益。在本文中，我们建议使用外部注意机制增强变压器架构，以带来外部知识和背景。通过将外部信息集成到预测过程中，我们希望减少对更大的模型的需求，并增加AI系统的民主化。我们发现所提出的外部注意机制可以显着提高现有AI系统的性能，使从业者可以轻松地将基础AI模型自定义到许多不同的下游应用程序。特别是，我们专注于勤杂朗语推理的任务，展示所提出的外部注意机制可以增加现有的变压器模型，并显着提高模型的推理能力。拟议的系统，知识外部关注推理（Kear），达到了开放的铜商QA研究基准的人类奇偶校验，其准确性为89.4 \％，与人类准确性为88.9 \％。

视觉问题回答（VQA）通常需要对视觉概念和语言语义的理解，这取决于外部知识。大多数现有方法利用了预训练的语言模型或/和非结构化文本，但是这些资源中的知识通常不完整且嘈杂。有些方法更喜欢使用经常具有强化结构知识的知识图（kgs），但是研究仍然相当初步。在本文中，我们提出了Lako，这是一种知识驱动的VQA方法，通过后期的文本注射。为了有效地纳入外部kg，我们将三元三元转移到文本中，并提出一种晚期注射机制。最后，我们将VQA作为文本生成任务，并具有有效的编码器范式。在使用OKVQA数据集的评估中，我们的方法可实现最新的结果。

知识图表（kg）作为从大型自然语言文本语料库中举行蒸馏信息的伟大工具。查询知识图表的自然语言问题对于这些信息的人类消费至关重要。通常通过将自然语言查询转换为结构化查询，然后在kg上触发结构化查询来解决此问题。在文献中的知识图中直接回答模型很少。查询转换模型和直接模型都需要与知识图表的域有关的特定培训数据。在这项工作中，我们将通过知识图表的自然语言问题转换为前提假设对的推理问题。使用培训的深度学习模型进行转换后的代理推理问题，我们为原始自然语言查询问题提供了解决方案。我们的方法在MetaQA数据集中实现了超过90％的准确性，击败现有的最先进。我们还提出了一种推论称为分层复发路径编码器（HRPE）的模型。可以微调推断模型以跨越跨越培训数据的域使用。我们的方法不需要大型域特定的培训数据来查询来自不同域的新知识图表。

在知识图上回答自然语言问题（KGQA）仍然是通过多跳推理理解复杂问题的巨大挑战。以前的努力通常利用与实体相关的文本语料库或知识图（kg）嵌入作为辅助信息来促进答案选择。但是，实体之间隐含的富裕语义远未得到很好的探索。本文提议通过利用关系路径的混合语义来改善多跳kgqa。具体而言，我们基于新颖的旋转和规模的实体链接链接预测框架，集成了关系路径的明确文本信息和隐式kg结构特征。在三个KGQA数据集上进行的广泛实验证明了我们方法的优势，尤其是在多跳场景中。进一步的调查证实了我们方法在问题和关系路径之间的系统协调，以识别答案实体。

Knowledge graphs (KG) have served as the key component of various natural language processing applications. Commonsense knowledge graphs (CKG) are a special type of KG, where entities and relations are composed of free-form text. However, previous works in KG completion and CKG completion suffer from long-tail relations and newly-added relations which do not have many know triples for training. In light of this, few-shot KG completion (FKGC), which requires the strengths of graph representation learning and few-shot learning, has been proposed to challenge the problem of limited annotated data. In this paper, we comprehensively survey previous attempts on such tasks in the form of a series of methods and applications. Specifically, we first introduce FKGC challenges, commonly used KGs, and CKGs. Then we systematically categorize and summarize existing works in terms of the type of KGs and the methods. Finally, we present applications of FKGC models on prediction tasks in different areas and share our thoughts on future research directions of FKGC.

Biomedical knowledge graphs (KG) are heterogenous networks consisting of biological entities as nodes and relations between them as edges. These entities and relations are extracted from millions of research papers and unified in a single resource. The goal of biomedical multi-hop question-answering over knowledge graph (KGQA) is to help biologist and scientist to get valuable insights by asking questions in natural language. Relevant answers can be found by first understanding the question and then querying the KG for right set of nodes and relationships to arrive at an answer. To model the question, language models such as RoBERTa and BioBERT are used to understand context from natural language question. One of the challenges in KGQA is missing links in the KG. Knowledge graph embeddings (KGE) help to overcome this problem by encoding nodes and edges in a dense and more efficient way. In this paper, we use a publicly available KG called Hetionet which is an integrative network of biomedical knowledge assembled from 29 different databases of genes, compounds, diseases, and more. We have enriched this KG dataset by creating a multi-hop biomedical question-answering dataset in natural language for testing the biomedical multi-hop question-answering system and this dataset will be made available to the research community. The major contribution of this research is an integrated system that combines language models with KG embeddings to give highly relevant answers to free-form questions asked by biologists in an intuitive interface. Biomedical multi-hop question-answering system is tested on this data and results are highly encouraging.

在本文中，我们提出了多跳问题回答的两个阶段模型。第一阶段是一个层次图网络，该网络用于对多跳问题进行推理，并能够使用文档的自然结构（即段落，问题，句子和实体）捕获不同级别的粒度。推理过程是转换为节点分类任务（即，段落节点和句子节点）。第二阶段是语言模型微调任务。在一句话中，第一阶段使用图形神经网络选择和连接支持句子作为一个段落，第二阶段在语言模型微调范式中找到答案跨度。

知识库问题的最现有的方法接听（KBQA）关注特定的基础知识库，原因是该方法的固有假设，或者因为在不同的知识库上评估它需要非琐碎的变化。然而，许多流行知识库在其潜在模式中的相似性份额可以利用，以便于跨知识库的概括。为了实现这一概念化，我们基于2级架构介绍了一个KBQA框架，该架构明确地将语义解析与知识库交互分开，促进了数据集和知识图中的转移学习。我们表明，具有不同潜在知识库的数据集预先灌注可以提供显着的性能增益并降低样本复杂性。我们的方法可实现LC-Quad（DBPedia），WEDQSP（FreeBase），简单问话（Wikidata）和MetaQA（WikiMovies-KG）的可比性或最先进的性能。