使用从预先接受训练的语言模型(LMS)和知识图表(LMS)和知识图表(kgs)回答问题的问题提出了两个挑战:给定QA上下文(问答选择),方法需要(i)从大型千克识别相关知识,(ii)对QA上下文和kg进行联合推理。在这项工作中,我们提出了一种新的模型,QA-GNN,它通过两个关键创新解决了上述挑战:(i)相关评分,我们使用LMS来估计KG节点相对于给定的QA上下文的重要性,以及(ii)联合推理,我们将QA上下文和kg连接到联合图,并通过图形神经网络相互更新它们的表示。我们评估了QA基准的模型(CommanSeaseQA,OpenBookQA)和生物医学(MedQa-USMLE)域名。QA-GNN优于现有的LM和LM + kg模型,并表现出可解释和结构化推理的能力,例如,正确处理问题的否定。
translated by 谷歌翻译
现有的kg增强模型用于问题回答主要专注于设计精心图形神经网络(GNN)以模拟知识图(KG)。但是,它们忽略了(i)有效地融合和推理过问题上下文表示和kg表示,并且(ii)在推理期间自动从嘈杂的KG中选择相关节点。在本文中,我们提出了一种新颖的型号,其通过LMS和GNN的联合推理和动态KGS修剪机制解决了上述限制。具体而言,ConntLK通过新的密集双向注意模块在LMS和GNN之间执行联合推理,其中每个问题令牌参加KG节点,每个KG节点都会参加问题令牌,并且两个模态表示熔断和通过多次熔断和更新。步互动。然后,动态修剪模块使用通过联合推理产生的注意重量来递归修剪无关的kg节点。我们在CommanSENSEQA和OpenBookQA数据集上的结果表明,我们的模态融合和知识修剪方法可以更好地利用相关知识来推理。
translated by 谷歌翻译
这项工作调查了以知识图(kg)形式的外部知识来源的理解问题的学习和推理的挑战。我们提出了一种新型的图形神经网络体系结构,称为动态相关图形网络(DRGN)。 DRGN根据问题和答案实体在给定的KG子图上运行,并使用节点之间的相关得分来动态建立新的边缘,以在图形网络中学习节点表示。相关性的这种显式用法作为图表具有以下优点,a)模型可以利用现有关系,重新缩放节点权重,并影响邻里节点的表示方式在kg子图中汇总的方式,b)恢复推理所需的千克中缺失的边缘。此外,作为副产品,由于考虑了问题节点与图形实体之间的相关性,我们的模型改善了处理负面问题。与最新发布的结果相比,我们提出的方法在两个质量检查基准CommonSenseQA和OpenBookQA上显示了竞争性能。
translated by 谷歌翻译
Question Answering (QA) is a task that entails reasoning over natural language contexts, and many relevant works augment language models (LMs) with graph neural networks (GNNs) to encode the Knowledge Graph (KG) information. However, most existing GNN-based modules for QA do not take advantage of rich relational information of KGs and depend on limited information interaction between the LM and the KG. To address these issues, we propose Question Answering Transformer (QAT), which is designed to jointly reason over language and graphs with respect to entity relations in a unified manner. Specifically, QAT constructs Meta-Path tokens, which learn relation-centric embeddings based on diverse structural and semantic relations. Then, our Relation-Aware Self-Attention module comprehensively integrates different modalities via the Cross-Modal Relative Position Bias, which guides information exchange between relevant entities of different modalities. We validate the effectiveness of QAT on commonsense question answering datasets like CommonsenseQA and OpenBookQA, and on a medical question answering dataset, MedQA-USMLE. On all the datasets, our method achieves state-of-the-art performance. Our code is available at http://github.com/mlvlab/QAT.
translated by 谷歌翻译
语言模型(LM)是否可以通过固有的关系推理能力在知识库中的地面问题解决方案(QA)任务?尽管以前仅使用LMS的模型在许多质量检查任务上都看到了一些成功,但最新的方法包括知识图(KG),以补充LMS的逻辑驱动的隐式知识。但是,有效从结构化数据(例如KGS)中提取信息,使LMS保持开放性问题,并且当前模型依靠图形技术来提取知识。在本文中,我们建议仅利用LMS将基于知识的问题的语言和知识与灵活性,覆盖范围和结构化推理相结合。具体而言,我们设计了一种知识构建方法,该方法可以通过动态跳跃来检索相关背景,该方法比传统的基于GNN的技术表达了更全面的。我们设计了一种深层融合机制,以进一步弥合语言和知识之间交换瓶颈的信息。广泛的实验表明,我们的模型始终证明了其对CommenSensenSENSENSESQA基准测试的最先进性能,从而展示了仅利用LMS将LMS稳健地质量质量质量质量质量固定到知识库的可能性。
translated by 谷歌翻译
预训练的语言模型(PTLM)已显示出在自然语言任务上表现良好。许多先前的作品都以通过知识图(KGS)标记的关系链接的实体的形式利用结构性常识来协助PTLM。检索方法使用kg作为单独的静态模块,该模块限制了覆盖范围,因为kgs包含有限的知识。生成方法训练PTLMS kg三倍以提高获得知识的规模。但是,对符号KG实体的培训限制了其在涉及自然语言文本的任务中的适用性,在这些任务中,它们忽略了整体上下文。为了减轻这种情况,我们提出了一个以句子为条件的常识性上下文化器(COSE-CO)作为输入,以使其在生成与输入文本的整体上下文相关的任务中通常可用。为了训练Cose-Co,我们提出了一个新的数据集,其中包括句子和常识知识对。 COSE-CO推断出的知识是多种多样的,并且包含了基础KG中不存在的新实体。我们增强了在多选质量质量检查和开放式常识性推理任务中产生的知识,从而改善了CSQA,ARC,QASC和OBQA数据集的当前最佳方法。我们还展示了其在改善释义生成任务的基线模型方面的适用性。
translated by 谷歌翻译
问题回答(QA)对知识库(KBS)的挑战是充满挑战的,因为所需的推理模式多样化,本质上是无限的,类型的推理模式。但是,我们假设以大型KB为基础,以回答各自子图中各个实体的查询类型所需的推理模式。利用不同子图的本地社区之间的这种结构相似性,我们引入了一个半参数模型(cbr-subg),(i)一个非参数组件,每个查询,每个查询,都会动态检索其他类似的$ k $ - $ - $ - $ - near-neart-tebrienk(KNN)培训查询以及查询特定的子图和(ii)训练的参数组件,该参数分量可以从KNN查询的子图中识别(潜在的)推理模式,然后将其应用于目标查询的子图。我们还提出了一种自适应子图收集策略,以选择特定于查询的compact子图,从而使我们可以扩展到包含数十亿个事实的完整freebase kb。我们表明,CBR-SUBG可以回答需要子图推理模式的查询,并在几个KBQA基准上的最佳模型竞争性能。我们的子图收集策略还会产生更多紧凑的子图(例如,webQSP的尺寸减小55 \%,而将答案召回的召回率增加4.85 \%)\ footNote {代码,模型和子码头可在\ url {https://github.com上获得。 /rajarshd/cbr-subg}}。
translated by 谷歌翻译
我们研究了学习因果推理对程序文本的挑战,以回答“如果...”何时需要外常识知识。我们提出了一个新颖的多跳图推理模型,以1)有效地从大知识图中提取常识子图;2)通过推理从常识子图获得的表示以及问题与上下文之间的上下文相互作用来预测因果答案。我们评估了WIQA基准测试的模型,并与最近的模型相比实现了最先进的性能。
translated by 谷歌翻译
Knowledge graph (KG) link prediction aims to infer new facts based on existing facts in the KG. Recent studies have shown that using the graph neighborhood of a node via graph neural networks (GNNs) provides more useful information compared to just using the query information. Conventional GNNs for KG link prediction follow the standard message-passing paradigm on the entire KG, which leads to over-smoothing of representations and also limits their scalability. On a large scale, it becomes computationally expensive to aggregate useful information from the entire KG for inference. To address the limitations of existing KG link prediction frameworks, we propose a novel retrieve-and-read framework, which first retrieves a relevant subgraph context for the query and then jointly reasons over the context and the query with a high-capacity reader. As part of our exemplar instantiation for the new framework, we propose a novel Transformer-based GNN as the reader, which incorporates graph-based attention structure and cross-attention between query and context for deep fusion. This design enables the model to focus on salient context information relevant to the query. Empirical results on two standard KG link prediction datasets demonstrate the competitive performance of the proposed method.
translated by 谷歌翻译
在知识图上回答自然语言问题(KGQA)仍然是通过多跳推理理解复杂问题的巨大挑战。以前的努力通常利用与实体相关的文本语料库或知识图(kg)嵌入作为辅助信息来促进答案选择。但是,实体之间隐含的富裕语义远未得到很好的探索。本文提议通过利用关系路径的混合语义来改善多跳kgqa。具体而言,我们基于新颖的旋转和规模的实体链接链接预测框架,集成了关系路径的明确文本信息和隐式kg结构特征。在三个KGQA数据集上进行的广泛实验证明了我们方法的优势,尤其是在多跳场景中。进一步的调查证实了我们方法在问题和关系路径之间的系统协调,以识别答案实体。
translated by 谷歌翻译
关于时间知识图(TKGQA)的问题回答最近发现兴趣越来越大。 TKGQA需要时间推理技术来从时间知识库中提取相关信息。唯一现有的TKGQA数据集,即cronquestions,由基于固定时间段内的事实组成,其中跨越同一时期的时间知识图(TKG)可以完全使用用于答案推断,允许使用TKGQA模型。即将根据过去事实回答问题的未来知识。但是,在现实世界的情况下,鉴于到目前为止的知识也很常见,我们希望TKGQA系统回答询问未来的问题。随着人类不断寻求未来计划,建立用于回答此类预测问题的TKGQA系统很重要。然而,这在先前的研究中仍未得到探索。在本文中,我们提出了一个新的任务:关于时间知识图的预测问题。我们还为此任务提出了一个大规模的TKGQA基准数据集,即预测。它包括三种类型的问题,即实体预测,不是和事实推理问题。对于我们数据集中的每个预测问题,QA模型只能在给定问题中注释的时间戳以进行答案推理之前访问TKG信息。我们发现,最先进的TKGQA方法在预测问题上的表现较差,并且他们无法回答不是问题和事实推理问题。为此,我们提出了一种TKGQA模型预测,该模型采用TKG预测模块进行未来推断,以回答所有三种类型的问题。实验结果表明,预测到实体预测问题的最新方法优于最近的TKGQA方法,并且在回答其他两种类型的问题方面也显示出很大的有效性。
translated by 谷歌翻译
Biomedical knowledge graphs (KG) are heterogenous networks consisting of biological entities as nodes and relations between them as edges. These entities and relations are extracted from millions of research papers and unified in a single resource. The goal of biomedical multi-hop question-answering over knowledge graph (KGQA) is to help biologist and scientist to get valuable insights by asking questions in natural language. Relevant answers can be found by first understanding the question and then querying the KG for right set of nodes and relationships to arrive at an answer. To model the question, language models such as RoBERTa and BioBERT are used to understand context from natural language question. One of the challenges in KGQA is missing links in the KG. Knowledge graph embeddings (KGE) help to overcome this problem by encoding nodes and edges in a dense and more efficient way. In this paper, we use a publicly available KG called Hetionet which is an integrative network of biomedical knowledge assembled from 29 different databases of genes, compounds, diseases, and more. We have enriched this KG dataset by creating a multi-hop biomedical question-answering dataset in natural language for testing the biomedical multi-hop question-answering system and this dataset will be made available to the research community. The major contribution of this research is an integrated system that combines language models with KG embeddings to give highly relevant answers to free-form questions asked by biologists in an intuitive interface. Biomedical multi-hop question-answering system is tested on this data and results are highly encouraging.
translated by 谷歌翻译
预训练的语言模型(PLM)在各种自然语言理解任务上取得了巨大的成功。另一方面,对PLM的简单微调对于特定于领域的任务可能是次优的,因为它们不可能涵盖所有域中的知识。尽管PLM的自适应预培训可以帮助他们获得特定于领域的知识,但需要大量的培训成本。此外,自适应预训练可能会通过造成灾难性忘记其常识来损害PLM在下游任务上的表现。为了克服PLM适应性适应性预训练的这种局限性,我们提出了一个新颖的域名适应框架,用于将PLMS创造为知识增强语言模型适应性(KALA),该框架调节了PLM的中间隐藏表示与域中的中间隐藏表示,由实体和实体和实体和实体和实体构成他们的关系事实。我们验证了Kala在问题答案中的性能,并在各个域的多个数据集上命名实体识别任务。结果表明,尽管在计算上有效,但我们的Kala在很大程度上优于适应性预训练。代码可在以下网址获得:https://github.com/nardien/kala/。
translated by 谷歌翻译
今天的大部分AI系统都专注于使用自我关注机制和变压器架构在大量多样化的数据中实现令人印象深刻的性能收益。在本文中,我们建议使用外部注意机制增强变压器架构,以带来外部知识和背景。通过将外部信息集成到预测过程中,我们希望减少对更大的模型的需求,并增加AI系统的民主化。我们发现所提出的外部注意机制可以显着提高现有AI系统的性能,使从业者可以轻松地将基础AI模型自定义到许多不同的下游应用程序。特别是,我们专注于勤杂朗语推理的任务,展示所提出的外部注意机制可以增加现有的变压器模型,并显着提高模型的推理能力。拟议的系统,知识外部关注推理(Kear),达到了开放的铜商QA研究基准的人类奇偶校验,其准确性为89.4 \%,与人类准确性为88.9 \%。
translated by 谷歌翻译
Multi-modal and multi-hop question answering aims to answer a question based on multiple input sources from different modalities. Previous methods retrieve the evidence separately and feed the retrieved evidence to a language model to generate the corresponding answer. However, these methods fail to build connections between candidates and thus cannot model the inter-dependent relation during retrieval. Moreover, the reasoning process over multi-modality candidates can be unbalanced without building alignments between different modalities. To address this limitation, we propose a Structured Knowledge and Unified Retrieval Generation based method (SKURG). We align the sources from different modalities via the shared entities and map them into a shared semantic space via structured knowledge. Then, we utilize a unified retrieval-generation decoder to integrate intermediate retrieval results for answer generation and adaptively determine the number of retrieval steps. We perform experiments on two multi-modal and multi-hop datasets: WebQA and MultimodalQA. The results demonstrate that SKURG achieves state-of-the-art performance on both retrieval and answer generation.
translated by 谷歌翻译
Machine reading comprehension (MRC) is a long-standing topic in natural language processing (NLP). The MRC task aims to answer a question based on the given context. Recently studies focus on multi-hop MRC which is a more challenging extension of MRC, which to answer a question some disjoint pieces of information across the context are required. Due to the complexity and importance of multi-hop MRC, a large number of studies have been focused on this topic in recent years, therefore, it is necessary and worth reviewing the related literature. This study aims to investigate recent advances in the multi-hop MRC approaches based on 31 studies from 2018 to 2022. In this regard, first, the multi-hop MRC problem definition will be introduced, then 31 models will be reviewed in detail with a strong focus on their multi-hop aspects. They also will be categorized based on their main techniques. Finally, a fine-grain comprehensive comparison of the models and techniques will be presented.
translated by 谷歌翻译
Recent improvements in KG-to-text generation are due to additional auxiliary pre-training tasks designed to give the fine-tune task a boost in performance. These tasks require extensive computational resources while only suggesting marginal improvements. Here, we demonstrate that by fusing graph-aware elements into existing pre-trained language models, we are able to outperform state-of-the-art models and close the gap imposed by additional pre-training tasks. We do so by proposing a mask structure to capture neighborhood information and a novel type encoder that adds a bias to the graph-attention weights depending on the connection type. Experiments on two KG-to-text benchmark datasets show our models are competitive while involving fewer parameters and no additional pre-training tasks. By formulating the problem as a framework, we can interchange the various proposed components and begin interpreting KG-to-text generative models based on the topological and type information found in a graph.
translated by 谷歌翻译
知识库问题的最现有的方法接听(KBQA)关注特定的基础知识库,原因是该方法的固有假设,或者因为在不同的知识库上评估它需要非琐碎的变化。然而,许多流行知识库在其潜在模式中的相似性份额可以利用,以便于跨知识库的概括。为了实现这一概念化,我们基于2级架构介绍了一个KBQA框架,该架构明确地将语义解析与知识库交互分开,促进了数据集和知识图中的转移学习。我们表明,具有不同潜在知识库的数据集预先灌注可以提供显着的性能增益并降低样本复杂性。我们的方法可实现LC-Quad(DBPedia),WEDQSP(FreeBase),简单问话(Wikidata)和MetaQA(WikiMovies-KG)的可比性或最先进的性能。
translated by 谷歌翻译
知识图表问题基于信息检索旨在通过从大型知识图表中检索答案来回答问题来回答(即,kgqa)。大多数现有方法首先粗略地检索可能包含候选答案的知识子图(KSG),然后搜索子图中的确切答案。然而,粗略检索的KSG可以包含数千个候选节点,因为查询中涉及的知识图通常是大规模的。为了解决这个问题,我们首先建议通过新的子图分区算法将检索到的ksg分区为几个较小的子ksgs,然后呈现一个图形增强学习,以便测量模型以从中选择排名的子ksgs。我们所提出的模型结合了新的子图匹配网络,以捕获问题和子图中的全局交互以及增强的双边多视角匹配模型,以捕获局部交互。最后,我们分别在全KSG和排名级分ksg上应用答案选择模型,以验证我们提出的图形增强学习的效果。多个基准数据集的实验结果表明了我们方法的有效性。
translated by 谷歌翻译
在本文中,我们提出了多跳问题回答的两个阶段模型。第一阶段是一个层次图网络,该网络用于对多跳问题进行推理,并能够使用文档的自然结构(即段落,问题,句子和实体)捕获不同级别的粒度。推理过程是转换为节点分类任务(即,段落节点和句子节点)。第二阶段是语言模型微调任务。在一句话中,第一阶段使用图形神经网络选择和连接支持句子作为一个段落,第二阶段在语言模型微调范式中找到答案跨度。
translated by 谷歌翻译