Dialogue systems can leverage large pre-trained language models and knowledge to generate fluent and informative responses. However, these models are still prone to produce hallucinated responses not supported by the input source, which greatly hinders their application. The heterogeneity between external knowledge and dialogue context challenges representation learning and source integration, and further contributes to unfaithfulness. To handle this challenge and generate more faithful responses, this paper presents RHO ($\rho$) utilizing the representations of linked entities and relation predicates from a knowledge graph (KG). We propose (1) local knowledge grounding to combine textual embeddings with the corresponding KG embeddings; and (2) global knowledge grounding to equip RHO with multi-hop reasoning abilities via the attention mechanism. In addition, we devise a response re-ranking technique based on walks over KG sub-graphs for better conversational reasoning. Experimental results on OpenDialKG show that our approach significantly outperforms state-of-the-art methods on both automatic and human evaluation by a large margin, especially in hallucination reduction (17.54% in FeQA).
translated by 谷歌翻译
Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.
translated by 谷歌翻译
与外部知识的接地对话系统是提高响应质量的一种有希望的方法。大多数现有的作品采用知识图(KGS)作为外部资源,关注对话的最后一句话中实体的贡献,以了解上下文理解和响应。然而,在多转变环境中隐含的知识与公斤关系之间的过渡规律之间的相关性是不足的。为此,我们提出了一个关系过渡意识知识的对话生成模型(RT-KGD)。具体而言,受到人类对话潜在逻辑的启发,我们的模型将对话级别的关系过渡规律与转向级实体语义信息相结合。以这种方式,知识之间的相互作用被认为是产生丰富的线索,以预测适当的知识并产生相干响应。自动评估和手动评估的实验结果表明,我们的模型表现优于最先进的基准。
translated by 谷歌翻译
知识基础问题回答(KBQA)旨在通过知识库(KB)回答问题。早期研究主要集中于回答有关KB的简单问题,并取得了巨大的成功。但是,他们在复杂问题上的表现远非令人满意。因此,近年来,研究人员提出了许多新颖的方法,研究了回答复杂问题的挑战。在这项调查中,我们回顾了KBQA的最新进展,重点是解决复杂问题,这些问题通常包含多个主题,表达复合关系或涉及数值操作。详细说明,我们从介绍复杂的KBQA任务和相关背景开始。然后,我们描述用于复杂KBQA任务的基准数据集,并介绍这些数据集的构建过程。接下来,我们提出两个复杂KBQA方法的主流类别,即基于语义解析的方法(基于SP)的方法和基于信息检索的方法(基于IR)。具体而言,我们通过流程设计说明了他们的程序,并讨论了它们的主要差异和相似性。之后,我们总结了这两类方法在回答复杂问题时会遇到的挑战,并解释了现有工作中使用的高级解决方案和技术。最后,我们结论并讨论了与复杂的KBQA有关的几个有希望的方向,以进行未来的研究。
translated by 谷歌翻译
本文对过去二十年来对自然语言生成(NLG)的研究提供了全面的审查,特别是与数据到文本生成和文本到文本生成深度学习方法有关,以及NLG的新应用技术。该调查旨在(a)给出关于NLG核心任务的最新综合,以及该领域采用的建筑;(b)详细介绍各种NLG任务和数据集,并提请注意NLG评估中的挑战,专注于不同的评估方法及其关系;(c)强调一些未来的强调和相对近期的研究问题,因为NLG和其他人工智能领域的协同作用而增加,例如计算机视觉,文本和计算创造力。
translated by 谷歌翻译
隐性知识,例如常识,是人类对话的关键。当前的神经反应生成(RG)模型经过训练以直接产生响应,省略了未阐明的隐式知识。在本文中,我们介绍了说话之前的思维(TBS),这是一种首先将隐式常识知识(思考)外部化的生成方法(思考),并使用这些知识来产生响应(speak)。我们期望外部化隐式知识可以更有效地学习,产生更多信息的响应,并实现了更多可解释的模型。我们分析了不同的选择,以收集知识一致的对话,代表隐式知识以及知识和对话之间的过渡。经验结果表明,TBS模型在大多数自动指标上优于端到端和知识增强的RG基准,并通过人类注释者评估,产生更有信息,具体和常识性遵循的响应。 TBS还产生了有意义的知识,并且与85 \%左右的对话有关。
translated by 谷歌翻译
Pre-trained Language Models (PLMs) which are trained on large text corpus through the self-supervised learning method, have yielded promising performance on various tasks in Natural Language Processing (NLP). However, though PLMs with huge parameters can effectively possess rich knowledge learned from massive training text and benefit downstream tasks at the fine-tuning stage, they still have some limitations such as poor reasoning ability due to the lack of external knowledge. Incorporating knowledge into PLMs has been tried to tackle these issues. In this paper, we present a comprehensive review of Knowledge-Enhanced Pre-trained Language Models (KE-PLMs) to provide a clear insight into this thriving field. We introduce appropriate taxonomies respectively for Natural Language Understanding (NLU) and Natural Language Generation (NLG) to highlight the focus of these two kinds of tasks. For NLU, we take several types of knowledge into account and divide them into four categories: linguistic knowledge, text knowledge, knowledge graph (KG), and rule knowledge. The KE-PLMs for NLG are categorized into KG-based and retrieval-based methods. Finally, we point out some promising future directions of KE-PLMs.
translated by 谷歌翻译
知识库问题应答(KBQA)旨在在外部知识库的帮助下回答自然语言问题。核心思想是找到内部知识与知识库的已知三元组之间的内部知识之间的联系。 KBQA任务管道包含几个步骤,包括实体识别,关系提取和实体链接。这种管道方法意味着任何过程中的错误将不可避免地传播到最终预测。为了解决上述问题,本文提出了一种具有预培训语言模型(PLM)和知识图(KG)的语料库生成 - 检索方法(CGRM)。首先,基于MT5模型,我们设计了两个新的预训练任务:基于段落的知识屏蔽语言建模和问题,以获取知识增强型T5(KT5)模型。其次,在用一系列启发式规则预处理知识图的预处理之后,KT5模型基于处理的三元组生成自然语言QA对。最后,我们通过检索合成数据集直接解决QA。我们在NLPCC-ICCPOL 2016 KBQA数据集上测试我们的方法,结果表明,我们的框架提高了KBQA的性能,直接向前的方法与最先进的方法竞争。
translated by 谷歌翻译
Large pre-trained language models have recently enabled open-ended generation frameworks (e.g., prompt-to-text NLG) to tackle a variety of tasks going beyond the traditional data-to-text generation. While this framework is more general, it is under-specified and often leads to a lack of controllability restricting their real-world usage. We propose a new grounded keys-to-text generation task: the task is to generate a factual description about an entity given a set of guiding keys, and grounding passages. To address this task, we introduce a new dataset, called EntDeGen. Inspired by recent QA-based evaluation measures, we propose an automatic metric, MAFE, for factual correctness of generated descriptions. Our EntDescriptor model is equipped with strong rankers to fetch helpful passages and generate entity descriptions. Experimental result shows a good correlation (60.14) between our proposed metric and human judgments of factuality. Our rankers significantly improved the factual correctness of generated descriptions (15.95% and 34.51% relative gains in recall and precision). Finally, our ablation study highlights the benefit of combining keys and groundings.
translated by 谷歌翻译
Conversational recommender systems (CRSs) often utilize external knowledge graphs (KGs) to introduce rich semantic information and recommend relevant items through natural language dialogues. However, original KGs employed in existing CRSs are often incomplete and sparse, which limits the reasoning capability in recommendation. Moreover, only few of existing studies exploit the dialogue context to dynamically refine knowledge from KGs for better recommendation. To address the above issues, we propose the Variational Reasoning over Incomplete KGs Conversational Recommender (VRICR). Our key idea is to incorporate the large dialogue corpus naturally accompanied with CRSs to enhance the incomplete KGs; and perform dynamic knowledge reasoning conditioned on the dialogue context. Specifically, we denote the dialogue-specific subgraphs of KGs as latent variables with categorical priors for adaptive knowledge graphs refactor. We propose a variational Bayesian method to approximate posterior distributions over dialogue-specific subgraphs, which not only leverages the dialogue corpus for restructuring missing entity relations but also dynamically selects knowledge based on the dialogue context. Finally, we infuse the dialogue-specific subgraphs to decode the recommendation and responses. We conduct experiments on two benchmark CRSs datasets. Experimental results confirm the effectiveness of our proposed method.
translated by 谷歌翻译
在本文中,我们建议利用对话的独特特征,共享参与者的常识性知识,以解决总结它们的困难。我们提出了病态的框架,该框架使用常识推论作为其他背景。与以前仅依赖于输入对话的工作相比,Sick使用外部知识模型来生成丰富的常识推断,并选择具有基于相似性选择方法的最可能的推理。基于生病的,病人++的理解为监督,在总结多任务学习环境中的对话时,添加了产生常识推断的任务。实验结果表明,通过注入常识性知识,我们的框架比现有方法产生更多信息和一致的摘要。
translated by 谷歌翻译
预训练的语言模型(PTLM)已显示出在自然语言任务上表现良好。许多先前的作品都以通过知识图(KGS)标记的关系链接的实体的形式利用结构性常识来协助PTLM。检索方法使用kg作为单独的静态模块,该模块限制了覆盖范围,因为kgs包含有限的知识。生成方法训练PTLMS kg三倍以提高获得知识的规模。但是,对符号KG实体的培训限制了其在涉及自然语言文本的任务中的适用性,在这些任务中,它们忽略了整体上下文。为了减轻这种情况,我们提出了一个以句子为条件的常识性上下文化器(COSE-CO)作为输入,以使其在生成与输入文本的整体上下文相关的任务中通常可用。为了训练Cose-Co,我们提出了一个新的数据集,其中包括句子和常识知识对。 COSE-CO推断出的知识是多种多样的,并且包含了基础KG中不存在的新实体。我们增强了在多选质量质量检查和开放式常识性推理任务中产生的知识,从而改善了CSQA,ARC,QASC和OBQA数据集的当前最佳方法。我们还展示了其在改善释义生成任务的基线模型方面的适用性。
translated by 谷歌翻译
Natural Language Processing (NLP) has been revolutionized by the use of Pre-trained Language Models (PLMs) such as BERT. Despite setting new records in nearly every NLP task, PLMs still face a number of challenges including poor interpretability, weak reasoning capability, and the need for a lot of expensive annotated data when applied to downstream tasks. By integrating external knowledge into PLMs, \textit{\underline{K}nowledge-\underline{E}nhanced \underline{P}re-trained \underline{L}anguage \underline{M}odels} (KEPLMs) have the potential to overcome the above-mentioned limitations. In this paper, we examine KEPLMs systematically through a series of studies. Specifically, we outline the common types and different formats of knowledge to be integrated into KEPLMs, detail the existing methods for building and evaluating KEPLMS, present the applications of KEPLMs in downstream tasks, and discuss the future research directions. Researchers will benefit from this survey by gaining a quick and comprehensive overview of the latest developments in this field.
translated by 谷歌翻译
会话推荐系统(CRS)旨在主动引起用户偏好,并通过自然语言对话推荐高质量的项目。通常,CRS由建议模块组成,以预测用户的首选项目和对话模块,以生成适当的响应。要开发有效的CR,必须无缝整合两个模块。现有作品要么设计语义一致性策略,要么共享两个模块之间的知识资源和表示。但是,这些方法仍然依靠不同的体系结构或技术来开发两个模块,因此很难进行有效的模块集成。为了解决这个问题,我们根据知识增强的及时学习提出了一个名为UNICRS的统一CRS模型。我们的方法将建议和对话子任务统一到及时学习范式中,并根据固定的预训练的语言模型(PLM)利用知识增强的提示来以统一的方法来实现两个子任务。在及时的设计中,我们包括融合的知识表示,特定于任务的软令牌和对话环境,它们可以提供足够的上下文信息以适应CRS任务的PLM。此外,对于建议子任务,我们还将生成的响应模板作为提示的重要组成部分结合起来,以增强两个子任务之间的信息交互。对两个公共CRS数据集进行的广泛实验证明了我们方法的有效性。
translated by 谷歌翻译
医学对话生成是一项重要但具有挑战性的任务。以前的大多数作品都依赖于注意力机制和大规模预处理的语言模型。但是,这些方法通常无法从长时间的对话历史中获取关键信息,从而产生准确和信息丰富的响应,因为医疗实体通常散布在多种话语中以及它们之间的复杂关系。为了减轻此问题,我们提出了一个具有关键信息召回(Medpir)的医疗响应生成模型,该模型建立在两个组件上,即知识吸引的对话图形编码器和召回增强的生成器。知识吸引的对话图编码器通过利用话语中的实体之间的知识关系,并使用图形注意力网络对话图来构建对话图。然后,召回增强的发电机通过在产生实际响应之前生成对话的摘要来增强这些关键信息的使用。两个大型医学对话数据集的实验结果表明,Medpir在BLEU分数和医疗实体F1度量中的表现优于强大的基准。
translated by 谷歌翻译
The goal of building dialogue agents that can converse with humans naturally has been a long-standing dream of researchers since the early days of artificial intelligence. The well-known Turing Test proposed to judge the ultimate validity of an artificial intelligence agent on the indistinguishability of its dialogues from humans'. It should come as no surprise that human-level dialogue systems are very challenging to build. But, while early effort on rule-based systems found limited success, the emergence of deep learning enabled great advance on this topic. In this thesis, we focus on methods that address the numerous issues that have been imposing the gap between artificial conversational agents and human-level interlocutors. These methods were proposed and experimented with in ways that were inspired by general state-of-the-art AI methodologies. But they also targeted the characteristics that dialogue systems possess.
translated by 谷歌翻译
由于缺乏培训数据和异质知识来源,知识接地的对话系统是挑战的。由于培训数据中涵盖的有限主题,现有系统在不良主题上表现不佳。此外,异构知识源使系统概括到其他任务的系统,因为不同知识表示中的知识来源需要不同的知识编码器。为了解决这些挑战,我们呈现插头,将不同知识来源均匀化为知识接地的对话生成任务的统一知识来源的语言模型。插头在对话生成任务上进行预先培训,调节统一的基本知识表示。它可以通过一些培训示例概括到不同下游知识接地的对话一代任务。两个基准测试的实证评估表明,我们的模型越好跨越不同的知识接地任务。它可以在完全监督的设置下实现具有最先进的方法的可比性,并且显着优于零拍摄和少量拍摄设置中的其他方法。
translated by 谷歌翻译
旨在用自然语言和谐地与人类交流的智能对话体系对于促进人工智能时代的人机互动的发展非常出色。有了逐渐复杂的人类计算机交互要求(例如,多模式输入,时间敏感性),传统的基于文本的对话系统很难满足对更加生动和方便的交互的需求。因此,视觉背景增强对话系统(VAD)有可能通过感知和理解多模式信息(即图像或视频中的视觉上下文,文本对话历史记录)与人类进行交流,已成为主要的研究范式。 VAD受益于视觉和文本上下文之间的一致性和互补性,具有产生引人入胜和背景感知响应的潜力。为了描述VAD的开发,我们首先表征VAD的概念和独特功能,然后介绍其通用系统体系结构以说明系统工作流程。随后,对一些研究挑战和代表性作品进行了详细研究,然后进行了权威基准摘要。我们通过提出一些开放问题和有前途的VAD研究趋势来结束本文,例如,在跨模式对话环境下,人机对话的认知机制以及知识增强的跨模式语义互动。
translated by 谷歌翻译
知识增强的预训练预审语言模型(Keplms)是预先接受的模型,具有从知识图中注入的关系三元组,以提高语言理解能力。为了保证有效的知识注入,之前的研究将模型与知识编码器集成,以表示从知识图表中检索的知识。知识检索和编码的操作带来了重要的计算负担,限制了在需要高推理速度的现实应用程序中使用这些模型。在本文中,我们提出了一种名为DKPLM的新型KEPLM,其在预训练,微调和推理阶段进行了预先训练的语言模型的知识注射过程,这有助于KEPLMS在现实世界场景中的应用。具体而言,我们首先检测知识感知的长尾实体作为知识注射的目标,增强了Keplms的语义理解能力,避免注入冗余信息。长尾实体的嵌入式被相关知识三元组形成的“伪令牌表示”取代。我们进一步设计了用于预培训的关系知识解码任务,以强制模型通过关系三重重建来真正了解注入的知识。实验表明,我们的模型在零拍摄知识探测任务和多种知识意识语言理解任务中显着优于其他KEPLS。我们进一步表明,由于分解机制,DKPLM具有比其他竞争模型更高的推理速度。
translated by 谷歌翻译
关系提取是一项重要但具有挑战性的任务,旨在从文本中提取所有隐藏的关系事实。随着深层语言模型的发展,关系提取方法在各种基准上都取得了良好的性能。但是,我们观察到以前方法的两个缺点:首先,在各种关系提取设置下没有统一的框架可以很好地工作;其次,有效利用外部知识作为背景信息。在这项工作中,我们提出了一种知识增强的生成模型来减轻这两个问题。我们的生成模型是一个统一的框架,可在各种关系提取设置下依次生成关系三胞胎,并明确利用来自知识图(KG)的相关知识来解决歧义。我们的模型在包括WebNLG,NYT10和Tacred在内的多个基准和设置上实现了卓越的性能。
translated by 谷歌翻译