我们在面向任务为导向的对话框（TOD）的端到端学习中提出了一种新问题，其中对话系统模仿故障排除代理，该故障排除代理通过诊断其问题（例如，汽车而未启动）帮助用户。这些对话框基于特定于域的流程图，该代理在对话期间应该遵循代理。我们的任务暴露了神经TOD的新颖技术挑战，例如在没有显式注释的情况下对流程图的话语接地，当用户询问澄清问题时，提及额外的手动页面，以及在测试时间遵循看不见的流程图。我们释放由2,738个对话框组成的数据集（浮雕），该对话框为12个不同的故障排除流程图。我们还设计了一个神经模型，扑腾，它使用检索增强的生成架构来训练对话框。我们的实验发现，Flonet可以对未来的流程图进行零射流传输，并为未来的研究设定强大的基线。

对话式AI中的现有研究主要将面向任务的对话框（TOD）和问题答案（QA）视为单独的任务。为了构建可以完成用户任务和支持信息寻求信息的对话代理的目标，构建一个可以访问各种外部知识的系统，构建一个处理TOD和QA的系统非常重要。在这项工作中，我们提出了一项新任务，开放式TOD（OB-TOD），将TOD与QA任务相结合，并将外部知识源扩展到包括明确的知识源（例如Web）和隐式知识源（例如，例如，预训练的语言模型）。我们创建了一个新的数据集ob-multiwoz，在这里，我们在其中丰富了Tod会议，并使用类似QA的信息寻求基于外部知识的经验。我们提出了一个统一的模型Opera（开放式末端到端任务对话框），可以适当地访问明确和隐性的外部知识，以解决定义的任务。实验结果表明，与闭环基线相比，Opera的表现出色，并说明了两种知识类型的价值。

这项工作提出了一个新的对话数据集，即cookdial，该数据集促进了对任务知识了解的面向任务的对话系统的研究。该语料库包含260个以人类对任务为导向的对话框，其中代理给出了配方文档，指导用户烹饪菜肴。 Cookdial中的对话框展示了两个独特的功能：（i）对话流与支持文档之间的程序对齐； （ii）复杂的代理决策涉及分割长句子，解释硬说明并在对话框上下文中解决核心。此外，我们在假定的面向任务的对话框系统中确定了三个具有挑战性的（子）任务：（1）用户问题理解，（2）代理操作框架预测和（3）代理响应生成。对于这些任务中的每一个，我们都会开发一个神经基线模型，我们在cookdial数据集上进行了评估。我们公开发布烹饪数据集，包括对话框和食谱文档的丰富注释，以刺激对特定于域的文档接地对话框系统的进一步研究。

Interview has been regarded as one of the most crucial step for recruitment. To fully prepare for the interview with the recruiters, job seekers usually practice with mock interviews between each other. However, such a mock interview with peers is generally far away from the real interview experience: the mock interviewers are not guaranteed to be professional and are not likely to behave like a real interviewer. Due to the rapid growth of online recruitment in recent years, recruiters tend to have online interviews, which makes it possible to collect real interview data from real interviewers. In this paper, we propose a novel application named EZInterviewer, which aims to learn from the online interview data and provides mock interview services to the job seekers. The task is challenging in two ways: (1) the interview data are now available but still of low-resource; (2) to generate meaningful and relevant interview dialogs requires thorough understanding of both resumes and job descriptions. To address the low-resource challenge, EZInterviewer is trained on a very small set of interview dialogs. The key idea is to reduce the number of parameters that rely on interview dialogs by disentangling the knowledge selector and dialog generator so that most parameters can be trained with ungrounded dialogs as well as the resume data that are not low-resource. Evaluation results on a real-world job interview dialog dataset indicate that we achieve promising results to generate mock interviews. With the help of EZInterviewer, we hope to make mock interview practice become easier for job seekers.

在寻求信息的对话中，用户与代理商进行对话，以提出一系列通常可以不足或过度指定的问题。理想的代理商首先将通过搜索其基本知识来源，然后与用户进行适当互动以解决它，从而确定他们处于这种情况。但是，大多数现有研究都无法或人为地纳入此类代理端计划。在这项工作中，我们介绍了Inscit（发音为Insight），这是一种用于与混合互动相互作用的信息寻求对话的数据集。它包含从805个人类对话中进行的4.7k用户代理转弯，代理商对Wikipedia进行搜索，并要求澄清或提供相关信息以解决用户查询。我们定义了两个子任务，即证据通过识别和响应产生，以及一种新的人类评估协议来评估模型绩效。我们根据对话知识识别和开放域问题的最新模型报告了两个强大的基线的结果。这两种模型都显着不足，并且没有产生连贯和信息丰富的反应，这表明未来的研究有足够的改进空间。

最近的知识接地对话框方法通过从外部文本文档中包含信息来生成响应。这些方法不需要在训练期间知道确切的文件，并依赖于使用检索系统来从大型索引获取相关文档。用于生成响应的文档被建模为潜在的变量，其先验概率需要估计。诸如rag等型号，在从索引中检索的文档上边缘化文档概率，以定义对端到端优化的日志似然丢失函数。在本文中，我们开发了上述技术的变分方法，据称，我们最大化证据下限（ELBO）。使用三个公开可用的开放式对话数据集的集合，我们展示了与地面真相响应的信息的后部分布如何允许在训练期间更好地逼近客观函数。为了克服与大型知识收集相关的抽样相关的挑战，我们开发了一种高效的方法来近似eLBO。据我们所知，我们是第一个适用于开放式无监督知识接地对话系统的变分培训。

人类通常通过利用关于他们正在交谈的人的主题和背景信息的先验知识来进行对话。然而，现有的会话代理和数据集不考虑此类综合信息，因此它们有一个限制生成知识和人格正确融合的话语。为解决此问题，我们介绍了一个呼叫进行定制对话（焦点）数据集，其中包括用户的角色和维基百科知识建立了自定义答案。为了评估预先训练的语言模型的信息和定制话语的能力，我们利用BART和GPT-2以及基于变压器的模型。我们评估了他们的生成能力，自动分数并对人类评估进行定性结果。我们仔细检查模型是否反映了我们提出的两个子任务，人物接地（PG）和知识接地（KG）的充分人物和知识。此外，我们表明我们的数据的话语通过接地质量评估来构建具有正确的知识和角色。

Chit-chat models are known to have several problems: they lack specificity, do not display a consistent personality and are often not very captivating. In this work we present the task of making chit-chat more engaging by conditioning on profile information. We collect data and train models to (i) condition on their given profile information; and (ii) information about the person they are talking to, resulting in improved dialogues, as measured by next utterance prediction. Since (ii) is initially unknown, our model is trained to engage its partner with personal topics, and we show the resulting dialogue can be used to predict profile information about the interlocutors.

In open-domain dialogue intelligent agents should exhibit the use of knowledge, however there are few convincing demonstrations of this to date. The most popular sequence to sequence models typically "generate and hope" generic utterances that can be memorized in the weights of the model when mapping from input utterance(s) to output, rather than employing recalled knowledge as context. Use of knowledge has so far proved difficult, in part because of the lack of a supervised learning benchmark task which exhibits knowledgeable open dialogue with clear grounding. To that end we collect and release a large dataset with conversations directly grounded with knowledge retrieved from Wikipedia. We then design architectures capable of retrieving knowledge, reading and conditioning on it, and finally generating natural responses. Our best performing dialogue models are able to conduct knowledgeable discussions on open-domain topics as evaluated by automatic metrics and human evaluations, while our new benchmark allows for measuring further improvements in this important research direction.

在典型的客户服务聊天方案中，客户联系支持中心以便帮助或提高投诉，人类代理商试图解决这些问题。在大多数情况下，在谈话结束时，要求代理人写一份简短的总结强调问题和建议的解决方案，通常是为了使其他可能需要处理同一客户或问题的其他代理商的利益。本文的目标是推进此任务的自动化。我们介绍了第一个大规模，高质量的客户服务对话框摘要数据集，接近6500人的注释摘要。数据基于现实世界的客户支持对话框，包括提取和抽象摘要。我们还介绍了一种特定于对话框的新无监督的提取摘要方法。

由于人类参与者的参与，收集培训对话系统的数据可能非常昂贵，并且需要广泛的注释。特别是在文档接地的对话系统中，人类专家需要仔细阅读非结构化文件以回答用户的问题。结果，现有的文档接地对话对话数据集相对较小，并且妨碍了对话系统的有效培训。在本文中，我们提出了一种通过生成对话模型在文档上接地的自动数据增强技术。对话模型由用户BOT和代理机器人组成，可以在给定输入文档的情况下合成不同的对话，然后用于训练下游模型。在补充原始数据集时，我们的方法可以实现对传统数据增强方法的显着改进。我们还在低资源环境中实现了良好的性能。

Knowledge-grounded dialogue systems powered by large language models often generate responses that, while fluent, are not attributable to a relevant source of information. Progress towards models that do not exhibit this issue requires evaluation metrics that can quantify its prevalence. To this end, we introduce the Benchmark for Evaluation of Grounded INteraction (BEGIN), comprised of 12k dialogue turns generated by neural dialogue systems trained on three knowledgegrounded dialogue corpora. We collect human annotations assessing the extent to which the models' responses can be attributed to the given background information. We then use BEGIN to analyze eight evaluation metrics. We find that these metrics rely on spurious correlations, do not reliably distinguish attributable abstractive responses from unattributable ones, and perform substantially worse when the knowledge source is longer. Our findings underscore the need for more sophisticated and robust evaluation metrics for knowledge-grounded dialogue. We make BEGIN publicly available at https://github.com/ google/BEGIN-dataset.

Many efforts have been made to construct dialog systems for different types of conversations, such as task-oriented dialog (TOD) and open-domain dialog (ODD). To better mimic human-level conversations that usually fuse various dialog modes, it is essential to build a system that can effectively handle both TOD and ODD and access different knowledge sources. To address the lack of available data for the fused task, we propose a framework for automatically generating dialogues that combine knowledge-grounded ODDs and TODs in various settings. Additionally, we introduce a unified model PivotBot that is capable of appropriately adopting TOD and ODD modes and accessing different knowledge sources in order to effectively tackle the fused task. Evaluation results demonstrate the superior ability of the proposed model to switch seamlessly between TOD and ODD tasks.

The goal of building dialogue agents that can converse with humans naturally has been a long-standing dream of researchers since the early days of artificial intelligence. The well-known Turing Test proposed to judge the ultimate validity of an artificial intelligence agent on the indistinguishability of its dialogues from humans'. It should come as no surprise that human-level dialogue systems are very challenging to build. But, while early effort on rule-based systems found limited success, the emergence of deep learning enabled great advance on this topic. In this thesis, we focus on methods that address the numerous issues that have been imposing the gap between artificial conversational agents and human-level interlocutors. These methods were proposed and experimented with in ways that were inspired by general state-of-the-art AI methodologies. But they also targeted the characteristics that dialogue systems possess.

对话研究的最终目标是开发可以在交互式设置中有效使用的系统。为此，我们在第9对话系统技术挑战中介绍了对话框的交互式评估。该曲目由两个子任务组成。第一个子任务涉及建立知识接地的响应生成模型。第二个子任务旨在通过与真实用户的交互式设置进行评估，旨在将对话模型扩展到静态数据集之外。我们的曲目挑战参与者开发强大的响应生成模型，并探索将它们扩展到与真实用户的来回互动的策略。从静态语料库到交互式评估的发展引入了独特的挑战，并促进了对开放域对话系统的更全面评估。本文概述了曲目，包括方法和结果。此外，它提供了有关如何最佳评估开放域对话框模型的见解

Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers. We introduce HOTPOTQA, a new dataset with 113k Wikipedia-based question-answer pairs with four key features: (1) the questions require finding and reasoning over multiple supporting documents to answer; (2) the questions are diverse and not constrained to any pre-existing knowledge bases or knowledge schemas; (3) we provide sentence-level supporting facts required for reasoning, allowing QA systems to reason with strong supervision and explain the predictions; (4) we offer a new type of factoid comparison questions to test QA systems' ability to extract relevant facts and perform necessary comparison. We show that HOTPOTQA is challenging for the latest QA systems, and the supporting facts enable models to improve performance and make explainable predictions.

我们介绍了Godel（接地开放对话语言模型），这是对话框的大型预训练的语言模型。与诸如Dialogpt之类的早期模型相比，Godel利用了一个新的扎根预训练阶段，旨在更好地支持将Godel适应广泛的下游对话框任务，这些任务需要当前对话外部的信息（例如，数据库或文档）到产生良好的回应。针对一系列基准测试的实验，这些基准涵盖了面向任务的对话框，对话质量质量检查和接地的开放式对话框，表明Godel在几次以上的微调设置中优于最先进的预训练的对话模型，就人类和自动评估。我们评估方法的一个新颖特征是引入了一个效用概念，该概念除了其交流特征（内在评估）外，还评估了响应的有用性（外部评估）。我们表明，外部评估提供了改进的通道间一致性和与自动指标的相关性。代码和数据处理脚本公开可用。

最近，通过“向导”模拟游戏收集了一类以任务为导向的对话（TOD）数据集。但是，《巫师》数据实际上是模拟的数据，因此与现实生活中的对话根本不同，这些对话更加嘈杂和随意。最近，Seretod挑战赛是组织的，并发布了Mobilecs数据集，该数据集由来自中国移动的真实用户和客户服务人员之间的真实世界对话框组成。基于Mobilecs数据集，Seretod挑战具有两个任务，不仅评估了对话系统本身的构建，而且还检查了对话框成绩单中的信息提取，这对于建立TOD的知识库至关重要。本文主要介绍了Mobilecs数据集对这两项任务的基线研究。我们介绍了如何构建两个基线，遇到的问题以及结果。我们预计基线可以促进令人兴奋的未来研究，以建立针对现实生活任务的人类机器人对话系统。

以任务为导向的对话系统（TDSS）主要在离线设置或人类评估中评估。评估通常仅限于单转或非常耗时。作为替代方案，模拟用户行为的用户模拟器使我们能够考虑一组广泛的用户目标，以生成类似人类的对话以进行模拟评估。使用现有的用户模拟器来评估TDSS是具有挑战性的，因为用户模拟器主要旨在优化TDSS的对话策略，并且评估功能有限。此外，对用户模拟器的评估是一个开放的挑战。在这项工作中，我们提出了一个用于端到端TDS评估的隐喻用户模拟器，如果它在与系统的交互中模拟用户的类似思维，则定义模拟器是隐喻的。我们还提出了一个基于测试人员的评估框架，以生成变体，即具有不同功能的对话系统。我们的用户模拟器构建了一个隐喻的用户模型，该模型通过参考遇到新项目时的先验知识来帮助模拟器进行推理。我们通过检查模拟器与变体之间的模拟相互作用来估计模拟器的质量。我们的实验是使用三个TDS数据集进行的。与基于议程的模拟器和三个数据集上的SEQ2SEQ模型相比，隐喻用户模拟器与手动评估的一致性更好。我们的测试人员框架展示了效率，并且可以更好地概括和可扩展性，因为它可以适用于多个域中的对话和多个任务，例如对话建议和电子商务对话。

Natural Language Generation (NLG) represents a large collection of tasks in the field of NLP. While many of these tasks have been tackled well by the cross-entropy (CE) loss, the task of dialog generation poses a few unique challenges for this loss function. First, CE loss assumes that for any given input, the only possible output is the one available as the ground truth in the training dataset. In general, this is not true for any task, as there can be multiple semantically equivalent sentences, each with a different surface form. This problem gets exaggerated further for the dialog generation task, as there can be multiple valid responses (for a given context) that not only have different surface forms but are also not semantically equivalent. Second, CE loss does not take the context into consideration while processing the response and, hence, it treats all ground truths with equal importance irrespective of the context. But, we may want our final agent to avoid certain classes of responses (e.g. bland, non-informative or biased responses) and give relatively higher weightage for more context-specific responses. To circumvent these shortcomings of the CE loss, in this paper, we propose a novel loss function, CORAL, that directly optimizes recently proposed estimates of human preference for generated responses. Using CORAL, we can train dialog generation models without assuming non-existence of response other than the ground-truth. Also, the CORAL loss is computed based on both the context and the response. Extensive comparisons on two benchmark datasets show that the proposed methods outperform strong state-of-the-art baseline models of different sizes.