零/几次转移到看不见的服务是面向任务的对话研究中的一个关键挑战。架构引导的对话（SGD）数据集引入了一个范式，以使模型通过模式以零摄影的方式支持任何服务，该模型将服务API描述为自然语言的模型。我们通过设计SGD -X来探索对话系统对模式中语言变化的鲁棒性 - 一种基准，该基准扩展了SGD的语义上相似但风格相似但在每个模式上具有相似风格的变体。我们观察到，两种顶级状态跟踪模型无法通过模式变体概括，这些模型通过联合目标准确性和用于测量模式灵敏度的新型指标来衡量。此外，我们提出了一种简单的模型数据扩展方法，以改善模式鲁棒性。

Virtual assistants such as Google Assistant, Alexa and Siri provide a conversational interface to a large number of services and APIs spanning multiple domains. Such systems need to support an ever-increasing number of services with possibly overlapping functionality. Furthermore, some of these services have little to no training data available. Existing public datasets for task-oriented dialogue do not sufficiently capture these challenges since they cover few domains and assume a single static ontology per domain. In this work, we introduce the the Schema-Guided Dialogue (SGD) dataset, containing over 16k multi-domain conversations spanning 16 domains. Our dataset exceeds the existing task-oriented dialogue corpora in scale, while also highlighting the challenges associated with building large-scale virtual assistants. It provides a challenging testbed for a number of tasks including language understanding, slot filling, dialogue state tracking and response generation. Along the same lines, we present a schema-guided paradigm for task-oriented dialogue, in which predictions are made over a dynamic set of intents and slots, provided as input, using their natural language descriptions. This allows a single dialogue system to easily support a large number of services and facilitates simple integration of new services without requiring additional training data. Building upon the proposed paradigm, we release a model for dialogue state tracking capable of zero-shot generalization to new APIs, while remaining competitive in the regular setting.

以任务为导向的对话系统通常采用对话状态跟踪器（DST）成功完成对话。最近的最新DST实现依赖于各种服务的模式来改善模型的鲁棒性并处理对新域的零击概括[1]，但是这种方法[2，3]通常需要多个大型变压器模型和长时间输入序列以表现良好。我们提出了一个基于多任务BERT的单个模型，该模型共同解决了意图预测的三个DST任务，请求的插槽预测和插槽填充。此外，我们提出了对对话历史和服务模式的高效和简约编码，该编码被证明可以进一步提高性能。对SGD数据集的评估表明，我们的方法的表现优于基线SGP-DST，比最新的方法相比表现良好，同时在计算上的效率更高。进行了广泛的消融研究，以检查我们模型成功的促成因素。

在与用户进行交流时，以任务为导向的对话系统必须根据对话历史记录在每个回合时跟踪用户的需求。这个称为对话状态跟踪（DST）的过程至关重要，因为它直接告知下游对话政策。近年来，DST引起了很大的兴趣，文本到文本范式作为受欢迎的方法。在本评论论文中，我们首先介绍任务及其相关的数据集。然后，考虑到最近出版的大量出版物，我们确定了2021 - 2022年研究的重点和研究进展。尽管神经方法已经取得了重大进展，但我们认为对话系统（例如概括性）的某些关键方面仍未得到充实。为了激励未来的研究，我们提出了几种研究途径。

Dialogue State Tracking (DST), a key component of task-oriented conversation systems, represents user intentions by determining the values of pre-defined slots in an ongoing dialogue. Existing approaches use hand-crafted templates and additional slot information to fine-tune and prompt large pre-trained language models and elicit slot values from the dialogue context. Significant manual effort and domain knowledge is required to design effective prompts, limiting the generalizability of these approaches to new domains and tasks. In this work, we propose DiSTRICT, a generalizable in-context tuning approach for DST that retrieves highly relevant training examples for a given dialogue to fine-tune the model without any hand-crafted templates. Experiments with the MultiWOZ benchmark datasets show that DiSTRICT outperforms existing approaches in various zero-shot and few-shot settings using a much smaller model, thereby providing an important advantage for real-world deployments that often have limited resource availability.

最近延伸预留下芬特的神经模型的神经模型继续实现新的最新导致对话状态跟踪（DST）基准的联合目标准确性（JGA）。但是，我们调查了他们的稳健性，因为它们在JGA中显示了急剧下降，以便与现实扰动的话语或对话框流动的对话。通过清单（Ribeiro等，2020），我们设计了一个名为CheckDST的度量集合，促进DST模型的比较，通过测试具有增强测试集的众所周知的弱点来促进革命性的全面尺寸。我们使用CheckDST评估最近的DST模型，并认为模型应该更全面地评估，而不是在JGA上追求最先进的JGA，因为更高的JGA不保证更好的整体稳健性。我们发现基于跨度的分类模型是有弹性的，不合适的命名实体，但对语言品种不强大，而那些基于自回归语言模型的人概括为语言变化，但往往会记住命名实体并往往是幻觉。由于它们各自的弱点，两种方法都不适合现实世界部署。我们认为CheckDst是未来研究的一个有用指南，用于开发面向任务的对话模型，体现了各种方法的优势。

用户模拟器（USS）通常用于通过增强学习训练面向任务的对话系统（DSS）。相互作用通常是在语义层面上以提高效率的，但是从语义动作到自然语言仍然存在差距，这会导致培训和部署环境之间的不匹配。在培训期间，将自然语言生成（NLG）模块与USS结合在一起可以部分解决此问题。但是，由于US的策略和NLG是单独优化的，因此在给定的情况下，这些模拟的用户话语可能不够自然。在这项工作中，我们提出了一个基于生成变压器的用户模拟器（Gentus）。 Gentus由编码器结构组成，这意味着它可以共同优化用户策略和自然语言。 Gentus既产生语义动作又产生自然语言话语，从而保留了解释性和增强语言的变化。另外，通过将输入和输出表示为单词序列以及使用大型的预训练语言模型，我们可以在功能表示中实现普遍性。我们通过自动指标和人类评估评估绅士。我们的结果表明，绅士会产生更多的自然语言，并能够以零拍的方式转移到看不见的本体论中。此外，通过加强学习为培训专业用户模拟器打开大门，可以进一步塑造其行为。

Diverse data formats and ontologies of task-oriented dialogue (TOD) datasets hinder us from developing general dialogue models that perform well on many datasets and studying knowledge transfer between datasets. To address this issue, we present ConvLab-3, a flexible dialogue system toolkit based on a unified TOD data format. In ConvLab-3, different datasets are transformed into one unified format and loaded by models in the same way. As a result, the cost of adapting a new model or dataset is significantly reduced. Compared to the previous releases of ConvLab (Lee et al., 2019b; Zhu et al., 2020b), ConvLab-3 allows developing dialogue systems with much more datasets and enhances the utility of the reinforcement learning (RL) toolkit for dialogue policies. To showcase the use of ConvLab-3 and inspire future work, we present a comprehensive study with various settings. We show the benefit of pre-training on other datasets for few-shot fine-tuning and RL, and encourage evaluating policy with diverse user simulators.

One of the biggest challenges of natural language generation (NLG) is the proper handling of named entities. Named entities are a common source of grammar mistakes such as wrong prepositions, wrong article handling, or incorrect entity inflection. Without factoring linguistic representation, such errors are often underrepresented when evaluating on a small set of arbitrarily picked argument values, or when translating a dataset from a linguistically simpler language, like English, to a linguistically complex language, like Russian. However, for some applications, broadly precise grammatical correctness is critical -- native speakers may find entity-related grammar errors silly, jarring, or even offensive. To enable the creation of more linguistically diverse NLG datasets, we release a Corpus of Linguistically Significant Entities (CLSE) annotated by linguist experts. The corpus includes 34 languages and covers 74 different semantic types to support various applications from airline ticketing to video games. To demonstrate one possible use of CLSE, we produce an augmented version of the Schema-Guided Dialog Dataset, SGD-CLSE. Using the CLSE's entities and a small number of human translations, we create a linguistically representative NLG evaluation benchmark in three languages: French (high-resource), Marathi (low-resource), and Russian (highly inflected language). We establish quality baselines for neural, template-based, and hybrid NLG systems and discuss the strengths and weaknesses of each approach.

Training dialogue systems often entails dealing with noisy training examples and unexpected user inputs. Despite their prevalence, there currently lacks an accurate survey of dialogue noise, nor is there a clear sense of the impact of each noise type on task performance. This paper addresses this gap by first constructing a taxonomy of noise encountered by dialogue systems. In addition, we run a series of experiments to show how different models behave when subjected to varying levels of noise and types of noise. Our results reveal that models are quite robust to label errors commonly tackled by existing denoising algorithms, but that performance suffers from dialogue-specific noise. Driven by these observations, we design a data cleaning algorithm specialized for conversational settings and apply it as a proof-of-concept for targeted dialogue denoising.

将对话状态跟踪（DST）概括为新数据特别具有挑战性，因为在培训过程中对丰富和细粒度的监督非常依赖。样本稀疏性，分布转移以及新概念和主题的发生经常导致推理期间的严重降级。在本文中，我们提出了一种培训策略，以构建提取性DST模型，而无需精细颗粒的手动跨度标签。两种新型的输入级辍学方法减轻了样品稀疏性的负面影响。我们提出了一种具有统一编码器的新模型体系结构，该架构通过利用注意机制来支持价值和插槽独立性。我们结合了三复制策略DST的优势和价值匹配，以从互补的预测中受益，而无需违反本体独立性的原则。我们的实验表明，可以在没有手动跨度标签的情况下训练提取的DST模型。我们的体系结构和培训策略提高了对样本稀疏，新概念和主题的鲁棒性，从而在一系列基准中提高了最先进的表现。我们进一步强调了我们的模型有效地从非拨号数据中学习的能力。

Dialogue state tracking (DST) aims to convert the dialogue history into dialogue states which consist of slot-value pairs. As condensed structural information memorizing all history information, the dialogue state in the last turn is typically adopted as the input for predicting the current state by DST models. However, these models tend to keep the predicted slot values unchanged, which is defined as state momentum in this paper. Specifically, the models struggle to update slot values that need to be changed and correct wrongly predicted slot values in the last turn. To this end, we propose MoNET to tackle state momentum via noise-enhanced training. First, the previous state of each turn in the training data is noised via replacing some of its slot values. Then, the noised previous state is used as the input to learn to predict the current state, improving the model's ability to update and correct slot values. Furthermore, a contrastive context matching framework is designed to narrow the representation distance between a state and its corresponding noised variant, which reduces the impact of noised state and makes the model better understand the dialogue history. Experimental results on MultiWOZ datasets show that MoNET outperforms previous DST methods. Ablations and analysis verify the effectiveness of MoNET in alleviating state momentum and improving anti-noise ability.

最近，通过“向导”模拟游戏收集了一类以任务为导向的对话（TOD）数据集。但是，《巫师》数据实际上是模拟的数据，因此与现实生活中的对话根本不同，这些对话更加嘈杂和随意。最近，Seretod挑战赛是组织的，并发布了Mobilecs数据集，该数据集由来自中国移动的真实用户和客户服务人员之间的真实世界对话框组成。基于Mobilecs数据集，Seretod挑战具有两个任务，不仅评估了对话系统本身的构建，而且还检查了对话框成绩单中的信息提取，这对于建立TOD的知识库至关重要。本文主要介绍了Mobilecs数据集对这两项任务的基线研究。我们介绍了如何构建两个基线，遇到的问题以及结果。我们预计基线可以促进令人兴奋的未来研究，以建立针对现实生活任务的人类机器人对话系统。

Even though machine learning has become the major scene in dialogue research community, the real breakthrough has been blocked by the scale of data available. To address this fundamental obstacle, we introduce the Multi-Domain Wizard-of-Oz dataset (MultiWOZ), a fully-labeled collection of human-human written conversations spanning over multiple domains and topics. At a size of 10k dialogues, it is at least one order of magnitude larger than all previous annotated task-oriented corpora. The contribution of this work apart from the open-sourced dataset labelled with dialogue belief states and dialogue actions is two-fold: firstly, a detailed description of the data collection procedure along with a summary of data structure and analysis is provided. The proposed data-collection pipeline is entirely based on crowd-sourcing without the need of hiring professional annotators; secondly, a set of benchmark results of belief tracking, dialogue act and response generation is reported, which shows the usability of the data and sets a baseline for future studies.

深度学习方法已实现了越来越复杂的话语的面向任务的语义解析。但是，单个模型通常仍在分别为每个任务进行培训和部署，需要为每个任务标记培训数据，这使得支持新任务的挑战，即使在单个业务垂直方面（例如，食品订购或旅行预订）也是如此。在本文中，我们描述了交叉顶部（交叉施加任务取消解析），这是一种在给定垂直方向中复杂语义解析的零摄像方法。通过利用用户从相同的垂直共享词汇和语义相似性请求的事实，对单个跨施加性解析器进行了培训，可以在垂直行业内使用任意数量的任意任务，看不见或看不见。我们表明，跨顶部可以在以前看不见的任务上实现高精度，而无需任何其他培训数据，从而提供了一种可扩展的方法来引导新任务的语义解析器。作为这项工作的一部分，我们发布了食物界数据集，该数据集是一个面向任务的解析数据集中的垂直垂直数据集，其话语和注释来自五个架构，每个架构都来自不同的餐厅菜单。

Task-oriented dialogue (TOD) systems have been applied in a range of domains to support human users to achieve specific goals. Systems are typically constructed for a single domain or language and do not generalise well beyond this. Their extension to other languages in particular is restricted by the lack of available training data for many of the world's languages. To support work on Natural Language Understanding (NLU) in TOD across multiple languages and domains simultaneously, we constructed MULTI3NLU++, a multilingual, multi-intent, multi-domain dataset. MULTI3NLU++ extends the English-only NLU++ dataset to include manual translations into a range of high, medium and low resource languages (Spanish, Marathi, Turkish and Amharic), in two domains (banking and hotels). MULTI3NLU++ inherits the multi-intent property of NLU++, where an utterance may be labelled with multiple intents, providing a more realistic representation of a user's goals and aligning with the more complex tasks that commercial systems aim to model. We use MULTI3NLU++ to benchmark state-of-the-art multilingual language models as well as Machine Translation and Question Answering systems for the NLU task of intent detection for TOD systems in the multilingual setting. The results demonstrate the challenging nature of the dataset, particularly in the low-resource language setting.

针对任务导向的对话系统的强大状态跟踪目前仍然限于一些流行语言。本文显示，给定以一种语言设置的大规模对话数据，我们可以使用机器翻译自动为其他语言生成有效的语义解析器。我们提出了对话数据集的自动翻译，并进行对齐，以确保插槽值的忠实翻译，并消除以前的基准中使用的昂贵人类监督。我们还提出了一种新的上下文语义解析模型，它编码正式的插槽和值，只有最后一个代理和用户话语。我们表明，简洁的表示降低了翻译误差的复合效果，而不会损害实践中的准确性。我们评估我们对几个对话状态跟踪基准的方法。在Risawoz，Crosswoz，Crosswoz-Zh和Multiwoz-Zh Datasets，我们将最先进的技术提高11％，17％，20％和0.3％，以共同的目标准确度。我们为所有三个数据集提供了全面的错误分析，显示错误注释可以模糊模型质量的判断。最后，我们使用推荐方法创建了Risawoz英语和德语数据集。在这些数据集中，准确性在原始的11％以内，表示可能的高精度多语言对话数据集，而无需依赖昂贵的人类注释。

Conversation designers continue to face significant obstacles when creating production quality task-oriented dialogue systems. The complexity and cost involved in schema development and data collection is often a major barrier for such designers, limiting their ability to create natural, user-friendly experiences. We frame the classification of user intent as the generation of a canonical form, a lightweight semantic representation using natural language. We show that canonical forms offer a promising alternative to traditional methods for intent classification. By tuning soft prompts for a frozen large language model, we show that canonical forms generalize very well to new, unseen domains in a zero- or few-shot setting. The method is also sample-efficient, reducing the complexity and effort of developing new task-oriented dialogue domains.

意图分类（IC）和插槽标签（SL）模型，它形成对话系统的基础，通常会在实际字环境中遇到噪声数据。在这项工作中，我们调查了强大的IC / SL模型是如何嘈杂的数据。我们在生产人机对话中发现七种常见噪声类型（缩写，套管，拼写错误，形态变体，释义，标点符号和同义词），我们收集并公开发布测试套件。在此测试套件上，我们表明普通噪声类型显着降低了最先进的基于伯特IC / SL模型的IC精度和SL F1性能。通过利用串噪声稳健性转移 - 对一种噪声类型的培训来提高另一种噪声类型的鲁棒性 - 我们设计综合数据增强方法，以增加所有七种噪声类型的模型性能+ 10.8％的IC精度和+15平均SL F1点。据我们所知，这是第一个展示单个IC / SL模型的工作，这是一个广泛的噪声现象。

Most research on task oriented dialog modeling is based on written text input. However, users interact with practical dialog systems often using speech as input. Typically, systems convert speech into text using an Automatic Speech Recognition (ASR) system, introducing errors. Furthermore, these systems do not address the differences in written and spoken language. The research on this topic is stymied by the lack of a public corpus. Motivated by these considerations, our goal in hosting the speech-aware dialog state tracking challenge was to create a public corpus or task which can be used to investigate the performance gap between the written and spoken forms of input, develop models that could alleviate this gap, and establish whether Text-to-Speech-based (TTS) systems is a reasonable surrogate to the more-labor intensive human data collection. We created three spoken versions of the popular written-domain MultiWoz task -- (a) TTS-Verbatim: written user inputs were converted into speech waveforms using a TTS system, (b) Human-Verbatim: humans spoke the user inputs verbatim, and (c) Human-paraphrased: humans paraphrased the user inputs. Additionally, we provided different forms of ASR output to encourage wider participation from teams that may not have access to state-of-the-art ASR systems. These included ASR transcripts, word time stamps, and latent representations of the audio (audio encoder outputs). In this paper, we describe the corpus, report results from participating teams, provide preliminary analyses of their results, and summarize the current state-of-the-art in this domain.