话语重写旨在恢复核心发挥，并从最新的多转话对话中省略信息。最近，在内部和室外重写设置中，标记而不是线性生成序列的方法已被证明更强。这是由于标记器的较小搜索空间，因为它只能从对话环境中复制令牌。但是，当必须将短语添加到源语言中时，单个上下文跨度不能涵盖这些方法时，这些方法可能会遭受较低的覆盖范围。这可能会以英语等语言发生，这些语言将令牌（例如介词）引入语法重写。我们提出了一个层次上下文标记器（HCT），该标记器通过预测插槽规则（例如，“ baster_”）来减轻此问题，其插槽后来充满了上下文跨度。 HCT（i）使用令牌级的编辑操作和开槽的规则标记源字符串，（ii）填充了对话环境中的跨度的结果规则插槽。此规则标记允许HCT一次添加外在代币和多个跨度。我们进一步集中了规则，以截断规则分布的长尾巴。几个基准测试的实验表明，HCT可以比2个BLEU点胜过最先进的重写系统。

谈话问题应答需要能够正确解释问题。然而，由于在日常谈话中难以理解共同参考和省略号的难度，目前的模型仍然不令人满意。尽管生成方法取得了显着的进展，但它们仍然被语义不完整陷入困境。本文提出了一种基于动作的方法来恢复问题的完整表达。具体地，我们首先在将相应的动作分配给每个候选跨度的同时定位问题中的共同引用或省略号的位置。然后，我们寻找与对话环境中的候选线索相关的匹配短语。最后，根据预测的操作，我们决定是否用匹配的信息替换共同参考或补充省略号。我们展示了我们对英语和中文发言权重写任务的方法的有效性，在RESTORATION-200K数据集中分别在3.9 \％和Rouge-L中提高了最先进的EM（完全匹配）。

Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.

The word alignment task, despite its prominence in the era of statistical machine translation (SMT), is niche and under-explored today. In this two-part tutorial, we argue for the continued relevance for word alignment. The first part provides a historical background to word alignment as a core component of the traditional SMT pipeline. We zero-in on GIZA++, an unsupervised, statistical word aligner with surprising longevity. Jumping forward to the era of neural machine translation (NMT), we show how insights from word alignment inspired the attention mechanism fundamental to present-day NMT. The second part shifts to a survey approach. We cover neural word aligners, showing the slow but steady progress towards surpassing GIZA++ performance. Finally, we cover the present-day applications of word alignment, from cross-lingual annotation projection, to improving translation.

本文介绍了一种称为JET（\ textbf {j} oint Learning Token \ textbf {e} Xtraction和\ textbf {t} ext Exenation）的模型（IUR）。与仅在提取或抽象数据集上工作的先前研究不同，我们设计了一个简单但有效的模型，适用于IUR的两种情况。我们的设计模拟了IUR的性质，在上下文中省略了令牌有助于恢复。由此，我们构建了一个识别省略令牌的选择器。为了支持选择器，我们设计了两种标签创建方法（软标签和硬标签），它们可以在没有注释数据的情况下使用省略的令牌。恢复是通过在关节学习的帮助者的帮助下使用发电机来完成的。在提取和抽象方案中的四个基准数据集上的有希望的结果表明，我们的模型比富裕和有限的培训数据设置中的验证的T5和非生成语言模型方法。 //github.com/shumpei19/jet}}}

Much recent work in task-oriented parsing has focused on finding a middle ground between flat slots and intents, which are inexpressive but easy to annotate, and powerful representations such as the lambda calculus, which are expressive but costly to annotate. This paper continues the exploration of task-oriented parsing by introducing a new dataset for parsing pizza and drink orders, whose semantics cannot be captured by flat slots and intents. We perform an extensive evaluation of deep-learning techniques for task-oriented parsing on this dataset, including different flavors of seq2seq systems and RNNGs. The dataset comes in two main versions, one in a recently introduced utterance-level hierarchical notation that we call TOP, and one whose targets are executable representations (EXR). We demonstrate empirically that training the parser to directly generate EXR notation not only solves the problem of entity resolution in one fell swoop and overcomes a number of expressive limitations of TOP notation, but also results in significantly greater parsing accuracy.

Grammatical Error Correction (GEC) is the task of automatically detecting and correcting errors in text. The task not only includes the correction of grammatical errors, such as missing prepositions and mismatched subject-verb agreement, but also orthographic and semantic errors, such as misspellings and word choice errors respectively. The field has seen significant progress in the last decade, motivated in part by a series of five shared tasks, which drove the development of rule-based methods, statistical classifiers, statistical machine translation, and finally neural machine translation systems which represent the current dominant state of the art. In this survey paper, we condense the field into a single article and first outline some of the linguistic challenges of the task, introduce the most popular datasets that are available to researchers (for both English and other languages), and summarise the various methods and techniques that have been developed with a particular focus on artificial error generation. We next describe the many different approaches to evaluation as well as concerns surrounding metric reliability, especially in relation to subjective human judgements, before concluding with an overview of recent progress and suggestions for future work and remaining challenges. We hope that this survey will serve as comprehensive resource for researchers who are new to the field or who want to be kept apprised of recent developments.

本文对过去二十年来对自然语言生成（NLG）的研究提供了全面的审查，特别是与数据到文本生成和文本到文本生成深度学习方法有关，以及NLG的新应用技术。该调查旨在（a）给出关于NLG核心任务的最新综合，以及该领域采用的建筑;（b）详细介绍各种NLG任务和数据集，并提请注意NLG评估中的挑战，专注于不同的评估方法及其关系;（c）强调一些未来的强调和相对近期的研究问题，因为NLG和其他人工智能领域的协同作用而增加，例如计算机视觉，文本和计算创造力。

文本到SQL解析是一项必不可少且具有挑战性的任务。文本到SQL解析的目的是根据关系数据库提供的证据将自然语言（NL）问题转换为其相应的结构性查询语言（SQL）。来自数据库社区的早期文本到SQL解析系统取得了显着的进展，重度人类工程和用户与系统的互动的成本。近年来，深层神经网络通过神经生成模型显着提出了这项任务，该模型会自动学习从输入NL问题到输出SQL查询的映射功能。随后，大型的预训练的语言模型将文本到SQL解析任务的最新作品带到了一个新级别。在这项调查中，我们对文本到SQL解析的深度学习方法进行了全面的评论。首先，我们介绍了文本到SQL解析语料库，可以归类为单转和多转。其次，我们提供了预先训练的语言模型和现有文本解析方法的系统概述。第三，我们向读者展示了文本到SQL解析所面临的挑战，并探索了该领域的一些潜在未来方向。

开放信息提取是一个重要的NLP任务，它针对从非结构化文本中提取结构化信息的目标，而无需限制关系类型或文本域。该调查文件涵盖了2007年至2022年的开放信息提取技术，重点是以前的调查未涵盖的新模型。我们从信息角度来源提出了一种新的分类方法，以适应最近的OIE技术的开发。此外，我们根据任务设置以及当前流行的数据集和模型评估指标总结了三种主要方法。鉴于全面的审查，从数据集，信息来源，输出表格，方法和评估指标方面显示了几个未来的方向。

随着未来以数据为中心的决策，对数据库的无缝访问至关重要。关于创建有效的文本到SQL（Text2SQL）模型以访问数据库的数据有广泛的研究。使用自然语言是可以通过有效访问数据库（尤其是对于非技术用户）来弥合数据和结果之间差距的最佳接口之一。它将打开门，并在精通技术技能或不太熟练的查询语言的用户中引起极大的兴趣。即使提出或研究了许多基于深度学习的算法，在现实工作场景中使用自然语言来解决数据查询问题仍然非常具有挑战性。原因是在不同的研究中使用不同的数据集，这带来了其局限性和假设。同时，我们确实缺乏对这些提议的模型及其对其训练的特定数据集的局限性的彻底理解。在本文中，我们试图介绍过去几年研究的24种神经网络模型的整体概述，包括其涉及卷积神经网络，经常性神经网络，指针网络，强化学习，生成模型等的架构。我们还概述11个数据集，这些数据集被广泛用于训练Text2SQL技术的模型。我们还讨论了无缝数据查询中文本2SQL技术的未来应用可能性。

We present SpanBERT, a pre-training method that is designed to better represent and predict spans of text. Our approach extends BERT by (1) masking contiguous random spans, rather than random tokens, and (2) training the span boundary representations to predict the entire content of the masked span, without relying on the individual token representations within it. Span-BERT consistently outperforms BERT and our better-tuned baselines, with substantial gains on span selection tasks such as question answering and coreference resolution. In particular, with the same training data and model size as BERT large , our single model obtains 94.6% and 88.7% F1 on SQuAD 1.1 and 2.0 respectively. We also achieve a new state of the art on the OntoNotes coreference resolution task (79.6% F1), strong performance on the TACRED relation extraction benchmark, and even gains on GLUE. 1 * Equal contribution. 1 Our code and pre-trained models are available at https://github.com/facebookresearch/ SpanBERT.

针对任务导向的对话系统的强大状态跟踪目前仍然限于一些流行语言。本文显示，给定以一种语言设置的大规模对话数据，我们可以使用机器翻译自动为其他语言生成有效的语义解析器。我们提出了对话数据集的自动翻译，并进行对齐，以确保插槽值的忠实翻译，并消除以前的基准中使用的昂贵人类监督。我们还提出了一种新的上下文语义解析模型，它编码正式的插槽和值，只有最后一个代理和用户话语。我们表明，简洁的表示降低了翻译误差的复合效果，而不会损害实践中的准确性。我们评估我们对几个对话状态跟踪基准的方法。在Risawoz，Crosswoz，Crosswoz-Zh和Multiwoz-Zh Datasets，我们将最先进的技术提高11％，17％，20％和0.3％，以共同的目标准确度。我们为所有三个数据集提供了全面的错误分析，显示错误注释可以模糊模型质量的判断。最后，我们使用推荐方法创建了Risawoz英语和德语数据集。在这些数据集中，准确性在原始的11％以内，表示可能的高精度多语言对话数据集，而无需依赖昂贵的人类注释。

深度神经语言模型的最新进展与大规模数据集的能力相结合，加速了自然语言生成系统的发展，这些系统在多种任务和应用程序上下文中产生流利和连贯的文本（在各种成功程度上）。但是，为所需的用户控制这些模型的输出仍然是一个开放的挑战。这不仅对于自定义生成语言的内容和样式至关重要，而且对于他们在现实世界中的安全可靠部署至关重要。我们提出了一项关于受约束神经语言生成的新兴主题的广泛调查，在该主题中，我们通过区分条件和约束（后者是在输出文本上而不是输入的可检验条件），正式定义和分类自然语言生成问题，目前是可检验的）约束文本生成任务，并查看受限文本生成的现有方法和评估指标。我们的目的是强调这个新兴领域的最新进展和趋势，以告知最有希望的方向和局限性，以推动受约束神经语言生成研究的最新作品。

许多自然语言处理任务，例如核心解决方案和语义角色标签，都需要选择文本跨度并就其做出决定。此类任务的典型方法是为所有可能的跨度评分，并贪婪地选择特定任务的下游处理的跨度。然而，这种方法并未纳入有关应选择哪种跨度的诱导偏见，例如，选定的跨度倾向于是句法成分。在本文中，我们提出了一种新型的基于语法的结构化选择模型，该模型学会了利用为此类问题提供的部分跨度注释。与以前的方法相比，我们的方法摆脱了启发式贪婪的跨度选择方案，使我们能够在一组最佳跨度上对下游任务进行建模。我们在两个流行的跨度预测任务上评估我们的模型：核心分辨率和语义角色标签。我们对两者都展示了经验改进。

In order to achieve deep natural language understanding, syntactic constituent parsing is a vital step, highly demanded by many artificial intelligence systems to process both text and speech. One of the most recent proposals is the use of standard sequence-to-sequence models to perform constituent parsing as a machine translation task, instead of applying task-specific parsers. While they show a competitive performance, these text-to-parse transducers are still lagging behind classic techniques in terms of accuracy, coverage and speed. To close the gap, we here extend the framework of sequence-to-sequence models for constituent parsing, not only by providing a more powerful neural architecture for improving their performance, but also by enlarging their coverage to handle the most complex syntactic phenomena: discontinuous structures. To that end, we design several novel linearizations that can fully produce discontinuities and, for the first time, we test a sequence-to-sequence model on the main discontinuous benchmarks, obtaining competitive results on par with task-specific discontinuous constituent parsers and achieving state-of-the-art scores on the (discontinuous) English Penn Treebank.

将对话状态跟踪（DST）概括为新数据特别具有挑战性，因为在培训过程中对丰富和细粒度的监督非常依赖。样本稀疏性，分布转移以及新概念和主题的发生经常导致推理期间的严重降级。在本文中，我们提出了一种培训策略，以构建提取性DST模型，而无需精细颗粒的手动跨度标签。两种新型的输入级辍学方法减轻了样品稀疏性的负面影响。我们提出了一种具有统一编码器的新模型体系结构，该架构通过利用注意机制来支持价值和插槽独立性。我们结合了三复制策略DST的优势和价值匹配，以从互补的预测中受益，而无需违反本体独立性的原则。我们的实验表明，可以在没有手动跨度标签的情况下训练提取的DST模型。我们的体系结构和培训策略提高了对样本稀疏，新概念和主题的鲁棒性，从而在一系列基准中提高了最先进的表现。我们进一步强调了我们的模型有效地从非拨号数据中学习的能力。

这项工作提出了一个新的对话数据集，即cookdial，该数据集促进了对任务知识了解的面向任务的对话系统的研究。该语料库包含260个以人类对任务为导向的对话框，其中代理给出了配方文档，指导用户烹饪菜肴。 Cookdial中的对话框展示了两个独特的功能：（i）对话流与支持文档之间的程序对齐； （ii）复杂的代理决策涉及分割长句子，解释硬说明并在对话框上下文中解决核心。此外，我们在假定的面向任务的对话框系统中确定了三个具有挑战性的（子）任务：（1）用户问题理解，（2）代理操作框架预测和（3）代理响应生成。对于这些任务中的每一个，我们都会开发一个神经基线模型，我们在cookdial数据集上进行了评估。我们公开发布烹饪数据集，包括对话框和食谱文档的丰富注释，以刺激对特定于域的文档接地对话框系统的进一步研究。

开放信息提取（OpenIE）促进了独立于域的大型语料库的关系事实的发现。该技术很好地适合许多开放世界的自然语言理解场景，例如自动知识基础构建，开放域问答和明确的推理。由于深度学习技术的快速发展，已经提出了许多神经开放式体系结构并取得了可观的性能。在这项调查中，我们提供了有关状态神经开放模型的广泛概述，其关键设计决策，优势和劣势。然后，我们讨论当前解决方案的局限性以及OpenIE问题本身的开放问题。最后，我们列出了最近的趋势，这些趋势可以帮助扩大其范围和适用性，从而为Openie的未来研究设定了有希望的方向。据我们所知，本文是有关此特定主题的第一篇评论。

We propose a transition-based approach that, by training a single model, can efficiently parse any input sentence with both constituent and dependency trees, supporting both continuous/projective and discontinuous/non-projective syntactic structures. To that end, we develop a Pointer Network architecture with two separate task-specific decoders and a common encoder, and follow a multitask learning strategy to jointly train them. The resulting quadratic system, not only becomes the first parser that can jointly produce both unrestricted constituent and dependency trees from a single model, but also proves that both syntactic formalisms can benefit from each other during training, achieving state-of-the-art accuracies in several widely-used benchmarks such as the continuous English and Chinese Penn Treebanks, as well as the discontinuous German NEGRA and TIGER datasets.