Abstractive dialogue summarization has received increasing attention recently. Despite the fact that most of the current dialogue summarization systems are trained to maximize the likelihood of human-written summaries and have achieved significant results, there is still a huge gap in generating high-quality summaries as determined by humans, such as coherence and faithfulness, partly due to the misalignment in maximizing a single human-written summary. To this end, we propose to incorporate different levels of human feedback into the training process. This will enable us to guide the models to capture the behaviors humans care about for summaries. Specifically, we ask humans to highlight the salient information to be included in summaries to provide the local feedback , and to make overall comparisons among summaries in terms of coherence, accuracy, coverage, concise and overall quality, as the global feedback. We then combine both local and global feedback to fine-tune the dialog summarization policy with Reinforcement Learning. Experiments conducted on multiple datasets demonstrate the effectiveness and generalization of our methods over the state-of-the-art supervised baselines, especially in terms of human judgments.

生成摘要中的事实不一致严重限制了抽象对话摘要的实际应用。尽管通过使用预先训练的模型实现了显着进展，但在人类评估期间发现了大量的幻觉含量。预先接受的模型最常见的是微调文本摘要的跨熵损失，这可能不是最佳策略。在这项工作中，我们为带注释数据提供了事实错误的类型，以突出显示错误的类型并远离对事实的二进制了解。我们进一步提出了一种培训策略，通过新颖的对比微调，改善了摘要的事实一致性和整体素质。基于我们的语言信息的错误类型，我们设计了各个目标的不同模块化目标。具体而言，我们利用硬阴性样本具有误差，以减少事实不一致的产生。为了捕获扬声器之间的关键信息，我们还设计了特定于对话的损失。使用人类评估和自动忠实度量指标，我们表明我们的模型在对话摘要，Samsum语料库中大大降低了各种事实错误。此外，我们的模型可以推广到会议概述，AMI语料库，它产生的分数明显高于两个数据集关于单词 - 重叠度量标准的基线。

我们报告了Dialogsum挑战的结果，即在INLG 2022上汇总现实生活中的对话的共同任务。四个团队参与了这项共享任务，并提交了他们的系统报告，探索了不同的方法来提高对话摘要的性能。尽管对于自动评估指标（例如Rouge分数），基线模型有很大的改进，但我们发现模型生成的输出与通过多个方面的人类评估之间的人类评估之间存在显着差距。这些发现表明了对话摘要的困难，并表明需要更细粒度的评估指标。

输出长度对于对话摘要系统至关重要。对话摘要长度由多个因素决定，包括对话复杂性，摘要目标和个人偏好。在这项工作中，我们从三个角度来对话摘要长度。首先，我们分析了现有模型的输出与相应的人类参考之间的长度差异，并发现摘要模型由于其预训练的目标而倾向于产生更多的详细摘要。其次，我们通过比较不同的模型设置来确定摘要长度预测的显着特征。第三，我们尝试使用长度意识的摘要，并在现有模型上显示出显着改进，如果汇总长度可以很好地整合。分析和实验是在流行的对话和Samsum数据集中进行的，以验证我们的发现。

Human evaluation is the foundation upon which the evaluation of both summarization systems and automatic metrics rests. However, existing human evaluation protocols and benchmarks for summarization either exhibit low inter-annotator agreement or lack the scale needed to draw statistically significant conclusions, and an in-depth analysis of human evaluation is lacking. In this work, we address the shortcomings of existing summarization evaluation along the following axes: 1) We propose a modified summarization salience protocol, Atomic Content Units (ACUs), which relies on fine-grained semantic units and allows for high inter-annotator agreement. 2) We curate the Robust Summarization Evaluation (RoSE) benchmark, a large human evaluation dataset consisting of over 22k summary-level annotations over state-of-the-art systems on three datasets. 3) We compare our ACU protocol with three other human evaluation protocols, underscoring potential confounding factors in evaluation setups. 4) We evaluate existing automatic metrics using the collected human annotations across evaluation protocols and demonstrate how our benchmark leads to more statistically stable and significant results. Furthermore, our findings have important implications for evaluating large language models (LLMs), as we show that LLMs adjusted by human feedback (e.g., GPT-3.5) may overfit unconstrained human evaluation, which is affected by the annotators' prior, input-agnostic preferences, calling for more robust, targeted evaluation methods.

在本文中，我们建议利用对话的独特特征，共享参与者的常识性知识，以解决总结它们的困难。我们提出了病态的框架，该框架使用常识推论作为其他背景。与以前仅依赖于输入对话的工作相比，Sick使用外部知识模型来生成丰富的常识推断，并选择具有基于相似性选择方法的最可能的推理。基于生病的，病人++的理解为监督，在总结多任务学习环境中的对话时，添加了产生常识推断的任务。实验结果表明，通过注入常识性知识，我们的框架比现有方法产生更多信息和一致的摘要。

在典型的客户服务聊天方案中，客户联系支持中心以便帮助或提高投诉，人类代理商试图解决这些问题。在大多数情况下，在谈话结束时，要求代理人写一份简短的总结强调问题和建议的解决方案，通常是为了使其他可能需要处理同一客户或问题的其他代理商的利益。本文的目标是推进此任务的自动化。我们介绍了第一个大规模，高质量的客户服务对话框摘要数据集，接近6500人的注释摘要。数据基于现实世界的客户支持对话框，包括提取和抽象摘要。我们还介绍了一种特定于对话框的新无监督的提取摘要方法。

Current state-of-the-art summarization models are trained with either maximum likelihood estimation (MLE) or reinforcement learning (RL). In this study, we investigate the third training paradigm and argue that inverse reinforcement learning (IRL) may be more suitable for text summarization. IRL focuses on estimating the reward function of an agent, given a set of observations of that agent's behavior. Generally, IRL provides advantages in situations where the reward function is not explicitly known or where it is difficult to define or interact with the environment directly. These situations are exactly what we observe in summarization. Thus, we introduce inverse reinforcement learning into text summarization and define a suite of sub-rewards that are important for summarization optimization. By simultaneously estimating the reward function and optimizing the summarization agent with expert demonstrations, we show that the model trained with IRL produces summaries that closely follow human behavior, in terms of better ROUGE, coverage, novelty, compression ratio and factuality when compared to the baselines trained with MLE and RL.

有几种原因，抽象对话摘要是一项有挑战性的任务。首先，谈话中的大多数重要信息通过与不同纹理样式的多方交互来跨越话语。其次，对话通常是非正式结构，其中不同的个人表达个人观点，与文本摘要不同，通常针对新闻文章等正式文件的任务。为解决这些问题，我们专注于来自各个扬声器和独特的句法结构之间的话语之间的关联。扬声器具有唯一的文本方式，可以包含语言信息，例如声音。因此，我们通过利用语言信息（即POS标记）来构建语法感知模型，这通过自然区分从各个扬声器发出的句子来减轻上述问题。我们采用了多任务学习的语法感知信息和对话摘要。据我们所知，我们的方法是第一种将多任务学习应用于对话摘要任务的方法。 Samsum语料库（大规模对话摘要语料库）的实验表明，我们的方法改善了香草模型。我们进一步分析了我们对基线模型的方法的成本和益处。

Controllable summarization allows users to generate customized summaries with specified attributes. However, due to the lack of designated annotations of controlled summaries, existing works have to craft pseudo datasets by adapting generic summarization benchmarks. Furthermore, most research focuses on controlling single attributes individually (e.g., a short summary or a highly abstractive summary) rather than controlling a mix of attributes together (e.g., a short and highly abstractive summary). In this paper, we propose MACSum, the first human-annotated summarization dataset for controlling mixed attributes. It contains source texts from two domains, news articles and dialogues, with human-annotated summaries controlled by five designed attributes (Length, Extractiveness, Specificity, Topic, and Speaker). We propose two simple and effective parameter-efficient approaches for the new task of mixed controllable summarization based on hard prompt tuning and soft prefix tuning. Results and analysis demonstrate that hard prompt models yield the best performance on all metrics and human evaluations. However, mixed-attribute control is still challenging for summarization tasks. Our dataset and code are available at https://github.com/psunlpgroup/MACSum.

对话是人类沟通与合作的重要组成部分。现有研究主要关注一对一时尚的短对话情景。然而，现实世界中的多人互动，例如会议或访谈，经常超过几千个字。仍然缺乏相应的研究和强大的工具来了解和处理这么长的对话。因此，在这项工作中，我们为长时间对话理解和总结提供了预先培训框架。考虑到长期交谈的性质，我们提出了一种基于窗口的去噪方法，用于生成预训练。对于对话框，它损坏了一个带有对话激发灵感噪声的文本窗口，并指导模型基于剩余对话的内容来重建此窗口。此外，为了更长的输入，我们增加了稀疏关注模型，这些模型以混合方式与传统的关注相结合。我们在长对话的五个数据集进行广泛的实验，涵盖对话摘要的任务，抽象问题回答和主题分割。实验，我们表明，我们的预先训练的模型DialogLM显着超越了数据集和任务的最先进的模型。我们的GitHub存储库（HTTPS:/github.com/microsoft/dialoglm上有源代码和所有预先训练的型号。

深度学习的最新进展，尤其是编码器架构的发明，已大大改善了抽象性摘要系统的性能。尽管大多数研究都集中在书面文件上，但我们观察到过去几年对对话和多方对话的总结越来越兴趣。一个可以可靠地将人类对话的音频或笔录转换为删节版本的系统，该版本在讨论中最重要的一点上可以在各种现实世界中，从商务会议到医疗咨询再到客户都有价值服务电话。本文着重于多党会议的抽象性摘要，对与此任务相关的挑战，数据集和系统进行了调查，并讨论了未来研究的有希望的方向。

Information overloading requires the need for summarizers to extract salient information from the text. Currently, there is an overload of dialogue data due to the rise of virtual communication platforms. The rise of Covid-19 has led people to rely on online communication platforms like Zoom, Slack, Microsoft Teams, Discord, etc. to conduct their company meetings. Instead of going through the entire meeting transcripts, people can use meeting summarizers to select useful data. Nevertheless, there is a lack of comprehensive surveys in the field of meeting summarizers. In this survey, we aim to cover recent meeting summarization techniques. Our survey offers a general overview of text summarization along with datasets and evaluation metrics for meeting summarization. We also provide the performance of each summarizer on a leaderboard. We conclude our survey with different challenges in this domain and potential research opportunities for future researchers.

Dialogue summarization has recently garnered significant attention due to its wide range of applications. However, existing methods for summarizing dialogues are suboptimal because they do not take into account the inherent structure of dialogue and rely heavily on labeled data, which can lead to poor performance in new domains. In this work, we propose DIONYSUS (dynamic input optimization in pre-training for dialogue summarization), a pre-trained encoder-decoder model for summarizing dialogues in any new domain. To pre-train DIONYSUS, we create two pseudo summaries for each dialogue example: one is produced by a fine-tuned summarization model, and the other is a collection of dialogue turns that convey important information. We then choose one of these pseudo summaries based on the difference in information distribution across different types of dialogues. This selected pseudo summary serves as the objective for pre-training DIONYSUS using a self-supervised approach on a large dialogue corpus. Our experiments show that DIONYSUS outperforms existing methods on six datasets, as demonstrated by its ROUGE scores in zero-shot and few-shot settings.

社区问题应答（CQA）FORA，如堆栈溢出和雅虎！答案包含丰富的资源，对广泛的基于社区的问题答案。每个问题线程都可以通过不同的角度接收大量答案。答案摘要的一个目标是产生反映答案视角范围的摘要。抽象答案概述的主要障碍是没有数据集，可以提供监督制作这些摘要。最近的作品提出了创建此类数据的启发式，但这些是嘈杂的，并且不会涵盖答案中存在的所有观点。这项工作介绍了4,631个CQA线程的新型数据集，用于答案摘要，由专业语言学家策划。我们的管道收集了答案概述所涉及的所有子特设的注释，包括选择与问题相关的答案句子，根据透视图对这些句子进行分组，总结每个视角，并生成整体摘要。我们在这些子组织上分析和基准最先进的模型，并为多视角数据增强引入了一种新的无监督方法，这进一步提高了根据自动评估的整体摘要性能。最后，我们提出了加强学习奖励，以改善事实一致性和答案覆盖范围和分析改进领域。

Narrative summarization aims to produce a distilled version of a narrative to describe its most salient events and characters. Summarizing a narrative is challenging as it requires an understanding of event causality and character behaviors. To encourage research in this direction, we propose NarraSum, a large-scale narrative summarization dataset. It contains 122K narrative documents, which are collected from plot descriptions of movies and TV episodes with diverse genres, and their corresponding abstractive summaries. Experiments show that there is a large performance gap between humans and the state-of-the-art summarization models on NarraSum. We hope that this dataset will promote future research in summarization, as well as broader studies of natural language understanding and generation. The dataset is available at https://github.com/zhaochaocs/narrasum.

对比学习模型在无监督的视觉表示学习中取得了巨大成功，这使得相同图像的不同视图的特征表示之间的相似性最大化，同时最小化不同图像的视图的特征表示之间的相似性。在文本摘要中，输出摘要是输入文档的较短形式，它们具有类似的含义。在本文中，我们提出了对监督抽象文本摘要的对比学习模型，在那里我们查看文档，它的金摘要及其模型生成的摘要，与相同的平均表示的不同视图，并在培训期间最大化它们之间的相似性。我们在三个不同的摘要数据集上改进了一个强序列到序列文本生成模型（即，BART）。人类评估还表明，与其对应物相比，我们的模型达到了更好的忠实性评级，没有对比的目标。

诸如学术文章和商业报告之类的长期文件一直是详细说明重要问题和需要额外关注的复杂主题的标准格式。自动汇总系统可以有效地将长文档置于简短而简洁的文本中，以封装最重要的信息，从而在帮助读者的理解中很重要。最近，随着神经体系结构的出现，已经做出了重大的研究工作，以推动自动文本摘要系统，以及有关将这些系统扩展到长期文档领域的挑战的大量研究。在这项调查中，我们提供了有关长期文档摘要的研究的全面概述，以及其研究环境的三个主要组成部分的系统评估：基准数据集，汇总模型和评估指标。对于每个组成部分，我们在长期汇总的背景下组织文献，并进行经验分析，以扩大有关当前研究进度的观点。实证分析包括一项研究基准数据集的内在特征，摘要模型的多维分析以及摘要评估指标的综述。根据总体发现，我们通过提出可能在这个快速增长的领域中提出未来探索的方向来得出结论。

Text summarization is a user-preference based task, i.e., for one document, users often have different priorities for summary. As a key aspect of customization in summarization, granularity is used to measure the semantic coverage between the summary and source document. However, developing systems that can generate summaries with customizable semantic coverage is still an under-explored topic. In this paper, we propose the first unsupervised multi-granularity summarization framework, GranuSum. We take events as the basic semantic units of the source documents and propose to rank these events by their salience. We also develop a model to summarize input documents with given events as anchors and hints. By inputting different numbers of events, GranuSum is capable of producing multi-granular summaries in an unsupervised manner. Meanwhile, we annotate a new benchmark GranuDUC that contains multiple summaries at different granularities for each document cluster. Experimental results confirm the substantial superiority of GranuSum on multi-granularity summarization over strong baselines. Further, by exploiting the event information, GranuSum also exhibits state-of-the-art performance under the conventional unsupervised abstractive setting. Dataset for this paper can be found at: https://github.com/maszhongming/GranuDUC

最先进的抽象摘要系统经常生成\ emph {幻觉};即，不直接从源文本中推断的内容。尽管被认为是不正确的，我们发现非常令人难潮的内容是事实，即与世界知识一致。这些事实幻觉通过提供有用的背景信息，可以在摘要中受益。在这项工作中，我们提出了一种新的检测方法，将事实与实体的非事实幻觉分开。我们的方法分别使用实体的先前和后验概率，分别是预训练和芬特的屏蔽语言模型。经验结果表明，我们的方法在精度和F1分数方面大大优于两种基线％，与人类判断强烈相关。百分比对事实分类任务。此外，我们显示我们的探测器，当用作离线增强学习（RL）算法中的奖励信号时，显着提高了摘要的事实性，同时保持抽象水平。