The past decade has witnessed the boom of human-machine interactions, particularly via dialog systems. In this paper, we study the task of response generation in open-domain multi-turn dialog systems. Many research efforts have been dedicated to building intelligent dialog systems, yet few shed light on deepening or widening the chatting topics in a conversational session, which would attract users to talk more. To this end, this paper presents a novel deep scheme consisting of three channels, namely global, wide, and deep ones. The global channel encodes the complete historical information within the given context, the wide one employs an attention-based recurrent neural network model to predict the keywords that may not appear in the historical context, and the deep one trains a Multi-layer Perceptron model to select some keywords for an in-depth discussion. Thereafter, our scheme integrates the outputs of these three channels to generate desired responses. To justify our model, we conducted extensive experiments to compare our model with several state-of-the-art baselines on two datasets: one is constructed by ourselves and the other is a public benchmark dataset. Experimental results demonstrate that our model yields promising performance by widening or deepening the topics of interest.
translated by 谷歌翻译
Attention mechanism has become a popular and widely used component in sequence-to-sequence models. However, previous research on neural gen-erative dialogue systems always generates universal responses, and the attention distribution learned by the model always attends to the same semantic aspect. To solve this problem, in this paper, we propose a novel Multi-Head Attention Mechanism (MHAM) for generative dialog systems, which aims at capturing multiple semantic aspects from the user utterance. Further, a regularizer is formulated to force different attention heads to concentrate on certain aspects. The proposed mechanism leads to more informative, diverse, and relevant response generated. Experimental results show that our proposed model outperforms several strong baselines.
translated by 谷歌翻译
While recent neural encoder-decoder models have shown great promise in mod-eling open-domain conversations, they often generate dull and generic responses. Unlike past work that has focused on diversifying the output of the decoder at word-level to alleviate this problem, we present a novel framework based on conditional variational autoencoders that captures the discourse-level diversity in the encoder. Our model uses latent variables to learn a distribution over potential conversational intents and generates diverse responses using only greedy de-coders. We have further developed a novel variant that is integrated with linguistic prior knowledge for better performance. Finally, the training procedure is improved by introducing a bag-of-word loss. Our proposed models have been validated to generate significantly more diverse responses than baseline approaches and exhibit competence in discourse-level decision-making. 1
translated by 谷歌翻译
开放域响应生成在近几年取得了显着进展,但有时会产生短暂且无信息的响应。我们提出了一种新的响应生成范例,即通过编辑生成响应,这显着增加了一代人的多样性和信息量。我们的假设是通过轻微修改现有的响应原型可以生成合理的响应。原型是从预定义的索引中检索出来的,并为生成提供了一个良好的起点,因为它具有语法性和信息性。我们设计了响应编辑模型,其中通过考虑原型上下文与当前上下文之间的差异来形成anedit向量,然后将编辑向量馈送到解码器以重新审视当前上下文的原型响应。大规模数据集上的实验结果表明,响应编辑模型在各个方面优于生成和基于检索的模型。
translated by 谷歌翻译
由于大量会话数据的可用性以及针对会话AI的神经方法的进展,开发智能开放域对话系统的兴趣日益浓厚。与传统的面向任务的机器人不同,开放域对话系统旨在通过满足人类对沟通,感情和社会归属的需求来与用户建立长期联系。本文回顾了最近关于neuralapproaches的工作,这些工作致力于解决开发此类系统的三个挑战:语义,一致性和交互性。语义学要求adialog系统不仅要理解对话框的内容,还要在对话过程中识别用户的社交需求。一致性要求系统展示一致的个性以赢得用户的信任和gaintheir的长期信心。互动性是指系统产生人际反应以实现特定社会目标的能力,如娱乐,整合和任务完成。我们选择的作品基于我们独特的观点,并不完整。尽管如此,我们希望这次讨论将激发新的研究,以开发更智能的对话系统。
translated by 谷歌翻译
在这项工作中,我们提出了一种神经对话响应生成方法,它不仅可以根据对话历史生成语义上合理的响应,还可以通过情感标签明确控制响应的情绪。我们提出的模型基于有条件的对抗性学习的范式;对情绪控制对话发生器的训练由对抗性鉴别器辅助,该鉴别器评估对话历史和给定情绪标签产生的反应的流畅性和可行性。由于我们的框架的灵活性,生成器可以是标准的序列到序列(SEQ2SEQ)模型或更复杂的模型,例如基于条件变异的基于编码器的SEQ2SEQ模型。使用自动和人道评估的实验结果都表明我们提出的框架能够产生语义上合理和情感控制的对话响应。
translated by 谷歌翻译
在过去的几年中,序列到序列(seq2seq)模型的神经抽象文本摘要已经获得了很多的普及。已经提出了许多有趣的技术来改进seq2seq模型,使得它们能够处理不同的挑战,例如显着性,流畅性和人类可读性,并生成高质量的摘要。一般而言,这些技术中的大多数在以下三个类别之一中不同:网络结构,参数推断和解码/生成。还有其他问题,例如培训模型的效率和并行性。在本文中,我们从网络结构,训练策略和摘要生成算法的角度提供了关于不同seq2seq模型的综合文献和技术调查,用于抽象文本摘要。许多模型首先被提出用于语言建模和生成任务,例如机器翻译,然后应用于抽象文本摘要。因此,我们还对这些模型进行了简要回顾。作为本次调查的一部分,我们还开发了一个开源库,即神经抽象文本摘要器(NATS)工具包,用于抽象文本摘要。在广泛使用的CNN /每日邮件数据集上进行了大量的实验,以检验几种不同神经网络组件的有效性。最后,我们在两个最近发布的数据集(即Newsroom和Bytecup)上对在NATS中实现的两个模型进行了基准测试。
translated by 谷歌翻译
我们针对开放域会话代理的现有编码器 - 解码器模型提出了三种增强,旨在有效地建模一致性和促进输出多样性:(1)我们引入一种一致性度量作为对话上下文与生成的响应之间的嵌入相似性,(2)我们根据相干性度量过滤我们的训练语料库,以获得局部相干和词汇多样化的上下文 - 响应对,(3)然后我们使用条件变量自动编码器模型训练响应生成器,该模型将相干性度量作为潜在变量并使用上下文门来保证与背景的主题一致性和促进双重多样性。在OpenSubtitles语料库上的实验表明,在BLEU评分以及连贯性和多样性的指标方面,竞争神经模型得到了实质性的改进。
translated by 谷歌翻译
构建开放式多圈对话系统是人工智能中最有趣和最具挑战性的任务之一。许多研究人员一直致力于建立这样的对话系统,但很少有人在正在进行的对话中对会话流进行建模。此外,人们在谈话中谈论高度相关的方面是常见的。主题是连贯的,自然漂移的,这表明了对话流建模的必要性。为此,我们提出了具有强化学习方法(RLCw)的多转换词驱动的会话系统,该方法努力选择具有最大未来信用的自适应提示词,从而提高生成的响应的质量。我们引入了一个新的方法来衡量提示词在有效性和相关性方面的质量。为了进一步优化长期对话模型,本文采用了强化方法。在real-realedataset上的实验表明,我们的模型在模拟转弯,多样性和人道评估方面始终优于一组竞争基线。
translated by 谷歌翻译
We propose Neural Responding Machine (NRM), a neural network-based response generator for Short-Text Conversation. NRM takes the general encoder-decoder framework: it formalizes the generation of response as a decoding process based on the latent representation of the input text, while both encoding and decoding are realized with recurrent neural networks (RNN). The NRM is trained with a large amount of one-round conversation data collected from a microblogging service. Empirical study shows that NRM can generate grammatically correct and content-wise appropriate responses to over 75% of the input text, outperforming state-of-the-arts in the same setting, including retrieval-based and SMT-based models.
translated by 谷歌翻译
具有基于文本或基于语音的对话界面的智能个人助理系统正变得越来越流行。大多数先前的研究使用基于检索或基于生成的方法。基于检索的方法具有返回流畅和信息性响应的优点,具有很大的多样性。检索到的响应更容易解释。但是,响应检索性能受响应存储库大小的限制。另一方面,尽管基于生成的方法可以在给定对话上下文的情况下返回高度一致的响应,但是它们可能返回具有不足的地面知识信息的普遍或一般响应。在本文中,我们构建了一个具有响应检索和生成能力的混合神经对话模型,并结合了这两种方法的优点。关于Twitter和Foursquare数据的实验结果表明,在自动评估指标和人工评估下,所提出的模型可以优于基于检索的方法和基于生成的方法(包括最近提出的知识接地神经对话模型)。我们的模型和研究发现提供了关于如何集成文本检索和文本生成模型以构建会话系统的新见解。
translated by 谷歌翻译
We propose simple and flexible training and decoding methods for influencing output style and topic in neural encoder-decoder based language generation. This capability is desirable in a variety of applications , including conversational systems, where successful agents need to produce language in a specific style and generate responses steered by a human puppeteer or external knowledge. We decompose the neural generation process into empirically easier sub-problems: a faithfulness model and a decoding method based on selective-sampling. We also describe training and sampling algorithms that bias the generation process with a specific language style restriction, or a topic restriction. Human evaluation results show that our proposed methods are able to restrict style and topic without degrading output quality in conversational tasks.
translated by 谷歌翻译
本文提出了一种新的模型,称为条件变换变异编码器(CTVAE),以提高使用条件变分自动编码器(CVAE)的会话响应生成的性能。在常规的CVA中,潜变量z的先验分布遵循多变量高斯分布,其中均值和方差由输入条件调制。以前的工作发现这种分布在实际应用中趋于变得不依赖于条件。在我们提出的CTVAE模型中,通过对输入条件和来自独立于独立的先验分布N(0; I)的样本的组合执行非线性变换来对潜在变量z进行采样。在我们的客观评估中,CTVAE模型在流畅性指标上优于CVAE模型,并且在多样性指标上超越了序列到序列(Seq2Seq)模型。在主观偏好测试中,我们提出的CTVAE模型在产生流畅性,信息性和主题相关反应方面比CVAE和Seq2Seq模型表现得更好。
translated by 谷歌翻译
Generating emotional language is a key step towards building empathetic natural language processing agents. However, a major challenge for this line of research is the lack of large-scale labeled training data, and previous studies are limited to only small sets of human annotated sentiment labels. Additionally, explicitly controlling the emotion and sentiment of generated text is also difficult. In this paper, we take a more radical approach: we exploit the idea of leveraging Twitter data that are naturally labeled with emojis. We collect a large corpus of Twitter conversations that include emojis in the response and assume the emojis convey the underlying emotions of the sentence. We investigate several conditional variational autoencoders training on these conversations , which allow us to use emojis to control the emotion of the generated text. Experimentally , we show in our quantitative and qualitative analyses that the proposed models can successfully generate high-quality abstractive conversation responses in accordance with designated emotions.
translated by 谷歌翻译
我们研究开放式域名对话生成与对话行为,旨在解释人们如何参与社交聊天。为了模仿人类的行为,我们建议管理人机交互的流程与对话行为的政策。政策和响应生成是从人 - 人对话中共同学习的,前者通过强化学习方法进一步优化。通过对话行为,我们在机器仿真和人机对话中对给定的对象和对话长度的响应质量的最新方法实现了显着的改进。
translated by 谷歌翻译
搜索通常依赖于关键字查询,但这些查询通常是语义上的。我们建议通过为用户提供基于关键字查询的自然语言问题来消除这一点,以消除他们的意图歧义。可以使用神经机器翻译技术来解决该关键词到任务的任务。然而,神经翻译模型需要大量的训练数据(关键词 - 问题对),这对于此任务是不可用的。本文的主要思想是从一小组手工标记的关键词 - 问题对中生成大量的合成训练数据。由于天然语言问题可以大量提供,我们开发模型来自动生成相应的关键字查询。此外,我们引入了各种过滤机制,以确保合成训练数据具有高质量。我们使用自动和手动评估证明了我们方法的可行性。这是在ICTIR'18的会议录中以相同标题发表的文章的扩展版本。
translated by 谷歌翻译
多样性在许多文本生成应用程序中起着至关重近年来,条件变分自动编码器(CVAE)已经显示出有希望完成此任务的性能。然而,他们经常遇到所谓的“消失”问题。先前的工作通过诸如加强编码器或弱化解码器等启发式方法来减轻这种问题,同时优化CVAE目标函数。然而,这些方法的优化方向是隐含的,很难找到适当的程度,应该应用这些方法。在本文中,我们提出了一个明确的优化目标,以补充CVAE,直接从KL消失。实际上,该目标术语指导编码器朝向解码器的“bestencoder”以增强表现力。引入标记网络来估计“最佳编码器”。它在CVAE的潜在空间中提供连续标签,以帮助在潜变量和目标之间建立紧密连接。整个提出的方法被命名为Self LabelingCVAE~(SLCVAE)。为了加速不同文本生成的研究,我们还提出了一个大的本地一对多数据集。在两个任务中进行了广泛的实验,这表明我们的方法在很大程度上改善了生成多样性,同时与现有技术相比实现了可比较的精度。
translated by 谷歌翻译
在本文中,我们探讨了深度神经网络在自然语言生成中的应用。具体来说,我们实现了两个序列到序列的神经变分模型 - 变分自动编码器(VAE)和变量编码器 - 解码器(VED)。用于文本生成的VAE难以训练,因为与损失函数的Kullback-Leibler(KL)发散项相关的问题消失为零。我们通过实施优化启发式(例如KL权重退火和字丢失)成功地训练VAE。我们还通过随机采样,线性插值和来自输入的邻域的采样来证明这种连续潜在空间的有效性。我们认为,如果VAE的设计不合适,可能会导致绕过连接,导致在训练期间忽略后期空间。我们通过实验证明了解码器隐藏状态初始化的例子,这种绕过连接将VAE降级为确定性模型,从而减少了生成的句子的多样性。我们发现传统的注意机制使用序列序列VED模型作为旁路连接,从而改进了模型的潜在空间。为了避免这个问题,我们提出了变分注意机制,其中关注上下文向量被建模为可以从分布中采样的随机变量。 Weshow凭经验使用自动评估指标,即熵和不同测量指标,我们的变分注意模型产生比确定性注意模型更多样化的输出句子。通过人类评估研究进行的定性分析证明,我们的模型同时产生的质量高,并且与确定性的注意力对应物产生的质量一样流畅。
translated by 谷歌翻译
We consider incorporating topic information into the sequence-to-sequenceframework to generate informative and interesting responses for chatbots. Tothis end, we propose a topic aware sequence-to-sequence (TA-Seq2Seq) model. Themodel utilizes topics to simulate prior knowledge of human that guides them toform informative and interesting responses in conversation, and leverages thetopic information in generation by a joint attention mechanism and a biasedgeneration probability. The joint attention mechanism summarizes the hiddenvectors of an input message as context vectors by message attention,synthesizes topic vectors by topic attention from the topic words of themessage obtained from a pre-trained LDA model, and let these vectors jointlyaffect the generation of words in decoding. To increase the possibility oftopic words appearing in responses, the model modifies the generationprobability of topic words by adding an extra probability item to bias theoverall distribution. Empirical study on both automatic evaluation metrics andhuman annotations shows that TA-Seq2Seq can generate more informative andinteresting responses, and significantly outperform the-state-of-the-artresponse generation models.
translated by 谷歌翻译
已经有几次尝试为achit-chat对话代理定义合理的动机,这可能导致引人入胜的对话。在这项工作中,我们探索了一个新的方向,即代理人专注于发现关于其对话者的信息。我们通过定义aquantitative metric来正式化这种方法。我们为代理提出了一种算法来最大化它。我们通过人工评估来验证这个想法,我们的系统优于各种基线。我们证明该指标确实与人类对参与度的判断相关联。
translated by 谷歌翻译