人工推理通常可以理解为两个系统之间的相互作用：直观和关联（“系统1”）和审议和逻辑（“系统2”）。神经序列模型 - 在执行复杂，结构化任务时越来越成功 - 表现出系统1的优点和故障模式：它们是快速和学习数据的模式，但通常不一致和不连贯。在这项工作中，我们通过添加系统2-Inspired逻辑推理，寻求一种轻量级，无培训的手段来改善现有系统1样序列模型。我们探讨了该主题的几种变体，其中通过符号推理模块检查来自神经序列模型的候选几代，可以通过符号推理模块来接受或拒绝几代人。我们的方法使用神经推理来介导神经系统1和逻辑系统2.导致强大的故事生成和接地的指示，表明这种方法可以增加神经基代的一致性和准确性。

Story generation and understanding -- as with all NLG/NLU tasks -- has seen a surge in neurosymbolic work. Researchers have recognized that, while large language models (LLMs) have tremendous utility, they can be augmented with symbolic means to be even better and to make up for any flaws that the neural networks might have. However, symbolic methods are extremely costly in terms of the amount of time and expertise needed to create them. In this work, we capitalize on state-of-the-art Code-LLMs, such as Codex, to bootstrap the use of symbolic methods for tracking the state of stories and aiding in story understanding. We show that our CoRRPUS system and abstracted prompting procedures can beat current state-of-the-art structured LLM techniques on pre-existing story understanding tasks (bAbI task 2 and Re^3) with minimal hand engineering. We hope that this work can help highlight the importance of symbolic representations and specialized prompting for LLMs as these models require some guidance for performing reasoning tasks properly.

Automated plot generation is the challenge of generating a sequence of events that will be perceived by readers as the plot of a coherent story. Traditional symbolic planners plan a story from a goal state and guarantee logical causal plot coherence but rely on a library of hand-crafted actions with their preconditions and effects. This closed world setting limits the length and diversity of what symbolic planners can generate. On the other hand, pre-trained neural language models can generate stories with great diversity, while being generally incapable of ending a story in a specified manner and can have trouble maintaining coherence. In this paper, we present an approach to story plot generation that unifies causal planning with neural language models. We propose to use commonsense knowledge extracted from large language models to recursively expand a story plot in a backward chaining fashion. Specifically, our system infers the preconditions for events in the story and then events that will cause those conditions to become true. We performed automatic evaluation to measure narrative coherence as indicated by the ability to answer questions about whether different events in the story are causally related to other events. Results indicate that our proposed method produces more coherent plotlines than several strong baselines.

Language models (LMs) often generate incoherent outputs: they refer to events and entity states that are incompatible with the state of the world described in their inputs. We introduce SituationSupervision, a family of approaches for improving coherence in LMs by training them to construct and condition on explicit representations of entities and their states. SituationSupervision has two components: an auxiliary situation modeling task that trains models to predict state representations in context, and a latent state inference procedure that imputes these states from partially annotated training data. SituationSupervision can be applied to both fine-tuning (by supervising LMs to encode state variables in their hidden representations) and prompting (by inducing LMs to interleave textual descriptions of entity states with output text). In both cases, SituationSupervision requires only a small number of state annotations to produce major coherence improvements (between 4-11%), showing that standard LMs can be sample-efficiently trained to model not just language but the situations it describes.

随着人工智能系统变得越来越强大和普遍，人们对机器的道德或缺乏道德的关注变得越来越关注。然而，向机器讲授道德是一项艰巨的任务，因为道德仍然是人类中最激烈的争论问题之一，更不用说AI了。但是，部署到数百万用户的现有AI系统已经在做出充满道德影响的决策，这构成了一个看似不可能的挑战：教学机器的道德意义，而人类继续努力努力。为了探索这一挑战，我们介绍了Delphi，这是一个基于深层神经网络的实验框架，直接训练了描述性道德判断，例如，“帮助朋友”通常是不错的，而“帮助朋友传播假新闻”不是。经验结果提供了对机器伦理的承诺和局限性的新见解。面对新的道德情况，德尔菲（Delphi）表现出强大的概括能力，而现成的神经网络模型表现出明显差的判断，包括不公正的偏见，证实了对明确教学机器的道德意义的必要性。然而，德尔菲并不完美，表现出对普遍性偏见和不一致的敏感性。尽管如此，我们还是展示了不完美的Delphi的积极用例，包括在其他不完美的AI系统中将其用作组件模型。重要的是，我们根据著名的道德理论来解释Delphi的运营化，这使我们提出了重要的未来研究问题。

我们探索使用大型预用语言模型作为少量语义解析器。语义解析中的目标是给定自然语言输入的结构化含义表示。但是，培训语言模型以生成自然语言。为了弥合差距，我们使用语言模型来解释进入一个类似于英语的受控的子宫内的输入，可以自动映射到目标含义表示表示。我们的结果表明，只有少量的数据和较少的代码转换为类似英语的代表，我们为快速启动语义解析器的蓝图导致了对多个社区任务的令人惊讶的有效性能，大大超过基线方法也在相同的限制上培训数据。

Many real-world applications of language models (LMs), such as code autocomplete and writing assistance, involve human-LM interaction, but the main LM benchmarks are non-interactive, where a system produces output without human intervention. To evaluate human-LM interaction, we develop a framework, Human-AI Language-based Interaction Evaluation (H-LINE), that expands non-interactive evaluation along three dimensions, capturing (i) the interactive process, not only the final output; (ii) the first-person subjective experience, not just a third-party assessment; and (iii) notions of preference beyond quality. We then design five tasks ranging from goal-oriented to open-ended to capture different forms of interaction. On four state-of-the-art LMs (three variants of OpenAI's GPT-3 and AI21's J1-Jumbo), we find that non-interactive performance does not always result in better human-LM interaction and that first-person and third-party metrics can diverge, suggesting the importance of examining the nuances of human-LM interaction.

自动化讲故事长期以来一直抓住了研究人员在日常生活中的叙述中的难以感受。但是，在用神经语言模型产生叙述时，保持一致性并保持对特定结束的特定结束挑战。在本文中，我们介绍了读者模型（Storm）的故事生成，这是一个框架，其中读者模型用于推理故事的推理应该进步。读者模型是人类读者相信关于虚构故事世界的概念，实体和关系的人。我们展示了如何作为知识图表所代表的明确读者模型提供故事一致性，并以实现给定的故事世界目标的形式提供可控性。实验表明，我们的模型产生了显着更加连贯和主题的故事，优于尺寸的基线，包括情节合理性并保持主题。我们的系统也优于在未订购的情况下在组成给定概念时占总引导的故事生成基线。

Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.

Controllable Text Generation (CTG) is emerging area in the field of natural language generation (NLG). It is regarded as crucial for the development of advanced text generation technologies that are more natural and better meet the specific constraints in practical applications. In recent years, methods using large-scale pre-trained language models (PLMs), in particular the widely used transformer-based PLMs, have become a new paradigm of NLG, allowing generation of more diverse and fluent text. However, due to the lower level of interpretability of deep neural networks, the controllability of these methods need to be guaranteed. To this end, controllable text generation using transformer-based PLMs has become a rapidly growing yet challenging new research hotspot. A diverse range of approaches have emerged in the recent 3-4 years, targeting different CTG tasks which may require different types of controlled constraints. In this paper, we present a systematic critical review on the common tasks, main approaches and evaluation methods in this area. Finally, we discuss the challenges that the field is facing, and put forward various promising future directions. To the best of our knowledge, this is the first survey paper to summarize CTG techniques from the perspective of PLMs. We hope it can help researchers in related fields to quickly track the academic frontier, providing them with a landscape of the area and a roadmap for future research.

我们介绍了Sparrow，这是一个寻求信息的对话代理，与提示的语言模型基线相比，训练有素，更有帮助，正确和无害。我们使用从人类反馈中的强化学习来培训我们的模型，以帮助人类评估者判断代理人的行为。首先，为了使我们的代理人更有帮助和无害，我们将良好对话的要求分解为代理人应遵循的自然语言规则，并分别向评估者询问每个规则。我们证明，这种崩溃使我们能够收集对代理行为的更多针对性的人类判断，并允许更有效的规则条件奖励模型。其次，我们的代理商在收集对模型声明的偏好判决时提供了支持事实主张的来源的证据。对于事实问题，麻雀提供的证据支持了78％的时间。比基线比基线更享受麻雀，同时对人类的对抗性探测更具弹性，在探测时只有8％的时间违反了我们的规则。最后，我们进行了广泛的分析，表明尽管我们的模型学会遵守我们的规则，但它可以表现出分布偏见。

Instruction tuning enables pretrained language models to perform new tasks from inference-time natural language descriptions. These approaches rely on vast amounts of human supervision in the form of crowdsourced datasets or user interactions. In this work, we introduce Unnatural Instructions: a large dataset of creative and diverse instructions, collected with virtually no human labor. We collect 64,000 examples by prompting a language model with three seed examples of instructions and eliciting a fourth. This set is then expanded by prompting the model to rephrase each instruction, creating a total of approximately 240,000 examples of instructions, inputs, and outputs. Experiments show that despite containing a fair amount of noise, training on Unnatural Instructions rivals the effectiveness of training on open-source manually-curated datasets, surpassing the performance of models such as T0++ and Tk-Instruct across various benchmarks. These results demonstrate the potential of model-generated data as a cost-effective alternative to crowdsourcing for dataset expansion and diversification.

我们探索如何产生一系列思想 - 一系列中间推理步骤 - 显着提高了大语言模型执行复杂推理的能力。特别是，我们通过一种称为“思想链”提示的简单方法在足够大的语言模型中自然出现这种推理能力，在此过程中，一些思想示范被作为提示的示例提供了。三种大语模型的实验表明，促使思想链提高了一系列算术，常识和象征性推理任务的性能。经验收益可能会引人注目。例如，仅使用八个思想范围的540B参数语言模型才能在数学单词问题的GSM8K基准上实现最新的精度，甚至超过了带有验证器的Fineted GPT-3。

深度神经语言模型的最新进展与大规模数据集的能力相结合，加速了自然语言生成系统的发展，这些系统在多种任务和应用程序上下文中产生流利和连贯的文本（在各种成功程度上）。但是，为所需的用户控制这些模型的输出仍然是一个开放的挑战。这不仅对于自定义生成语言的内容和样式至关重要，而且对于他们在现实世界中的安全可靠部署至关重要。我们提出了一项关于受约束神经语言生成的新兴主题的广泛调查，在该主题中，我们通过区分条件和约束（后者是在输出文本上而不是输入的可检验条件），正式定义和分类自然语言生成问题，目前是可检验的）约束文本生成任务，并查看受限文本生成的现有方法和评估指标。我们的目的是强调这个新兴领域的最新进展和趋势，以告知最有希望的方向和局限性，以推动受约束神经语言生成研究的最新作品。

基于神经语言模型的自动化故事生成方法遭受了两个重要的限制。首先，基于语言模型的故事生成器通常不适用于给定的目标或结束。其次，当故事变长时，他们经常失去一致。我们提出了一种新的自动化故事生成方法，将问题视为生成的问答之一。我们所提出的故事生成系统从封装故事的最终事件的句子开始。然后系统迭代地（1）分析描述最新事件的文本，（2）生成关于“为什么”一个字符正在执行他们在事件中执行的事情的问题，然后（3）尝试生成另一个前面的回答这个问题的事件。

抽象推理是智能系统的关键能力。大型语言模型在抽象推理任务上实现了高度的性能，但表现出许多缺陷。但是，人类的抽象推理也是不完美的，并且取决于我们对推理问题内容的知识和信念。例如，人类对在日常情况下基于逻辑规则的逻辑规则比关于抽象属性的任意规则更可靠地理解。语言模型的培训经验类似地赋予了他们先前的期望，这些期望反映了人类的知识和信念。因此，我们假设语言模型会显示出类似人类的内容对抽象推理问题的影响。我们在三个逻辑推理任务中探讨了这一假设：自然语言推论，判断三段论的逻辑有效性和ison选择任务（Wason，1968）。我们发现，最新的大语言模型（具有7或700亿个参数； Hoffman等，2022）反映了这些任务中人类在人类中观察到的许多相同模式 - 像人类一样，模型对可信情况的理由更有效地理由不现实或抽象的。我们的发现对理解这些认知效应以及有助于语言模型表现的因素具有影响。

当前的语言模型可以产生高质量的文本。他们只是复制他们之前看到的文本，或者他们学习了普遍的语言抽象吗？要取笑这些可能性，我们介绍了乌鸦，这是一套评估生成文本的新颖性，专注于顺序结构（n-gram）和句法结构。我们将这些分析应用于四种神经语言模型（LSTM，变压器，变换器-XL和GPT-2）。对于本地结构 - 例如，单个依赖性 - 模型生成的文本比来自每个模型的测试集的人类生成文本的基线显着不那么新颖。对于大规模结构 - 例如，总句结构 - 模型生成的文本与人生成的基线一样新颖甚至更新颖，但模型仍然有时复制，在某些情况下，在训练集中重复超过1000字超过1,000字的通道。我们还表现了广泛的手动分析，表明GPT-2的新文本通常在形态学和语法中形成良好，但具有合理的语义问题（例如，是自相矛盾）。

大型预先训练的生成语言模型的出现为AI故事的常见框架通过采样模型来创建持续故事的序列。然而，单独的抽样对故事产生不足。特别是，很难指导语言模型来创建故事以达到特定的目标事件。我们提出了两种在深增强学习和奖励塑造的自动化技术，以控制计算机生成的故事的情节。首先利用近端策略优化来微调现有的基于变换器的语言模型，以生成文本持续，而且是寻求目标。第二种提取来自展开故事的知识图，该故事由策略网络使用，具有图注意选择由语言模型生成的候选继续。我们报告了与故事如何实现给定的目标事件以及与基线和消融相比的一致性和整体故事质量的人类参与者排名的自动化指标报告。

神经文本生成的主导范式是自回归语言模型的左右解码。然而，复杂的词汇约束下的受约束或可控发生的产生需要远见计划未来可行的未来路径。从A *搜索算法绘制灵感，我们提出了一种神经系统A * esque，一种解码算法包含未来成本的启发式估计。我们开发了高效的寻找高效，对大规模语言模型有效，使我们的方法成为诸如光束搜索和顶-K采样等共同技术的替代品。为了使受约束的产生，我们构建了神经系统解码（Lu等，2021），将其灵活性结合到与未来约束满足的* esque估计结合起来的逻辑限制。我们的方法在五代任务中优于竞争力的基线，并在表格到文本生成，受限机器翻译和关键字的生成中实现了新的最先进的性能。在需要复杂约束满足或少量拍摄或零拍摄设置的任务上，改进尤其显着。神经系统A * esque说明了用于改进和实现大规模语言模型的新功能的解码的力量。

Pragmatics is an essential part of communication, but it remains unclear what mechanisms underlie human pragmatic communication and whether NLP systems capture pragmatic language understanding. To investigate both these questions, we perform a fine-grained comparison of language models and humans on seven pragmatic phenomena, using zero-shot prompting on an expert-curated set of English materials. We ask whether models (1) select pragmatic interpretations of speaker utterances, (2) make similar error patterns as humans, and (3) use similar linguistic cues as humans to solve the tasks. We find that the largest models achieve high accuracy and match human error patterns: within incorrect responses, models favor the literal interpretation of an utterance over heuristic-based distractors. We also find evidence that models and humans are sensitive to similar linguistic cues. Our results suggest that even paradigmatic pragmatic phenomena may be solved without explicit representations of other agents' mental states, and that artificial models can be used to gain mechanistic insights into human pragmatic processing.