语言模型（LMS）在多大程度上在答案时在多大程度上建立场景的“心理模型”（例如，关于特定伦理困境的问题）？虽然认知科学表明，心理模型在人类问题解决中发挥着基本作用，但目前尚不清楚现有LMS的高问答性能是由类似的模型建设进行支持 - 如果不是，那是否可以解释他们众所周知的灾难性的失败。我们观察到Magaw是一种现有的基于T5的LM，当探测时提供了一些有用但是情境问题的有用但不足的心理模型（估计精度= 43％，有用= 21％，一致性= 42％）。我们提出梦想，一种采用情境问题作为输入，以产生精神模型的表现，没有任何其他任务的心理模型培训数据。它通过来自现有NLP资源的遥远监督来继承其社会型号。我们的分析显示，与金刚鹦鹉相比，梦想可以产生明显更好的精神模型（估计精度= 67％，有用= 37％，一致性= 71％）。最后，梦想生成的心理模型可以用作情境QA任务的其他背景。此附加上下文将MACAW零拍摄模型的答案精度提高到三个不同数据集上的+ 1％和+ 4％（绝对）。

When people think of everyday things like an "egg," they typically have a mental image associated with it. This commonsense knowledge helps us understand how these everyday things work and how to interact with them. For example, when someone tries to make a fried egg, they know that it has a shell and that it can be cracked open to reveal the egg white and yolk inside. However, if a system does not have a coherent picture of such everyday things, thinking that the egg yolk surrounds the shell, then it might have to resort to ridiculous approaches such as trying to scrape the egg yolk off the shell into the pan. Do language models have a coherent picture of such everyday things? To investigate this, we propose a benchmark dataset consisting of 100 everyday things, their parts, and the relationships between these parts. We observe that state-of-the-art pre-trained language models (LMs) like GPT-3 and Macaw have fragments of knowledge about these entities, but they fail to produce consistent parts mental models. We propose a simple extension to these LMs where we apply a constraint satisfaction layer on top of raw predictions from LMs to produce more consistent and accurate parts mental models of everyday things.

随着人工智能系统变得越来越强大和普遍，人们对机器的道德或缺乏道德的关注变得越来越关注。然而，向机器讲授道德是一项艰巨的任务，因为道德仍然是人类中最激烈的争论问题之一，更不用说AI了。但是，部署到数百万用户的现有AI系统已经在做出充满道德影响的决策，这构成了一个看似不可能的挑战：教学机器的道德意义，而人类继续努力努力。为了探索这一挑战，我们介绍了Delphi，这是一个基于深层神经网络的实验框架，直接训练了描述性道德判断，例如，“帮助朋友”通常是不错的，而“帮助朋友传播假新闻”不是。经验结果提供了对机器伦理的承诺和局限性的新见解。面对新的道德情况，德尔菲（Delphi）表现出强大的概括能力，而现成的神经网络模型表现出明显差的判断，包括不公正的偏见，证实了对明确教学机器的道德意义的必要性。然而，德尔菲并不完美，表现出对普遍性偏见和不一致的敏感性。尽管如此，我们还是展示了不完美的Delphi的积极用例，包括在其他不完美的AI系统中将其用作组件模型。重要的是，我们根据著名的道德理论来解释Delphi的运营化，这使我们提出了重要的未来研究问题。

相同上下文的可能后果可能会因我们所指的情况而异。但是，当前在自然语言处理中的研究并不集中于多种可能情况下的常识性推理。本研究通过短篇小说文字提出与候选人答案相同的结尾的多个问题来构成这项任务。我们由此产生的数据集，可能的故事，包括超过1.3k的故事文本超过4.5k的问题。我们发现，即使是目前的强训练性语言模型也很难始终如一地回答问题，这强调了无监督环境中最高的准确性（60.2％）远远落后于人类准确性（92.5％）。通过与现有数据集进行比较，我们观察到数据集中的问题包含答案选项中的最小注释伪像。此外，我们的数据集还包括需要反事实推理的示例，以及需要读者的反应和虚构信息的示例，这表明我们的数据集可以作为对未来常识性推理的未来研究的挑战性测试。

Context is vital for commonsense moral reasoning. "Lying to a friend" is wrong if it is meant to deceive them, but may be morally okay if it is intended to protect them. Such nuanced but salient contextual information can potentially flip the moral judgment of an action. Thus, we present ClarifyDelphi, an interactive system that elicits missing contexts of a moral situation by generating clarification questions such as "Why did you lie to your friend?". Our approach is inspired by the observation that questions whose potential answers lead to diverging moral judgments are the most informative. We learn to generate questions using Reinforcement Learning, by maximizing the divergence between moral judgements of hypothetical answers to a question. Human evaluation shows that our system generates more relevant, informative and defeasible questions compared to other question generation baselines. ClarifyDelphi assists informed moral reasoning processes by seeking additional morally consequential context to disambiguate social and moral situations.

在维持预审预定序列模型的灵活性的同时，是否有利于常识性推理，这仍然是一个悬而未决的问题。为了调查这个问题，我们开发了生成的知识提示，该提示包括从语言模型中生成知识，然后在回答问题时提供知识作为附加输入。我们的方法不需要特定于任务的监督知识集成或访问结构化的知识库，但它可以提高四个常识性推理任务上的大规模，最先进的模型的性能，从而实现最先进-ART结果取决于数值常识（NumerSense），通用常识性（Commonsenseqa 2.0）和科学常识（QASC）基准。产生的知识促使大型语言模型是灵活的外部知识来源，以改善常识性推理。我们的代码可从https://github.com/liujch1998/gkp获得

我们介绍了Sparrow，这是一个寻求信息的对话代理，与提示的语言模型基线相比，训练有素，更有帮助，正确和无害。我们使用从人类反馈中的强化学习来培训我们的模型，以帮助人类评估者判断代理人的行为。首先，为了使我们的代理人更有帮助和无害，我们将良好对话的要求分解为代理人应遵循的自然语言规则，并分别向评估者询问每个规则。我们证明，这种崩溃使我们能够收集对代理行为的更多针对性的人类判断，并允许更有效的规则条件奖励模型。其次，我们的代理商在收集对模型声明的偏好判决时提供了支持事实主张的来源的证据。对于事实问题，麻雀提供的证据支持了78％的时间。比基线比基线更享受麻雀，同时对人类的对抗性探测更具弹性，在探测时只有8％的时间违反了我们的规则。最后，我们进行了广泛的分析，表明尽管我们的模型学会遵守我们的规则，但它可以表现出分布偏见。

Many real-world applications of language models (LMs), such as code autocomplete and writing assistance, involve human-LM interaction, but the main LM benchmarks are non-interactive, where a system produces output without human intervention. To evaluate human-LM interaction, we develop a framework, Human-AI Language-based Interaction Evaluation (H-LINE), that expands non-interactive evaluation along three dimensions, capturing (i) the interactive process, not only the final output; (ii) the first-person subjective experience, not just a third-party assessment; and (iii) notions of preference beyond quality. We then design five tasks ranging from goal-oriented to open-ended to capture different forms of interaction. On four state-of-the-art LMs (three variants of OpenAI's GPT-3 and AI21's J1-Jumbo), we find that non-interactive performance does not always result in better human-LM interaction and that first-person and third-party metrics can diverge, suggesting the importance of examining the nuances of human-LM interaction.

最近已被证明大型语言模型在各种任务集中获得合理的零射普通化（Brown等，2020）。它已经假设这是语言模型的隐式多任务学习的结果，在语言模型中的预押（Radford等，2019）。可以通过明确的多任务学习直接引起零拍常规化？为了以缩放测试这个问题，我们开发一个系统，以便轻松地将任何自然语言任务映射到人类可读的提示表单中。我们转换一组大量的监督数据集，每个数据集都有多个提示，具有不同的措辞。这些提示的数据集允许基准测试模型执行完全看不见的任务的能力。我们介绍了一个普拉克尔编码器 - 解码器模型（Raffel等，2020; Lester等，2021），覆盖各种任务。该模型在多个标准数据集中达到强大的零点性能，通常优于其尺寸的型号超过16倍。此外，我们的方法对来自Big-替补基准测试的任务子集具有强烈性能，优于其尺寸的6倍。所有提示和培训的型号都可以在https://github.com/ bigscience-workshop / protectsource / httpsource / https：//huggingface.co/bigscience/t0pp。

Despite the success of large language models (LLMs) in various natural language processing (NLP) tasks, the stored knowledge in these models may inevitably be incomplete, out-of-date, or incorrect. This motivates the need to utilize external knowledge to assist LLMs. Unfortunately, current methods for incorporating external knowledge often require additional training or fine-tuning, which can be costly and may not be feasible for LLMs. To address this issue, we propose a novel post-processing approach, rethinking with retrieval (RR), which retrieves relevant external knowledge based on the decomposed reasoning steps obtained from the chain-of-thought (CoT) prompting. This lightweight approach does not require additional training or fine-tuning and is not limited by the input length of LLMs. We evaluate the effectiveness of RR through extensive experiments with GPT-3 on three complex reasoning tasks: commonsense reasoning, temporal reasoning, and tabular reasoning. Our results show that RR can produce more faithful explanations and improve the performance of LLMs.

动机，情感和行动是人类活动中相关的基本因素。尽管长期以来一直认为动机和情感是探索人们如何在人类活动中采取行动的核心，但几乎没有研究支持分析人类精神状态与行动之间的关系。我们介绍了第一项研究，该研究研究了基于语言的人类活动中建模动机，情感和行动的生存能力，即逗号（人类活动的认知框架）。在逗号的指导下，我们定义了三个自然语言处理任务（情感理解，动机理解和有条件的动作生成），并通过自动从故事常识中提取样本来建立一个具有挑战性的数据集冰雹。 NLP应用程序的实验结果证明了建模关系的有效性。此外，与现有方法相比，受逗号启发的模型可以更好地揭示动机，情感和行动之间的基本关系。

Pragmatics is an essential part of communication, but it remains unclear what mechanisms underlie human pragmatic communication and whether NLP systems capture pragmatic language understanding. To investigate both these questions, we perform a fine-grained comparison of language models and humans on seven pragmatic phenomena, using zero-shot prompting on an expert-curated set of English materials. We ask whether models (1) select pragmatic interpretations of speaker utterances, (2) make similar error patterns as humans, and (3) use similar linguistic cues as humans to solve the tasks. We find that the largest models achieve high accuracy and match human error patterns: within incorrect responses, models favor the literal interpretation of an utterance over heuristic-based distractors. We also find evidence that models and humans are sensitive to similar linguistic cues. Our results suggest that even paradigmatic pragmatic phenomena may be solved without explicit representations of other agents' mental states, and that artificial models can be used to gain mechanistic insights into human pragmatic processing.

When answering a question, people often draw upon their rich world knowledge in addition to the particular context. Recent work has focused primarily on answering questions given some relevant document or context, and required very little general background. To investigate question answering with prior knowledge, we present COMMONSENSEQA: a challenging new dataset for commonsense question answering. To capture common sense beyond associations, we extract from CON-CEPTNET (Speer et al., 2017) multiple target concepts that have the same semantic relation to a single source concept. Crowd-workers are asked to author multiple-choice questions that mention the source concept and discriminate in turn between each of the target concepts. This encourages workers to create questions with complex semantics that often require prior knowledge. We create 12,247 questions through this procedure and demonstrate the difficulty of our task with a large number of strong baselines. Our best baseline is based on BERT-large (Devlin et al., 2018) and obtains 56% accuracy, well below human performance, which is 89%.

As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (which is time-consuming and expensive) or existing data sources (which are not always available). Here, we automatically generate evaluations with LMs. We explore approaches with varying amounts of human effort, from instructing LMs to write yes/no questions to making complex Winogender schemas with multiple stages of LM-based generation and filtering. Crowdworkers rate the examples as highly relevant and agree with 90-100% of labels, sometimes more so than corresponding human-written datasets. We generate 154 datasets and discover new cases of inverse scaling where LMs get worse with size. Larger LMs repeat back a dialog user's preferred answer ("sycophancy") and express greater desire to pursue concerning goals like resource acquisition and goal preservation. We also find some of the first examples of inverse scaling in RL from Human Feedback (RLHF), where more RLHF makes LMs worse. For example, RLHF makes LMs express stronger political views (on gun rights and immigration) and a greater desire to avoid shut down. Overall, LM-written evaluations are high-quality and let us quickly discover many novel LM behaviors.

Visual understanding goes well beyond object recognition. With one glance at an image, we can effortlessly imagine the world beyond the pixels: for instance, we can infer people's actions, goals, and mental states. While this task is easy for humans, it is tremendously difficult for today's vision systems, requiring higher-order cognition and commonsense reasoning about the world. We formalize this task as Visual Commonsense Reasoning. Given a challenging question about an image, a machine must answer correctly and then provide a rationale justifying its answer.Next, we introduce a new dataset, VCR, consisting of 290k multiple choice QA problems derived from 110k movie scenes. The key recipe for generating non-trivial and highquality problems at scale is Adversarial Matching, a new approach to transform rich annotations into multiple choice questions with minimal bias. Experimental results show that while humans find VCR easy (over 90% accuracy), state-of-the-art vision models struggle (∼45%).To move towards cognition-level understanding, we present a new reasoning engine, Recognition to Cognition Networks (R2C), that models the necessary layered inferences for grounding, contextualization, and reasoning. R2C helps narrow the gap between humans and machines (∼65%); still, the challenge is far from solved, and we provide analysis that suggests avenues for future work.

本文探讨了提高语言模型的零次学习能力的简单方法。我们表明，指令调整 - 通过对说明书中所述的任务集合微调语言模型 - 大幅提升零射门上看不见任务中的表现。我们采取预训练的语言模型和指令调整它通过自然语言指令模板语言表达了60NLP任务137B参数。我们评估这种指令调整模型，我们称之为FLAN，在看不见的任务类型。FLAN显着改善其未修饰的对应的性能和超过25的20个任务，我们评估零射门175BGPT-3。FLAN甚至GPT-3通过在安利，RTE，BoolQ，AI2-ARC，OpenbookQA和StoryCloze大比分胜过几拍。消融研究显示任务和模型的规模，这个数字是指令调整取得成功的关键组成部分。

情绪分析中最突出的任务是为文本分配情绪，并了解情绪如何在语言中表现出来。自然语言处理的一个重要观察结果是，即使没有明确提及情感名称，也可以通过单独参考事件来隐式传达情绪。在心理学中，被称为评估理论的情感理论类别旨在解释事件与情感之间的联系。评估可以被形式化为变量，通过他们认为相关的事件的人们的认知评估来衡量认知评估。其中包括评估事件是否是新颖的，如果该人认为自己负责，是否与自己的目标以及许多其他人保持一致。这样的评估解释了哪些情绪是基于事件开发的，例如，新颖的情况会引起惊喜或不确定后果的人可能引起恐惧。我们在文本中分析了评估理论对情绪分析的适用性，目的是理解注释者是否可以可靠地重建评估概念，如果可以通过文本分类器预测，以及评估概念是否有助于识别情感类别。为了实现这一目标，我们通过要求人们发短信描述触发特定情绪并披露其评估的事件来编译语料库。然后，我们要求读者重建文本中的情感和评估。这种设置使我们能够衡量是否可以纯粹从文本中恢复情绪和评估，并为判断模型的绩效指标提供人体基准。我们将文本分类方法与人类注释者的比较表明，两者都可以可靠地检测出具有相似性能的情绪和评估。我们进一步表明，评估概念改善了文本中情绪的分类。

致致辞问题答案（CQA）旨在测试模型是否可以回答有关每个人都知道的勤杂朗语言的问题。结合外部知识库的事先作品已经显示了有希望的结果，但知识库是昂贵的构造，并且通常限于固定的一组关系。在本文中，我们专注于更好地利用\ Texit {隐式知识}存储在预先接受预先接受的语言模型中。虽然研究人员发现嵌入在预先接受预先训练的语言模型中的知识，但可以通过填写仔细设计的提取和文本分类的谨慎设计的空白来提取，但如果我们可以在输入和输入的CQA中采用此范例，仍然不清楚输出采取更灵活的形式。为此，我们调查了四种翻译方法，可以将自然问题转化为渗出风格的句子，从语言模型中更好地征求致辞知识，包括基于句法的模型，无监督的神经模型和两个监督的神经模型。此外，要结合不同的翻译方法，我们建议鼓励模型预测与未标记数据不同翻译问题的一致性。我们展示了我们在零拍摄设置中三个CQA数据集上的方法的有效性。我们表明，我们的方法与知识库改进的模型互补，并结合它们可以导致最先进的零射击性能。分析还揭示了不同的强化翻译方法的明显特征，并为什么结合它们导致巨大改进提供了洞察。

预训练的语言模型（PTLM）已显示出在自然语言任务上表现良好。许多先前的作品都以通过知识图（KGS）标记的关系链接的实体的形式利用结构性常识来协助PTLM。检索方法使用kg作为单独的静态模块，该模块限制了覆盖范围，因为kgs包含有限的知识。生成方法训练PTLMS kg三倍以提高获得知识的规模。但是，对符号KG实体的培训限制了其在涉及自然语言文本的任务中的适用性，在这些任务中，它们忽略了整体上下文。为了减轻这种情况，我们提出了一个以句子为条件的常识性上下文化器（COSE-CO）作为输入，以使其在生成与输入文本的整体上下文相关的任务中通常可用。为了训练Cose-Co，我们提出了一个新的数据集，其中包括句子和常识知识对。 COSE-CO推断出的知识是多种多样的，并且包含了基础KG中不存在的新实体。我们增强了在多选质量质量检查和开放式常识性推理任务中产生的知识，从而改善了CSQA，ARC，QASC和OBQA数据集的当前最佳方法。我们还展示了其在改善释义生成任务的基线模型方面的适用性。

叙事中的事件可以通过其参与者的基本状态理解为一致的整体。通常，这些参与者在叙述中没有明确提及，而是通过常识性或推论填写。理解叙述的模型应该能够推断出这些隐性参与者状态，以及有关这些状态对叙事的影响的原因。为了促进这一目标，我们介绍了一个新的众包参与者指出的数据集意大利面。该数据集包含有效的，可推断的参与者状态；对国家的反事实扰动；如果反事实是真实的，那么故事的变化将是必要的。我们介绍了三项基于州的推理任务，这些任务测试了一个故事何时由故事启用，修改一个反事实状态的故事，并解释给定经过修订的故事的最有可能的状态变化。我们的基准测试实验表明，尽管当今的LLM能够在某种程度上推理有关州的推理，但仍有很大的改进空间，这表明了未来研究的潜在途径。