Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP), owing to their excellent understanding and generation abilities. Remarkably, what further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. While many downstream applications provide the model with an informational context to aid its performance on the underlying task, how the model's world knowledge interacts with the factual information presented in the context remains under explored. As a desirable behavior, an LLM should give precedence to the context whenever it contains task-relevant information that conflicts with the model's memorized knowledge. This enables model predictions to be grounded in the context, which can then be used to update or correct specific model predictions without frequent retraining. By contrast, when the context is irrelevant to the task, the model should ignore it and fall back on its internal knowledge. In this paper, we undertake a first joint study of the aforementioned two properties, namely controllability and robustness, in the context of LLMs. We demonstrate that state-of-the-art T5 and PaLM (both pretrained and finetuned) could exhibit poor controllability and robustness, which do not scale with increasing model size. As a solution, we propose a novel method - Knowledge Aware FineTuning (KAFT) - to strengthen both controllability and robustness by incorporating counterfactual and irrelevant contexts to standard supervised datasets. Our comprehensive evaluation showcases the utility of KAFT across model architectures and sizes.

Question answering models commonly have access to two sources of "knowledge" during inference time: (1) parametric knowledge - the factual knowledge encoded in the model weights, and (2) contextual knowledge - external knowledge (e.g., a Wikipedia passage) given to the model to generate a grounded answer. Having these two sources of knowledge entangled together is a core issue for generative QA models as it is unclear whether the answer stems from the given non-parametric knowledge or not. This unclarity has implications on issues of trust, interpretability and factuality. In this work, we propose a new paradigm in which QA models are trained to disentangle the two sources of knowledge. Using counterfactual data augmentation, we introduce a model that predicts two answers for a given question: one based on given contextual knowledge and one based on parametric knowledge. Our experiments on the Natural Questions dataset show that this approach improves the performance of QA models by making them more robust to knowledge conflicts between the two knowledge sources, while generating useful disentangled answers.

最近已被证明大型语言模型在各种任务集中获得合理的零射普通化（Brown等，2020）。它已经假设这是语言模型的隐式多任务学习的结果，在语言模型中的预押（Radford等，2019）。可以通过明确的多任务学习直接引起零拍常规化？为了以缩放测试这个问题，我们开发一个系统，以便轻松地将任何自然语言任务映射到人类可读的提示表单中。我们转换一组大量的监督数据集，每个数据集都有多个提示，具有不同的措辞。这些提示的数据集允许基准测试模型执行完全看不见的任务的能力。我们介绍了一个普拉克尔编码器 - 解码器模型（Raffel等，2020; Lester等，2021），覆盖各种任务。该模型在多个标准数据集中达到强大的零点性能，通常优于其尺寸的型号超过16倍。此外，我们的方法对来自Big-替补基准测试的任务子集具有强烈性能，优于其尺寸的6倍。所有提示和培训的型号都可以在https://github.com/ bigscience-workshop / protectsource / httpsource / https：//huggingface.co/bigscience/t0pp。

预测任务标签和为其预测生成自由文本阐述的自律化模型可以实现与NLP系统更直观的交互。然而，这些模型目前正在接受大量人为的自由文本解释，每个任务都会阻碍更广泛的使用。我们建议使用少数培训例子研究更现实的自律化建立。我们出示2月 - 一个标准化的四个现有英语数据集和相关指标。我们通过2月份广泛探索自然语言提示来确定正确的提示方法。然后，通过使用此提示并缩放模型大小，我们证明了几次拍摄自合合理化的进展。我们展示了这项任务的完善房间仍然有充足的改进空间：人类注册人评估的生成解释的平均合理性最多为51％，而人类解释的合理性是76％。我们希望2月份与我们的拟议方法一起促使社区承担几次拍摄的自我合理化挑战。

即使是最大的神经网络也会出错，随着世界的变化，曾经纠正的预测可能变得无效。模型编辑器对基础模型（预训练）模型的行为进行本地更新，以注入更新的知识或纠正不良行为。现有的模型编辑已经显示出希望，但也没有足够的表现力：他们难以准确地对编辑的预期范围进行建模（受编辑影响的示例），从而导致与编辑相关的测试输入的预测不准确，并且经常在之后完全失败。许多编辑。作为一个较高容量的替代方案，我们建议使用检索型反面模型（SERAC）提出半参数编辑，该模型（SERAC）存储在明确的内存中，并学会对它们进行推理以根据需要调节基本模型的预测。为了实现对模型编辑器的更严格评估，我们介绍了三个具有挑战性的语言模型编辑问题，基于问题回答，事实检查和对话生成。我们发现，只有SERAC才能在所有三个问题上实现高性能，从而超过了现有的方法，可以通过大量利润进行模型编辑。代码，数据和其他项目信息将在https://sites.google.com/view/serac-editing上提供。

在维持预审预定序列模型的灵活性的同时，是否有利于常识性推理，这仍然是一个悬而未决的问题。为了调查这个问题，我们开发了生成的知识提示，该提示包括从语言模型中生成知识，然后在回答问题时提供知识作为附加输入。我们的方法不需要特定于任务的监督知识集成或访问结构化的知识库，但它可以提高四个常识性推理任务上的大规模，最先进的模型的性能，从而实现最先进-ART结果取决于数值常识（NumerSense），通用常识性（Commonsenseqa 2.0）和科学常识（QASC）基准。产生的知识促使大型语言模型是灵活的外部知识来源，以改善常识性推理。我们的代码可从https://github.com/liujch1998/gkp获得

大型语言模型在各种任务上显示出令人印象深刻的几次结果。但是，当知识是此类结果的关键时，就像问题回答和事实检查之类的任务一样，似乎需要存储知识的大量参数计数。众所周知，检索增强模型可以在不需要多个参数的情况下在知识密集的任务上表现出色，但是目前尚不清楚它们是否在几个弹药设置中工作。在这项工作中，我们介绍了地图集，这是一个经过精心设计和预先训练的增强语言模型，能够通过很少的培训示例学习知识密集型任务。我们对包括MMLU，苏格兰短裙和归类等各种任务进行评估，并研究文档索引内容的影响，表明它可以很容易地进行更新。值得注意的是，在自然问题上仅使用64个示例在自然问题上达到超过42 \％的准确性，尽管参数少了50倍，但比540B参数模型的表现优于540b参数模型。

我们介绍了Sparrow，这是一个寻求信息的对话代理，与提示的语言模型基线相比，训练有素，更有帮助，正确和无害。我们使用从人类反馈中的强化学习来培训我们的模型，以帮助人类评估者判断代理人的行为。首先，为了使我们的代理人更有帮助和无害，我们将良好对话的要求分解为代理人应遵循的自然语言规则，并分别向评估者询问每个规则。我们证明，这种崩溃使我们能够收集对代理行为的更多针对性的人类判断，并允许更有效的规则条件奖励模型。其次，我们的代理商在收集对模型声明的偏好判决时提供了支持事实主张的来源的证据。对于事实问题，麻雀提供的证据支持了78％的时间。比基线比基线更享受麻雀，同时对人类的对抗性探测更具弹性，在探测时只有8％的时间违反了我们的规则。最后，我们进行了广泛的分析，表明尽管我们的模型学会遵守我们的规则，但它可以表现出分布偏见。

Natural Language Generation (NLG) has improved exponentially in recent years thanks to the development of sequence-to-sequence deep learning technologies such as Transformer-based language models. This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation and data-to-text generation. However, it is also apparent that deep learning based generation is prone to hallucinate unintended text, which degrades the system performance and fails to meet user expectations in many real-world scenarios. To address this issue, many studies have been presented in measuring and mitigating hallucinated texts, but these have never been reviewed in a comprehensive manner before. In this survey, we thus provide a broad overview of the research progress and challenges in the hallucination problem in NLG. The survey is organized into two parts: (1) a general overview of metrics, mitigation methods, and future directions; and (2) an overview of task-specific research progress on hallucinations in the following downstream tasks, namely abstractive summarization, dialogue generation, generative question answering, data-to-text generation, machine translation, and visual-language generation. This survey serves to facilitate collaborative efforts among researchers in tackling the challenge of hallucinated texts in NLG.

Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the limitations of relying solely on their parameters to encode a wealth of world knowledge. This paper aims to understand LMs' strengths and limitations in memorizing factual knowledge, by conducting large-scale knowledge probing experiments of 10 models and 4 augmentation methods on PopQA, our new open-domain QA dataset with 14k questions. We find that LMs struggle with less popular factual knowledge, and that scaling fails to appreciably improve memorization of factual knowledge in the tail. We then show that retrieval-augmented LMs largely outperform orders of magnitude larger LMs, while unassisted LMs remain competitive in questions about high-popularity entities. Based on those findings, we devise a simple, yet effective, method for powerful and efficient retrieval-augmented LMs, which retrieves non-parametric memories only when necessary. Experimental results show that this significantly improves models' performance while reducing the inference costs.

我们研究语言模型是否可以评估自己主张的有效性，并预测他们能够正确回答的问题。我们首先表明，当以正确的格式提供时，较大的模型在多样化的多项选择和True/False问题上进行了很好的校准。因此，我们可以通过要求模型首先提出答案，然后评估其答案正确的概率“ p（true）”来对开放式采样任务进行自我评估。我们发现在各种任务中，P（true）的表现，校准和缩放令人鼓舞。当我们允许模型考虑自己的许多样本之前，在预测一种特定可能性的有效性之前，自我评估的性能进一步改善。接下来，我们研究是否可以培训模型来预测“ P（ik）”，即“我知道”问题的概率，而无需参考任何特定提出的答案。模型在预测P（IK）方面表现良好，并且在跨任务中部分概括，尽管它们在新任务上的P（IK）校准方面遇到了困难。预测的p（IK）概率在存在相关的原始材料的情况下以及对数学单词问题解决方案的提示也适当增加。我们希望这些观察结果为培训更诚实的模型提供了基础，并研究了诚实对模型模仿人类写作以外的其他目标培训的案例的普遍性。

推理是人类认知和智力的关键支柱。在过去的十年中，我们目睹了自然语言处理的巨大收益和大型语言模型的前所未有的缩放。最近的工作表征了很少射击技术的能力，例如思想链，可以在大语言模型中模仿人类的推理。这个标志性的功能很少，连同不断扩展的语言模型相结合，打开了解决各种任务的可能性的远景，例如数学单词问题，代码完成和常识性推理。促使思想链（COT）通过提供中间步骤并敦促模型遵循相同的过程，从而进一步推动了模型的性能。尽管具有令人信服的性能，但在这些模型中推理能力的起源却很少探索。这项工作启动了对大语言模型中推理机制的更深入了解的初步步骤。我们的工作围绕查询模型，同时在提示中控制除一个组件以外的所有组件外：符号，模式和文本。然后，我们分析查询之间的性能差异。我们的结果表明，在提示中存在事实模式对于COT的成功并不是必需的。尽管如此，我们从经验上表明，仅依靠模式也不足以获得高质量的结果。我们认为文本具有常识性知识和意义。我们详尽的经验分析提供了定性的例子，说明了文本和模式之间的共生关系。这种对COT的系统理解使我们能够设计简洁的思想链，被称为CCOT，在其中修剪文本和模式只能保留其关键角色，同时以PAR或更高的求解任务率交付。

Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, but the quality bar for medical and clinical applications is high. Today, attempts to assess models' clinical knowledge typically rely on automated evaluations on limited benchmarks. There is no standard to evaluate model predictions and reasoning across a breadth of tasks. To address this, we present MultiMedQA, a benchmark combining six existing open question answering datasets spanning professional medical exams, research, and consumer queries; and HealthSearchQA, a new free-response dataset of medical questions searched online. We propose a framework for human evaluation of model answers along multiple axes including factuality, precision, possible harm, and bias. In addition, we evaluate PaLM (a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM, on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA, MedMCQA, PubMedQA, MMLU clinical topics), including 67.6% accuracy on MedQA (US Medical License Exam questions), surpassing prior state-of-the-art by over 17%. However, human evaluation reveals key gaps in Flan-PaLM responses. To resolve this we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, recall of knowledge, and medical reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal important limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLM models for clinical applications.

Question Answering (QA) is a longstanding challenge in natural language processing. Existing QA works mostly focus on specific question types, knowledge domains, or reasoning skills. The specialty in QA research hinders systems from modeling commonalities between tasks and generalization for wider applications. To address this issue, we present ProQA, a unified QA paradigm that solves various tasks through a single model. ProQA takes a unified structural prompt as the bridge and improves the QA-centric ability by structural prompt-based pre-training. Through a structurally designed prompt-based input schema, ProQA concurrently models the knowledge generalization for all QA tasks while keeping the knowledge customization for every specific QA task. Furthermore, ProQA is pre-trained with structural prompt-formatted large-scale synthesized corpus, which empowers the model with the commonly-required QA ability. Experimental results on 11 QA benchmarks demonstrate that ProQA consistently boosts performance on both full data fine-tuning, few-shot learning, and zero-shot testing scenarios. Furthermore, ProQA exhibits strong ability in both continual learning and transfer learning by taking the advantages of the structural prompt.

Neural language models (LMs) have achieved impressive results on various language-based reasoning tasks by utilizing latent knowledge encoded in their own pretrained parameters. To make this reasoning process more explicit, recent works retrieve a rationalizing LM's internal knowledge by training or prompting it to generate free-text rationales, which can be used to guide task predictions made by either the same LM or a separate reasoning LM. However, rationalizing LMs require expensive rationale annotation and/or computation, without any assurance that their generated rationales improve LM task performance or faithfully reflect LM decision-making. In this paper, we propose PINTO, an LM pipeline that rationalizes via prompt-based learning, and learns to faithfully reason over rationales via counterfactual regularization. First, PINTO maps out a suitable reasoning process for the task input by prompting a frozen rationalizing LM to generate a free-text rationale. Second, PINTO's reasoning LM is fine-tuned to solve the task using the generated rationale as context, while regularized to output less confident predictions when the rationale is perturbed. Across four datasets, we show that PINTO significantly improves the generalization ability of the reasoning LM, yielding higher performance on both in-distribution and out-of-distribution test sets. Also, we find that PINTO's rationales are more faithful to its task predictions than those generated by competitive baselines.

Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can't detect. We propose circumventing this issue by directly finding latent knowledge inside the internal activations of a language model in a purely unsupervised way. Specifically, we introduce a method for accurately answering yes-no questions given only unlabeled model activations. It works by finding a direction in activation space that satisfies logical consistency properties, such as that a statement and its negation have opposite truth values. We show that despite using no supervision and no model outputs, our method can recover diverse knowledge represented in large language models: across 6 models and 10 question-answering datasets, it outperforms zero-shot accuracy by 4\% on average. We also find that it cuts prompt sensitivity in half and continues to maintain high accuracy even when models are prompted to generate incorrect answers. Our results provide an initial step toward discovering what language models know, distinct from what they say, even when we don't have access to explicit ground truth labels.

Large language models (LLMs) have shown impressive results across a variety of tasks while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial for both system developers and users in this setting. We propose and study Attributed QA as a key first step in the development of attributed LLMs. We develop a reproducable evaluation framework for the task, using human annotations as a gold standard and a correlated automatic metric that we show is suitable for development settings. We describe and benchmark a broad set of architectures for the task. Our contributions give some concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third key question (How to build LLMs with attribution?).

Natural Language Processing (NLP) has been revolutionized by the use of Pre-trained Language Models (PLMs) such as BERT. Despite setting new records in nearly every NLP task, PLMs still face a number of challenges including poor interpretability, weak reasoning capability, and the need for a lot of expensive annotated data when applied to downstream tasks. By integrating external knowledge into PLMs, \textit{\underline{K}nowledge-\underline{E}nhanced \underline{P}re-trained \underline{L}anguage \underline{M}odels} (KEPLMs) have the potential to overcome the above-mentioned limitations. In this paper, we examine KEPLMs systematically through a series of studies. Specifically, we outline the common types and different formats of knowledge to be integrated into KEPLMs, detail the existing methods for building and evaluating KEPLMS, present the applications of KEPLMs in downstream tasks, and discuss the future research directions. Researchers will benefit from this survey by gaining a quick and comprehensive overview of the latest developments in this field.

The emergence of large pretrained models has enabled language models to achieve superior performance in common NLP tasks, including language modeling and question answering, compared to previous static word representation methods. Augmenting these models with a retriever to retrieve the related text and documents as supporting information has shown promise in effectively solving NLP problems in a more interpretable way given that the additional knowledge is injected explicitly rather than being captured in the models' parameters. In spite of the recent progress, our analysis on retriever-augmented language models shows that this class of language models still lack reasoning over the retrieved documents. In this paper, we study the strengths and weaknesses of different retriever-augmented language models such as REALM, kNN-LM, FiD, ATLAS, and Flan-T5 in reasoning over the selected documents in different tasks. In particular, we analyze the reasoning failures of each of these models and study how the models' failures in reasoning are rooted in the retriever module as well as the language model.

Recent work has shown that fine-tuning large pre-trained language models on a collection of tasks described via instructions, a.k.a. instruction-tuning, improves their zero and few-shot generalization to unseen tasks. However, there is a limited understanding of the performance trade-offs of different decisions made during the instruction-tuning process. These decisions include the scale and diversity of the instruction-tuning benchmark, different task sampling strategies, fine-tuning with and without demonstrations, training using specialized datasets for reasoning and dialogue, and finally, the fine-tuning objectives themselves. In this paper, we characterize the effect of instruction-tuning decisions on downstream task performance when scaling both model and benchmark sizes. To this end, we create OPT-IML Bench: a large benchmark for Instruction Meta-Learning (IML) of 2000 NLP tasks consolidated into task categories from 8 existing benchmarks, and prepare an evaluation framework to measure three types of model generalizations: to tasks from fully held-out categories, to held-out tasks from seen categories, and to held-out instances from seen tasks. Through the lens of this framework, we first present insights about instruction-tuning decisions as applied to OPT-30B and further exploit these insights to train OPT-IML 30B and 175B, which are instruction-tuned versions of OPT. OPT-IML demonstrates all three generalization abilities at both scales on four different evaluation benchmarks with diverse tasks and input formats -- PromptSource, FLAN, Super-NaturalInstructions, and UnifiedSKG. Not only does it significantly outperform OPT on all benchmarks but is also highly competitive with existing models fine-tuned on each specific benchmark. We release OPT-IML at both scales, together with the OPT-IML Bench evaluation framework.