扩展语言模型已被证明可以预测提高各种下游任务的性能和样本效率。相反，本文讨论了一种不可预测的现象，我们将其称为大语言模型的新兴能力。如果在较小的模型中不存在，而是在较大的模型中存在，那么我们认为它可以突然出现。因此，不仅可以通过推断较小模型的性能来预测紧急能力。这种出现的存在意味着额外的扩展可以进一步扩大语言模型的能力范围。

Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models.

大型语言模型在零拍摄设置中的许多自然语言处理（NLP）任务中表现出令人印象深刻的性能。我们询问这些模型是否展示了致辞语言 - NLP应用的关键组成部分 - 通过评估四个偶数基准的模型。我们发现大型语言模型的令人印象深刻的零射击性能主要是由于我们的基准测试中的数据集偏差。我们还表明，零拍摄性能对基准的超参数和相似性敏感到预训练数据集。此外，当在几次拍摄设置中评估模型时，我们没有观察大量改进。最后，与以前的工作相比，我们发现利用明确的致辞知识并没有产生重大改善。

本文探讨了提高语言模型的零次学习能力的简单方法。我们表明，指令调整 - 通过对说明书中所述的任务集合微调语言模型 - 大幅提升零射门上看不见任务中的表现。我们采取预训练的语言模型和指令调整它通过自然语言指令模板语言表达了60NLP任务137B参数。我们评估这种指令调整模型，我们称之为FLAN，在看不见的任务类型。FLAN显着改善其未修饰的对应的性能和超过25的20个任务，我们评估零射门175BGPT-3。FLAN甚至GPT-3通过在安利，RTE，BoolQ，AI2-ARC，OpenbookQA和StoryCloze大比分胜过几拍。消融研究显示任务和模型的规模，这个数字是指令调整取得成功的关键组成部分。

We demonstrate that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even becoming competitive with prior state-ofthe-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous nonsparse language model, and test its performance in the few-shot setting. For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model. GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks. We also identify some datasets where GPT-3's few-shot learning still struggles, as well as some datasets where GPT-3 faces methodological issues related to training on large web corpora.

我们探索如何产生一系列思想 - 一系列中间推理步骤 - 显着提高了大语言模型执行复杂推理的能力。特别是，我们通过一种称为“思想链”提示的简单方法在足够大的语言模型中自然出现这种推理能力，在此过程中，一些思想示范被作为提示的示例提供了。三种大语模型的实验表明，促使思想链提高了一系列算术，常识和象征性推理任务的性能。经验收益可能会引人注目。例如，仅使用八个思想范围的540B参数语言模型才能在数学单词问题的GSM8K基准上实现最新的精度，甚至超过了带有验证器的Fineted GPT-3。

具有更多数据，计算和参数的缩放语言模型在自然语言处理方面取得了重大进展。例如，由于缩放，GPT-3能够在内心学习任务上实现强烈结果。但是，培训这些大密度模型需要大量的计算资源。在本文中，我们提出并开发了名为Glam（通用语言模型）的语言模型系列，它使用稀疏激活的专家架构来规模模型容量，同时与致密变体相比，也产生显着更少的训练成本。最大的Glam具有1.2万亿参数，比GPT-3大约为7倍。它仅消耗了用于训练GPT-3的1/3的能量，并且需要一半的计算拖鞋进行推理，同时仍然在29个NLP任务中实现更好的整体零射击和一次性性能。

With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few training examples. It has been a new trend exploring ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress, challenges, and future work in ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques of ICL, including training strategies, prompting strategies, and so on. Finally, we present the challenges of ICL and provide potential directions for further research. We hope our work can encourage more research on uncovering how ICL works and improving ICL in future work.

稀疏的专家模型是一个三十年来的概念，作为深度学习中流行的建筑。这类体系结构包括专家的混合物，交换变压器，路由网络，基础层等，所有这些都以一个统一的想法，即每个示例都由参数的一个子集进行。通过这样做，稀疏度将参数计数与每个示例的计算分解，从而允许使用极大但有效的模型。最终的模型显示了各种领域的显着改善，例如自然语言处理，计算机视觉和语音识别。我们回顾了稀疏专家模型的概念，提供了对常见算法的基本描述，将深度学习时代的进步进行上下文化，并通过突出未来工作的领域来结束。

大型语言模型在各种任务上显示出令人印象深刻的几次结果。但是，当知识是此类结果的关键时，就像问题回答和事实检查之类的任务一样，似乎需要存储知识的大量参数计数。众所周知，检索增强模型可以在不需要多个参数的情况下在知识密集的任务上表现出色，但是目前尚不清楚它们是否在几个弹药设置中工作。在这项工作中，我们介绍了地图集，这是一个经过精心设计和预先训练的增强语言模型，能够通过很少的培训示例学习知识密集型任务。我们对包括MMLU，苏格兰短裙和归类等各种任务进行评估，并研究文档索引内容的影响，表明它可以很容易地进行更新。值得注意的是，在自然问题上仅使用64个示例在自然问题上达到超过42 \％的准确性，尽管参数少了50倍，但比540B参数模型的表现优于540b参数模型。

专家层（MOES）的混合物通过条件计算实现语言模型的高效缩放。本文提出了一个详细的实证研究，自回归鞋语言模型与广泛的设置中的密集模型相比：在域外语言建模，零和少量射击和全部微调。除了微调外，我们发现Moes基本上更加计算效率。在更适度的培训预算下，MOES可以使用$ \ SIM值4倍的计算，符合密集模型的性能。该差距在比例下变窄，但我们最大的MOE模型（1.1T参数）始终如一地优于计算等效的密集模型（6.7b参数）。总体而言，这种表现差距在任务和域中有很大差异，表明MOE和密集模型以不值得研究的方式概括不同的方式。我们使我们的代码和模型公开可用于研究使用。

Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, but the quality bar for medical and clinical applications is high. Today, attempts to assess models' clinical knowledge typically rely on automated evaluations on limited benchmarks. There is no standard to evaluate model predictions and reasoning across a breadth of tasks. To address this, we present MultiMedQA, a benchmark combining six existing open question answering datasets spanning professional medical exams, research, and consumer queries; and HealthSearchQA, a new free-response dataset of medical questions searched online. We propose a framework for human evaluation of model answers along multiple axes including factuality, precision, possible harm, and bias. In addition, we evaluate PaLM (a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM, on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA, MedMCQA, PubMedQA, MMLU clinical topics), including 67.6% accuracy on MedQA (US Medical License Exam questions), surpassing prior state-of-the-art by over 17%. However, human evaluation reveals key gaps in Flan-PaLM responses. To resolve this we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, recall of knowledge, and medical reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal important limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLM models for clinical applications.

Mathematical reasoning is a fundamental aspect of human intelligence and is applicable in various fields, including science, engineering, finance, and everyday life. The development of artificial intelligence (AI) systems capable of solving math problems and proving theorems has garnered significant interest in the fields of machine learning and natural language processing. For example, mathematics serves as a testbed for aspects of reasoning that are challenging for powerful deep learning models, driving new algorithmic and modeling advances. On the other hand, recent advances in large-scale neural language models have opened up new benchmarks and opportunities to use deep learning for mathematical reasoning. In this survey paper, we review the key tasks, datasets, and methods at the intersection of mathematical reasoning and deep learning over the past decade. We also evaluate existing benchmarks and methods, and discuss future research directions in this domain.

大型语言模型经常经过数十万个计算天的训练，已经显示出零和少数学习的显着功能。鉴于它们的计算成本，如果没有大量资本，这些模型很难复制。对于通过API可用的少数产品，没有访问完整的模型权重，因此很难学习。我们提供开放训练的预训练变压器（OPT），这是一套仅解码器预训练的变压器，范围从12500万到175b参数，我们旨在与感兴趣的研究人员完全和负责任地分享。我们表明，OPT-175B与GPT-3相当，而仅需要1/7碳足迹才能开发。我们还释放了日志，详细介绍了我们面临的基础架构挑战，以及用于尝试所有发布模型的代码。

Reasoning is a fundamental aspect of human intelligence that plays a crucial role in activities such as problem solving, decision making, and critical thinking. In recent years, large language models (LLMs) have made significant progress in natural language processing, and there is observation that these models may exhibit reasoning abilities when they are sufficiently large. However, it is not yet clear to what extent LLMs are capable of reasoning. This paper provides a comprehensive overview of the current state of knowledge on reasoning in LLMs, including techniques for improving and eliciting reasoning in these models, methods and benchmarks for evaluating reasoning abilities, findings and implications of previous research in this field, and suggestions on future directions. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful discussion and future work.

Scaling up language models has led to unprecedented performance gains, but little is understood about how the training dynamics change as models get larger. How do language models of different sizes learn during pre-training? Why do larger language models demonstrate more desirable behaviors? In this paper, we analyze the intermediate training checkpoints of differently sized OPT models (Zhang et al.,2022)--from 125M to 175B parameters--on next-token prediction, sequence-level generation, and downstream tasks. We find that 1) at a given perplexity and independent of model sizes, a similar subset of training tokens see the most significant reduction in loss, with the rest stagnating or showing double-descent behavior; 2) early in training, all models learn to reduce the perplexity of grammatical sequences that contain hallucinations, with small models halting at this suboptimal distribution and larger ones eventually learning to assign these sequences lower probabilities; 3) perplexity is a strong predictor of in-context learning performance on 74 multiple-choice tasks from BIG-Bench, and this holds independent of the model size. Together, these results show that perplexity is more predictive of model behaviors than model size or training computation.

Current large language models can perform reasonably well on complex tasks that require step-by-step reasoning with few-shot learning. Are these models applying reasoning skills they have learnt during pre-training and reason outside of their training context, or are they simply memorizing their training corpus at finer granularity and have learnt to better understand their context? To tease apart these possibilities, we introduce ALERT, a benchmark and suite of analyses for assessing language models' reasoning ability comparing pre-trained and finetuned models on complex tasks that require reasoning skills to solve. ALERT provides a test bed to asses any language model on fine-grained reasoning skills, which spans over 20 datasets and covers 10 different reasoning skills. We leverage ALERT to further investigate the role of finetuning. With extensive empirical analysis we find that language models learn more reasoning skills such as textual entailment, abductive reasoning, and analogical reasoning during finetuning stage compared to pretraining state. We also find that when language models are finetuned they tend to overfit to the prompt template, which hurts the robustness of models causing generalization problems.

几乎没有射击的内在学习（ICL）使预训练的语言模型能够通过为输入的一部分提供少量的培训示例来执行以前的任务，而无需任何基于梯度的培训。 ICL会产生大量的计算，内存和存储成本，因为它每次进行预测时都涉及处理所有培训示例。参数有效的微调（PEFT）（例如，适配器模块，提示调谐，稀疏更新方法等）提供了替代范式，其中训练了一组少量参数以启用模型来执行新任务。在本文中，我们严格地比较了几个ICL和PEFT，并证明后者提供了更好的准确性，并大大降低了计算成本。在此过程中，我们引入了一种称为（IA）$^3 $的新PEFT方法，该方法通过学习的向量来扩展激活，从而获得更强的性能，同时仅引入相对少量的新参数。我们还提出了一个基于称为T-FEW的T0模型的简单食谱，可以将其应用于新任务，而无需特定于任务的调整或修改。我们通过将T-FEW应用于木筏基准，首次实现超人性能，并以6％的绝对性能优于最先进的方法来验证T-FEW对完全看不见的任务的有效性。我们实验中使用的所有代码均可公开使用。

我们表明，在将直接转换应用到数据集之后，自回归语言模型可以学会填充文本，这简单地将文本的跨度从文档的中间移动到了其末尾。虽然近年来这种数据增强引起了人们的极大兴趣，但我们提供了广泛的证据，表明以这种方式转换的数据很大一部分并不会损害原始的左右生成能力，这是通过困惑和抽样评估来衡量的广泛的尺度。鉴于培训模型对中间的有用性，简单性和效率（FIM），我们建议默认情况下使用FIM培训未来的自回归语言模型。为此，我们在关键的超参数上运行一系列消融，例如数据转换频率，转换的结构以及选择填充跨度的方法。我们使用这些消融来规定强大的默认设置和最佳实践来训练FIM模型。我们发布了最佳的填充模型，该模型在API中培训了最佳实践，并发布了我们的填充基准，以帮助未来的研究。

鉴于大型语言模型的广泛能力，应该有可能朝着一般的文本的助手工作，这些助手与人类价值一致，这意味着它是有帮助，诚实的和无害的。在此方向上的初始遗传，我们研究简单的基线技术和评估，例如提示。我们发现，从模型规模增加适度的干预措施的好处，概括为各种对准评估，并不会损害大型模型的性能。接下来，我们调查与对齐，比较仿制，二进制歧视和排名偏好建模相关的几个培训目标的缩放趋势。我们发现排名优先级模型比模仿学习更好地表现得多，并且通常以模型大小更有利地缩放。相比之下，二进制歧视通常与模仿学习非常类似地执行和缩放。最后，我们研究了一种“偏好模型预训练阶段的培训阶段，其目的是在对人偏好的芬明时提高样本效率。