As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (which is time-consuming and expensive) or existing data sources (which are not always available). Here, we automatically generate evaluations with LMs. We explore approaches with varying amounts of human effort, from instructing LMs to write yes/no questions to making complex Winogender schemas with multiple stages of LM-based generation and filtering. Crowdworkers rate the examples as highly relevant and agree with 90-100% of labels, sometimes more so than corresponding human-written datasets. We generate 154 datasets and discover new cases of inverse scaling where LMs get worse with size. Larger LMs repeat back a dialog user's preferred answer ("sycophancy") and express greater desire to pursue concerning goals like resource acquisition and goal preservation. We also find some of the first examples of inverse scaling in RL from Human Feedback (RLHF), where more RLHF makes LMs worse. For example, RLHF makes LMs express stronger political views (on gun rights and immigration) and a greater desire to avoid shut down. Overall, LM-written evaluations are high-quality and let us quickly discover many novel LM behaviors.

Many high-level skills that are required for computer vision tasks, such as parsing questions, comparing and contrasting semantics, and writing descriptions, are also required in other domains such as natural language processing. In this paper, we ask whether this makes it possible to learn those skills from text data and then use them to complete vision tasks without ever training on visual training data. Key to our approach is exploiting the joint embedding space of contrastively trained vision and language encoders. In practice, there can be systematic differences between embedding spaces for different modalities in contrastive models, and we analyze how these differences affect our approach and study a variety of strategies to mitigate this concern. We produce models using only text training data on three tasks: image captioning, visual entailment and visual question answering, and evaluate them on standard benchmarks using images. We find that this kind of transfer is possible and results in only a small drop in performance relative to models trained on images. We also showcase a variety of stylistic image captioning models that were trained using no image data and no human-curated language data, but instead text data from books, the web, or language models.

Reinforcement learning (RL) operating on attack graphs leveraging cyber terrain principles are used to develop reward and state associated with determination of surveillance detection routes (SDR). This work extends previous efforts on developing RL methods for path analysis within enterprise networks. This work focuses on building SDR where the routes focus on exploring the network services while trying to evade risk. RL is utilized to support the development of these routes by building a reward mechanism that would help in realization of these paths. The RL algorithm is modified to have a novel warm-up phase which decides in the initial exploration which areas of the network are safe to explore based on the rewards and penalty scale factor.

我们提出了Unified-io，该模型执行了跨越经典计算机视觉任务的各种AI任务，包括姿势估计，对象检测，深度估计和图像生成，视觉和语言任务，例如区域字幕和引用表达理解，并引用表达理解，进行自然语言处理任务，例如回答和释义。由于与每个任务有关的异质输入和输出，包括RGB图像，每个像素映射，二进制掩码，边界框和语言，开发一个统一模型引起了独特的挑战。我们通过将每个受支持的输入和输出均匀地均匀地统一到一系列离散的词汇令牌来实现这一统一。在所有任务中，这种共同的表示使我们能够在视觉和语言字段中的80多个不同数据集上培训单个基于变压器的体系结构。 Unified-io是第一个能够在砂砾基准上执行所有7个任务的模型，并在NYUV2-DEPTH，Imagenet，VQA2.0，OK-VQA，SWIG，SWIG，VIZWIZ，BOOLQ，BOOLQ和SCITAIL，带有NYUV2-DEPTH，Imagenet，VQA2.0，诸如NYUV2-DEPTH，ImageNet，vqa2.0等16个不同的基准中产生强大的结果。没有任务或基准特定的微调。 unified-io的演示可在https://unified-io.allenai.org上获得。

通用视觉（GPV）系统是旨在解决各种视觉任务的模型，而无需进行架构更改。如今，GPV主要从大型完全监督的数据集中学习技能和概念。通过获取数据以迅速学习每个技能的每个概念，将GPV扩展到数万个概念都变得令人望而却步。这项工作提出了一种有效且廉价的替代方法：从监督数据集中学习技能，从Web图像搜索中学习概念，并利用GPV的关键特征：跨技能传递视觉知识的能力。我们使用跨越10K+视觉概念的1M+图像的数据集来演示3个基准上的两个现有GPV（GPV-1和VL-T5）的Webly Supumented概念扩展：5个基于可可的数据集（80个主要概念），这是一个新的策划系列，这是一个新的策划系列。基于OpenImages和VisualGenome存储库（〜500个概念）以及Web衍生的数据集（10K+概念）的5个数据集。我们还提出了一种新的体系结构GPV-2，该架构支持各种任务 - 从分类和本地化等视觉任务到Qu Viewer+语言任务，例如QA和字幕，再到更多的利基市场，例如人类对象互动检测。 GPV-2从Web数据中受益匪浅，并且在这些基准测试中胜过GPV-1和VL-T5。我们的数据，代码和Web演示可在https://prior.allenai.org/projects/gpv2上获得。

我们呈现Nureality，一个虚拟现实'VR'环境，旨在测试车辆行为在城市交叉路口自主车辆和行人之间的相互作用中沟通意图的效果。在这个项目中，我们专注于表达行为作为行人的手段，即易于认识到AV运动的潜在意图。 VR是用于测试这些情况的理想工具，因为它可以被沉浸，并将受试者放入这些潜在的危险情景中而没有风险。 Nureality提供了一种新颖的和沉浸式虚拟现实环境，包括众多视觉细节（道路和建筑纹理，停放的汽车，摇曳的树肢）以及听觉细节（鸟儿唧唧喳喳，距离距离的汽车）。在这些文件中，我们呈现Nureality环境，其10个独特的车辆行为场景，以及每个场景的虚幻引擎和Autodesk Maya源文件。这些文件在www.nureality.org上公开发布为开源，以支持学术界，研究临界公平互动。

与人类沟通对AIS有挑战性，因为它需要对世界的共同理解，复杂的语义（例如，隐喻或类似物），并且在多码模态手势（例如，指向手指，或图中的箭头）。我们在基于图案的基础上的绘画和猜测的语境中调查了这些挑战，这对研究界构成了一种新的挑战。在ICONARY中，猜测者试图通过编写图标来识别抽屉绘制的短语，以及抽屉迭代地修改绘图以帮助猜测响应的猜测。这次来回经常使用规范场景，视觉隐喻或图标组成来表达具有挑战性的词语，使其成为AI中混合语言和视觉/象征性通信的理想测试。我们提出模型进行图标，并在人类球员之间的55,000多场比赛中培训。我们的型号是熟练的玩家，能够在语言模型中雇用世界知识，以便在训练期间与看不见的文字一起玩。精英人类球员优于我们的模型，特别是在绘图任务中，留下了未来研究的重要缺口。我们将数据集，代码和评估设置释放为对社区的挑战http://www.github.com/allenai/conary。

Masked language modeling (MLM) pre-training methods such as BERT corrupt the input by replacing some tokens with [MASK] and then train a model to reconstruct the original tokens. While they produce good results when transferred to downstream NLP tasks, they generally require large amounts of compute to be effective. As an alternative, we propose a more sample-efficient pre-training task called replaced token detection. Instead of masking the input, our approach corrupts it by replacing some tokens with plausible alternatives sampled from a small generator network. Then, instead of training a model that predicts the original identities of the corrupted tokens, we train a discriminative model that predicts whether each token in the corrupted input was replaced by a generator sample or not. Thorough experiments demonstrate this new pre-training task is more efficient than MLM because the task is defined over all input tokens rather than just the small subset that was masked out. As a result, the contextual representations learned by our approach substantially outperform the ones learned by BERT given the same model size, data, and compute. The gains are particularly strong for small models; for example, we train a model on one GPU for 4 days that outperforms GPT (trained using 30x more compute) on the GLUE natural language understanding benchmark. Our approach also works well at scale, where it performs comparably to RoBERTa and XLNet while using less than 1/4 of their compute and outperforms them when using the same amount of compute.

Large pre-trained neural networks such as BERT have had great recent success in NLP, motivating a growing body of research investigating what aspects of language they are able to learn from unlabeled data. Most recent analysis has focused on model outputs (e.g., language model surprisal) or internal vector representations (e.g., probing classifiers). Complementary to these works, we propose methods for analyzing the attention mechanisms of pre-trained models and apply them to BERT. BERT's attention heads exhibit patterns such as attending to delimiter tokens, specific positional offsets, or broadly attending over the whole sentence, with heads in the same layer often exhibiting similar behaviors. We further show that certain attention heads correspond well to linguistic notions of syntax and coreference. For example, we find heads that attend to the direct objects of verbs, determiners of nouns, objects of prepositions, and coreferent mentions with remarkably high accuracy. Lastly, we propose an attention-based probing classifier and use it to further demonstrate that substantial syntactic information is captured in BERT's attention. 1 Code will be released at https://github.com/ clarkkev/attention-analysis.2 We use the English base-sized model.

We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Our word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pretrained on a large text corpus. We show that these representations can be easily added to existing models and significantly improve the state of the art across six challenging NLP problems, including question answering, textual entailment and sentiment analysis. We also present an analysis showing that exposing the deep internals of the pre-trained network is crucial, allowing downstream models to mix different types of semi-supervision signals.