差异隐私（DP）提供了正式的隐私保证，以防止对手可以访问机器学习模型，从而从提取有关单个培训点的信息。最受欢迎的DP训练方法是差异私有随机梯度下降（DP-SGD），它通过在训练过程中注入噪声来实现这种保护。然而，以前的工作发现，DP-SGD通常会导致标准图像分类基准的性能显着降解。此外，一些作者假设DP-SGD在大型模型上固有地表现不佳，因为保留隐私所需的噪声规范与模型维度成正比。相反，我们证明了过度参数化模型上的DP-SGD可以比以前想象的要好得多。将仔细的超参数调整与简单技术结合起来，以确保信号传播并提高收敛速率，我们获得了新的SOTA，而没有额外数据的CIFAR-10，在81.4％的81.4％下（8，10^{ - 5}） - 使用40 -layer wide-Resnet，比以前的SOTA提高了71.7％。当对预训练的NFNET-F3进行微调时，我们在ImageNet（0.5，8*10^{ - 7}）下达到了83.8％的TOP-1精度。此外，我们还在（8，8 \ cdot 10^{ - 7}）下达到了86.7％的TOP-1精度，DP仅比当前的非私人SOTA仅4.3％。我们认为，我们的结果是缩小私人图像分类和非私有图像分类之间准确性差距的重要一步。

大多数现实世界的应用需要处理传感器噪声或预测性不确定性等旋能性，其中正式规格的所需行为是固有的概率。尽管正式核查确保神经网络的可靠性，但概率规格方向的进展受到限制。在这个方向上，我们首先介绍神经网络的概率规范的一般性，它捕获了概率网络（例如，贝叶斯神经网络，MC-Dropout Networks）和不确定输入（通过传感器噪声或其他扰动而产生的输入）。然后，我们提出了一种通过概括拉格朗日二元性的概念来验证这些规范的一般技术，替换具有“功能乘法器”的标准拉格朗日乘法器，其可以是给定层上激活的任意功能。我们表明，功能乘法器的最佳选择导致精确的验证（即，声音和完全验证），以及特定形式的乘法器，我们开发了易诊的实际验证算法。我们通过将它们应用于贝叶斯神经网络（BNNS）和MC差动网络，以及认证属性，以及诸如对分发超出（OOD）数据的抗逆性鲁棒性和鲁棒检测的认证性能来验证我们的算法。在这些任务中，与现有工作相比，我们能够提供明显更强烈的保证 - 例如，对于在CiFar-10上培训的VGG-64 MC-Tropout CNN，我们改进了认证的AUC（真实AUC的验证的下限）对于鲁棒的OOD检测（在CIFAR-100上）起价$ 0 \％\ lightarrow 29 \％$。同样，对于在MNIST培训的BNN，我们从60.2美元\％\ lightarrow 74.6 \％$提高了强大的准确性。此外，在一种新颖的规范 - 分布稳健的检测 - 我们从5 \％\ lightarrow 23 \％$的5 \％$。

Reading comprehension of legal text can be a particularly challenging task due to the length and complexity of legal clauses and a shortage of expert-annotated datasets. To address this challenge, we introduce the Merger Agreement Understanding Dataset (MAUD), an expert-annotated reading comprehension dataset based on the American Bar Association's 2021 Public Target Deal Points Study, with over 39,000 examples and over 47,000 total annotations. Our fine-tuned Transformer baselines show promising results, with models performing well above random on most questions. However, on a large subset of questions, there is still room for significant improvement. As the only expert-annotated merger agreement dataset, MAUD is valuable as a benchmark for both the legal profession and the NLP community.

Recently, learning-based controllers have been shown to push mobile robotic systems to their limits and provide the robustness needed for many real-world applications. However, only classical optimization-based control frameworks offer the inherent flexibility to be dynamically adjusted during execution by, for example, setting target speeds or actuator limits. We present a framework to overcome this shortcoming of neural controllers by conditioning them on an auxiliary input. This advance is enabled by including a feature-wise linear modulation layer (FiLM). We use model-free reinforcement-learning to train quadrotor control policies for the task of navigating through a sequence of waypoints in minimum time. By conditioning the policy on the maximum available thrust or the viewing direction relative to the next waypoint, a user can regulate the aggressiveness of the quadrotor's flight during deployment. We demonstrate in simulation and in real-world experiments that a single control policy can achieve close to time-optimal flight performance across the entire performance envelope of the robot, reaching up to 60 km/h and 4.5g in acceleration. The ability to guide a learned controller during task execution has implications beyond agile quadrotor flight, as conditioning the control policy on human intent helps safely bringing learning based systems out of the well-defined laboratory environment into the wild.

Double-blind peer review is considered a pillar of academic research because it is perceived to ensure a fair, unbiased, and fact-centered scientific discussion. Yet, experienced researchers can often correctly guess from which research group an anonymous submission originates, biasing the peer-review process. In this work, we present a transformer-based, neural-network architecture that only uses the text content and the author names in the bibliography to atttribute an anonymous manuscript to an author. To train and evaluate our method, we created the largest authorship-identification dataset to date. It leverages all research papers publicly available on arXiv amounting to over 2 million manuscripts. In arXiv-subsets with up to 2,000 different authors, our method achieves an unprecedented authorship attribution accuracy, where up to 95% of papers are attributed correctly. Thanks to our method, we are not only able to predict the author of an anonymous work but we also identify weaknesses of the double-blind review process by finding the key aspects that make a paper attributable. We believe that this work gives precious insights into how a submission can remain anonymous in order to support an unbiased double-blind review process.

Standard language model training employs gold human documents or human-human interaction data, and treats all training data as positive examples. Growing evidence shows that even with very large amounts of positive training data, issues remain that can be alleviated with relatively small amounts of negative data -- examples of what the model should not do. In this work, we propose a novel procedure to train with such data called the CRINGE loss (ContRastive Iterative Negative GEneration). We show the effectiveness of this approach across three different experiments on the tasks of safe generation, contradiction avoidance, and open-domain dialogue. Our models outperform multiple strong baselines and are conceptually simple, easy to train and implement.

The use of multilingual language models for tasks in low and high-resource languages has been a success story in deep learning. In recent times, Arabic has been receiving widespread attention on account of its dialectal variance. While prior research studies have tried to adapt these multilingual models for dialectal variants of Arabic, it still remains a challenging problem owing to the lack of sufficient monolingual dialectal data and parallel translation data of such dialectal variants. It remains an open problem on whether the limited dialectical data can be used to improve the models trained in Arabic on its dialectal variants. First, we show that multilingual-BERT (mBERT) incrementally pretrained on Arabic monolingual data takes less training time and yields comparable accuracy when compared to our custom monolingual Arabic model and beat existing models (by an avg metric of +$6.41$). We then explore two continual pre-training methods-- (1) using small amounts of dialectical data for continual finetuning and (2) parallel Arabic to English data and a Translation Language Modeling loss function. We show that both approaches help improve performance on dialectal classification tasks ($+4.64$ avg. gain) when used on monolingual models.

We present a new NLP task and dataset from the domain of the U.S. civil procedure. Each instance of the dataset consists of a general introduction to the case, a particular question, and a possible solution argument, accompanied by a detailed analysis of why the argument applies in that case. Since the dataset is based on a book aimed at law students, we believe that it represents a truly complex task for benchmarking modern legal language models. Our baseline evaluation shows that fine-tuning a legal transformer provides some advantage over random baseline models, but our analysis reveals that the actual ability to infer legal arguments remains a challenging open research question.

在许多现实世界中，当不二维测量值时，可能会提供自由旋转3D刚体（例如卫星）的图像观察。但是，图像数据的高维度排除了学习动力学和缺乏解释性的使用，从而降低了标准深度学习方法的有用性。在这项工作中，我们提出了一个物理知识的神经网络模型，以估计和预测图像序列中的3D旋转动力学。我们使用多阶段预测管道实现了这一目标，该管道将单个图像映射到潜在表示同构为$ \ Mathbf {so}（3）$，从潜在对计算角速度，并使用Hamiltonian Motion使用Hamiltonian运动方程来预测未来的潜在状态博学的哈密顿人的代表。我们证明了方法对新的旋转刚体数据集的功效，该数据集具有旋转立方体和矩形棱镜序列，并具有均匀且不均匀的密度。

ICECUBE是一种用于检测1 GEV和1 PEV之间大气和天体中微子的光学传感器的立方公斤阵列，该阵列已部署1.45 km至2.45 km的南极的冰盖表面以下1.45 km至2.45 km。来自ICE探测器的事件的分类和重建在ICeCube数据分析中起着核心作用。重建和分类事件是一个挑战，这是由于探测器的几何形状，不均匀的散射和冰中光的吸收，并且低于100 GEV的光，每个事件产生的信号光子数量相对较少。为了应对这一挑战，可以将ICECUBE事件表示为点云图形，并将图形神经网络（GNN）作为分类和重建方法。 GNN能够将中微子事件与宇宙射线背景区分开，对不同的中微子事件类型进行分类，并重建沉积的能量，方向和相互作用顶点。基于仿真，我们提供了1-100 GEV能量范围的比较与当前ICECUBE分析中使用的当前最新最大似然技术，包括已知系统不确定性的影响。对于中微子事件分类，与当前的IceCube方法相比，GNN以固定的假阳性速率（FPR）提高了信号效率的18％。另外，GNN在固定信号效率下将FPR的降低超过8（低于半百分比）。对于能源，方向和相互作用顶点的重建，与当前最大似然技术相比，分辨率平均提高了13％-20％。当在GPU上运行时，GNN能够以几乎是2.7 kHz的中位数ICECUBE触发速率的速率处理ICECUBE事件，这打开了在在线搜索瞬态事件中使用低能量中微子的可能性。