神经辐射场（NERFS）表现出惊人的能力，可以从新颖的观点中综合3D场景的图像。但是，他们依赖于基于射线行进的专门体积渲染算法，这些算法与广泛部署的图形硬件的功能不匹配。本文介绍了基于纹理多边形的新的NERF表示形式，该表示可以有效地与标准渲染管道合成新型图像。 NERF表示为一组多边形，其纹理代表二进制不相处和特征向量。用Z-Buffer对多边形的传统渲染产生了每个像素的图像，该图像由在片段着色器中运行的小型，观点依赖的MLP来解释，以产生最终的像素颜色。这种方法使NERF可以使用传统的Polygon栅格化管道渲染，该管道提供了庞大的像素级并行性，从而在包括移动电话在内的各种计算平台上实现了交互式帧速率。

神经辐射场（NERF）是一种普遍的视图综合技术，其表示作为连续体积函数的场景，由多层的感知来参数化，其提供每个位置处的体积密度和视图相关的发射辐射。虽然基于NERF的技术在代表精细的几何结构时，具有平稳变化的视图依赖性外观，但它们通常无法精确地捕获和再现光泽表面的外观。我们通过引入Ref-nerf来解决这些限制，该ref-nerf替换了nerf的视图依赖性输出辐射的参数化，使用反射辐射的表示和使用空间不同场景属性的集合来构造该函数的表示。我们展示了与正常载体上的规范器一起，我们的模型显着提高了镜面反射的现实主义和准确性。此外，我们表明我们的模型的外向光线的内部表示是可解释的，可用于场景编辑。

神经辐射字段（NERF）是一种用于高质量新颖观看综合的技术从一系列姿势输入图像。与大多数视图合成方法一样，NERF使用TONEMAPPED的低动态范围（LDR）作为输入;这些图像已经通过流畅的相机管道处理，平滑细节，剪辑突出显示，并扭曲了原始传感器数据的简单噪声分布。我们修改NERF以直接在线性原始图像直接培训，保持场景的完整动态范围。通过从生成的NERF渲染原始输出图像，我们可以执行新颖的高动态范围（HDR）视图综合任务。除了改变相机的观点外，我们还可以在事实之后操纵焦点，曝光和调度率。虽然单个原始图像显然比后处理的原始图像显着更大，但我们表明NERF对原始噪声的零平均分布非常强大。当优化许多嘈杂的原始输入（25-200）时，NERF会产生一个场景表示，如此准确的，即其呈现的新颖视图优于在同一宽基线输入图像上运行的专用单个和多像深生物丹机。因此，我们调用Rawnerf的方法可以从近黑暗中捕获的极其嘈杂的图像中重建场景。

虽然神经辐射场（NERF）已经证明了令人印象深刻的视图合成结果对物体和小型空间区域的结果，但它们在“无界”场景上挣扎，其中相机可以在任何方向上点，并且内容在任何距离处都存在。在此设置中，现有的形式的类似形式模型通常会产生模糊或低分辨率渲染（由于附近和远处物体的不平衡细节和规模），慢慢训练，并且由于任务的固有歧义而可能表现出伪影从一小部分图像重建大场景。我们介绍了MIP-NERF（一个NERF变体，用于解决采样和混叠的NERF变体），其使用非线性场景参数化，在线蒸馏和基于新的失真的常规程序来克服无限性场景所呈现的挑战。我们的模型，我们将“MIP-NERF 360”为瞄准相机围绕一点旋转360度的瞄准场景，与MIP NERF相比将平均平方误差减少54％，并且能够产生逼真的合成视图和用于高度复杂，无限性的现实景区的详细深度图。

The rendering procedure used by neural radiance fields (NeRF) samples a scene with a single ray per pixel and may therefore produce renderings that are excessively blurred or aliased when training or testing images observe scene content at different resolutions. The straightforward solution of supersampling by rendering with multiple rays per pixel is impractical for NeRF, because rendering each ray requires querying a multilayer perceptron hundreds of times. Our solution, which we call "mip-NeRF" (à la "mipmap"), extends NeRF to represent the scene at a continuously-valued scale. By efficiently rendering anti-aliased conical frustums instead of rays, mip-NeRF reduces objectionable aliasing artifacts and significantly improves NeRF's ability to represent fine details, while also being 7% faster than NeRF and half the size. Compared to NeRF, mip-NeRF reduces average error rates by 17% on the dataset presented with NeRF and by 60% on a challenging multiscale variant of that dataset that we present. Mip-NeRF is also able to match the accuracy of a brute-force supersampled NeRF on our multiscale dataset while being 22× faster.

Supervised Question Answering systems (QA systems) rely on domain-specific human-labeled data for training. Unsupervised QA systems generate their own question-answer training pairs, typically using secondary knowledge sources to achieve this outcome. Our approach (called PIE-QG) uses Open Information Extraction (OpenIE) to generate synthetic training questions from paraphrased passages and uses the question-answer pairs as training data for a language model for a state-of-the-art QA system based on BERT. Triples in the form of <subject, predicate, object> are extracted from each passage, and questions are formed with subjects (or objects) and predicates while objects (or subjects) are considered as answers. Experimenting on five extractive QA datasets demonstrates that our technique achieves on-par performance with existing state-of-the-art QA systems with the benefit of being trained on an order of magnitude fewer documents and without any recourse to external reference data sources.

This paper presents a machine learning approach to multidimensional item response theory (MIRT), a class of latent factor models that can be used to model and predict student performance from observed assessment data. Inspired by collaborative filtering, we define a general class of models that includes many MIRT models. We discuss the use of penalized joint maximum likelihood (JML) to estimate individual models and cross-validation to select the best performing model. This model evaluation process can be optimized using batching techniques, such that even sparse large-scale data can be analyzed efficiently. We illustrate our approach with simulated and real data, including an example from a massive open online course (MOOC). The high-dimensional model fit to this large and sparse dataset does not lend itself well to traditional methods of factor interpretation. By analogy to recommender-system applications, we propose an alternative "validation" of the factor model, using auxiliary information about the popularity of items consulted during an open-book exam in the course.

We introduce Argoverse 2 (AV2) - a collection of three datasets for perception and forecasting research in the self-driving domain. The annotated Sensor Dataset contains 1,000 sequences of multimodal data, encompassing high-resolution imagery from seven ring cameras, and two stereo cameras in addition to lidar point clouds, and 6-DOF map-aligned pose. Sequences contain 3D cuboid annotations for 26 object categories, all of which are sufficiently-sampled to support training and evaluation of 3D perception models. The Lidar Dataset contains 20,000 sequences of unlabeled lidar point clouds and map-aligned pose. This dataset is the largest ever collection of lidar sensor data and supports self-supervised learning and the emerging task of point cloud forecasting. Finally, the Motion Forecasting Dataset contains 250,000 scenarios mined for interesting and challenging interactions between the autonomous vehicle and other actors in each local scene. Models are tasked with the prediction of future motion for "scored actors" in each scenario and are provided with track histories that capture object location, heading, velocity, and category. In all three datasets, each scenario contains its own HD Map with 3D lane and crosswalk geometry - sourced from data captured in six distinct cities. We believe these datasets will support new and existing machine learning research problems in ways that existing datasets do not. All datasets are released under the CC BY-NC-SA 4.0 license.

The celebrated FedAvg algorithm of McMahan et al. (2017) is based on three components: client sampling (CS), data sampling (DS) and local training (LT). While the first two are reasonably well understood, the third component, whose role is to reduce the number of communication rounds needed to train the model, resisted all attempts at a satisfactory theoretical explanation. Malinovsky et al. (2022) identified four distinct generations of LT methods based on the quality of the provided theoretical communication complexity guarantees. Despite a lot of progress in this area, none of the existing works were able to show that it is theoretically better to employ multiple local gradient-type steps (i.e., to engage in LT) than to rely on a single local gradient-type step only in the important heterogeneous data regime. In a recent breakthrough embodied in their ProxSkip method and its theoretical analysis, Mishchenko et al. (2022) showed that LT indeed leads to provable communication acceleration for arbitrarily heterogeneous data, thus jump-starting the $5^{\rm th}$ generation of LT methods. However, while these latest generation LT methods are compatible with DS, none of them support CS. We resolve this open problem in the affirmative. In order to do so, we had to base our algorithmic development on new algorithmic and theoretical foundations.

Graph clustering is a fundamental problem in unsupervised learning, with numerous applications in computer science and in analysing real-world data. In many real-world applications, we find that the clusters have a significant high-level structure. This is often overlooked in the design and analysis of graph clustering algorithms which make strong simplifying assumptions about the structure of the graph. This thesis addresses the natural question of whether the structure of clusters can be learned efficiently and describes four new algorithmic results for learning such structure in graphs and hypergraphs. All of the presented theoretical results are extensively evaluated on both synthetic and real-word datasets of different domains, including image classification and segmentation, migration networks, co-authorship networks, and natural language processing. These experimental results demonstrate that the newly developed algorithms are practical, effective, and immediately applicable for learning the structure of clusters in real-world data.