由于大规模数据集的可用性，通常在特定位置和良好的天气条件下收集的大规模数据集，近年来，自动驾驶汽车的感知进展已加速。然而，为了达到高安全要求，这些感知系统必须在包括雪和雨在内的各种天气条件下进行稳健运行。在本文中，我们提出了一个新数据集，以通过新颖的数据收集过程启用强大的自动驾驶 - 在不同场景（Urban，Highway，乡村，校园），天气，雪，雨，阳光下，沿着15公里的路线反复记录数据），时间（白天/晚上）以及交通状况（行人，骑自行车的人和汽车）。该数据集包括来自摄像机和激光雷达传感器的图像和点云，以及高精度GPS/ins以在跨路线上建立对应关系。该数据集包括使用Amodal掩码捕获部分遮挡和3D边界框的道路和对象注释。我们通过分析基准在道路和对象，深度估计和3D对象检测中的性能来证明该数据集的独特性。重复的路线为对象发现，持续学习和异常检测打开了新的研究方向。链接到ITHACA365：https：//ithaca365.mae.cornell.edu/

估计物体的距离是自动驾驶的一项安全至关重要的任务。专注于短距离对象，现有方法和数据集忽略了同样重要的远程对象。在本文中，我们引入了一项具有挑战性且探索不足的任务，我们将其称为长距离距离估计，以及两个数据集，以验证为此任务开发的新方法。然后，我们提出了第一个框架，即通过使用场景中已知距离的引用来准确估算远程对象的距离。从人类感知中汲取灵感，R4D通过将目标对象连接到所有引用来构建图形。图中的边缘编码一对目标和参考对象之间的相对距离信息。然后使用注意模块权衡参考对象的重要性，并将它们组合到一个目标对象距离预测中。与现有基准相比，这两个数据集的实验通过显示出显着改善，证明了R4D的有效性和鲁棒性。我们正在寻求制作提出的数据集，Waymo OpenDataSet-远程标签，可在Waymo.com/open/download上公开可用。

由于其在自主驾驶中的应用，因此基于单眼图像的3D感知已成为一个活跃的研究领域。与基于激光雷达的技术相比，单眼3D感知（包括检测和跟踪）的方法通常会产生较低的性能。通过系统的分析，我们确定了每个对象深度估计精度是界限性能的主要因素。在这种观察过程中，我们提出了一种多级融合方法，该方法将不同的表示（RGB和伪LIDAR）和跨多个对象（Tracklets）的时间信息结合在一起，以增强对目标深度估计。我们提出的融合方法实现了Waymo打开数据集，KITTI检测数据集和Kitti MOT数据集的每个对象深度估计的最新性能。我们进一步证明，通过简单地用融合增强的深度替换估计的深度，我们可以在单眼3D感知任务（包括检测和跟踪）方面取得重大改进。

自动驾驶汽车必须在3D中检测其他车辆和行人，以计划安全路线并避免碰撞。基于深度学习的最先进的3D对象探测器已显示出有希望的准确性，但容易过度拟合域特质，使它们在新环境中失败 - 如果自动驾驶汽车旨在自动操作，则是一个严重的问题。在本文中，我们提出了一种新颖的学习方法，该方法通过在目标域中的伪标记上微调检测器，从而大大减少这一差距，我们的方法在车辆停放时会根据先前记录的驾驶序列的重播而生成的差距。在这些重播中，随着时间的推移会跟踪对象，并且检测被插值和外推 - 至关重要的是利用未来的信息来捕获硬病例。我们在五个自动驾驶数据集上显示，对这些伪标签上的对象检测器进行微调大大减少了域间隙到新的驾驶环境，从而极大地提高了准确性和检测可靠性。

New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments, including deep learning training and inference characterization on MIG, GPU sharing characterization, and framework compatibility with MIG. The results of these experiments provide new insights and guidance for users to effectively employ MIG, and lay the foundation for further research on the orchestration of hybrid training and inference workloads on MIGs. The code and results are released on https://github.com/MLSysOps/MIGProfiler. This work is still in progress and more results will be published soon.

Learning to predict masked tokens in a sequence has been shown to be a powerful pretraining objective for large-scale language models. After training, such masked language models can provide distributions of tokens conditioned on bidirectional context. In this short draft, we show that such bidirectional conditionals often demonstrate considerable inconsistencies, i.e., they can not be derived from a coherent joint distribution when considered together. We empirically quantify such inconsistencies in the simple scenario of bigrams for two common styles of masked language models: T5-style and BERT-style. For example, we show that T5 models often confuse its own preference regarding two similar bigrams. Such inconsistencies may represent a theoretical pitfall for the research work on sampling sequences based on the bidirectional conditionals learned by BERT-style MLMs. This phenomenon also means that T5-style MLMs capable of infilling will generate discrepant results depending on how much masking is given, which may represent a particular trust issue.

This paper presents a practical global optimization algorithm for the K-center clustering problem, which aims to select K samples as the cluster centers to minimize the maximum within-cluster distance. This algorithm is based on a reduced-space branch and bound scheme and guarantees convergence to the global optimum in a finite number of steps by only branching on the regions of centers. To improve efficiency, we have designed a two-stage decomposable lower bound, the solution of which can be derived in a closed form. In addition, we also propose several acceleration techniques to narrow down the region of centers, including bounds tightening, sample reduction, and parallelization. Extensive studies on synthetic and real-world datasets have demonstrated that our algorithm can solve the K-center problems to global optimal within 4 hours for ten million samples in the serial mode and one billion samples in the parallel mode. Moreover, compared with the state-of-the-art heuristic methods, the global optimum obtained by our algorithm can averagely reduce the objective function by 25.8% on all the synthetic and real-world datasets.

Through a study of multi-gas mixture datasets, we show that in multi-component spectral analysis, the number of functional or non-functional principal components required to retain the essential information is the same as the number of independent constituents in the mixture set. Due to the mutual in-dependency among different gas molecules, near one-to-one projection from the principal component to the mixture constituent can be established, leading to a significant simplification of spectral quantification. Further, with the knowledge of the molar extinction coefficients of each constituent, a complete principal component set can be extracted from the coefficients directly, and few to none training samples are required for the learning model. Compared to other approaches, the proposed methods provide fast and accurate spectral quantification solutions with a small memory size needed.

Neural operators, which emerge as implicit solution operators of hidden governing equations, have recently become popular tools for learning responses of complex real-world physical systems. Nevertheless, the majority of neural operator applications has thus far been data-driven, which neglects the intrinsic preservation of fundamental physical laws in data. In this paper, we introduce a novel integral neural operator architecture, to learn physical models with fundamental conservation laws automatically guaranteed. In particular, by replacing the frame-dependent position information with its invariant counterpart in the kernel space, the proposed neural operator is by design translation- and rotation-invariant, and consequently abides by the conservation laws of linear and angular momentums. As applications, we demonstrate the expressivity and efficacy of our model in learning complex material behaviors from both synthetic and experimental datasets, and show that, by automatically satisfying these essential physical laws, our learned neural operator is not only generalizable in handling translated and rotated datasets, but also achieves state-of-the-art accuracy and efficiency as compared to baseline neural operator models.

Diagram object detection is the key basis of practical applications such as textbook question answering. Because the diagram mainly consists of simple lines and color blocks, its visual features are sparser than those of natural images. In addition, diagrams usually express diverse knowledge, in which there are many low-frequency object categories in diagrams. These lead to the fact that traditional data-driven detection model is not suitable for diagrams. In this work, we propose a gestalt-perception transformer model for diagram object detection, which is based on an encoder-decoder architecture. Gestalt perception contains a series of laws to explain human perception, that the human visual system tends to perceive patches in an image that are similar, close or connected without abrupt directional changes as a perceptual whole object. Inspired by these thoughts, we build a gestalt-perception graph in transformer encoder, which is composed of diagram patches as nodes and the relationships between patches as edges. This graph aims to group these patches into objects via laws of similarity, proximity, and smoothness implied in these edges, so that the meaningful objects can be effectively detected. The experimental results demonstrate that the proposed GPTR achieves the best results in the diagram object detection task. Our model also obtains comparable results over the competitors in natural image object detection.