近年来，自主驾驶LIDAR数据的3D对象检测一直在迈出卓越的进展。在最先进的方法中，已经证明了将点云进行编码为鸟瞰图（BEV）是有效且有效的。与透视图不同，BEV在物体之间保留丰富的空间和距离信息;虽然在BEV中相同类型的更远物体不会较小，但它们包含稀疏点云特征。这一事实使用共享卷积神经网络削弱了BEV特征提取。为了解决这一挑战，我们提出了范围感知注意网络（RAANET），提取更强大的BEV功能并产生卓越的3D对象检测。范围感知的注意力（RAA）卷曲显着改善了近距离的特征提取。此外，我们提出了一种新的辅助损耗，用于密度估计，以进一步增强覆盖物体的Raanet的检测精度。值得注意的是，我们提出的RAA卷积轻量级，并兼容，以集成到用于BEV检测的任何CNN架构中。 Nuscenes DataSet上的广泛实验表明，我们的提出方法优于基于LIDAR的3D对象检测的最先进的方法，具有16 Hz的实时推断速度，为LITE版本为22 Hz。该代码在匿名GitHub存储库HTTPS://github.com/Anonymous0522 / ange上公开提供。

近年来，由于深度学习技术的发展，LiDar Point Clouds的3D对象检测取得了长足的进步。尽管基于体素或基于点的方法在3D对象检测中很受欢迎，但它们通常涉及耗时的操作，例如有关体素的3D卷积或点之间的球查询，从而使所得网络不适合时间关键应用程序。另一方面，基于2D视图的方法具有较高的计算效率，而通常比基于体素或基于点的方法获得的性能低。在这项工作中，我们提出了一个基于实时视图的单阶段3D对象检测器，即CVFNET完成此任务。为了在苛刻的效率条件下加强跨视图的学习，我们的框架提取了不同视图的特征，并以有效的渐进式方式融合了它们。我们首先提出了一个新颖的点范围特征融合模块，该模块在多个阶段深入整合点和范围视图特征。然后，当将所获得的深点视图转换为鸟类视图时，特殊的切片柱旨在很好地维护3D几何形状。为了更好地平衡样品比率，提出了一个稀疏的柱子检测头，将检测集中在非空网上。我们对流行的Kitti和Nuscenes基准进行了实验，并以准确性和速度来实现最先进的性能。

3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications. In this paper, we extend our preliminary work PointRCNN to a novel and strong point-cloud-based 3D object detection framework, the part-aware and aggregation neural network (Part-A 2 net). The whole framework consists of the part-aware stage and the part-aggregation stage. Firstly, the part-aware stage for the first time fully utilizes free-of-charge part supervisions derived from 3D ground-truth boxes to simultaneously predict high quality 3D proposals and accurate intra-object part locations. The predicted intra-object part locations within the same proposal are grouped by our new-designed RoI-aware point cloud pooling module, which results in an effective representation to encode the geometry-specific features of each 3D proposal. Then the part-aggregation stage learns to re-score the box and refine the box location by exploring the spatial relationship of the pooled intra-object part locations. Extensive experiments are conducted to demonstrate the performance improvements from each component of our proposed framework. Our Part-A 2 net outperforms all existing 3D detection methods and achieves new state-of-the-art on KITTI 3D object detection dataset by utilizing only the LiDAR point cloud data. Code is available at https://github.com/sshaoshuai/PointCloudDet3D.

We present AVOD, an Aggregate View Object Detection network for autonomous driving scenarios. The proposed neural network architecture uses LIDAR point clouds and RGB images to generate features that are shared by two subnetworks: a region proposal network (RPN) and a second stage detector network. The proposed RPN uses a novel architecture capable of performing multimodal feature fusion on high resolution feature maps to generate reliable 3D object proposals for multiple object classes in road scenes. Using these proposals, the second stage detection network performs accurate oriented 3D bounding box regression and category classification to predict the extents, orientation, and classification of objects in 3D space. Our proposed architecture is shown to produce state of the art results on the KITTI 3D object detection benchmark [1] while running in real time with a low memory footprint, making it a suitable candidate for deployment on autonomous vehicles. Code is at: https://github.com/kujason/avod

We address the problem of real-time 3D object detection from point clouds in the context of autonomous driving. Computation speed is critical as detection is a necessary component for safety. Existing approaches are, however, expensive in computation due to high dimensionality of point clouds. We utilize the 3D data more efficiently by representing the scene from the Bird's Eye View (BEV), and propose PIXOR, a proposal-free, single-stage detector that outputs oriented 3D object estimates decoded from pixelwise neural network predictions. The input representation, network architecture, and model optimization are especially designed to balance high accuracy and real-time efficiency. We validate PIXOR on two datasets: the KITTI BEV object detection benchmark, and a large-scale 3D vehicle detection benchmark. In both datasets we show that the proposed detector surpasses other state-of-the-art methods notably in terms of Average Precision (AP), while still runs at > 28 FPS.

In this paper, we propose a novel 3D object detector that can exploit both LIDAR as well as cameras to perform very accurate localization. Towards this goal, we design an end-to-end learnable architecture that exploits continuous convolutions to fuse image and LIDAR feature maps at different levels of resolution. Our proposed continuous fusion layer encode both discrete-state image features as well as continuous geometric information. This enables us to design a novel, reliable and efficient end-to-end learnable 3D object detector based on multiple sensors. Our experimental evaluation on both KITTI as well as a large scale 3D object detection benchmark shows significant improvements over the state of the art.

我们呈现PIFENET，一种高效准确的实时3D探测器，用于点云的行人检测。我们解决了在检测行人时遇到的3D对象检测框架的两个挑战：Partion云中的柱特征的表达力量和小型行人的小占领区。首先，我们引入了一个可堆叠的柱子感知注意力（PAA）模块，用于增强的柱子特征提取，同时抑制点云中的噪声。通过将多点感知池，点亮，通道和任务感知注意与到一个简单的模块集成到一个简单的模块，在需要几乎额外的计算资源的同时提高表示功能。我们还存在Mini-Bifpn，一个小而有效的特征网络，创建双向信息流和多级串尺度特征融合，以更好地集成多分辨率功能。我们的方法在Kitti Peistrian Bev和3D排行榜中排名第一，同时以每秒26帧（FPS）运行，并在Nuscenes检测基准上实现最先进的性能。

它得到了很好的认识到，从深度感知的LIDAR点云和语义富有的立体图像中融合互补信息将有利于3D对象检测。然而，探索稀疏3D点和密集2D像素之间固有的不自然相互作用并不重要。为了简化这种困难，最近的建议通常将3D点投影到2D图像平面上以对图像数据进行采样，然后聚合点处的数据。然而，这种方法往往遭受点云和RGB图像的分辨率之间的不匹配，导致次优性能。具体地，作为多模态数据聚合位置的稀疏点导致高分辨率图像的严重信息丢失，这反过来破坏了多传感器融合的有效性。在本文中，我们呈现VPFNET - 一种新的架构，可以在“虚拟”点处巧妙地对齐和聚合点云和图像数据。特别地，它们的密度位于3D点和2D像素的密度之间，虚拟点可以很好地桥接两个传感器之间的分辨率间隙，从而保持更多信息以进行处理。此外，我们还研究了可以应用于点云和RGB图像的数据增强技术，因为数据增强对迄今为止对3D对象探测器的贡献不可忽略。我们对Kitti DataSet进行了广泛的实验，与最先进的方法相比，观察到了良好的性能。值得注意的是，我们的VPFNET在KITTI测试集上实现了83.21 \％中等3D AP和91.86 \％适度的BEV AP，自2021年5月21日起排名第一。网络设计也考虑了计算效率 - 我们可以实现FPS 15对单个NVIDIA RTX 2080TI GPU。该代码将用于复制和进一步调查。

Aiming at highly accurate object detection for connected and automated vehicles (CAVs), this paper presents a Deep Neural Network based 3D object detection model that leverages a three-stage feature extractor by developing a novel LIDAR-Camera fusion scheme. The proposed feature extractor extracts high-level features from two input sensory modalities and recovers the important features discarded during the convolutional process. The novel fusion scheme effectively fuses features across sensory modalities and convolutional layers to find the best representative global features. The fused features are shared by a two-stage network: the region proposal network (RPN) and the detection head (DH). The RPN generates high-recall proposals, and the DH produces final detection results. The experimental results show the proposed model outperforms more recent research on the KITTI 2D and 3D detection benchmark, particularly for distant and highly occluded instances.

人的大脑可以毫不费力地识别和定位对象，而基于激光雷达点云的当前3D对象检测方法仍然报告了较低的性能，以检测闭塞和远处的对象：点云的外观由于遮挡而变化很大，并且在沿线的固有差异沿点固有差异变化。传感器的距离。因此，设计功能表示对此类点云至关重要。受到人类联想识别的启发，我们提出了一个新颖的3D检测框架，该框架通过域的适应来使对象完整特征。我们弥合感知域之间的差距，其中特征是从具有亚最佳表示的真实场景中得出的，以及概念域，其中功能是从由不批准对象组成的增强场景中提取的，并具有丰富的详细信息。研究了一种可行的方法，可以在没有外部数据集的情况下构建概念场景。我们进一步介绍了一个基于注意力的重新加权模块，该模块可适应地增强更翔实区域的特征。该网络的功能增强能力将被利用，而无需在推理过程中引入额外的成本，这是各种3D检测框架中的插件。我们以准确性和速度都在Kitti 3D检测基准上实现了新的最先进性能。关于Nuscenes和Waymo数据集的实验也验证了我们方法的多功能性。

LIDAR的准确3D对象检测对于自动驾驶至关重要。现有的研究全都基于平坦的假设。但是，实际的道路可能会在陡峭的部分中很复杂，从而打破了前提。在这种情况下，当前方法由于难以正确检测到倾斜的地形上的物体而受到性能降解。在这项工作中，我们提出了DET6D，这是第一个没有空间和姿势局限性的自由度3D对象检测器，以改善地形鲁棒性。我们通过建立在整个空间范围内检测对象的能力来选择基于点的框架。为了预测包括音高和滚动在内的全程姿势，我们设计了一个利用当地地面约束的地面方向分支。鉴于长尾非平板场景数据收集和6D姿势注释的难度，我们提出了斜坡，这是一种数据增强方法，用于从平面场景中记录的现有数据集中合成非平板地形。各种数据集的实验证明了我们方法在不同地形上的有效性和鲁棒性。我们进一步进行了扩展实验，以探索网络如何预测两个额外的姿势。提出的模块是现有基于点的框架的插件。该代码可在https://github.com/hitsz-nrsl/de6d上找到。

这项工作旨在通过使用路边激光射击环境的3D感知来应对自动驾驶的挑战。我们设计了一个3D对象检测模型，该模型可以实时检测路边激光雷达的交通参与者。我们的模型使用现有的3D检测器作为基线并提高其准确性。为了证明我们提出的模块的有效性，我们在三个不同的车辆和基础设施数据集上训练和评估模型。为了显示我们探测器的域适应能力，我们在来自中国的基础架构数据集上训练它，并在德国记录的其他数据集上进行转移学习。我们为检测器中每个模块进行几套实验和消融研究，这些实验表明我们的模型的表现优于基线，而推理速度为45 Hz（22 ms）。我们对基于激光雷达的3D探测器做出了重大贡献，可用于智能城市应用程序，以提供连接和自动化的车辆具有深远的视野。连接到路边传感器的车辆可以获取有关拐角处其他车辆的信息，以改善其道路和操纵计划并提高道路交通安全性。

Object detection in point clouds is an important aspect of many robotics applications such as autonomous driving. In this paper we consider the problem of encoding a point cloud into a format appropriate for a downstream detection pipeline. Recent literature suggests two types of encoders; fixed encoders tend to be fast but sacrifice accuracy, while encoders that are learned from data are more accurate, but slower. In this work we propose PointPillars, a novel encoder which utilizes PointNets to learn a representation of point clouds organized in vertical columns (pillars). While the encoded features can be used with any standard 2D convolutional detection architecture, we further propose a lean downstream network. Extensive experimentation shows that PointPillars outperforms previous encoders with respect to both speed and accuracy by a large margin. Despite only using lidar, our full detection pipeline significantly outperforms the state of the art, even among fusion methods, with respect to both the 3D and bird's eye view KITTI benchmarks. This detection performance is achieved while running at 62 Hz: a 2 -4 fold runtime improvement. A faster version of our method matches the state of the art at 105 Hz. These benchmarks suggest that PointPillars is an appropriate encoding for object detection in point clouds.

Three-dimensional objects are commonly represented as 3D boxes in a point-cloud. This representation mimics the well-studied image-based 2D bounding-box detection but comes with additional challenges. Objects in a 3D world do not follow any particular orientation, and box-based detectors have difficulties enumerating all orientations or fitting an axis-aligned bounding box to rotated objects. In this paper, we instead propose to represent, detect, and track 3D objects as points. Our framework, CenterPoint, first detects centers of objects using a keypoint detector and regresses to other attributes, including 3D size, 3D orientation, and velocity. In a second stage, it refines these estimates using additional point features on the object. In CenterPoint, 3D object tracking simplifies to greedy closest-point matching. The resulting detection and tracking algorithm is simple, efficient, and effective. CenterPoint achieved state-of-theart performance on the nuScenes benchmark for both 3D detection and tracking, with 65.5 NDS and 63.8 AMOTA for a single model. On the Waymo Open Dataset, Center-Point outperforms all previous single model methods by a large margin and ranks first among all Lidar-only submissions. The code and pretrained models are available at https://github.com/tianweiy/CenterPoint.

We present a new two-stage 3D object detection framework, named sparse-to-dense 3D Object Detector (STD). The first stage is a bottom-up proposal generation network that uses raw point cloud as input to generate accurate proposals by seeding each point with a new spherical anchor. It achieves a high recall with less computation compared with prior works. Then, PointsPool is applied for generating proposal features by transforming their interior point features from sparse expression to compact representation, which saves even more computation time. In box prediction, which is the second stage, we implement a parallel intersection-over-union (IoU) branch to increase awareness of localization accuracy, resulting in further improved performance. We conduct experiments on KITTI dataset, and evaluate our method in terms of 3D object and Bird's Eye View (BEV) detection. Our method outperforms other stateof-the-arts by a large margin, especially on the hard set, with inference speed more than 10 FPS.

Radar, the only sensor that could provide reliable perception capability in all weather conditions at an affordable cost, has been widely accepted as a key supplement to camera and LiDAR in modern advanced driver assistance systems (ADAS) and autonomous driving systems. Recent state-of-the-art works reveal that fusion of radar and LiDAR can lead to robust detection in adverse weather, such as fog. However, these methods still suffer from low accuracy of bounding box estimations. This paper proposes a bird's-eye view (BEV) fusion learning for an anchor box-free object detection system, which uses the feature derived from the radar range-azimuth heatmap and the LiDAR point cloud to estimate the possible objects. Different label assignment strategies have been designed to facilitate the consistency between the classification of foreground or background anchor points and the corresponding bounding box regressions. Furthermore, the performance of the proposed object detector can be further enhanced by employing a novel interactive transformer module. We demonstrated the superior performance of the proposed methods in this paper using the recently published Oxford Radar RobotCar (ORR) dataset. We showed that the accuracy of our system significantly outperforms the other state-of-the-art methods by a large margin.

在鸟眼中学习强大的表现（BEV），以进行感知任务，这是趋势和吸引行业和学术界的广泛关注。大多数自动驾驶算法的常规方法在正面或透视视图中执行检测，细分，跟踪等。随着传感器配置变得越来越复杂，从不同的传感器中集成了多源信息，并在统一视图中代表功能至关重要。 BEV感知继承了几个优势，因为代表BEV中的周围场景是直观和融合友好的。对于BEV中的代表对象，对于随后的模块，如计划和/或控制是最可取的。 BEV感知的核心问题在于（a）如何通过从透视视图到BEV来通过视图转换来重建丢失的3D信息； （b）如何在BEV网格中获取地面真理注释； （c）如何制定管道以合并来自不同来源和视图的特征； （d）如何适应和概括算法作为传感器配置在不同情况下各不相同。在这项调查中，我们回顾了有关BEV感知的最新工作，并对不同解决方案进行了深入的分析。此外，还描述了该行业的BEV方法的几种系统设计。此外，我们推出了一套完整的实用指南，以提高BEV感知任务的性能，包括相机，激光雷达和融合输入。最后，我们指出了该领域的未来研究指示。我们希望该报告能阐明社区，并鼓励对BEV感知的更多研究。我们保留一个活跃的存储库来收集最新的工作，并在https://github.com/openperceptionx/bevperception-survey-recipe上提供一包技巧的工具箱。

LiDAR-based 3D Object detectors have achieved impressive performances in many benchmarks, however, multisensors fusion-based techniques are promising to further improve the results. PointPainting, as a recently proposed framework, can add the semantic information from the 2D image into the 3D LiDAR point by the painting operation to boost the detection performance. However, due to the limited resolution of 2D feature maps, severe boundary-blurring effect happens during re-projection of 2D semantic segmentation into the 3D point clouds. To well handle this limitation, a general multimodal fusion framework MSF has been proposed to fuse the semantic information from both the 2D image and 3D points scene parsing results. Specifically, MSF includes three main modules. First, SOTA off-the-shelf 2D/3D semantic segmentation approaches are employed to generate the parsing results for 2D images and 3D point clouds. The 2D semantic information is further re-projected into the 3D point clouds with calibrated parameters. To handle the misalignment between the 2D and 3D parsing results, an AAF module is proposed to fuse them by learning an adaptive fusion score. Then the point cloud with the fused semantic label is sent to the following 3D object detectors. Furthermore, we propose a DFF module to aggregate deep features in different levels to boost the final detection performance. The effectiveness of the framework has been verified on two public large-scale 3D object detection benchmarks by comparing with different baselines. The experimental results show that the proposed fusion strategies can significantly improve the detection performance compared to the methods using only point clouds and the methods using only 2D semantic information. Most importantly, the proposed approach significantly outperforms other approaches and sets new SOTA results on the nuScenes testing benchmark.

合作感允许连接的自动驾驶汽车（CAV）与附近的其他骑士相互作用，以增强对周围物体的感知以提高安全性和可靠性。它可以弥补常规车辆感知的局限性，例如盲点，低分辨率和天气影响。合作感知中间融合方法的有效特征融合模型可以改善特征选择和信息聚集，以进一步提高感知精度。我们建议具有可训练的特征选择模块的自适应特征融合模型。我们提出的模型之一是通过空间自适应特征融合（S-Adafusion）在OPV2V数据集的两个子集上的所有其他最先进的模型：默认的Carla Towns用于车辆检测和用于域适应的Culver City。此外，先前的研究仅测试了合作感的车辆检测。但是，行人在交通事故中更有可能受到重伤。我们使用CODD数据集评估了车辆和行人检测的合作感的性能。与CODD数据集中的车辆和行人检测相比，我们的架构达到的平均精度（AP）高。实验表明，与常规感知过程相比，合作感也可以提高行人检测准确性。

基于查询的变压器在许多图像域任务中构建长期注意力方面表现出了巨大的潜力，但是由于点云数据的压倒性大小，在基于激光雷达的3D对象检测中很少考虑。在本文中，我们提出了CenterFormer，这是一个基于中心的变压器网络，用于3D对象检测。 CenterFormer首先使用中心热图在基于标准的Voxel点云编码器之上选择中心候选者。然后，它将中心候选者的功能用作变压器中的查询嵌入。为了进一步从多个帧中汇总功能，我们通过交叉注意设计一种方法来融合功能。最后，添加回归头以预测输出中心功能表示形式上的边界框。我们的设计降低了变压器结构的收敛难度和计算复杂性。结果表明，与无锚对象检测网络的强基线相比，有了显着改善。 CenterFormer在Waymo Open数据集上实现了单个模型的最新性能，验证集的MAPH为73.7％，测试集的MAPH上有75.6％的MAPH，大大优于所有先前发布的CNN和基于变压器的方法。我们的代码可在https://github.com/tusimple/centerformer上公开获取