Recently, over-height vehicle strike frequently occurs, causing great economic cost and serious safety problems. Hence, an alert system which can accurately discover any possible height limiting devices in advance is necessary to be employed in modern large or medium sized cars, such as touring cars. Detecting and estimating the height limiting devices act as the key point of a successful height limit alert system. Though there are some works research height limit estimation, existing methods are either too computational expensive or not accurate enough. In this paper, we propose a novel stereo-based pipeline named SHLE for height limit estimation. Our SHLE pipeline consists of two stages. In stage 1, a novel devices detection and tracking scheme is introduced, which accurately locate the height limit devices in the left or right image. Then, in stage 2, the depth is temporally measured, extracted and filtered to calculate the height limit device. To benchmark the height limit estimation task, we build a large-scale dataset named "Disparity Height", where stereo images, pre-computed disparities and ground-truth height limit annotations are provided. We conducted extensive experiments on "Disparity Height" and the results show that SHLE achieves an average error below than 10cm though the car is 70m away from the devices. Our method also outperforms all compared baselines and achieves state-of-the-art performance. Code is available at https://github.com/Yang-Kaixing/SHLE.

Visual perception plays an important role in autonomous driving. One of the primary tasks is object detection and identification. Since the vision sensor is rich in color and texture information, it can quickly and accurately identify various road information. The commonly used technique is based on extracting and calculating various features of the image. The recent development of deep learning-based method has better reliability and processing speed and has a greater advantage in recognizing complex elements. For depth estimation, vision sensor is also used for ranging due to their small size and low cost. Monocular camera uses image data from a single viewpoint as input to estimate object depth. In contrast, stereo vision is based on parallax and matching feature points of different views, and the application of deep learning also further improves the accuracy. In addition, Simultaneous Location and Mapping (SLAM) can establish a model of the road environment, thus helping the vehicle perceive the surrounding environment and complete the tasks. In this paper, we introduce and compare various methods of object detection and identification, then explain the development of depth estimation and compare various methods based on monocular, stereo, and RDBG sensors, next review and compare various methods of SLAM, and finally summarize the current problems and present the future development trends of vision technologies.

3D object detection is an essential task in autonomous driving. Recent techniques excel with highly accurate detection rates, provided the 3D input data is obtained from precise but expensive LiDAR technology. Approaches based on cheaper monocular or stereo imagery data have, until now, resulted in drastically lower accuracies -a gap that is commonly attributed to poor image-based depth estimation. However, in this paper we argue that it is not the quality of the data but its representation that accounts for the majority of the difference. Taking the inner workings of convolutional neural networks into consideration, we propose to convert image-based depth maps to pseudo-LiDAR representations -essentially mimicking the LiDAR signal. With this representation we can apply different existing LiDAR-based detection algorithms. On the popular KITTI benchmark, our approach achieves impressive improvements over the existing state-of-the-art in image-based performance -raising the detection accuracy of objects within the 30m range from the previous state-of-the-art of 22% to an unprecedented 74%. At the time of submission our algorithm holds the highest entry on the KITTI 3D object detection leaderboard for stereo-image-based approaches. Our code is publicly available at https: //github.com/mileyan/pseudo_lidar.

3D object detection is vital as it would enable us to capture objects' sizes, orientation, and position in the world. As a result, we would be able to use this 3D detection in real-world applications such as Augmented Reality (AR), self-driving cars, and robotics which perceive the world the same way we do as humans. Monocular 3D Object Detection is the task to draw 3D bounding box around objects in a single 2D RGB image. It is localization task but without any extra information like depth or other sensors or multiple images. Monocular 3D object detection is an important yet challenging task. Beyond the significant progress in image-based 2D object detection, 3D understanding of real-world objects is an open challenge that has not been explored extensively thus far. In addition to the most closely related studies.

在鸟眼中学习强大的表现（BEV），以进行感知任务，这是趋势和吸引行业和学术界的广泛关注。大多数自动驾驶算法的常规方法在正面或透视视图中执行检测，细分，跟踪等。随着传感器配置变得越来越复杂，从不同的传感器中集成了多源信息，并在统一视图中代表功能至关重要。 BEV感知继承了几个优势，因为代表BEV中的周围场景是直观和融合友好的。对于BEV中的代表对象，对于随后的模块，如计划和/或控制是最可取的。 BEV感知的核心问题在于（a）如何通过从透视视图到BEV来通过视图转换来重建丢失的3D信息； （b）如何在BEV网格中获取地面真理注释； （c）如何制定管道以合并来自不同来源和视图的特征； （d）如何适应和概括算法作为传感器配置在不同情况下各不相同。在这项调查中，我们回顾了有关BEV感知的最新工作，并对不同解决方案进行了深入的分析。此外，还描述了该行业的BEV方法的几种系统设计。此外，我们推出了一套完整的实用指南，以提高BEV感知任务的性能，包括相机，激光雷达和融合输入。最后，我们指出了该领域的未来研究指示。我们希望该报告能阐明社区，并鼓励对BEV感知的更多研究。我们保留一个活跃的存储库来收集最新的工作，并在https://github.com/openperceptionx/bevperception-survey-recipe上提供一包技巧的工具箱。

作为许多自主驾驶和机器人活动的基本组成部分，如自我运动估计，障碍避免和场景理解，单眼深度估计（MDE）引起了计算机视觉和机器人社区的极大关注。在过去的几十年中，已经开发了大量方法。然而，据我们所知，对MDE没有全面调查。本文旨在通过审查1970年至2021年之间发布的197个相关条款来弥补这一差距。特别是，我们为涵盖各种方法的MDE提供了全面的调查，介绍了流行的绩效评估指标并汇总公开的数据集。我们还总结了一些代表方法的可用开源实现，并比较了他们的表演。此外，我们在一些重要的机器人任务中审查了MDE的应用。最后，我们通过展示一些有希望的未来研究方向来结束本文。预计本调查有助于读者浏览该研究领域。

基于LIDAR的传感驱动器电流自主车辆。尽管进展迅速，但目前的激光雷达传感器在分辨率和成本方面仍然落后于传统彩色相机背后的二十年。对于自主驾驶，这意味着靠近传感器的大物体很容易可见，但远方或小物体仅包括一个测量或两个。这是一个问题，尤其是当这些对象结果驾驶危险时。另一方面，在车载RGB传感器中清晰可见这些相同的对象。在这项工作中，我们提出了一种将RGB传感器无缝熔化成基于LIDAR的3D识别方法。我们的方法采用一组2D检测来生成密集的3D虚拟点，以增加否则稀疏的3D点云。这些虚拟点自然地集成到任何基于标准的LIDAR的3D探测器以及常规激光雷达测量。由此产生的多模态检测器简单且有效。大规模NUSCENES数据集的实验结果表明，我们的框架通过显着的6.6地图改善了强大的中心点基线，并且优于竞争融合方法。代码和更多可视化可在https://tianweiy.github.io/mvp/上获得

Surround-view fisheye perception under valet parking scenes is fundamental and crucial in autonomous driving. Environmental conditions in parking lots perform differently from the common public datasets, such as imperfect light and opacity, which substantially impacts on perception performance. Most existing networks based on public datasets may generalize suboptimal results on these valet parking scenes, also affected by the fisheye distortion. In this article, we introduce a new large-scale fisheye dataset called Fisheye Parking Dataset(FPD) to promote the research in dealing with diverse real-world surround-view parking cases. Notably, our compiled FPD exhibits excellent characteristics for different surround-view perception tasks. In addition, we also propose our real-time distortion-insensitive multi-task framework Fisheye Perception Network (FPNet), which improves the surround-view fisheye BEV perception by enhancing the fisheye distortion operation and multi-task lightweight designs. Extensive experiments validate the effectiveness of our approach and the dataset's exceptional generalizability.

摄像机是自动化驱动系统中的主要传感器。它们提供高信息密度，并对检测为人类视野提供的道路基础设施线索最优。环绕式摄像机系统通常包括具有190 {\ DEG} +视野的四个鱼眼相机，覆盖在车辆周围的整个360 {\ DEG}集中在近场传感上。它们是低速，高精度和近距离传感应用的主要传感器，如自动停车，交通堵塞援助和低速应急制动。在这项工作中，我们提供了对这种视觉系统的详细调查，在可以分解为四个模块化组件的架构中，设置调查即可识别，重建，重建和重组。我们共同称之为4R架构。我们讨论每个组件如何完成特定方面，并提供一个位置论证，即它们可以协同组织以形成用于低速自动化的完整感知系统。我们通过呈现来自以前的作品的结果，并通过向此类系统提出架构提案来支持此参数。定性结果在视频中呈现在HTTPS://youtu.be/ae8bcof7777uy中。

对于许多应用程序，包括自动驾驶，机器人抓握和增强现实，单眼3D对象检测是一项基本但非常重要的任务。现有的领先方法倾向于首先估算输入图像的深度，并基于点云检测3D对象。该例程遭受了深度估计和对象检测之间固有的差距。此外，预测误差积累也会影响性能。在本文中，提出了一种名为MonopCN的新方法。引入单频道的洞察力是，我们建议在训练期间模拟基于点云的探测器的特征学习行为。因此，在推理期间，学习的特征和预测将与基于点云的检测器相似。为了实现这一目标，我们建议一个场景级仿真模块，一个ROI级别的仿真模块和一个响应级仿真模块，这些模块逐渐用于检测器的完整特征学习和预测管道。我们将我们的方法应用于著名的M3D-RPN检测器和CADDN检测器，并在Kitti和Waymo Open数据集上进行了广泛的实验。结果表明，我们的方法始终提高不同边缘的不同单眼探测器的性能，而无需更改网络体系结构。我们的方法最终达到了最先进的性能。

它得到了很好的认识到，从深度感知的LIDAR点云和语义富有的立体图像中融合互补信息将有利于3D对象检测。然而，探索稀疏3D点和密集2D像素之间固有的不自然相互作用并不重要。为了简化这种困难，最近的建议通常将3D点投影到2D图像平面上以对图像数据进行采样，然后聚合点处的数据。然而，这种方法往往遭受点云和RGB图像的分辨率之间的不匹配，导致次优性能。具体地，作为多模态数据聚合位置的稀疏点导致高分辨率图像的严重信息丢失，这反过来破坏了多传感器融合的有效性。在本文中，我们呈现VPFNET - 一种新的架构，可以在“虚拟”点处巧妙地对齐和聚合点云和图像数据。特别地，它们的密度位于3D点和2D像素的密度之间，虚拟点可以很好地桥接两个传感器之间的分辨率间隙，从而保持更多信息以进行处理。此外，我们还研究了可以应用于点云和RGB图像的数据增强技术，因为数据增强对迄今为止对3D对象探测器的贡献不可忽略。我们对Kitti DataSet进行了广泛的实验，与最先进的方法相比，观察到了良好的性能。值得注意的是，我们的VPFNET在KITTI测试集上实现了83.21 \％中等3D AP和91.86 \％适度的BEV AP，自2021年5月21日起排名第一。网络设计也考虑了计算效率 - 我们可以实现FPS 15对单个NVIDIA RTX 2080TI GPU。该代码将用于复制和进一步调查。

Compared to typical multi-sensor systems, monocular 3D object detection has attracted much attention due to its simple configuration. However, there is still a significant gap between LiDAR-based and monocular-based methods. In this paper, we find that the ill-posed nature of monocular imagery can lead to depth ambiguity. Specifically, objects with different depths can appear with the same bounding boxes and similar visual features in the 2D image. Unfortunately, the network cannot accurately distinguish different depths from such non-discriminative visual features, resulting in unstable depth training. To facilitate depth learning, we propose a simple yet effective plug-and-play module, One Bounding Box Multiple Objects (OBMO). Concretely, we add a set of suitable pseudo labels by shifting the 3D bounding box along the viewing frustum. To constrain the pseudo-3D labels to be reasonable, we carefully design two label scoring strategies to represent their quality. In contrast to the original hard depth labels, such soft pseudo labels with quality scores allow the network to learn a reasonable depth range, boosting training stability and thus improving final performance. Extensive experiments on KITTI and Waymo benchmarks show that our method significantly improves state-of-the-art monocular 3D detectors by a significant margin (The improvements under the moderate setting on KITTI validation set are $\mathbf{1.82\sim 10.91\%}$ mAP in BEV and $\mathbf{1.18\sim 9.36\%}$ mAP in 3D}. Codes have been released at https://github.com/mrsempress/OBMO.

In this work, we study 3D object detection from RGB-D data in both indoor and outdoor scenes. While previous methods focus on images or 3D voxels, often obscuring natural 3D patterns and invariances of 3D data, we directly operate on raw point clouds by popping up RGB-D scans. However, a key challenge of this approach is how to efficiently localize objects in point clouds of large-scale scenes (region proposal). Instead of solely relying on 3D proposals, our method leverages both mature 2D object detectors and advanced 3D deep learning for object localization, achieving efficiency as well as high recall for even small objects. Benefited from learning directly in raw point clouds, our method is also able to precisely estimate 3D bounding boxes even under strong occlusion or with very sparse points. Evaluated on KITTI and SUN RGB-D 3D detection benchmarks, our method outperforms the state of the art by remarkable margins while having real-time capability. * Majority of the work done as an intern at Nuro, Inc. depth to point cloud 2D region (from CNN) to 3D frustum 3D box (from PointNet)

自动化驾驶系统（广告）开辟了汽车行业的新领域，为未来的运输提供了更高的效率和舒适体验的新可能性。然而，在恶劣天气条件下的自主驾驶已经存在，使自动车辆（AVS）长时间保持自主车辆（AVS）或更高的自主权。本文评估了天气在分析和统计方式中为广告传感器带来的影响和挑战，并对恶劣天气条件进行了解决方案。彻底报道了关于对每种天气的感知增强的最先进技术。外部辅助解决方案如V2X技术，当前可用的数据集，模拟器和天气腔室的实验设施中的天气条件覆盖范围明显。通过指出各种主要天气问题，自主驾驶场目前正在面临，近年来审查硬件和计算机科学解决方案，这项调查概述了在不利的天气驾驶条件方面的障碍和方向的障碍和方向。

以视觉为中心的BEV感知由于其固有的优点，最近受到行业和学术界的关注，包括展示世界自然代表和融合友好。随着深度学习的快速发展，已经提出了许多方法来解决以视觉为中心的BEV感知。但是，最近没有针对这个小说和不断发展的研究领域的调查。为了刺激其未来的研究，本文对以视觉为中心的BEV感知及其扩展进行了全面调查。它收集并组织了最近的知识，并对常用算法进行了系统的综述和摘要。它还为几项BEV感知任务提供了深入的分析和比较结果，从而促进了未来作品的比较并激发了未来的研究方向。此外，还讨论了经验实现细节并证明有利于相关算法的开发。

鉴于其经济性与多传感器设置相比，从单眼输入中感知的3D对象对于机器人系统至关重要。它非常困难，因为单个图像无法提供预测绝对深度值的任何线索。通过双眼方法进行3D对象检测，我们利用了相机自我运动提供的强几何结构来进行准确的对象深度估计和检测。我们首先对此一般的两视案例进行了理论分析，并注意两个挑战：1）来自多个估计的累积错误，这些估计使直接预测棘手； 2）由静态摄像机和歧义匹配引起的固有难题。因此，我们建立了具有几何感知成本量的立体声对应关系，作为深度估计的替代方案，并以单眼理解进一步补偿了它，以解决第二个问题。我们的框架（DFM）命名为深度（DFM），然后使用已建立的几何形状将2D图像特征提升到3D空间并检测到其3D对象。我们还提出了一个无姿势的DFM，以使其在摄像头不可用时可用。我们的框架在Kitti基准测试上的优于最先进的方法。详细的定量和定性分析也验证了我们的理论结论。该代码将在https://github.com/tai-wang/depth-from-motion上发布。

Multiview检测使用多个校准摄像机，并具有重叠的视野来定位遮挡的行人。在该领域，现有方法通常采用``人类建模 - 聚合''策略。为了找到强大的行人表示，有些人直观地使用检测到的2D边界框的位置，而另一些则使用投影到地面上的整个框架功能。但是，前者不考虑人类的外表，并导致许多歧义，而后者由于缺乏人类躯干和头部的准确高度而遭受投影错误。在本文中，我们提出了一种基于人类点云建模的新行人代表方案。具体而言，使用射线跟踪进行整体人类深度估计，我们将行人建模为直立的，薄的纸板点云。然后，我们通过多个视图汇总了行人纸板的点云以进行最终决定。与现有表示形式相比，提出的方法明确利用人类的外观并通过相对准确的高度估计大大减少投影误差。在两个标准评估基准上，提出的方法取得了非常具竞争力的结果。

自治机器人目前是最受欢迎的人工智能问题之一，在过去十年中，从自动驾驶汽车和人形系统到交付机器人和无人机，这是一项最受欢迎的智能问题。部分问题是获得一个机器人，以模仿人类的感知，我们的视觉感，用诸如神经网络等数学模型用相机和大脑的眼睛替换眼睛。开发一个能够在没有人为干预的情况下驾驶汽车的AI和一个小型机器人在城市中递送包裹可能看起来像不同的问题，因此来自感知和视觉的观点来看，这两个问题都有几种相似之处。我们目前的主要解决方案通过使用计算机视觉技术，机器学习和各种算法来实现对环境感知的关注，使机器人理解环境或场景，移动，调整其轨迹并执行其任务（维护，探索，等。）无需人为干预。在这项工作中，我们从头开始开发一个小型自动车辆，能够仅使用视觉信息理解场景，通过工业环境导航，检测人员和障碍，或执行简单的维护任务。我们审查了基本问题的最先进问题，并证明了小规模采用的许多方法类似于来自特斯拉或Lyft等公司的真正自动驾驶汽车中使用的方法。最后，我们讨论了当前的机器人和自主驾驶状态以及我们在这一领域找到的技术和道德限制。

3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications. In this paper, we extend our preliminary work PointRCNN to a novel and strong point-cloud-based 3D object detection framework, the part-aware and aggregation neural network (Part-A 2 net). The whole framework consists of the part-aware stage and the part-aggregation stage. Firstly, the part-aware stage for the first time fully utilizes free-of-charge part supervisions derived from 3D ground-truth boxes to simultaneously predict high quality 3D proposals and accurate intra-object part locations. The predicted intra-object part locations within the same proposal are grouped by our new-designed RoI-aware point cloud pooling module, which results in an effective representation to encode the geometry-specific features of each 3D proposal. Then the part-aggregation stage learns to re-score the box and refine the box location by exploring the spatial relationship of the pooled intra-object part locations. Extensive experiments are conducted to demonstrate the performance improvements from each component of our proposed framework. Our Part-A 2 net outperforms all existing 3D detection methods and achieves new state-of-the-art on KITTI 3D object detection dataset by utilizing only the LiDAR point cloud data. Code is available at https://github.com/sshaoshuai/PointCloudDet3D.

最近，融合了激光雷达点云和相机图像，提高了3D对象检测的性能和稳健性，因为这两种方式自然具有强烈的互补性。在本文中，我们通过引入新型级联双向融合〜（CB融合）模块和多模态一致性〜（MC）损耗来提出用于多模态3D对象检测的EPNet ++。更具体地说，所提出的CB融合模块提高点特征的丰富语义信息，以级联双向交互融合方式具有图像特征，导致更全面且辨别的特征表示。 MC损失明确保证预测分数之间的一致性，以获得更全面且可靠的置信度分数。基蒂，JRDB和Sun-RGBD数据集的实验结果展示了通过最先进的方法的EPNet ++的优越性。此外，我们强调一个关键但很容易被忽视的问题，这是探讨稀疏场景中的3D探测器的性能和鲁棒性。广泛的实验存在，EPNet ++优于现有的SOTA方法，在高稀疏点云壳中具有显着的边距，这可能是降低LIDAR传感器的昂贵成本的可用方向。代码将来会发布。