结合同时定位和映射(SLAM)估计和动态场景建模可以高效地在动态环境中获得机器人自主权。机器人路径规划和障碍避免任务依赖于场景中动态对象运动的准确估计。本文介绍了VDO-SLAM,这是一种强大的视觉动态对象感知SLAM系统,用于利用语义信息,使得能够在场景中进行准确的运动估计和跟踪动态刚性物体,而无需任何先前的物体形状或几何模型的知识。所提出的方法识别和跟踪环境中的动态对象和静态结构,并将这些信息集成到统一的SLAM框架中。这导致机器人轨迹的高度准确估计和对象的全部SE(3)运动以及环境的时空地图。该系统能够从对象的SE(3)运动中提取线性速度估计,为复杂的动态环境中的导航提供重要功能。我们展示了所提出的系统对许多真实室内和室外数据集的性能,结果表明了对最先进的算法的一致和实质性的改进。可以使用源代码的开源版本。
translated by 谷歌翻译
Simultaneous Localization & Mapping (SLAM) is the process of building a mutual relationship between localization and mapping of the subject in its surrounding environment. With the help of different sensors, various types of SLAM systems have developed to deal with the problem of building the relationship between localization and mapping. A limitation in the SLAM process is the lack of consideration of dynamic objects in the mapping of the environment. We propose the Dynamic Object Tracking SLAM (DyOb-SLAM), which is a Visual SLAM system that can localize and map the surrounding dynamic objects in the environment as well as track the dynamic objects in each frame. With the help of a neural network and a dense optical flow algorithm, dynamic objects and static objects in an environment can be differentiated. DyOb-SLAM creates two separate maps for both static and dynamic contents. For the static features, a sparse map is obtained. For the dynamic contents, a trajectory global map is created as output. As a result, a frame to frame real-time based dynamic object tracking system is obtained. With the pose calculation of the dynamic objects and camera, DyOb-SLAM can estimate the speed of the dynamic objects with time. The performance of DyOb-SLAM is observed by comparing it with a similar Visual SLAM system, VDO-SLAM and the performance is measured by calculating the camera and object pose errors as well as the object speed error.
translated by 谷歌翻译
动态对象感知的SLAM(DOS)利用对象级信息以在动态环境中启用强大的运动估计。现有方法主要集中于识别和排除优化的动态对象。在本文中,我们表明,基于功能的视觉量大系统也可以通过利用两个观察结果来受益于动态铰接式对象的存在:(1)随着时间的推移,铰接对象的每个刚性部分的3D结构保持一致; (2)同一刚性零件上的点遵循相同的运动。特别是,我们提出了Airdos,这是一种动态的对象感知系统,该系统将刚度和运动限制引入模型铰接对象。通过共同优化相机姿势,对象运动和对象3D结构,我们可以纠正摄像头姿势估计,防止跟踪损失,并为动态对象和静态场景生成4D时空图。实验表明,我们的算法改善了在挑战拥挤的城市环境中的视觉大满贯算法的鲁棒性。据我们所知,Airdos是第一个动态对象感知的大满贯系统,该系统表明可以通过合并动态铰接式对象来改善相机姿势估计。
translated by 谷歌翻译
经典的视觉同时定位和映射(SLAM)算法通常假设环境是刚性的。此假设限制了这些算法的适用性,因为它们无法准确估算包含移动物体的现实生活场景中的相机姿势和世界结构(例如汽车,自行车,行人等)。为了解决这个问题,我们提出了Twistlam:一种语义,动态和立体声猛击系统,可以跟踪环境中的动态对象。我们的算法根据其语义类创建积分群。得益于通过机械关节建模的集群间约束(语义类的功能)的定义,因此,新颖的约束束调整能够共同估计移动物体的姿势和速度以及古典世界结构和摄像机轨迹。我们对公共Kitti数据集的多个序列进行了评估,并定量证明它与最新方法相比改进了相机和对象跟踪。
translated by 谷歌翻译
在本文中,我们提出了一个紧密耦合的视觉惯性对象级多效性动态大满贯系统。即使在极其动态的场景中,它也可以为摄像机姿势,速度,IMU偏见并构建一个密集的3D重建对象级映射图。我们的系统可以通过稳健的传感器和对象跟踪,可以强牢固地跟踪和重建任意对象的几何形状,其语义和运动的几何形状,其语义和运动的几何形状,并通过逐步融合相关的颜色,深度,语义和前景对象概率概率。此外,当对象在视野视野外丢失或移动时,我们的系统可以在重新观察时可靠地恢复其姿势。我们通过定量和定性测试现实世界数据序列来证明我们方法的鲁棒性和准确性。
translated by 谷歌翻译
视觉同时定位和映射(VSLAM)在计算机视觉和机器人社区中取得了巨大进展,并已成功用于许多领域,例如自主机器人导航和AR/VR。但是,VSLAM无法在动态和复杂的环境中实现良好的定位。许多出版物报告说,通过与VSLAM结合语义信息,语义VSLAM系统具有近年来解决上述问题的能力。然而,尚无关于语义VSLAM的全面调查。为了填补空白,本文首先回顾了语义VSLAM的发展,并明确着眼于其优势和差异。其次,我们探讨了语义VSLAM的三个主要问题:语义信息的提取和关联,语义信息的应用以及语义VSLAM的优势。然后,我们收集和分析已广泛用于语义VSLAM系统的当前最新SLAM数据集。最后,我们讨论未来的方向,该方向将为语义VSLAM的未来发展提供蓝图。
translated by 谷歌翻译
同时本地化和映射(SLAM)是自动移动机器人中的基本问题之一,在该机器人需要重建以前看不见的环境的同时,同时在地图上进行了本身。特别是,Visual-Slam使用移动机器人中的各种传感器来收集和感测地图的表示。传统上,基于几何模型的技术被用来解决大满贯问题,在充满挑战的环境下,该问题往往容易出错。诸如深度学习技术之类的计算机视觉方面的最新进展提供了一种数据驱动的方法来解决视觉范围问题。这篇综述总结了使用各种基于学习的方法的视觉 - 峰领域的最新进展。我们首先提供了基于几何模型的方法的简洁概述,然后进行有关SLAM当前范式的技术评论。然后,我们介绍了从移动机器人那里收集感官输入并执行场景理解的各种基于学习的方法。讨论并将基于深度学习的语义理解中的当前范式讨论并置于视觉峰的背景下。最后,我们讨论了在视觉 - 峰中基于学习的方法方向上的挑战和进一步的机会。
translated by 谷歌翻译
作为许多自主驾驶和机器人活动的基本组成部分,如自我运动估计,障碍避免和场景理解,单眼深度估计(MDE)引起了计算机视觉和机器人社区的极大关注。在过去的几十年中,已经开发了大量方法。然而,据我们所知,对MDE没有全面调查。本文旨在通过审查1970年至2021年之间发布的197个相关条款来弥补这一差距。特别是,我们为涵盖各种方法的MDE提供了全面的调查,介绍了流行的绩效评估指标并汇总公开的数据集。我们还总结了一些代表方法的可用开源实现,并比较了他们的表演。此外,我们在一些重要的机器人任务中审查了MDE的应用。最后,我们通过展示一些有希望的未来研究方向来结束本文。预计本调查有助于读者浏览该研究领域。
translated by 谷歌翻译
从单眼视频中估算移动摄像头的姿势是一个具有挑战性的问题,尤其是由于动态环境中移动对象的存在,在动态环境中,现有摄像头姿势估计方法的性能易于几何一致的像素。为了应对这一挑战,我们为视频提供了一种强大的密度间接结构,该结构是基于由成对光流初始化的致密对应的。我们的关键想法是将远程视频对应性优化为密集的点轨迹,并使用它来学习对运动分割的强大估计。提出了一种新型的神经网络结构来处理不规则的点轨迹数据。然后,在远程点轨迹的一部分中,通过全局捆绑式调整估算和优化摄像头姿势,这些轨迹被归类为静态。 MPI Sintel数据集的实验表明,与现有最新方法相比,我们的系统产生的相机轨迹明显更准确。此外,我们的方法能够在完全静态的场景上保留相机姿势的合理准确性,该场景始终优于端到端深度学习的强大最新密度对应方法,这证明了密集间接方法的潜力基于光流和点轨迹。由于点轨迹表示是通用的,因此我们进一步介绍了具有动态对象的复杂运动的野外单眼视频的比较。代码可在https://github.com/bytedance/particle-sfm上找到。
translated by 谷歌翻译
完全自主移动机器人的现实部署取决于能够处理动态环境的强大的大满贯(同时本地化和映射)系统,其中对象在机器人的前面移动以及不断变化的环境,在此之后移动或更换对象。机器人已经绘制了现场。本文介绍了更换式SLAM,这是一种在动态和不断变化的环境中强大的视觉猛烈抨击的方法。这是通过使用与长期数据关联算法结合的贝叶斯过滤器来实现的。此外,它采用了一种有效的算法,用于基于对象检测的动态关键点过滤,该对象检测正确识别了不动态的边界框中的特征,从而阻止了可能导致轨道丢失的功能的耗竭。此外,开发了一个新的数据集,其中包含RGB-D数据,专门针对评估对象级别的变化环境,称为PUC-USP数据集。使用移动机器人,RGB-D摄像头和运动捕获系统创建了六个序列。这些序列旨在捕获可能导致跟踪故障或地图损坏的不同情况。据我们所知,更换 - 峰是第一个对动态和不断变化的环境既有坚固耐用的视觉大满贯系统,又不假设给定的相机姿势或已知地图,也能够实时运行。使用基准数据集对所提出的方法进行了评估,并将其与其他最先进的方法进行了比较,证明是高度准确的。
translated by 谷歌翻译
基于学习的视觉探针计(VO)算法在常见的静态场景上实现了显着的性能,受益于高容量模型和大量注释的数据,但在动态,填充的环境中往往会失败。语义细分在估计摄像机动作之前主要用于丢弃动态关联,但以丢弃静态功能为代价,并且很难扩展到看不见的类别。在本文中,我们利用相机自我运动和运动分割之间的相互依赖性,并表明两者都可以在单个基于学习的框架中共同完善。特别是,我们提出了Dytanvo,这是第一个涉及动态环境的基于学习的VO方法。它需要实时两个连续的单眼帧,并以迭代方式预测相机的自我运动。我们的方法在现实世界动态环境中的最先进的VOUTESS的平均提高27.7%,甚至在动态视觉SLAM系统中进行竞争性,从而优化了后端的轨迹。在很多看不见的环境上进行的实验也证明了我们的方法的普遍性。
translated by 谷歌翻译
Ego-pose estimation and dynamic object tracking are two critical problems for autonomous driving systems. The solutions to these problems are generally based on their respective assumptions, \ie{the static world assumption for simultaneous localization and mapping (SLAM) and the accurate ego-pose assumption for object tracking}. However, these assumptions are challenging to hold in dynamic road scenarios, where SLAM and object tracking become closely correlated. Therefore, we propose DL-SLOT, a dynamic LiDAR SLAM and object tracking method, to simultaneously address these two coupled problems. This method integrates the state estimations of both the autonomous vehicle and the stationary and dynamic objects in the environment into a unified optimization framework. First, we used object detection to identify all points belonging to potentially dynamic objects. Subsequently, a LiDAR odometry was conducted using the filtered point cloud. Simultaneously, we proposed a sliding window-based object association method that accurately associates objects according to the historical trajectories of tracked objects. The ego-states and those of the stationary and dynamic objects are integrated into the sliding window-based collaborative graph optimization. The stationary objects are subsequently restored from the potentially dynamic object set. Finally, a global pose-graph is implemented to eliminate the accumulated error. Experiments on KITTI datasets demonstrate that our method achieves better accuracy than SLAM and object tracking baseline methods. This confirms that solving SLAM and object tracking simultaneously is mutually advantageous, dramatically improving the robustness and accuracy of SLAM and object tracking in dynamic road scenarios.
translated by 谷歌翻译
我们提出了场景运动的新颖双流表示,将光流分​​解为由摄像机运动引起的静态流场和另一个由场景中对象的运动引起的动态流场。基于此表示形式,我们提出了一个动态的大满贯,称为Deflowslam,它利用图像中的静态和动态像素来求解相机的姿势,而不是像其他动态SLAM系统一样简单地使用静态背景像素。我们提出了一个动态更新模块,以一种自我监督的方式训练我们的Deflowslam,其中密集的束调节层采用估计的静态流场和由动态掩码控制的权重,并输出优化的静态流动场的残差,相机姿势的残差,和反度。静态和动态流场是通过将当前图像翘曲到相邻图像来估计的,并且可以通过将两个字段求和来获得光流。广泛的实验表明,在静态场景和动态场景中,Deflowslam可以很好地推广到静态和动态场景,因为它表现出与静态和动态较小的场景中最先进的Droid-Slam相当的性能,同时在高度动态的环境中表现出明显优于Droid-Slam。代码和数据可在项目网页上找到:\ urlstyle {tt} \ textColor {url_color} {\ url {https://zju3dv.github.io/deflowslam/}}}。
translated by 谷歌翻译
现代视觉惯性导航系统(VINS)面临着实际部署中的一个关键挑战:他们需要在高度动态的环境中可靠且强大地运行。当前最佳解决方案仅根据对象类别的语义将动态对象过滤为异常值。这样的方法不缩放,因为它需要语义分类器来包含所有可能移动的对象类;这很难定义,更不用说部署。另一方面,许多现实世界的环境以墙壁和地面等平面形式表现出强大的结构规律,这也是至关重要的。我们呈现RP-VIO,一种单眼视觉惯性内径系统,可以利用这些平面的简单几何形状,以改善充满活力环境的鲁棒性和准确性。由于现有数据集具有有限数量的动态元素,因此我们还提供了一种高动态的光致态度合成数据集,用于更有效地对现代VINS系统的功能的评估。我们评估我们在该数据集中的方法,以及来自标准数据集的三个不同序列,包括两个真实的动态序列,并在最先进的单眼视觉惯性内径系统上显示出鲁棒性和准确性的显着提高。我们还显示在模拟中,通过简单的动态特征掩蔽方法改进。我们的代码和数据集是公开可用的。
translated by 谷歌翻译
This paper presents ORB-SLAM, a feature-based monocular SLAM system that operates in real time, in small and large, indoor and outdoor environments. The system is robust to severe motion clutter, allows wide baseline loop closing and relocalization, and includes full automatic initialization. Building on excellent algorithms of recent years, we designed from scratch a novel system that uses the same features for all SLAM tasks: tracking, mapping, relocalization, and loop closing. A survival of the fittest strategy that selects the points and keyframes of the reconstruction leads to excellent robustness and generates a compact and trackable map that only grows if the scene content changes, allowing lifelong operation. We present an exhaustive evaluation in 27 sequences from the most popular datasets. ORB-SLAM achieves unprecedented performance with respect to other state-of-the-art monocular SLAM approaches. For the benefit of the community, we make the source code public.
translated by 谷歌翻译
a) Stereo input: trajectory and sparse reconstruction of an urban environment with multiple loop closures. (b) RGB-D input: keyframes and dense pointcloud of a room scene with one loop closure. The pointcloud is rendered by backprojecting the sensor depth maps from estimated keyframe poses. No fusion is performed.
translated by 谷歌翻译
This paper presents ORB-SLAM3, the first system able to perform visual, visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models.The first main novelty is a feature-based tightly-integrated visual-inertial SLAM system that fully relies on Maximum-a-Posteriori (MAP) estimation, even during the IMU initialization phase. The result is a system that operates robustly in real time, in small and large, indoor and outdoor environments, and is two to ten times more accurate than previous approaches.The second main novelty is a multiple map system that relies on a new place recognition method with improved recall. Thanks to it, ORB-SLAM3 is able to survive to long periods of poor visual information: when it gets lost, it starts a new map that will be seamlessly merged with previous maps when revisiting mapped areas. Compared with visual odometry systems that only use information from the last few seconds, ORB-SLAM3 is the first system able to reuse in all the algorithm stages all previous information. This allows to include in bundle adjustment co-visible keyframes, that provide high parallax observations boosting accuracy, even if they are widely separated in time or if they come from a previous mapping session.Our experiments show that, in all sensor configurations, ORB-SLAM3 is as robust as the best systems available in the literature, and significantly more accurate. Notably, our stereo-inertial SLAM achieves an average accuracy of 3.5 cm in the EuRoC drone and 9 mm under quick hand-held motions in the room of TUM-VI dataset, a setting representative of AR/VR scenarios. For the benefit of the community we make public the source code.
translated by 谷歌翻译
Visually impaired people usually find it hard to travel independently in many public places such as airports and shopping malls due to the problems of obstacle avoidance and guidance to the desired location. Therefore, in the highly dynamic indoor environment, how to improve indoor navigation robot localization and navigation accuracy so that they guide the visually impaired well becomes a problem. One way is to use visual SLAM. However, typical visual SLAM either assumes a static environment, which may lead to less accurate results in dynamic environments or assumes that the targets are all dynamic and removes all the feature points above, sacrificing computational speed to a large extent with the available computational power. This paper seeks to explore marginal localization and navigation systems for indoor navigation robotics. The proposed system is designed to improve localization and navigation accuracy in highly dynamic environments by identifying and tracking potentially moving objects and using vector field histograms for local path planning and obstacle avoidance. The system has been tested on a public indoor RGB-D dataset, and the results show that the new system improves accuracy and robustness while reducing computation time in highly dynamic indoor scenes.
translated by 谷歌翻译
单眼语义同时定位和映射(SLAM)的有效对象级别表示仍然缺乏广泛接受的解决方案。在本文中,我们提出了基于结构点的有效表示的使用,以基于姿势格式的配方在单眼语义大满贯系统中用作地标的几何形状。特别是,为姿势图中的地标节点提出了一个反深度参数化,以存储对象位置,方向和大小/比例。所提出的配方是一般的,可以应用于不同的几何形状。在本文中,我们关注的是室内环境,其中人工制品通常具有平面矩形形状,例如窗户,门,橱柜等。模拟中的实验表现出良好的性能,尤其是在对象几何重建中。
translated by 谷歌翻译
由于其许多潜在应用,从视频中估算人类运动是一个活跃的研究领域。大多数最先进的方法可以预测单个图像的人类形状和姿势估计,并且不利用视频中可用的时间信息。许多“野生”运动序列被移动的摄像机捕获,这为估计增加了混合的摄像头和人类运动的并发症。因此,我们介绍了Bodyslam,这是一种单眼大满贯系统,共同估计人体的位置,形状和姿势以及摄像机轨迹。我们还引入了一种新型的人类运动模型,以限制顺序身体姿势并观察场景的规模。通过通过移动的单眼相机捕获的人类运动的视频序列进行的一系列实验,我们证明了Bodyslam与单独估计这些估计相比,可以改善所有人体参数和相机的估计。
translated by 谷歌翻译