轨迹预测已在许多领域广泛追求,并且已经探索了许多基于模型和模型的方法。前者包括基于规则的,几何或基于优化的模型,后者主要由深度学习方法组成。在本文中,我们提出了一种基于新的神经微分方程模型的新方法,结合了两种方法。我们的新模型(神经社会物理或NSP)是一个深层神经网络,我们在其中使用具有可学习参数的显式物理模型。显式物理模型在建模行人行为时是强大的感应偏见,而网络的其余部分就系统参数估计和动力学随机性建模提供了强大的数据拟合能力。我们将NSP与6个数据集上的15种深度学习方法进行了比较,并将最新性能提高了5.56%-70%。此外,我们表明NSP在预测截然不同的情况下的合理轨迹方面具有更好的概括性,其中密度的密度是测试数据的2-5倍。最后,我们表明NSP中的物理模型可以为行人行为提供合理的解释,而不是黑盒深度学习。可用代码:https://github.com/realcrane/human-trajectory-prediction-via-noral-social-physics。
translated by 谷歌翻译
预测行人运动对于人类行为分析以及安全有效的人类代理相互作用至关重要。但是,尽管取得了重大进展,但对于捕捉人类导航决策的不确定性和多模式的现有方法仍然具有挑战性。在本文中,我们提出了SocialVae,这是一种新颖的人类轨迹预测方法。社会节的核心是一种时间上的变性自动编码器体系结构,它利用随机反复的神经网络进行预测,结合社会注意力机制和向后的后近似值,以更好地提取行人导航策略。我们表明,社交活动改善了几个步行轨迹预测基准的最新性能,包括ETH/UCY基准,Stanford Drone DataSet和Sportvu NBA运动数据集。代码可在以下网址获得:https://github.com/xupei0610/socialvae。
translated by 谷歌翻译
在人群情景中,根据许多外部因素,预测行人的轨迹是一个复杂和具有挑战性的任务。场景的拓扑和行人之间的相互作用只是其中一些。由于数据 - 科学和数据收集技术的进步,深入学习方法最近成为众多域中的研究热点。因此,越来越多的研究人员对预测行人的轨迹应用这些方法并不令人惊讶。本文将这些相对较新的深度学习算法与基于经典知识的模型进行了比较,这些算法被广泛用于模拟行人动态。它为两种方法提供了全面的文献综述,探索了技术和应用面向差异,并解决了未来的问题以及未来的发展方向。我们的调查指出,由于深度学习算法的高准确性,现在,基于知识的模型来预测局部轨迹的内容是可疑的。然而,深度学习算法用于大规模模拟的能力和集体动态的描述仍有待证明。此外,比较表明,两种方法(混合方法)的组合似乎很有希望克服像深度学习方法的缺失解释性等缺点。
translated by 谷歌翻译
行人轨迹预测是自动驾驶的重要技术,近年来已成为研究热点。以前的方法主要依靠行人的位置关系来模型社交互动,这显然不足以代表实际情况中的复杂病例。此外,大多数现有工作通常通常将场景交互模块作为独立分支介绍,并在轨迹生成过程中嵌入社交交互功能,而不是同时执行社交交互和场景交互,这可能破坏轨迹预测的合理性。在本文中,我们提出了一个名为社会软关注图卷积网络(SSAGCN)的一个新的预测模型,旨在同时处理行人和环境之间的行人和场景相互作用之间的社交互动。详细说明,在建模社交互动时,我们提出了一种新的\ EMPH {社会软关注功能},其充分考虑了行人之间的各种交互因素。并且它可以基于各种情况下的不同因素来区分行人周围的人行力的影响。对于物理互动,我们提出了一个新的\ emph {顺序场景共享机制}。每个时刻在每个时刻对一个代理的影响可以通过社会柔和关注与其他邻居共享,因此场景的影响在空间和时间尺寸中都是扩展。在这些改进的帮助下,我们成功地获得了社会和身体上可接受的预测轨迹。公共可用数据集的实验证明了SSAGCN的有效性,并取得了最先进的结果。
translated by 谷歌翻译
Understanding human motion behavior is critical for autonomous moving platforms (like self-driving cars and social robots) if they are to navigate human-centric environments. This is challenging because human motion is inherently multimodal: given a history of human motion paths, there are many socially plausible ways that people could move in the future. We tackle this problem by combining tools from sequence prediction and generative adversarial networks: a recurrent sequence-to-sequence model observes motion histories and predicts future behavior, using a novel pooling mechanism to aggregate information across people. We predict socially plausible futures by training adversarially against a recurrent discriminator, and encourage diverse predictions with a novel variety loss. Through experiments on several datasets we demonstrate that our approach outperforms prior work in terms of accuracy, variety, collision avoidance, and computational complexity.
translated by 谷歌翻译
Pedestrians follow different trajectories to avoid obstacles and accommodate fellow pedestrians. Any autonomous vehicle navigating such a scene should be able to foresee the future positions of pedestrians and accordingly adjust its path to avoid collisions. This problem of trajectory prediction can be viewed as a sequence generation task, where we are interested in predicting the future trajectory of people based on their past positions. Following the recent success of Recurrent Neural Network (RNN) models for sequence prediction tasks, we propose an LSTM model which can learn general human movement and predict their future trajectories. This is in contrast to traditional approaches which use hand-crafted functions such as Social forces. We demonstrate the performance of our method on several public datasets. Our model outperforms state-of-the-art methods on some of these datasets . We also analyze the trajectories predicted by our model to demonstrate the motion behaviour learned by our model.
translated by 谷歌翻译
轨迹预测旨在预测代理商可能的未来位置,考虑到他们的观察以及视频背景。这是许多自主平台所要求的,如跟踪,检测,机器人导航,自动驾驶汽车和许多其他电脑视觉应用。无论是代理人的内部人格因素,与社区的互动行为,还是周围环境的影响,所有这些都可能代表对代理商的未来计划的影响。然而,许多以前的方法模型和预测具有相同策略或“单曲”特征分布的代理商的行为,使其具有挑战性地给出足够的风格差异的预测。该稿件提出了利用风格假设和程式化预测的两个子网的多种式网络(MSN),以共同地以新颖的分类方式提供代理多种准式预测。我们使用代理人的终点计划及其交互上下文作为行为分类的基础,以便通过网络中的一系列样式通道自适应地学习多种不同的行为样式。然后,我们假设目标代理将根据这些分类样式中的每一个规划他们未来的行为,从而利用不同的风格频道,以便并行地提供具有重要风格差异的一系列预测。实验表明,所提出的MSN在两个广泛使用的数据集上以最新的最先进的方法优于10 \%-20 \%,并且定性地提出了更好的多样式特性。
translated by 谷歌翻译
Reasoning about human motion is an important prerequisite to safe and socially-aware robotic navigation. As a result, multi-agent behavior prediction has become a core component of modern human-robot interactive systems, such as self-driving cars. While there exist many methods for trajectory forecasting, most do not enforce dynamic constraints and do not account for environmental information (e.g., maps). Towards this end, we present Trajectron++, a modular, graph-structured recurrent model that forecasts the trajectories of a general number of diverse agents while incorporating agent dynamics and heterogeneous data (e.g., semantic maps). Trajectron++ is designed to be tightly integrated with robotic planning and control frameworks; for example, it can produce predictions that are optionally conditioned on ego-agent motion plans. We demonstrate its performance on several challenging real-world trajectory forecasting datasets, outperforming a wide array of state-ofthe-art deterministic and generative methods.
translated by 谷歌翻译
这本数字本书包含在物理模拟的背景下与深度学习相关的一切实际和全面的一切。尽可能多,所有主题都带有Jupyter笔记本的形式的动手代码示例,以便快速入门。除了标准的受监督学习的数据中,我们将看看物理丢失约束,更紧密耦合的学习算法,具有可微分的模拟,以及加强学习和不确定性建模。我们生活在令人兴奋的时期:这些方法具有从根本上改变计算机模拟可以实现的巨大潜力。
translated by 谷歌翻译
We introduce a Deep Stochastic IOC 1 RNN Encoderdecoder framework, DESIRE, for the task of future predictions of multiple interacting agents in dynamic scenes. DESIRE effectively predicts future locations of objects in multiple scenes by 1) accounting for the multi-modal nature of the future prediction (i.e., given the same context, future may vary), 2) foreseeing the potential future outcomes and make a strategic prediction based on that, and 3) reasoning not only from the past motion history, but also from the scene context as well as the interactions among the agents. DESIRE achieves these in a single end-to-end trainable neural network model, while being computationally efficient. The model first obtains a diverse set of hypothetical future prediction samples employing a conditional variational autoencoder, which are ranked and refined by the following RNN scoring-regression module. Samples are scored by accounting for accumulated future rewards, which enables better long-term strategic decisions similar to IOC frameworks. An RNN scene context fusion module jointly captures past motion histories, the semantic scene context and interactions among multiple agents. A feedback mechanism iterates over the ranking and refinement to further boost the prediction accuracy. We evaluate our model on two publicly available datasets: KITTI and Stanford Drone Dataset. Our experiments show that the proposed model significantly improves the prediction accuracy compared to other baseline methods.
translated by 谷歌翻译
预测动态场景中的行人轨迹仍然是各种应用中的关键问题,例如自主驾驶和社会意识的机器人。由于人类和人类对象的相互作用和人类随机性引起的未来不确定性,这种预测是挑战。基于生成式模型的方法通过采样潜在变量来处理未来的不确定性。然而,很少有研究探索了潜在变量的产生。在这项工作中,我们提出了具有伪Oracle(TPPO)的轨迹预测器,这是一种基于模型的基于模型的轨迹预测因子。第一个伪甲骨文是行人的移动方向,第二个是从地面真理轨迹估计的潜在变量。社会注意力模块用于基于行人移动方向与未来轨迹之间的相关性聚集邻居的交互。这种相关性受到行人的未来轨迹往往受到前方行人的影响。提出了一种潜在的变量预测器来估计观察和地面轨迹的潜在可变分布。此外,在训练期间,这两个分布之间的间隙最小化。因此,潜在的变量预测器可以估计观察到的轨迹的潜变量,以近似从地面真理轨迹估计。我们将TPPO与在几个公共数据集上的相关方法进行比较。结果表明,TPPO优于最先进的方法,具有低平均和最终位移误差。作为测试期间的采样时间下降,消融研究表明预测性能不会显着降低。
translated by 谷歌翻译
作为自主驱动系统的核心技术,行人轨迹预测可以显着提高主动车辆安全性的功能,减少道路交通损伤。在交通场景中,当遇到迎面而来的人时,行人可能会立即转动或停止,这通常会导致复杂的轨迹。为了预测这种不可预测的轨迹,我们可以深入了解行人之间的互动。在本文中,我们提出了一种名为Spatial Interaction Transformer(SIT)的新型生成方法,其通过注意机制学习行人轨迹的时空相关性。此外,我们介绍了条件变形Autiachoder(CVAE)框架来模拟未来行人的潜在行动状态。特别是,基于大规模的TRAFC数据集NUSCENES [2]的实验显示,坐下的性能优于最先进的(SOTA)方法。对挑战性的Eth和UCY数据集的实验评估概述了我们提出的模型的稳健性
translated by 谷歌翻译
近年来,人类运动轨迹预测是许多领域自治系统的重要任务。通过不同社区提出的多种新方法,缺乏标准化的基准和客观比较越来越成为评估进度并指导进一步研究的主要局限性。现有基准的范围和灵活性有限,无法进行相关实验,并说明了代理和环境的上下文提示。在本文中,我们提出了地图集,这是一个系统地评估人类运动轨迹预测算法的基准。 Atlas提供数据预处理功能,超参数优化,具有流行的数据集,并具有灵活性,可以进行设置和进行不充分的相关实验,以分析方法的准确性和鲁棒性。在ATLAS的示例应用中,我们比较了五个流行的模型和基于学习的预测指标,并发现,如果适当应用,基于物理的早期方法仍然具有竞争力。这样的结果证实了像Atlas这样的基准的必要性。
translated by 谷歌翻译
Motion prediction systems aim to capture the future behavior of traffic scenarios enabling autonomous vehicles to perform safe and efficient planning. The evolution of these scenarios is highly uncertain and depends on the interactions of agents with static and dynamic objects in the scene. GNN-based approaches have recently gained attention as they are well suited to naturally model these interactions. However, one of the main challenges that remains unexplored is how to address the complexity and opacity of these models in order to deal with the transparency requirements for autonomous driving systems, which includes aspects such as interpretability and explainability. In this work, we aim to improve the explainability of motion prediction systems by using different approaches. First, we propose a new Explainable Heterogeneous Graph-based Policy (XHGP) model based on an heterograph representation of the traffic scene and lane-graph traversals, which learns interaction behaviors using object-level and type-level attention. This learned attention provides information about the most important agents and interactions in the scene. Second, we explore this same idea with the explanations provided by GNNExplainer. Third, we apply counterfactual reasoning to provide explanations of selected individual scenarios by exploring the sensitivity of the trained model to changes made to the input data, i.e., masking some elements of the scene, modifying trajectories, and adding or removing dynamic agents. The explainability analysis provided in this paper is a first step towards more transparent and reliable motion prediction systems, important from the perspective of the user, developers and regulatory agencies. The code to reproduce this work is publicly available at https://github.com/sancarlim/Explainable-MP/tree/v1.1.
translated by 谷歌翻译
我们提出了一种新颖的场景表示,其编码达到距离 - 沿着可行轨迹的场景中的任何位置之间的距离。我们证明,该环境现场表示可以直接指导2D迷宫或3D室内场景中代理的动态行为。我们的环境领域是一种连续表示,通过使用离散采样的培训数据通过神经隐式功能学习。我们展示其在2D迷宫中的代理导航应用,3D室内环境中的人为轨迹预测。为了为人类生产物理似品和自然的轨迹,我们还学习了一种生成模型,该模型预测了人类通常出现的区域,并强制执行要在这些区域内定义的环境场。广泛的实验表明,所提出的方法可以有效准确地产生可行和合理的轨迹。
translated by 谷歌翻译
轨迹预测面临着困难,以捕获具有多样性和准确性的未来动力学的多模式性质。在本文中,我们提出了一种分布歧视(DISDIS)方法,可以通过区分潜在分布来预测个性化运动模式。由于每个人的习惯,每个人的运动模式都被个性化,我们的disdis学会了潜在分布来代表不同的运动模式,并通过对比度歧视来优化它。这种分布歧视鼓励潜在分布更具歧视性。我们的方法可以与现有的多模式随机预测模型集成为插件模块,以了解更具歧视性的潜在分布。为了评估潜在分布,我们进一步提出了一个新的度量标准,概率累积最小距离(PCMD)曲线,该曲线累计计算了分类概率的最小距离。对ETH和UCY数据集的实验结果显示了我们方法的有效性。
translated by 谷歌翻译
Interacting systems are prevalent in nature, from dynamical systems in physics to complex societal dynamics. The interplay of components can give rise to complex behavior, which can often be explained using a simple model of the system's constituent parts. In this work, we introduce the neural relational inference (NRI) model: an unsupervised model that learns to infer interactions while simultaneously learning the dynamics purely from observational data. Our model takes the form of a variational auto-encoder, in which the latent code represents the underlying interaction graph and the reconstruction is based on graph neural networks. In experiments on simulated physical systems, we show that our NRI model can accurately recover ground-truth interactions in an unsupervised manner. We further demonstrate that we can find an interpretable structure and predict complex dynamics in real motion capture and sports tracking data.
translated by 谷歌翻译
Autonomous navigation in crowded spaces poses a challenge for mobile robots due to the highly dynamic, partially observable environment. Occlusions are highly prevalent in such settings due to a limited sensor field of view and obstructing human agents. Previous work has shown that observed interactive behaviors of human agents can be used to estimate potential obstacles despite occlusions. We propose integrating such social inference techniques into the planning pipeline. We use a variational autoencoder with a specially designed loss function to learn representations that are meaningful for occlusion inference. This work adopts a deep reinforcement learning approach to incorporate the learned representation for occlusion-aware planning. In simulation, our occlusion-aware policy achieves comparable collision avoidance performance to fully observable navigation by estimating agents in occluded spaces. We demonstrate successful policy transfer from simulation to the real-world Turtlebot 2i. To the best of our knowledge, this work is the first to use social occlusion inference for crowd navigation.
translated by 谷歌翻译
建模人行走的动力是对计算机视觉的长期兴趣的问题。许多涉及行人轨迹预测的以前的作品将一组特定的单个动作定义为隐式模型组动作。在本文中,我们介绍了一个名为GP-GRAPH的新颖架构,该架构具有集体的小组表示,用于在拥挤的环境中有效的人行道轨迹预测,并且与所有类型的现有方法兼容。 GP-GRAPH的一个关键思想是将个人和小组关系的关系作为图表表示。为此,GP-Graph首先学会将每个行人分配给最可能的行为组。然后,使用此分配信息,GP编写是图形的组内和组间相互作用,分别考虑了组和群体关系中的人类关系。要具体,对于小组内相互作用,我们掩盖了相关组中的行人图边缘。我们还建议小组合并和不致密操作,以代表一个具有多个行人作为一个图节点的小组。最后,GP-GRAPH从两个组相互作用的综合特征中渗透了一个可获得社会上可接受的未来轨迹的概率图。此外,我们介绍了一个小组潜在的矢量抽样,以确保对一系列可能的未来轨迹的集体推断。进行了广泛的实验来验证我们的体系结构的有效性,该实验证明了通过公开可用的基准测试的绩效一致。代码可在https://github.com/inhwanbae/gpgraph上公开获取。
translated by 谷歌翻译
揭开多个代理之间的相互作用与过去的轨迹之间的相互作用至关重要。但是,以前的作品主要考虑与有限的关系推理的静态,成对的相互作用。为了促进更全面的互动建模和关系推理,我们提出了Dyngroupnet,这是一个动态群体感知的网络,i)可以在高度动态的场景中建模时间变化的交互; ii)捕获配对和小组互动; iii)理由互动强度和类别没有直接监督。基于Dyngroupnet,我们进一步设计了一个预测系统,以预测具有动态关系推理的社会合理轨迹。提出的预测系统利用高斯混合模型,多个抽样和预测细化,分别促进预测多样性,训练稳定性和轨迹平滑度。广泛的实验表明:1)dyngroupnet可以捕获随时间变化的群体行为,在轨迹预测过程中推断时间变化的交互类别和相互作用强度,而无需在物理模拟数据集上进行任何关系监督; 2)dyngroupnet优于最先进的轨迹预测方法,其显着改善22.6%/28.0%,26.9%/34.9%,5.1%/13.0%的ADE/FDE在NBA,NFL足球和SDD Datasets上的ADE/FDE并在ETH-COY数据集上实现最先进的性能。
translated by 谷歌翻译