Siamese network based trackers formulate tracking as convolutional feature cross-correlation between a target template and a search region. However, Siamese trackers still have an accuracy gap compared with state-of-theart algorithms and they cannot take advantage of features from deep networks, such as ResNet-50 or deeper. In this work we prove the core reason comes from the lack of strict translation invariance. By comprehensive theoretical analysis and experimental validations, we break this restriction through a simple yet effective spatial aware sampling strategy and successfully train a ResNet-driven Siamese tracker with significant performance gain. Moreover, we propose a new model architecture to perform layer-wise and depthwise aggregations, which not only further improves the accuracy but also reduces the model size. We conduct extensive ablation studies to demonstrate the effectiveness of the proposed tracker, which obtains currently the best results on five large tracking benchmarks, including OTB2015, VOT2018, UAV123, LaSOT, and TrackingNet. Our model will be released to facilitate further researches.
translated by 谷歌翻译
基于无人机(UAV)基于无人机的视觉对象跟踪已实现了广泛的应用,并且由于其多功能性和有效性而引起了智能运输系统领域的越来越多的关注。作为深度学习革命性趋势的新兴力量,暹罗网络在基于无人机的对象跟踪中闪耀,其准确性,稳健性和速度有希望的平衡。由于开发了嵌入式处理器和深度神经网络的逐步优化,暹罗跟踪器获得了广泛的研究并实现了与无人机的初步组合。但是,由于无人机在板载计算资源和复杂的现实情况下,暹罗网络的空中跟踪仍然在许多方面都面临严重的障碍。为了进一步探索基于无人机的跟踪中暹罗网络的部署,这项工作对前沿暹罗跟踪器进行了全面的审查,以及使用典型的无人机板载处理器进行评估的详尽无人用分析。然后,进行板载测试以验证代表性暹罗跟踪器在现实世界无人机部署中的可行性和功效。此外,为了更好地促进跟踪社区的发展,这项工作分析了现有的暹罗跟踪器的局限性,并进行了以低弹片评估表示的其他实验。最后,深入讨论了基于无人机的智能运输系统的暹罗跟踪的前景。领先的暹罗跟踪器的统一框架,即代码库及其实验评估的结果,请访问https://github.com/vision4robotics/siamesetracking4uav。
translated by 谷歌翻译
准确且强大的视觉对象跟踪是最具挑战性和最基本的计算机视觉问题之一。它需要在图像序列中估计目标的轨迹,仅给出其初始位置和分段,或者在边界框的形式中粗略近似。判别相关滤波器(DCF)和深度暹罗网络(SNS)被出现为主导跟踪范式,这导致了重大进展。在过去十年的视觉对象跟踪快速演变之后,该调查介绍了90多个DCFS和暹罗跟踪器的系统和彻底审查,基于九个跟踪基准。首先,我们介绍了DCF和暹罗跟踪核心配方的背景理论。然后,我们在这些跟踪范式中区分和全面地审查共享以及具体的开放研究挑战。此外,我们彻底分析了DCF和暹罗跟踪器对九个基准的性能,涵盖了视觉跟踪的不同实验方面:数据集,评估度量,性能和速度比较。通过提出根据我们的分析提出尊重开放挑战的建议和建议来完成调查。
translated by 谷歌翻译
在本文中,我们介绍了Siammask,这是一个实时使用相同简单方法实时执行视觉对象跟踪和视频对象分割的框架。我们通过通过二进制细分任务来增强其损失,从而改善了流行的全面暹罗方法的离线培训程序。离线训练完成后,SiamMask只需要一个单个边界框来初始化,并且可以同时在高框架速率下进行视觉对象跟踪和分割。此外,我们表明可以通过简单地以级联的方式重新使用多任务模型来扩展框架以处理多个对象跟踪和细分。实验结果表明,我们的方法具有较高的处理效率,每秒约55帧。它可以在视觉对象跟踪基准测试中产生实时最新结果,同时以高速进行视频对象分割基准测试以高速显示竞争性能。
translated by 谷歌翻译
基于模板的鉴别性跟踪器是目前主导的跟踪范例由于其稳健性,但不限于边界框跟踪和有限的转换模型,这降低了它们的本地化准确性。我们提出了一个判别的单次分割跟踪器 - D3S2,其缩小了视觉对象跟踪和视频对象分段之间的差距。单次网络应用两个具有互补的几何属性的目标模型,一个不变的变换,包括非刚性变形,另一个假设刚性对象同时实现强大的在线目标分段。通过解耦对象和特征比例估计,进一步提高了整体跟踪可靠性。没有每数据集FineTuning,并且仅用于分段作为主要输出,D3S2胜过最近的短期跟踪基准Vot2020上的所有已发布的跟踪器,并非常接近GOT-10K上的最先进的跟踪器, TrackingNet,OTB100和Lasot。 D3S2优于视频对象分段基准上的前导分割跟踪器SIAMMASK,并与顶部视频对象分段算法进行操作。
translated by 谷歌翻译
Visual object tracking has been a fundamental topic in recent years and many deep learning based trackers have achieved state-of-the-art performance on multiple benchmarks. However, most of these trackers can hardly get top performance with real-time speed. In this paper, we propose the Siamese region proposal network (Siamese-RPN) which is end-to-end trained off-line with large-scale image pairs. Specifically, it consists of Siamese subnetwork for feature extraction and region proposal subnetwork including the classification branch and regression branch. In the inference phase, the proposed framework is formulated as a local one-shot detection task. We can pre-compute the template branch of the Siamese subnetwork and formulate the correlation layers as trivial convolution layers to perform online tracking. Benefit from the proposal refinement, traditional multi-scale test and online fine-tuning can be discarded. The Siamese-RPN runs at 160 FPS while achieving leading performance in VOT2015, VOT2016 and VOT2017 real-time challenges.
translated by 谷歌翻译
基于暹罗的跟踪器在Visual Object跟踪任务上实现了有希望的性能。大多数现有的基于暹罗的跟踪器包含两个单独的跟踪分支,包括分类分支和边界框回归分支。此外,图像分割提供了obetain更准确的目标区域的替代方法。在本文中,我们提出了一种具有两个阶段的新型跟踪器:检测和分割。检测阶段能够通过暹罗网络定位目标。然后,通过在第一阶段中的粗状态估计,通过分割模块获得更准确的跟踪结果。我们对四个基准进行实验。我们的方法可以实现最先进的结果,在VOT2016,VOT2018上的51.3美元\%$ 52.6 $ \%$分别在VOT2018和VOT2019数据集上的39.0 $ \%$。
translated by 谷歌翻译
我们呈现恐惧,新颖,快速,高效,准确,强大的暹罗视觉跟踪器。我们介绍了对象模型适配的架构块,称为双模板表示,以及像素 - 明智的融合块,以实现模型的额外灵活性和效率。双模板模块仅包含单个学习参数的时间信息,而像素-Wise融合块与标准相关模块相比,像素-Wise融合块对具有较少参数的判别特征进行了更多的辨别特征。通过用新型模块插入复杂的骨干,恐惧-M和恐惧-L跟踪器在既准确性和效率的几个学术基准上超过大多数暹粒例子。使用轻质骨干,优化的版本恐惧-XS提供了超过10倍的跟踪跟踪,而不是当前暹罗跟踪器,同时保持最先进的结果。 GEAF-XS跟踪器比LightTrack [62]更小2.4倍,比LightTrack [62]更高。此外,我们通过在能量消耗和执行速度上引入基准来扩展模型效率的定义。源代码,预先训练的模型和评估协议将根据要求提供
translated by 谷歌翻译
最近,我们看到了基于深神经网络(DNN)的视觉跟踪解决方案的快速发展。一些跟踪器将基于DNN的解决方案与判别相关滤波器(DCF)相结合,以提取语义特征并成功地提供最新的跟踪准确性。但是,这些解决方案是高度计算密集型的,需要长时间处理时间,从而导致无抵押的实时性能。为了提供高精度和可靠的实时性能,我们提出了一个名为Siamvgg的新颖跟踪器。它结合了卷积神经网络(CNN)主链和互相关操作员,并利用示例图像中的功能以进行更准确的对象跟踪。 Siamvgg的体系结构是根据VGG-16自定义的,其参数由示例性图像和所需的输入视频帧共享。我们在OTB-2013/50/100和Dot 2015/2016/2017数据集中证明了拟议的暹罗,具有STATE-ORT-TEA-ART精度,同时保持在GTX 1080TI上运行的50 FPS的体面实时性能。与Dot2017挑战中的ECO和C-COT相比,我们的设计可以实现预期平均重叠(EAO)的预期平均重叠(EAO)。
translated by 谷歌翻译
While recent years have witnessed astonishing improvements in visual tracking robustness, the advancements in tracking accuracy have been limited. As the focus has been directed towards the development of powerful classifiers, the problem of accurate target state estimation has been largely overlooked. In fact, most trackers resort to a simple multi-scale search in order to estimate the target bounding box. We argue that this approach is fundamentally limited since target estimation is a complex task, requiring highlevel knowledge about the object.We address this problem by proposing a novel tracking architecture, consisting of dedicated target estimation and classification components. High level knowledge is incorporated into the target estimation through extensive offline learning. Our target estimation component is trained to predict the overlap between the target object and an estimated bounding box. By carefully integrating target-specific information, our approach achieves previously unseen bounding box accuracy. We further introduce a classification component that is trained online to guarantee high discriminative power in the presence of distractors. Our final tracking framework sets a new state-of-the-art on five challenging benchmarks. On the new large-scale Track-ingNet dataset, our tracker ATOM achieves a relative gain of 15% over the previous best approach, while running at over 30 FPS. Code and models are available at https: //github.com/visionml/pytracking.
translated by 谷歌翻译
传统上通过在线学习物体的外观模型来解决任意对象跟踪问题,作为视频本身的唯一培训数据。尽管这些方法的成功,但他们的在线方法本身限制了他们可以学习的模型的丰富性。最近,已经进行了几次尝试利用深度卷积网络的表现力。然而,当预先知道对象轨道时,有必要在线执行随机梯度下降以调整网络的权重,严重影响系统的速度。本文在ILSVRC15数据集上装备了一种基本的跟踪算法,该历程在ILSVRC15数据集上训练了训练的端到端,用于视频中的对象检测。我们的跟踪器以超出实时的帧速率运行,尽管其极端简单,但在多个基准中实现了最先进的性能。
translated by 谷歌翻译
In recent years, Siamese network based trackers have significantly advanced the state-of-the-art in real-time tracking. Despite their success, Siamese trackers tend to suffer from high memory costs, which restrict their applicability to mobile devices with tight memory budgets. To address this issue, we propose a distilled Siamese tracking framework to learn small, fast and accurate trackers (students), which capture critical knowledge from large Siamese trackers (teachers) by a teacher-students knowledge distillation model. This model is intuitively inspired by the one teacher vs. multiple students learning method typically employed in schools. In particular, our model contains a single teacher-student distillation module and a student-student knowledge sharing mechanism. The former is designed using a tracking-specific distillation strategy to transfer knowledge from a teacher to students. The latter is utilized for mutual learning between students to enable in-depth knowledge understanding. Extensive empirical evaluations on several popular Siamese trackers demonstrate the generality and effectiveness of our framework. Moreover, the results on five tracking benchmarks show that the proposed distilled trackers achieve compression rates of up to 18$\times$ and frame-rates of $265$ FPS, while obtaining comparable tracking accuracy compared to base models.
translated by 谷歌翻译
最近,基于模板的跟踪器已成为领先的跟踪算法,在效率和准确性方面具有希望的性能。然而,查询特征与给定模板之间的相关操作仅利用准确的目标本地化,导致状态估计误差,特别是当目标遭受严重可变形变化时。为了解决这个问题,已经提出了基于分段的跟踪器,以便使用每像素匹配来有效地提高可变形物体的跟踪性能。然而,大多数现有跟踪器仅指初始帧中的目标特征,从而缺乏处理具有挑战性因素的辨别能力,例如,类似的分心,背景杂乱,外观变化等。在此目的,我们提出了一种动态的紧凑型存储器嵌入以增强基于分段的可变形视觉跟踪方法的辨别。具体而言,我们初始化与第一帧中的目标功能嵌入的内存嵌入。在跟踪过程中,与现有内存具有高相关的当前目标特征被更新为在线嵌入的内存。为了进一步提高可变形对象的分割精度,我们采用了点对集的匹配策略来测量像素 - 方向查询特征和整个模板之间的相关性,以捕获更详细的变形信息。关于六个具有挑战性的跟踪基准的广泛评估,包括VOT2016,VOT2018,VOT2019,GOT-10K,TrackingNet和莱斯特展示了我们对近期近似追踪者的方法的优势。此外,我们的方法优于基于出色的基于分段的跟踪器,即DVIS2017基准测试。
translated by 谷歌翻译
更复杂和强大的神经网络模型的设计在视觉对象跟踪中具有显着提升的最先进。这些前进可以归因于更深的网络,或引入新的构建块,例如变形金刚。然而,在追求增加的跟踪性能时,有效的跟踪架构令人惊讶地注意到很少的关注。在本文中,我们介绍了用于实时视觉对象跟踪的高效变压器的示例变压器。 E.T.Track我们的视觉跟踪器包含示例变换器层,在CPU上以47 FPS运行。这比其他基于变压器的型号快8倍,使其成为唯一基于实时变压器的跟踪器。与可在标准CPU上实时运行的轻量级跟踪器相比,E.T.Track始终如一地优于锯齿,OTB-100,NFS,TrackingNet和Vot-ST2020数据集上的所有其他方法。代码很快将在https://github.com/visionml/pytracking上发布。
translated by 谷歌翻译
最近的跟踪器采用变压器来组合或替换广泛使用的重新NET作为其新的骨干网络。尽管他们的跟踪器在常规场景中运行良好,但是他们只是将2D功能弄平为序列,以更好地匹配变压器。我们认为这些操作忽略了目标对象的空间先验,这可能仅导致次优结果。此外,许多作品表明,自我注意力实际上是一个低通滤波器,它与输入功能或键/查询无关。也就是说,它可能会抑制输入功能的高频组成部分,并保留甚至放大低频信息。为了解决这些问题,在本文中,我们提出了一个统一的空间频率变压器,该变压器同时建模高斯空间先验和高频强调(GPHA)。具体而言,高斯空间先验是使用双重多层感知器(MLP)生成的,并注入了通过将查询和自我注意的关键特征乘产生的相似性矩阵。输出将被馈入软磁层,然后分解为两个组件,即直接信号和高频信号。低通和高通的分支被重新缩放并组合以实现全通,因此,高频特征将在堆叠的自发层中得到很好的保护。我们进一步将空间频率变压器整合到暹罗跟踪框架中,并提出一种新颖的跟踪算法,称为SFTRANST。基于跨级融合的SwintransFormer被用作骨干,还使用多头交叉意见模块来增强搜索和模板功能之间的相互作用。输出将被馈入跟踪头以进行目标定位。短期和长期跟踪基准的广泛实验都证明了我们提出的框架的有效性。
translated by 谷歌翻译
我们介绍了一个基于仅用于跟踪的变压器的暹罗样的双分支网络。给定模板和搜索映像,我们将它们分成非重叠补丁,并基于其在注意窗口中的其他人的匹配结果提取每个补丁的特征向量。对于每个令牌,我们估计它是否包含目标对象和相应的大小。该方法的优点是,该特征从匹配中学到,最终匹配。因此,功能与目标跟踪任务对齐。该方法实现更好或比较的结果作为首先使用CNN提取特征的最佳性能,然后使用变压器熔断它们。它优于GOT-10K和VOT2020基准上的最先进的方法。此外,该方法在一个GPU上实现了实时推理速度(约为40美元的FPS)。代码和模型将被释放。
translated by 谷歌翻译
In this paper we illustrate how to perform both visual object tracking and semi-supervised video object segmentation, in real-time, with a single simple approach. Our method, dubbed SiamMask, improves the offline training procedure of popular fully-convolutional Siamese approaches for object tracking by augmenting their loss with a binary segmentation task. Once trained, SiamMask solely relies on a single bounding box initialisation and operates online, producing class-agnostic object segmentation masks and rotated bounding boxes at 55 frames per second. Despite its simplicity, versatility and fast speed, our strategy allows us to establish a new state of the art among real-time trackers on VOT-2018, while at the same time demonstrating competitive performance and the best speed for the semisupervised video object segmentation task on DAVIS-2016 and DAVIS-2017. The project website is http://www. robots.ox.ac.uk/ ˜qwang/SiamMask.
translated by 谷歌翻译
The current strive towards end-to-end trainable computer vision systems imposes major challenges for the task of visual tracking. In contrast to most other vision problems, tracking requires the learning of a robust target-specific appearance model online, during the inference stage. To be end-to-end trainable, the online learning of the target model thus needs to be embedded in the tracking architecture itself. Due to the imposed challenges, the popular Siamese paradigm simply predicts a target feature template, while ignoring the background appearance information during inference. Consequently, the predicted model possesses limited target-background discriminability.We develop an end-to-end tracking architecture, capable of fully exploiting both target and background appearance information for target model prediction. Our architecture is derived from a discriminative learning loss by designing a dedicated optimization process that is capable of predicting a powerful model in only a few iterations. Furthermore, our approach is able to learn key aspects of the discriminative loss itself. The proposed tracker sets a new state-of-the-art on 6 tracking benchmarks, achieving an EAO score of 0.440 on VOT2018, while running at over 40 FPS. The code and models are available at https: //github.com/visionml/pytracking.
translated by 谷歌翻译
对人类对象相互作用的理解在第一人称愿景(FPV)中至关重要。遵循相机佩戴者操纵的对象的视觉跟踪算法可以提供有效的信息,以有效地建模此类相互作用。在过去的几年中,计算机视觉社区已大大提高了各种目标对象和场景的跟踪算法的性能。尽管以前有几次尝试在FPV域中利用跟踪器,但仍缺少对最先进跟踪器的性能的有条理分析。这项研究差距提出了一个问题,即应使用当前的解决方案``现成''还是应进行更多特定领域的研究。本文旨在为此类问题提供答案。我们介绍了FPV中单个对象跟踪的首次系统研究。我们的研究广泛分析了42个算法的性能,包括通用对象跟踪器和基线FPV特定跟踪器。分析是通过关注FPV设置的不同方面,引入新的绩效指标以及与FPV特定任务有关的。这项研究是通过引入Trek-150(由150个密集注释的视频序列组成的新型基准数据集)来实现的。我们的结果表明,FPV中的对象跟踪对当前的视觉跟踪器构成了新的挑战。我们强调了导致这种行为的因素,并指出了可能的研究方向。尽管遇到了困难,但我们证明了跟踪器为需要短期对象跟踪的FPV下游任务带来好处。我们预计,随着新的和FPV特定的方法学会得到研究,通用对象跟踪将在FPV中受欢迎。
translated by 谷歌翻译
Maintaining the identity of multiple objects in real-time video is a challenging task, as it is not always feasible to run a detector on every frame. Thus, motion estimation systems are often employed, which either do not scale well with the number of targets or produce features with limited semantic information. To solve the aforementioned problems and allow the tracking of dozens of arbitrary objects in real-time, we propose SiamMOTION. SiamMOTION includes a novel proposal engine that produces quality features through an attention mechanism and a region-of-interest extractor fed by an inertia module and powered by a feature pyramid network. Finally, the extracted tensors enter a comparison head that efficiently matches pairs of exemplars and search areas, generating quality predictions via a pairwise depthwise region proposal network and a multi-object penalization module. SiamMOTION has been validated on five public benchmarks, achieving leading performance against current state-of-the-art trackers. Code available at: https://github.com/lorenzovaquero/SiamMOTION
translated by 谷歌翻译