从测试阶段的单个初始示例跟踪视觉对象已被广泛地作为一个/几次射击问题，即初始适应的一次性学习和在线适应的少量学习。近期几次拍摄的在线适应方法通过在离线阶段的复杂元学习优化中，从大量注释的训练数据中纳入了现有知识。这有助于在线深度跟踪器实现快速适应并降低跟踪的过度风险。在本文中，我们提出了一个简单但有效的递归最小二乘估计估计者辅助在线学习方法，但在不需要离线培训的情况下进行了几次拍摄的在线适应。它允许内置的内存保留机制进行模型，以记住关于之前看到的对象的知识，因此可以安全地从训练中安全地移除所看到的数据。这也与在防止灾难性遗忘的新出现的连续学习领域带有某些相似之处。这种机制使我们能够揭示现代在线深度跟踪器的力量，而不会产生过多的计算成本。我们根据在线学习家庭中的两个网络评估我们的方法，即在RT-MDNET中的多层的rceptrons和DIMP中的卷积神经网络。对若干具有挑战性的跟踪基准的一致性改进展示了其有效性和效率。

准确且强大的视觉对象跟踪是最具挑战性和最基本的计算机视觉问题之一。它需要在图像序列中估计目标的轨迹，仅给出其初始位置和分段，或者在边界框的形式中粗略近似。判别相关滤波器（DCF）和深度暹罗网络（SNS）被出现为主导跟踪范式，这导致了重大进展。在过去十年的视觉对象跟踪快速演变之后，该调查介绍了90多个DCFS和暹罗跟踪器的系统和彻底审查，基于九个跟踪基准。首先，我们介绍了DCF和暹罗跟踪核心配方的背景理论。然后，我们在这些跟踪范式中区分和全面地审查共享以及具体的开放研究挑战。此外，我们彻底分析了DCF和暹罗跟踪器对九个基准的性能，涵盖了视觉跟踪的不同实验方面：数据集，评估度量，性能和速度比较。通过提出根据我们的分析提出尊重开放挑战的建议和建议来完成调查。

The current strive towards end-to-end trainable computer vision systems imposes major challenges for the task of visual tracking. In contrast to most other vision problems, tracking requires the learning of a robust target-specific appearance model online, during the inference stage. To be end-to-end trainable, the online learning of the target model thus needs to be embedded in the tracking architecture itself. Due to the imposed challenges, the popular Siamese paradigm simply predicts a target feature template, while ignoring the background appearance information during inference. Consequently, the predicted model possesses limited target-background discriminability.We develop an end-to-end tracking architecture, capable of fully exploiting both target and background appearance information for target model prediction. Our architecture is derived from a discriminative learning loss by designing a dedicated optimization process that is capable of predicting a powerful model in only a few iterations. Furthermore, our approach is able to learn key aspects of the discriminative loss itself. The proposed tracker sets a new state-of-the-art on 6 tracking benchmarks, achieving an EAO score of 0.440 on VOT2018, while running at over 40 FPS. The code and models are available at https: //github.com/visionml/pytracking.

While recent years have witnessed astonishing improvements in visual tracking robustness, the advancements in tracking accuracy have been limited. As the focus has been directed towards the development of powerful classifiers, the problem of accurate target state estimation has been largely overlooked. In fact, most trackers resort to a simple multi-scale search in order to estimate the target bounding box. We argue that this approach is fundamentally limited since target estimation is a complex task, requiring highlevel knowledge about the object.We address this problem by proposing a novel tracking architecture, consisting of dedicated target estimation and classification components. High level knowledge is incorporated into the target estimation through extensive offline learning. Our target estimation component is trained to predict the overlap between the target object and an estimated bounding box. By carefully integrating target-specific information, our approach achieves previously unseen bounding box accuracy. We further introduce a classification component that is trained online to guarantee high discriminative power in the presence of distractors. Our final tracking framework sets a new state-of-the-art on five challenging benchmarks. On the new large-scale Track-ingNet dataset, our tracker ATOM achieves a relative gain of 15% over the previous best approach, while running at over 30 FPS. Code and models are available at https: //github.com/visionml/pytracking.

对人类对象相互作用的理解在第一人称愿景（FPV）中至关重要。遵循相机佩戴者操纵的对象的视觉跟踪算法可以提供有效的信息，以有效地建模此类相互作用。在过去的几年中，计算机视觉社区已大大提高了各种目标对象和场景的跟踪算法的性能。尽管以前有几次尝试在FPV域中利用跟踪器，但仍缺少对最先进跟踪器的性能的有条理分析。这项研究差距提出了一个问题，即应使用当前的解决方案``现成''还是应进行更多特定领域的研究。本文旨在为此类问题提供答案。我们介绍了FPV中单个对象跟踪的首次系统研究。我们的研究广泛分析了42个算法的性能，包括通用对象跟踪器和基线FPV特定跟踪器。分析是通过关注FPV设置的不同方面，引入新的绩效指标以及与FPV特定任务有关的。这项研究是通过引入Trek-150（由150个密集注释的视频序列组成的新型基准数据集）来实现的。我们的结果表明，FPV中的对象跟踪对当前的视觉跟踪器构成了新的挑战。我们强调了导致这种行为的因素，并指出了可能的研究方向。尽管遇到了困难，但我们证明了跟踪器为需要短期对象跟踪的FPV下游任务带来好处。我们预计，随着新的和FPV特定的方法学会得到研究，通用对象跟踪将在FPV中受欢迎。

基于无人机（UAV）基于无人机的视觉对象跟踪已实现了广泛的应用，并且由于其多功能性和有效性而引起了智能运输系统领域的越来越多的关注。作为深度学习革命性趋势的新兴力量，暹罗网络在基于无人机的对象跟踪中闪耀，其准确性，稳健性和速度有希望的平衡。由于开发了嵌入式处理器和深度神经网络的逐步优化，暹罗跟踪器获得了广泛的研究并实现了与无人机的初步组合。但是，由于无人机在板载计算资源和复杂的现实情况下，暹罗网络的空中跟踪仍然在许多方面都面临严重的障碍。为了进一步探索基于无人机的跟踪中暹罗网络的部署，这项工作对前沿暹罗跟踪器进行了全面的审查，以及使用典型的无人机板载处理器进行评估的详尽无人用分析。然后，进行板载测试以验证代表性暹罗跟踪器在现实世界无人机部署中的可行性和功效。此外，为了更好地促进跟踪社区的发展，这项工作分析了现有的暹罗跟踪器的局限性，并进行了以低弹片评估表示的其他实验。最后，深入讨论了基于无人机的智能运输系统的暹罗跟踪的前景。领先的暹罗跟踪器的统一框架，即代码库及其实验评估的结果，请访问https://github.com/vision4robotics/siamesetracking4uav。

传统上通过在线学习物体的外观模型来解决任意对象跟踪问题，作为视频本身的唯一培训数据。尽管这些方法的成功，但他们的在线方法本身限制了他们可以学习的模型的丰富性。最近，已经进行了几次尝试利用深度卷积网络的表现力。然而，当预先知道对象轨道时，有必要在线执行随机梯度下降以调整网络的权重，严重影响系统的速度。本文在ILSVRC15数据集上装备了一种基本的跟踪算法，该历程在ILSVRC15数据集上训练了训练的端到端，用于视频中的对象检测。我们的跟踪器以超出实时的帧速率运行，尽管其极端简单，但在多个基准中实现了最先进的性能。

在本文中，我们介绍了Siammask，这是一个实时使用相同简单方法实时执行视觉对象跟踪和视频对象分割的框架。我们通过通过二进制细分任务来增强其损失，从而改善了流行的全面暹罗方法的离线培训程序。离线训练完成后，SiamMask只需要一个单个边界框来初始化，并且可以同时在高框架速率下进行视觉对象跟踪和分割。此外，我们表明可以通过简单地以级联的方式重新使用多任务模型来扩展框架以处理多个对象跟踪和细分。实验结果表明，我们的方法具有较高的处理效率，每秒约55帧。它可以在视觉对象跟踪基准测试中产生实时最新结果，同时以高速进行视频对象分割基准测试以高速显示竞争性能。

跟踪需要为推理阶段构建目标的判别模型。实现这一目标的有效方法是在线学习，可以舒适地占据截肢培训的型号。最近的研究表明，由于其像素级别歧视，视觉跟踪从统一视觉跟踪和分割的统一中受益匪浅。但是，对这种统一模型进行在线学习产生巨大挑战。分段模型不能轻易地从视觉跟踪方案中给出的先前信息学习。在本文中，我们提出了TrackM1P：一种新的元学习方法，优化了仅从部分信息学习以解决强加的挑战。我们的模型能够广泛利用有限的事先信息，因此具有比其他在线学习方法更强大的目标 - 背景辨别性。凭经验，我们表明我们的模型在竞争模型上实现了最先进的性能和切实改善。我们的模式实现了VOT2019，VOT2018，VOT2018和VOT2016数据集的66.0％，67.1％，68.5％的平均重叠增长了6.4％，7.3％，高于我们基线的6.4％。代码将公开可用。

基于模板的鉴别性跟踪器是目前主导的跟踪范例由于其稳健性，但不限于边界框跟踪和有限的转换模型，这降低了它们的本地化准确性。我们提出了一个判别的单次分割跟踪器 - D3S2，其缩小了视觉对象跟踪和视频对象分段之间的差距。单次网络应用两个具有互补的几何属性的目标模型，一个不变的变换，包括非刚性变形，另一个假设刚性对象同时实现强大的在线目标分段。通过解耦对象和特征比例估计，进一步提高了整体跟踪可靠性。没有每数据集FineTuning，并且仅用于分段作为主要输出，D3S2胜过最近的短期跟踪基准Vot2020上的所有已发布的跟踪器，并非常接近GOT-10K上的最先进的跟踪器， TrackingNet，OTB100和Lasot。 D3S2优于视频对象分段基准上的前导分割跟踪器SIAMMASK，并与顶部视频对象分段算法进行操作。

Continual Learning (CL) is a field dedicated to devise algorithms able to achieve lifelong learning. Overcoming the knowledge disruption of previously acquired concepts, a drawback affecting deep learning models and that goes by the name of catastrophic forgetting, is a hard challenge. Currently, deep learning methods can attain impressive results when the data modeled does not undergo a considerable distributional shift in subsequent learning sessions, but whenever we expose such systems to this incremental setting, performance drop very quickly. Overcoming this limitation is fundamental as it would allow us to build truly intelligent systems showing stability and plasticity. Secondly, it would allow us to overcome the onerous limitation of retraining these architectures from scratch with the new updated data. In this thesis, we tackle the problem from multiple directions. In a first study, we show that in rehearsal-based techniques (systems that use memory buffer), the quantity of data stored in the rehearsal buffer is a more important factor over the quality of the data. Secondly, we propose one of the early works of incremental learning on ViTs architectures, comparing functional, weight and attention regularization approaches and propose effective novel a novel asymmetric loss. At the end we conclude with a study on pretraining and how it affects the performance in Continual Learning, raising some questions about the effective progression of the field. We then conclude with some future directions and closing remarks.

Siamese network based trackers formulate tracking as convolutional feature cross-correlation between a target template and a search region. However, Siamese trackers still have an accuracy gap compared with state-of-theart algorithms and they cannot take advantage of features from deep networks, such as ResNet-50 or deeper. In this work we prove the core reason comes from the lack of strict translation invariance. By comprehensive theoretical analysis and experimental validations, we break this restriction through a simple yet effective spatial aware sampling strategy and successfully train a ResNet-driven Siamese tracker with significant performance gain. Moreover, we propose a new model architecture to perform layer-wise and depthwise aggregations, which not only further improves the accuracy but also reduces the model size. We conduct extensive ablation studies to demonstrate the effectiveness of the proposed tracker, which obtains currently the best results on five large tracking benchmarks, including OTB2015, VOT2018, UAV123, LaSOT, and TrackingNet. Our model will be released to facilitate further researches.

In recent years, Discriminative Correlation Filter (DCF) based methods have significantly advanced the state-of-theart in tracking. However, in the pursuit of ever increasing tracking performance, their characteristic speed and realtime capability have gradually faded. Further, the increasingly complex models, with massive number of trainable parameters, have introduced the risk of severe over-fitting. In this work, we tackle the key causes behind the problems of computational complexity and over-fitting, with the aim of simultaneously improving both speed and performance.We revisit the core DCF formulation and introduce: (i) a factorized convolution operator, which drastically reduces the number of parameters in the model; (ii) a compact generative model of the training sample distribution, that significantly reduces memory and time complexity, while providing better diversity of samples; (iii) a conservative model update strategy with improved robustness and reduced complexity. We perform comprehensive experiments on four benchmarks: VOT2016, UAV123, OTB-2015, and Temple-Color. When using expensive deep features, our tracker provides a 20-fold speedup and achieves a 13.0% relative gain in Expected Average Overlap compared to the top ranked method [12] in the VOT2016 challenge. Moreover, our fast variant, using hand-crafted features, operates at 60 Hz on a single CPU, while obtaining 65.0% AUC on OTB-2015.

为了跟踪视频中的目标，当前的视觉跟踪器通常采用贪婪搜索每个帧中目标对象定位，也就是说，将选择最大响应分数的候选区域作为每个帧的跟踪结果。但是，我们发现这可能不是一个最佳选择，尤其是在遇到挑战性的跟踪方案（例如重闭塞和快速运动）时。为了解决这个问题，我们建议维护多个跟踪轨迹并将光束搜索策略应用于视觉跟踪，以便可以识别出更少的累积错误的轨迹。因此，本文介绍了一种新型的基于梁搜索策略的新型多代理增强学习策略，称为横梁。它主要是受图像字幕任务的启发，该任务将图像作为输入，并使用Beam搜索算法生成多种描述。因此，我们通过多个并行决策过程来将跟踪提出作为样本选择问题，每个过程旨在将一个样本作为每个帧的跟踪结果选择。每个维护的轨迹都与代理商相关联，以执行决策并确定应采取哪些操作来更新相关信息。处理所有帧时，我们将最大累积分数作为跟踪结果选择轨迹。在七个流行的跟踪基准数据集上进行了广泛的实验证实了所提出的算法的有效性。

Artificial neural networks thrive in solving the classification problem for a particular rigid task, acquiring knowledge through generalized learning behaviour from a distinct training phase. The resulting network resembles a static entity of knowledge, with endeavours to extend this knowledge without targeting the original task resulting in a catastrophic forgetting. Continual learning shifts this paradigm towards networks that can continually accumulate knowledge over different tasks without the need to retrain from scratch. We focus on task incremental classification, where tasks arrive sequentially and are delineated by clear boundaries. Our main contributions concern (1) a taxonomy and extensive overview of the state-of-the-art; (2) a novel framework to continually determine the stability-plasticity trade-off of the continual learner; (3) a comprehensive experimental comparison of 11 state-of-the-art continual learning methods and 4 baselines. We empirically scrutinize method strengths and weaknesses on three benchmarks, considering Tiny Imagenet and large-scale unbalanced iNaturalist and a sequence of recognition datasets. We study the influence of model capacity, weight decay and dropout regularization, and the order in which the tasks are presented, and qualitatively compare methods in terms of required memory, computation time and storage.

估计目标范围在视觉对象跟踪中构成了基本挑战。通常，跟踪器以箱子为中心，并且完全依靠边界框来定义场景中的目标。实际上，对象通常具有复杂的形状，并且与图像轴不符。在这些情况下，边界框不能提供对目标的准确描述，并且通常包含大多数背景像素。我们提出了一个以细分为中心的跟踪管道，该管道不仅会产生高度准确的分割掩码，而且还可以使用分割掩码而不是边界框来使用内部。因此，我们的跟踪器能够更好地学习目标表示形式，该目标表示明确将场景中的目标与背景内容区分开来。为了实现具有挑战性的跟踪方案的必要鲁棒性，我们提出了一个单独的实例本地化组件，该组件用于在产生输出掩码时用于调节分割解码器。我们从分段掩码中推断出一个边界框，验证我们的跟踪器在挑战跟踪数据集方面，并在LASOT上实现新的最新状态，并以69.7％的速度获得了AUC得分。由于大多数跟踪数据集不包含掩码注释，因此我们无法使用它们来评估预测的分割掩码。相反，我们在两个流行的视频对象细分数据集上验证了分割质量。

Visual object tracking has been a fundamental topic in recent years and many deep learning based trackers have achieved state-of-the-art performance on multiple benchmarks. However, most of these trackers can hardly get top performance with real-time speed. In this paper, we propose the Siamese region proposal network (Siamese-RPN) which is end-to-end trained off-line with large-scale image pairs. Specifically, it consists of Siamese subnetwork for feature extraction and region proposal subnetwork including the classification branch and regression branch. In the inference phase, the proposed framework is formulated as a local one-shot detection task. We can pre-compute the template branch of the Siamese subnetwork and formulate the correlation layers as trivial convolution layers to perform online tracking. Benefit from the proposal refinement, traditional multi-scale test and online fine-tuning can be discarded. The Siamese-RPN runs at 160 FPS while achieving leading performance in VOT2015, VOT2016 and VOT2017 real-time challenges.

最近的跟踪器采用变压器来组合或替换广泛使用的重新NET作为其新的骨干网络。尽管他们的跟踪器在常规场景中运行良好，但是他们只是将2D功能弄平为序列，以更好地匹配变压器。我们认为这些操作忽略了目标对象的空间先验，这可能仅导致次优结果。此外，许多作品表明，自我注意力实际上是一个低通滤波器，它与输入功能或键/查询无关。也就是说，它可能会抑制输入功能的高频组成部分，并保留甚至放大低频信息。为了解决这些问题，在本文中，我们提出了一个统一的空间频率变压器，该变压器同时建模高斯空间先验和高频强调（GPHA）。具体而言，高斯空间先验是使用双重多层感知器（MLP）生成的，并注入了通过将查询和自我注意的关键特征乘产生的相似性矩阵。输出将被馈入软磁层，然后分解为两个组件，即直接信号和高频信号。低通和高通的分支被重新缩放并组合以实现全通，因此，高频特征将在堆叠的自发层中得到很好的保护。我们进一步将空间频率变压器整合到暹罗跟踪框架中，并提出一种新颖的跟踪算法，称为SFTRANST。基于跨级融合的SwintransFormer被用作骨干，还使用多头交叉意见模块来增强搜索和模板功能之间的相互作用。输出将被馈入跟踪头以进行目标定位。短期和长期跟踪基准的广泛实验都证明了我们提出的框架的有效性。

无人驾驶飞机（UAV）跟踪对于诸如交货和农业等广泛应用具有重要意义。该领域的先前基准分析主要集中在小规模的跟踪问题上，同时忽略了数据模式的类型，目标类别和方案的多样性以及所涉及的评估协议的数量，从而极大地隐藏了深度无人机跟踪的巨大功能。在这项工作中，我们提出了迄今为止最大的公共无人机跟踪基准Webuav-3M，以促进深度无人机跟踪器的开发和评估。 Webuav-3M在4,500个视频中包含超过330万帧，并提供223个高度多样化的目标类别。每个视频都通过有效且可扩展的半自动目标注释（SATA）管道密集注释。重要的是，要利用语言和音频的互补优势，我们通过提供自然语言规格和音频描述来丰富Webuav-3M。我们认为，这种增加将大大促进未来的研究，以探索语言功能和音频提示，用于多模式无人机跟踪。此外，构建了scenario约束（UTUSC）评估协议和七个具有挑战性的场景子测验集，以使社区能够开发，适应和评估各种类型的高级跟踪器。我们提供了43个代表性跟踪器的广泛评估和详细分析，并设想了深度无人机跟踪及其他领域的未来研究方向。数据集，工具包和基线结果可在\ url {https://github.com/983632847/webuav-3m}中获得。

尽管机器学习在视觉对象跟踪的任务上进行了广泛采用，但最近基于学习的方法在很大程度上忽略了一个事实，即视觉跟踪是其本质上的序列级任务。他们在很大程度上依赖框架级训练，这不可避免地会导致数据分布和任务目标的培训和测试之间的不一致。这项工作介绍了基于强化学习的视觉跟踪序列训练策略，并讨论了数据采样，学习目标和数据增强的序列级设计如何提高跟踪算法的准确性和稳健性。我们对包括LASOT，TrackingNet和GoT-10K在内的标准基准测试的实验表明，四个代表性跟踪模型，SiamRPN ++，Siamattn，Transt和TRDIMP，通过在不修改建筑架构的情况下将提出的方法纳入训练中，从而不断改进。