主动学习是自动化机器学习系统的重要技术。与旨在自动化神经网络体系结构设计的神经体系结构搜索（NAS）相反，主动学习旨在自动化培训数据选择。对于训练长尾巴的任务尤其重要，在该任务中，在该任务中，稀疏的样品分布稀疏。主动学习通过逐步培训模型，以有效的数据选择来减轻昂贵的数据注释问题。它没有注释所有未标记的样本，而是迭代选择并注释最有价值的样本。主动学习在图像分类中很受欢迎，但在对象检测中尚未得到充分探索。当前的大多数对象检测方法都通过不同的设置进行评估，因此很难公平地比较其性能。为了促进该领域的研究，本文贡献了一个活跃的学习基准框架，称为Albench，用于评估对象检测中的主动学习。该Albench框架在自动深层模型训练系统上开发，易于使用，与不同的主动学习算法兼容，并确保使用相同的培训和测试协议。我们希望这种自动化的基准系统能够帮助研究人员轻松复制文学的表现，并与先前的艺术进行客观的比较。该代码将通过GitHub发布。

As an important data selection schema, active learning emerges as the essential component when iterating an Artificial Intelligence (AI) model. It becomes even more critical given the dominance of deep neural network based models, which are composed of a large number of parameters and data hungry, in application. Despite its indispensable role for developing AI models, research on active learning is not as intensive as other research directions. In this paper, we present a review of active learning through deep active learning approaches from the following perspectives: 1) technical advancements in active learning, 2) applications of active learning in computer vision, 3) industrial systems leveraging or with potential to leverage active learning for data iteration, 4) current limitations and future research directions. We expect this paper to clarify the significance of active learning in a modern AI model manufacturing process and to bring additional research attention to active learning. By addressing data automation challenges and coping with automated machine learning systems, active learning will facilitate democratization of AI technologies by boosting model production at scale.

Object detection requires substantial labeling effort for learning robust models. Active learning can reduce this effort by intelligently selecting relevant examples to be annotated. However, selecting these examples properly without introducing a sampling bias with a negative impact on the generalization performance is not straightforward and most active learning techniques can not hold their promises on real-world benchmarks. In our evaluation paper, we focus on active learning techniques without a computational overhead besides inference, something we refer to as zero-cost active learning. In particular, we show that a key ingredient is not only the score on a bounding box level but also the technique used for aggregating the scores for ranking images. We outline our experimental setup and also discuss practical considerations when using active learning for object detection.

接受注释较弱的对象探测器是全面监督者的负担得起的替代方案。但是，它们之间仍然存在显着的性能差距。我们建议通过微调预先训练的弱监督检测器来缩小这一差距，并使用``Box-In-box''（bib'（bib）自动从训练集中自动选择了一些完全注销的样品，这是一种新颖的活跃学习专门针对弱势监督探测器的据可查的失败模式而设计的策略。 VOC07和可可基准的实验表明，围嘴表现优于其他活跃的学习技术，并显着改善了基本的弱监督探测器的性能，而每个类别仅几个完全宣布的图像。围嘴达到了完全监督的快速RCNN的97％，在VOC07上仅10％的全已通量图像。在可可（COCO）上，平均每类使用10张全面通量的图像，或同等的训练集的1％，还减少了弱监督检测器和完全监督的快速RCN之间的性能差距（In AP）以上超过70％ ，在性能和数据效率之间表现出良好的权衡。我们的代码可在https://github.com/huyvvo/bib上公开获取。

Active learning as a paradigm in deep learning is especially important in applications involving intricate perception tasks such as object detection where labels are difficult and expensive to acquire. Development of active learning methods in such fields is highly computationally expensive and time consuming which obstructs the progression of research and leads to a lack of comparability between methods. In this work, we propose and investigate a sandbox setup for rapid development and transparent evaluation of active learning in deep object detection. Our experiments with commonly used configurations of datasets and detection architectures found in the literature show that results obtained in our sandbox environment are representative of results on standard configurations. The total compute time to obtain results and assess the learning behavior can thereby be reduced by factors of up to 14 when comparing with Pascal VOC and up to 32 when comparing with BDD100k. This allows for testing and evaluating data acquisition and labeling strategies in under half a day and contributes to the transparency and development speed in the field of active learning for object detection.

主动学习（al）试图通过标记最少的样本来最大限度地提高模型的性能增益。深度学习（DL）是贪婪的数据，需要大量的数据电源来优化大量参数，因此模型了解如何提取高质量功能。近年来，由于互联网技术的快速发展，我们处于信息种类的时代，我们有大量的数据。通过这种方式，DL引起了研究人员的强烈兴趣，并已迅速发展。与DL相比，研究人员对Al的兴趣相对较低。这主要是因为在DL的崛起之前，传统的机器学习需要相对较少的标记样品。因此，早期的Al很难反映其应得的价值。虽然DL在各个领域取得了突破，但大多数这一成功都是由于大量现有注释数据集的宣传。然而，收购大量高质量的注释数据集消耗了很多人力，这在某些领域不允许在需要高专业知识，特别是在语音识别，信息提取，医学图像等领域中， al逐渐受到适当的关注。自然理念是AL是否可用于降低样本注释的成本，同时保留DL的强大学习能力。因此，已经出现了深度主动学习（DAL）。虽然相关的研究非常丰富，但它缺乏对DAL的综合调查。本文要填补这一差距，我们为现有工作提供了正式的分类方法，以及全面和系统的概述。此外，我们还通过申请的角度分析并总结了DAL的发展。最后，我们讨论了DAL中的混乱和问题，为DAL提供了一些可能的发展方向。

The performance of deep neural networks improves with more annotated data. The problem is that the budget for annotation is limited. One solution to this is active learning, where a model asks human to annotate data that it perceived as uncertain. A variety of recent methods have been proposed to apply active learning to deep networks but most of them are either designed specific for their target tasks or computationally inefficient for large networks. In this paper, we propose a novel active learning method that is simple but task-agnostic, and works efficiently with the deep networks. We attach a small parametric module, named "loss prediction module," to a target network, and learn it to predict target losses of unlabeled inputs. Then, this module can suggest data that the target model is likely to produce a wrong prediction. This method is task-agnostic as networks are learned from a single loss regardless of target tasks. We rigorously validate our method through image classification, object detection, and human pose estimation, with the recent network architectures. The results demonstrate that our method consistently outperforms the previous methods over the tasks.

Recent aerial object detection models rely on a large amount of labeled training data, which requires unaffordable manual labeling costs in large aerial scenes with dense objects. Active learning is effective in reducing the data labeling cost by selectively querying the informative and representative unlabelled samples. However, existing active learning methods are mainly with class-balanced setting and image-based querying for generic object detection tasks, which are less applicable to aerial object detection scenario due to the long-tailed class distribution and dense small objects in aerial scenes. In this paper, we propose a novel active learning method for cost-effective aerial object detection. Specifically, both object-level and image-level informativeness are considered in the object selection to refrain from redundant and myopic querying. Besides, an easy-to-use class-balancing criterion is incorporated to favor the minority objects to alleviate the long-tailed class distribution problem in model training. To fully utilize the queried information, we further devise a training loss to mine the latent knowledge in the undiscovered image regions. Extensive experiments are conducted on the DOTA-v1.0 and DOTA-v2.0 benchmarks to validate the effectiveness of the proposed method. The results show that it can save more than 75% of the labeling cost to reach the same performance compared to the baselines and state-of-the-art active object detection methods. Code is available at https://github.com/ZJW700/MUS-CDB

主动学习旨在选择最具信息丰富的样本，以利用有限的注释预算。大多数现有的工作通过分别在每个数据集上多次重复耗时的模型训练和批量数据选择，遵循麻烦的管道。通过提出本文提出新的一般和有效的主动学习（GEAL）方法，挑战该地位QUO。利用预先培训的大型数据集预先培训的公开模型，我们的方法可以在不同的数据集中对具有相同模型的单通推断进行数据选择过程。为了捕获图像内的微妙本地信息，我们提出了从预先训练网络的中间特征中容易地提取的知识集群。而不是麻烦的批量选择策略，通过在细粒度知识集群级别执行K中心贪婪来选择所有数据样本。整个过程只需要单通式模型推论而不培训或监督，使我们的方法在时间复杂程度明显优于现有技术，从而长达数百次。广泛的实验越来越展示了我们对物体检测，语义分割，深度估计和图像分类方法的有希望的性能。

大规模数据集在计算机视觉中起着至关重要的作用。但是当前的数据集盲目注释而没有与样品区分的区分，从而使数据收集效率低下且不计。开放的问题是如何积极地构建大型数据集。尽管先进的主动学习算法可能是答案，但我们在实验上发现它们在分发数据广泛的现实注释方案中是la脚的。因此，这项工作为现实的数据集注释提供了一个新颖的主动学习框架。配备了此框架，我们构建了一个高质量的视觉数据集 - 竹子，由69m的图像分类注释，带有119K类别，带有809个类别的28m对象边界框注释。我们通过从几个知识库中整合的层次分类法来组织这些类别。分类注释比Imagenet22K大四倍，检测的注释比Object365大三倍。与ImagEnet22K和Objects365相比，预先训练的竹子在各种下游任务中实现了卓越的性能（分类的6.2％增长，检测到2.1％的增长）。我们认为，我们的积极学习框架和竹子对于将来的工作至关重要。

昂贵注释的要求是培训良好的实例细分模型的重大负担。在本文中，我们提出了一个经济活跃的学习环境，称为主动监督实例细分（API），该实例分段（API）从框级注释开始，并迭代地在盒子内划分一个点，并询问它是否属于对象。API的关键是找到最大程度地提高分段准确性的最佳点，以有限的注释预算。我们制定此设置，并提出几种基于不确定性的抽样策略。与其他学习策略相比，使用这些策略开发的模型可以在具有挑战性的MS-Coco数据集上获得一致的性能增长。结果表明，API集成了主动学习和基于点的监督的优势，是标签有效实例分割的有效学习范式。

深度神经网络对物体检测达到了高精度，但它们的成功铰链大量标记数据。为了减少标签依赖性，已经提出了各种主动学习策略，通常基于探测器的置信度。但是，这些方法偏向于高性能类，并且可以导致获取的数据集不是测试集数据的代表不好。在这项工作中，我们提出了一个统一的主动学习框架，这考虑了探测器的不确定性和鲁棒性，确保网络在所有类中表现良好。此外，我们的方法利用自动标记来抑制潜在的分布漂移，同时提高模型的性能。 Pascal VOC07 ​​+ 12和MS-Coco的实验表明，我们的方法始终如一地优于各种活跃的学习方法，在地图中产生高达7.7％，或降低标记成本的82％。代码将在接受纸张时发布。

The field of object detection has made significant advances riding on the wave of region-based ConvNets, but their training procedure still includes many heuristics and hyperparameters that are costly to tune. We present a simple yet surprisingly effective online hard example mining (OHEM) algorithm for training region-based ConvNet detectors. Our motivation is the same as it has always beendetection datasets contain an overwhelming number of easy examples and a small number of hard examples. Automatic selection of these hard examples can make training more effective and efficient. OHEM is a simple and intuitive algorithm that eliminates several heuristics and hyperparameters in common use. But more importantly, it yields consistent and significant boosts in detection performance on benchmarks like PASCAL VOC 2007 and 2012. Its effectiveness increases as datasets become larger and more difficult, as demonstrated by the results on the MS COCO dataset. Moreover, combined with complementary advances in the field, OHEM leads to state-of-the-art results of 78.9% and 76.3% mAP on PASCAL VOC 2007 and 2012 respectively.

研究表明，当训练数据缺少注释时，对象检测器的性能下降，即稀疏注释数据。当代方法专注于缺少地面实话注释的代理，无论是伪标签的形式还是通过在训练期间重新称重梯度。在这项工作中，我们重新审视了稀疏注释物体检测的制定。我们观察到稀疏注释的物体检测可以被认为是区域级的半监督对象检测问题。在此洞察力上，我们提出了一种基于区域的半监督算法，它自动识别包含未标记的前景对象的区域。我们的算法然后以不同的方式处理标记和未标记的前景区域，在半监督方法中进行常见做法。为了评估所提出的方法的有效性，我们对普斯卡尔库尔和可可数据集的稀疏注释方法常用的五种分裂进行详尽的实验，并实现最先进的性能。除此之外，我们还表明，我们的方法在标准半监督设置上实现了竞争性能，证明了我们的方法的实力和广泛适用性。

大型标记数据集的可用性是深度学习成功的关键组成部分。但是，大型数据集上的标签通常很耗时且昂贵。主动学习是一个研究领域，通过选择最重要的标签样本来解决昂贵的标签问题。基于多样性的采样算法被称为基于表示的主动学习方法的组成部分。在本文中，我们介绍了一种新的基于多样性的初始数据集选择算法，以选择有效学习环境中初始标记的最有用的样本集。自我监督的表示学习用于考虑初始数据集选择算法中样品的多样性。此外，我们提出了一种新型的主动学习查询策略，该策略使用基于多样性的基于一致性的嵌入方式采样。通过考虑基于一致性的嵌入方案中多样性的一致性信息，该方法可以在半监督的学习环境中选择更多信息的样本来标记。比较实验表明，通过利用未标记的数据的多样性，与先前的主动学习方法相比，该提出的方法在CIFAR-10和CALTECH-101数据集上取得了令人信服的结果。

由于准备点云的标记数据用于训练语义分割网络是一个耗时的过程，因此已经引入了弱监督的方法，以从一小部分数据中学习。这些方法通常是基于对比损失的学习，同时自动从一组稀疏的用户注销标签中得出每个点伪标签。在本文中，我们的关键观察是，选择要注释的样品的选择与这些样品的使用方式一样重要。因此，我们介绍了一种对3D场景进行弱监督分割的方法，该方法将自我训练与主动学习结合在一起。主动学习选择注释点可能会导致训练有素的模型的性能改进，而自我培训则可以有效利用用户提供的标签来学习模型。我们证明我们的方法会导致一种有效的方法，该方法可改善场景细分对以前的作品和基线，同时仅需要少量的用户注释。

Open-set object detection (OSOD) aims to detect the known categories and identify unknown objects in a dynamic world, which has achieved significant attentions. However, previous approaches only consider this problem in data-abundant conditions, while neglecting the few-shot scenes. In this paper, we seek a solution for the few-shot open-set object detection (FSOSOD), which aims to quickly train a detector based on few samples while detecting all known classes and identifying unknown classes. The main challenge for this task is that few training samples induce the model to overfit on the known classes, resulting in a poor open-set performance. We propose a new FSOSOD algorithm to tackle this issue, named Few-shOt Open-set Detector (FOOD), which contains a novel class weight sparsification classifier (CWSC) and a novel unknown decoupling learner (UDL). To prevent over-fitting, CWSC randomly sparses parts of the normalized weights for the logit prediction of all classes, and then decreases the co-adaptability between the class and its neighbors. Alongside, UDL decouples training the unknown class and enables the model to form a compact unknown decision boundary. Thus, the unknown objects can be identified with a confidence probability without any pseudo-unknown samples for training. We compare our method with several state-of-the-art OSOD methods in few-shot scenes and observe that our method improves the recall of unknown classes by 5%-9% across all shots in VOC-COCO dataset setting.

实例对象检测在智能监视，视觉导航，人机交互，智能服务和其他字段中扮演重要作用。灵感来自深度卷积神经网络（DCNN）的巨大成功，基于DCNN的实例对象检测已成为一个有前途的研究主题。为了解决DCNN始终需要大规模注释数据集来监督其培训的问题，而手动注释是耗尽和耗时的，我们提出了一种基于共同训练的新框架，称为克自我标记和检测（Gram-SLD） 。建议的克拉姆-SLD可以自动注释大量数据，具有非常有限的手动标记的关键数据并实现竞争性能。在我们的框架中，克朗损失被定义并用于构造两个完全冗余和独立的视图和一个关键的样本选择策略以及自动注释策略，可以全面考虑精度并回忆，以产生高质量的伪标签。 Public Gmu厨房数据集的实验，活动视觉数据集和自制的Bhid-Item DataSetDemonstrite，只有5％的标记训练数据，我们的克斯LLD比较了对象检测中的竞争性能（少于2％的地图丢失）通过完全监督的方法。在具有复杂和变化环境的实际应用中，所提出的方法可以满足实例对象检测的实时和准确性要求。

In this paper, we introduce a new large-scale object detection dataset, Objects365, which has 365 object categories over 600K training images. More than 10 million, high-quality bounding boxes are manually labeled through a three-step, carefully designed annotation pipeline. It is the largest object detection dataset (with full annotation) so far and establishes a more challenging benchmark for the community. Objects365 can serve as a better feature learning dataset for localization-sensitive tasks like object detection and semantic segmentation. The Objects365 pre-trained models significantly outperform ImageNet pre-trained models with 5.6 points gain (42 vs 36.4) based on the standard setting of 90K iterations on COCO benchmark. Even compared with much long training time like 540K iterations, our Objects365 pretrained model with 90K iterations still have 2.7 points gain (42 vs 39.3). Meanwhile, the finetuning time can be greatly reduced (up to 10 times) when reaching the same accuracy. Better generalization ability of Object365 has also been verified on CityPersons, VOC segmentation, and ADE tasks. The dataset as well as the pretrainedmodels have been released at www.objects365.org. * indicates equal contribution.

While deep learning succeeds in a wide range of tasks, it highly depends on the massive collection of annotated data which is expensive and time-consuming. To lower the cost of data annotation, active learning has been proposed to interactively query an oracle to annotate a small proportion of informative samples in an unlabeled dataset. Inspired by the fact that the samples with higher loss are usually more informative to the model than the samples with lower loss, in this paper we present a novel deep active learning approach that queries the oracle for data annotation when the unlabeled sample is believed to incorporate high loss. The core of our approach is a measurement Temporal Output Discrepancy (TOD) that estimates the sample loss by evaluating the discrepancy of outputs given by models at different optimization steps. Our theoretical investigation shows that TOD lower-bounds the accumulated sample loss thus it can be used to select informative unlabeled samples. On basis of TOD, we further develop an effective unlabeled data sampling strategy as well as an unsupervised learning criterion for active learning. Due to the simplicity of TOD, our methods are efficient, flexible, and task-agnostic. Extensive experimental results demonstrate that our approach achieves superior performances than the state-of-the-art active learning methods on image classification and semantic segmentation tasks. In addition, we show that TOD can be utilized to select the best model of potentially the highest testing accuracy from a pool of candidate models.