主动学习旨在选择最具信息丰富的样本，以利用有限的注释预算。大多数现有的工作通过分别在每个数据集上多次重复耗时的模型训练和批量数据选择，遵循麻烦的管道。通过提出本文提出新的一般和有效的主动学习（GEAL）方法，挑战该地位QUO。利用预先培训的大型数据集预先培训的公开模型，我们的方法可以在不同的数据集中对具有相同模型的单通推断进行数据选择过程。为了捕获图像内的微妙本地信息，我们提出了从预先训练网络的中间特征中容易地提取的知识集群。而不是麻烦的批量选择策略，通过在细粒度知识集群级别执行K中心贪婪来选择所有数据样本。整个过程只需要单通式模型推论而不培训或监督，使我们的方法在时间复杂程度明显优于现有技术，从而长达数百次。广泛的实验越来越展示了我们对物体检测，语义分割，深度估计和图像分类方法的有希望的性能。

几乎所有用于计算机视觉任务的最先进的神经网络都受到（1）在目标数据集上的大规模数据集和（2）FINETUNING上的预培训（1）预培训。该策略有助于减少对目标数据集的依赖，并提高目标任务的收敛速率和泛化。虽然对大型数据集进行预训练非常有用，但其最重要的缺点是高培训成本。要解决此问题，我们提出了有效的过滤方法，以从训练前的数据集中选择相关子集。此外，我们发现，在训练前的图像分辨率降低图像分辨率在成本和性能之间提供了很大的权衡。我们通过在无监督和监督设置中的想象中进行预测，并在各种目标数据集和任务集合中进行预测，通过预先培训来验证我们的技术。我们提出的方法大大降低了预训练成本并提供了强大的性能提升。最后，我们通过在我们的子集上调整可用模型来提高标准ImageNet预培训1-3％，并在从更大的规模数据集中过滤的数据集上进行预训练。

Jitendra Malik once said, "Supervision is the opium of the AI researcher". Most deep learning techniques heavily rely on extreme amounts of human labels to work effectively. In today's world, the rate of data creation greatly surpasses the rate of data annotation. Full reliance on human annotations is just a temporary means to solve current closed problems in AI. In reality, only a tiny fraction of data is annotated. Annotation Efficient Learning (AEL) is a study of algorithms to train models effectively with fewer annotations. To thrive in AEL environments, we need deep learning techniques that rely less on manual annotations (e.g., image, bounding-box, and per-pixel labels), but learn useful information from unlabeled data. In this thesis, we explore five different techniques for handling AEL.

The performance of deep neural networks improves with more annotated data. The problem is that the budget for annotation is limited. One solution to this is active learning, where a model asks human to annotate data that it perceived as uncertain. A variety of recent methods have been proposed to apply active learning to deep networks but most of them are either designed specific for their target tasks or computationally inefficient for large networks. In this paper, we propose a novel active learning method that is simple but task-agnostic, and works efficiently with the deep networks. We attach a small parametric module, named "loss prediction module," to a target network, and learn it to predict target losses of unlabeled inputs. Then, this module can suggest data that the target model is likely to produce a wrong prediction. This method is task-agnostic as networks are learned from a single loss regardless of target tasks. We rigorously validate our method through image classification, object detection, and human pose estimation, with the recent network architectures. The results demonstrate that our method consistently outperforms the previous methods over the tasks.

Active learning aims to develop label-efficient algorithms by sampling the most representative queries to be labeled by an oracle. We describe a pool-based semisupervised active learning algorithm that implicitly learns this sampling mechanism in an adversarial manner. Unlike conventional active learning algorithms, our approach is task agnostic, i.e., it does not depend on the performance of the task for which we are trying to acquire labeled data. Our method learns a latent space using a variational autoencoder (VAE) and an adversarial network trained to discriminate between unlabeled and labeled data. The minimax game between the VAE and the adversarial network is played such that while the VAE tries to trick the adversarial network into predicting that all data points are from the labeled pool, the adversarial network learns how to discriminate between dissimilarities in the latent space. We extensively evaluate our method on various image classification and semantic segmentation benchmark datasets and establish a new state of the art on CIFAR10/100, Caltech-256, ImageNet, Cityscapes, and BDD100K. Our results demonstrate that our adversarial approach learns an effective low dimensional latent space in large-scale settings and provides for a computationally efficient sampling method. 1

As an important data selection schema, active learning emerges as the essential component when iterating an Artificial Intelligence (AI) model. It becomes even more critical given the dominance of deep neural network based models, which are composed of a large number of parameters and data hungry, in application. Despite its indispensable role for developing AI models, research on active learning is not as intensive as other research directions. In this paper, we present a review of active learning through deep active learning approaches from the following perspectives: 1) technical advancements in active learning, 2) applications of active learning in computer vision, 3) industrial systems leveraging or with potential to leverage active learning for data iteration, 4) current limitations and future research directions. We expect this paper to clarify the significance of active learning in a modern AI model manufacturing process and to bring additional research attention to active learning. By addressing data automation challenges and coping with automated machine learning systems, active learning will facilitate democratization of AI technologies by boosting model production at scale.

在域适应领域，模型性能与目标域注释的数量之间存在权衡。积极的学习，最大程度地提高了模型性能，几乎没有信息的标签数据，以方便这种情况。在这项工作中，我们提出了D2ADA，这是用于语义分割的一般活动域的适应框架。为了使模型使用最小查询标签调整到目标域，我们提出了在目标域中具有高概率密度的样品的获取标签，但源域中的概率密度较低，与现有源域标记的数据互补。为了进一步提高标签效率，我们设计了动态的调度策略，以调整域探索和模型不确定性之间的标签预算。广泛的实验表明，我们的方法的表现优于现有的活跃学习和域适应基线，这两个基准测试基准，GTA5-> CityScapes和Synthia-> CityScapes。对于目标域注释不到5％，我们的方法与完全监督的结果可比结果。我们的代码可在https://github.com/tsunghan-wu/d2ada上公开获取。

标记大量数据很昂贵。主动学习旨在通过要求注释未标记的集合中最有用的数据来解决这个问题。我们提出了一种新颖的活跃学习方法，该方法利用自我监督的借口任务和独特的数据采样器来选择既困难又具有代表性的数据。我们发现，简单的自我监督借口任务（例如旋转预测）的损失与下游任务损失密切相关。在主动学习迭代之前，对未标记的集合进行了借口任务学习者进行培训，并且未标记的数据被分类并通过其借口任务损失分组成批处理。在每个主动的学习迭代中，主要任务模型用于批评要注释的批次中最不确定的数据。我们评估了有关各种图像分类和分割基准测试的方法，并在CIFAR10，CALTECH-101，IMAGENET和CITYSCAPES上实现引人注目的性能。我们进一步表明，我们的方法在不平衡的数据集上表现良好，并且可以有效地解决冷启动问题的解决方案，在这种问题中，主动学习性能受到随机采样的初始标记集的影响。

We propose LiDAL, a novel active learning method for 3D LiDAR semantic segmentation by exploiting inter-frame uncertainty among LiDAR frames. Our core idea is that a well-trained model should generate robust results irrespective of viewpoints for scene scanning and thus the inconsistencies in model predictions across frames provide a very reliable measure of uncertainty for active sample selection. To implement this uncertainty measure, we introduce new inter-frame divergence and entropy formulations, which serve as the metrics for active selection. Moreover, we demonstrate additional performance gains by predicting and incorporating pseudo-labels, which are also selected using the proposed inter-frame uncertainty measure. Experimental results validate the effectiveness of LiDAL: we achieve 95% of the performance of fully supervised learning with less than 5% of annotations on the SemanticKITTI and nuScenes datasets, outperforming state-of-the-art active learning methods. Code release: https://github.com/hzykent/LiDAL.

近年来，已经开发了几种无监督和自我监督的方法，以从大规模未标记的数据集中学习视觉功能。然而，它们的主要缺点是，如果简单地旋转或相机的视角更改，这些方法几乎无法识别同一对象的视觉特征。为了克服此限制，同时利用有用的监督来源，我们考虑了视频对象轨道。遵循直觉，轨道中的两个补丁应该在学习的特征空间中具有相似的视觉表示形式，我们采用了一种无监督的基于群集的方法，并约束此类表示为同一类别，因为它们可能属于同一对象或对象零件。与先前的工作相比，不同数据集上两个下游任务的实验结果证明了我们在线深度聚类（ODCT）方法的有效性，而视频轨道一致性（ODCT）方法没有利用时间信息。此外，我们表明，与依靠昂贵和精确的轨道注释相比，利用无监督的类不知所措但嘈杂的轨道生成器的产量提高了准确性。

While deep learning succeeds in a wide range of tasks, it highly depends on the massive collection of annotated data which is expensive and time-consuming. To lower the cost of data annotation, active learning has been proposed to interactively query an oracle to annotate a small proportion of informative samples in an unlabeled dataset. Inspired by the fact that the samples with higher loss are usually more informative to the model than the samples with lower loss, in this paper we present a novel deep active learning approach that queries the oracle for data annotation when the unlabeled sample is believed to incorporate high loss. The core of our approach is a measurement Temporal Output Discrepancy (TOD) that estimates the sample loss by evaluating the discrepancy of outputs given by models at different optimization steps. Our theoretical investigation shows that TOD lower-bounds the accumulated sample loss thus it can be used to select informative unlabeled samples. On basis of TOD, we further develop an effective unlabeled data sampling strategy as well as an unsupervised learning criterion for active learning. Due to the simplicity of TOD, our methods are efficient, flexible, and task-agnostic. Extensive experimental results demonstrate that our approach achieves superior performances than the state-of-the-art active learning methods on image classification and semantic segmentation tasks. In addition, we show that TOD can be utilized to select the best model of potentially the highest testing accuracy from a pool of candidate models.

大规模数据集在计算机视觉中起着至关重要的作用。但是当前的数据集盲目注释而没有与样品区分的区分，从而使数据收集效率低下且不计。开放的问题是如何积极地构建大型数据集。尽管先进的主动学习算法可能是答案，但我们在实验上发现它们在分发数据广泛的现实注释方案中是la脚的。因此，这项工作为现实的数据集注释提供了一个新颖的主动学习框架。配备了此框架，我们构建了一个高质量的视觉数据集 - 竹子，由69m的图像分类注释，带有119K类别，带有809个类别的28m对象边界框注释。我们通过从几个知识库中整合的层次分类法来组织这些类别。分类注释比Imagenet22K大四倍，检测的注释比Object365大三倍。与ImagEnet22K和Objects365相比，预先训练的竹子在各种下游任务中实现了卓越的性能（分类的6.2％增长，检测到2.1％的增长）。我们认为，我们的积极学习框架和竹子对于将来的工作至关重要。

我们通过以端到端的方式对大规模未标记的数据集进行分类，呈现扭曲，简单和理论上可解释的自我监督的表示学习方法。我们使用Softmax操作终止的暹罗网络，以产生两个增强图像的双类分布。没有监督，我们强制执行不同增强的班级分布。但是，只需最小化增强之间的分歧将导致折叠解决方案，即，输出所有图像的相同类概率分布。在这种情况下，留下有关输入图像的信息。为了解决这个问题，我们建议最大化输入和课程预测之间的互信息。具体地，我们最小化每个样品的分布的熵，使每个样品的课程预测是对每个样品自信的预测，并最大化平均分布的熵，以使不同样品的预测变得不同。以这种方式，扭曲可以自然地避免没有特定设计的折叠解决方案，例如非对称网络，停止梯度操作或动量编码器。因此，扭曲优于各种任务的最先进的方法。特别是，在半监督学习中，扭曲令人惊讶地表现出令人惊讶的是，使用Reset-50作为骨干的1％ImageNet标签实现61.2％的顶级精度，以前的最佳结果为6.2％。代码和预先训练的模型是给出的：https://github.com/byteDance/twist

我们研究了用于半监控学习（SSL）的无监督数据选择，其中可以提供大规模的未标记数据集，并且为标签采集预算小额数据子集。现有的SSL方法专注于学习一个有效地集成了来自给定小标记数据和大型未标记数据的信息的模型，而我们专注于选择正确的数据以用于SSL的注释，而无需任何标签或任务信息。直观地，要标记的实例应统称为下游任务的最大多样性和覆盖范围，并且单独具有用于SSL的最大信息传播实用程序。我们以三步数据为中心的SSL方法形式化这些概念，使稳定性和精度的纤维液改善8％的CiFar-10（标记为0.08％）和14％的Imagenet -1k（标记为0.2％）。它也是一种具有各种SSL方法的通用框架，提供一致的性能增益。我们的工作表明，在仔细选择注释数据上花费的小计算带来了大注释效率和模型性能增益，而无需改变学习管道。我们完全无监督的数据选择可以轻松扩展到其他弱监督的学习设置。

主动学习（al）试图通过标记最少的样本来最大限度地提高模型的性能增益。深度学习（DL）是贪婪的数据，需要大量的数据电源来优化大量参数，因此模型了解如何提取高质量功能。近年来，由于互联网技术的快速发展，我们处于信息种类的时代，我们有大量的数据。通过这种方式，DL引起了研究人员的强烈兴趣，并已迅速发展。与DL相比，研究人员对Al的兴趣相对较低。这主要是因为在DL的崛起之前，传统的机器学习需要相对较少的标记样品。因此，早期的Al很难反映其应得的价值。虽然DL在各个领域取得了突破，但大多数这一成功都是由于大量现有注释数据集的宣传。然而，收购大量高质量的注释数据集消耗了很多人力，这在某些领域不允许在需要高专业知识，特别是在语音识别，信息提取，医学图像等领域中， al逐渐受到适当的关注。自然理念是AL是否可用于降低样本注释的成本，同时保留DL的强大学习能力。因此，已经出现了深度主动学习（DAL）。虽然相关的研究非常丰富，但它缺乏对DAL的综合调查。本文要填补这一差距，我们为现有工作提供了正式的分类方法，以及全面和系统的概述。此外，我们还通过申请的角度分析并总结了DAL的发展。最后，我们讨论了DAL中的混乱和问题，为DAL提供了一些可能的发展方向。

昂贵注释的要求是培训良好的实例细分模型的重大负担。在本文中，我们提出了一个经济活跃的学习环境，称为主动监督实例细分（API），该实例分段（API）从框级注释开始，并迭代地在盒子内划分一个点，并询问它是否属于对象。API的关键是找到最大程度地提高分段准确性的最佳点，以有限的注释预算。我们制定此设置，并提出几种基于不确定性的抽样策略。与其他学习策略相比，使用这些策略开发的模型可以在具有挑战性的MS-Coco数据集上获得一致的性能增长。结果表明，API集成了主动学习和基于点的监督的优势，是标签有效实例分割的有效学习范式。

The generalisation performance of a convolutional neural networks (CNN) is majorly predisposed by the quantity, quality, and diversity of the training images. All the training data needs to be annotated in-hand before, in many real-world applications data is easy to acquire but expensive and time-consuming to label. The goal of the Active learning for the task is to draw most informative samples from the unlabeled pool which can used for training after annotation. With total different objective, self-supervised learning which have been gaining meteoric popularity by closing the gap in performance with supervised methods on large computer vision benchmarks. self-supervised learning (SSL) these days have shown to produce low-level representations that are invariant to distortions of the input sample and can encode invariance to artificially created distortions, e.g. rotation, solarization, cropping etc. self-supervised learning (SSL) approaches rely on simpler and more scalable frameworks for learning. In this paper, we unify these two families of approaches from the angle of active learning using self-supervised learning mainfold and propose Deep Active Learning using BarlowTwins(DALBT), an active learning method for all the datasets using combination of classifier trained along with self-supervised loss framework of Barlow Twins to a setting where the model can encode the invariance of artificially created distortions, e.g. rotation, solarization, cropping etc.

接受注释较弱的对象探测器是全面监督者的负担得起的替代方案。但是，它们之间仍然存在显着的性能差距。我们建议通过微调预先训练的弱监督检测器来缩小这一差距，并使用``Box-In-box''（bib'（bib）自动从训练集中自动选择了一些完全注销的样品，这是一种新颖的活跃学习专门针对弱势监督探测器的据可查的失败模式而设计的策略。 VOC07和可可基准的实验表明，围嘴表现优于其他活跃的学习技术，并显着改善了基本的弱监督探测器的性能，而每个类别仅几个完全宣布的图像。围嘴达到了完全监督的快速RCNN的97％，在VOC07上仅10％的全已通量图像。在可可（COCO）上，平均每类使用10张全面通量的图像，或同等的训练集的1％，还减少了弱监督检测器和完全监督的快速RCN之间的性能差距（In AP）以上超过70％ ，在性能和数据效率之间表现出良好的权衡。我们的代码可在https://github.com/huyvvo/bib上公开获取。

带有像素天标签的注释图像是耗时和昂贵的过程。最近，DataSetGan展示了有希望的替代方案 - 通过利用一小组手动标记的GaN生成的图像来通过生成的对抗网络（GAN）来综合大型标记数据集。在这里，我们将DataSetGan缩放到ImageNet类别的规模。我们从ImageNet上训练的类条件生成模型中拍摄图像样本，并为所有1K类手动注释每个类的5张图像。通过在Biggan之上培训有效的特征分割架构，我们将Bigan转换为标记的DataSet生成器。我们进一步表明，VQGan可以类似地用作数据集生成器，利用已经注释的数据。我们通过在各种设置中标记一组8K实图像并在各种设置中评估分段性能来创建一个新的想象因基准。通过广泛的消融研究，我们展示了利用大型生成的数据集来培训在像素 - 明智的任务上培训不同的监督和自我监督的骨干模型的大增益。此外，我们证明，使用我们的合成数据集进行预培训，以改善在几个下游数据集上的标准Imagenet预培训，例如Pascal-VOC，MS-Coco，Citycapes和Chink X射线以及任务（检测，细分）。我们的基准将公开并维护一个具有挑战性的任务的排行榜。项目页面：https://nv-tlabs.github.io/big-dataseTgan/

Active learning as a paradigm in deep learning is especially important in applications involving intricate perception tasks such as object detection where labels are difficult and expensive to acquire. Development of active learning methods in such fields is highly computationally expensive and time consuming which obstructs the progression of research and leads to a lack of comparability between methods. In this work, we propose and investigate a sandbox setup for rapid development and transparent evaluation of active learning in deep object detection. Our experiments with commonly used configurations of datasets and detection architectures found in the literature show that results obtained in our sandbox environment are representative of results on standard configurations. The total compute time to obtain results and assess the learning behavior can thereby be reduced by factors of up to 14 when comparing with Pascal VOC and up to 32 when comparing with BDD100k. This allows for testing and evaluating data acquisition and labeling strategies in under half a day and contributes to the transparency and development speed in the field of active learning for object detection.