昂贵注释的要求是培训良好的实例细分模型的重大负担。在本文中，我们提出了一个经济活跃的学习环境，称为主动监督实例细分（API），该实例分段（API）从框级注释开始，并迭代地在盒子内划分一个点，并询问它是否属于对象。API的关键是找到最大程度地提高分段准确性的最佳点，以有限的注释预算。我们制定此设置，并提出几种基于不确定性的抽样策略。与其他学习策略相比，使用这些策略开发的模型可以在具有挑战性的MS-Coco数据集上获得一致的性能增长。结果表明，API集成了主动学习和基于点的监督的优势，是标签有效实例分割的有效学习范式。

我们提出了一个令人尴尬的简单点注释方案，以收集弱监督，例如分割。除了边界框外，我们还收集了在每个边界框内均匀采样的一组点的二进制标签。我们表明，为完整的掩模监督开发的现有实例细分模型可以通过我们的方案收集基于点的监督而无缝培训。值得注意的是，接受了可可，Pascal VOC，CityScapes和LVI的面具R-CNN，每个物体只有10个带注释的随机点可实现94％ - 占其完全监督的性能的98％，为弱化的实例细分定下了强大的基线。新点注释方案的速度比注释完整的对象掩码快5倍，使高质量实例分割在实践中更容易访问。受基于点的注释形式的启发，我们提出了对Pointrend实例分割模块的修改。对于每个对象，称为隐式pointrend的新体系结构生成一个函数的参数，该函数可以使最终的点级掩码预测。隐式Pointrend更加简单，并使用单点级掩蔽丢失。我们的实验表明，新模块更适合基于点的监督。

接受注释较弱的对象探测器是全面监督者的负担得起的替代方案。但是，它们之间仍然存在显着的性能差距。我们建议通过微调预先训练的弱监督检测器来缩小这一差距，并使用``Box-In-box''（bib'（bib）自动从训练集中自动选择了一些完全注销的样品，这是一种新颖的活跃学习专门针对弱势监督探测器的据可查的失败模式而设计的策略。 VOC07和可可基准的实验表明，围嘴表现优于其他活跃的学习技术，并显着改善了基本的弱监督探测器的性能，而每个类别仅几个完全宣布的图像。围嘴达到了完全监督的快速RCNN的97％，在VOC07上仅10％的全已通量图像。在可可（COCO）上，平均每类使用10张全面通量的图像，或同等的训练集的1％，还减少了弱监督检测器和完全监督的快速RCN之间的性能差距（In AP）以上超过70％ ，在性能和数据效率之间表现出良好的权衡。我们的代码可在https://github.com/huyvvo/bib上公开获取。

As an important data selection schema, active learning emerges as the essential component when iterating an Artificial Intelligence (AI) model. It becomes even more critical given the dominance of deep neural network based models, which are composed of a large number of parameters and data hungry, in application. Despite its indispensable role for developing AI models, research on active learning is not as intensive as other research directions. In this paper, we present a review of active learning through deep active learning approaches from the following perspectives: 1) technical advancements in active learning, 2) applications of active learning in computer vision, 3) industrial systems leveraging or with potential to leverage active learning for data iteration, 4) current limitations and future research directions. We expect this paper to clarify the significance of active learning in a modern AI model manufacturing process and to bring additional research attention to active learning. By addressing data automation challenges and coping with automated machine learning systems, active learning will facilitate democratization of AI technologies by boosting model production at scale.

Recent aerial object detection models rely on a large amount of labeled training data, which requires unaffordable manual labeling costs in large aerial scenes with dense objects. Active learning is effective in reducing the data labeling cost by selectively querying the informative and representative unlabelled samples. However, existing active learning methods are mainly with class-balanced setting and image-based querying for generic object detection tasks, which are less applicable to aerial object detection scenario due to the long-tailed class distribution and dense small objects in aerial scenes. In this paper, we propose a novel active learning method for cost-effective aerial object detection. Specifically, both object-level and image-level informativeness are considered in the object selection to refrain from redundant and myopic querying. Besides, an easy-to-use class-balancing criterion is incorporated to favor the minority objects to alleviate the long-tailed class distribution problem in model training. To fully utilize the queried information, we further devise a training loss to mine the latent knowledge in the undiscovered image regions. Extensive experiments are conducted on the DOTA-v1.0 and DOTA-v2.0 benchmarks to validate the effectiveness of the proposed method. The results show that it can save more than 75% of the labeling cost to reach the same performance compared to the baselines and state-of-the-art active object detection methods. Code is available at https://github.com/ZJW700/MUS-CDB

由于准备点云的标记数据用于训练语义分割网络是一个耗时的过程，因此已经引入了弱监督的方法，以从一小部分数据中学习。这些方法通常是基于对比损失的学习，同时自动从一组稀疏的用户注销标签中得出每个点伪标签。在本文中，我们的关键观察是，选择要注释的样品的选择与这些样品的使用方式一样重要。因此，我们介绍了一种对3D场景进行弱监督分割的方法，该方法将自我训练与主动学习结合在一起。主动学习选择注释点可能会导致训练有素的模型的性能改进，而自我培训则可以有效利用用户提供的标签来学习模型。我们证明我们的方法会导致一种有效的方法，该方法可改善场景细分对以前的作品和基线，同时仅需要少量的用户注释。

In contrast to fully supervised methods using pixel-wise mask labels, box-supervised instance segmentation takes advantage of simple box annotations, which has recently attracted increasing research attention. This paper presents a novel single-shot instance segmentation approach, namely Box2Mask, which integrates the classical level-set evolution model into deep neural network learning to achieve accurate mask prediction with only bounding box supervision. Specifically, both the input image and its deep features are employed to evolve the level-set curves implicitly, and a local consistency module based on a pixel affinity kernel is used to mine the local context and spatial relations. Two types of single-stage frameworks, i.e., CNN-based and transformer-based frameworks, are developed to empower the level-set evolution for box-supervised instance segmentation, and each framework consists of three essential components: instance-aware decoder, box-level matching assignment and level-set evolution. By minimizing the level-set energy function, the mask map of each instance can be iteratively optimized within its bounding box annotation. The experimental results on five challenging testbeds, covering general scenes, remote sensing, medical and scene text images, demonstrate the outstanding performance of our proposed Box2Mask approach for box-supervised instance segmentation. In particular, with the Swin-Transformer large backbone, our Box2Mask obtains 42.4% mask AP on COCO, which is on par with the recently developed fully mask-supervised methods. The code is available at: https://github.com/LiWentomng/boxlevelset.

深度学习方法需要大量的注释数据以优化参数。例如，附加具有准确边界框注释的数据集对于现代对象检测任务至关重要。但是，具有这样的像素准确性的标签是费力且耗时的，并且精心制作的标记程序对于降低人造噪声是必不可少的，涉及注释审查和接受测试。在本文中，我们关注嘈杂的位置注释对对象检测方法的性能的影响，并旨在减少噪声的不利影响。首先，当将噪声引入边界框注释中时，一阶段和两阶段检测器都会在实验上观察到明显的性能降解。例如，我们的合成噪声导致可可测试分裂的FCO探测器的性能从38.9％的AP降低到33.6％的AP，对于更快的R-CNN而言，COCO检测器的性能从38.9％的AP下降到37.8％的AP和33.7％的AP。其次，提出了一种基于贝叶斯过滤器进行预测合奏的自我纠正技术，以更好地利用教师学习范式后的嘈杂位置注释。合成和现实世界情景的实验始终证明了我们方法的有效性，例如，我们的方法将FCOS检测器的降解性能从33.6％的AP提高到可可的35.6％AP。

当前的3D分割方法很大程度上依赖于大规模的点状数据集，众所周知，这些数据集众所周知。很少有尝试规避需要每点注释的需求。在这项工作中，我们研究了弱监督的3D语义实例分割。关键的想法是利用3D边界框标签，更容易，更快地注释。确实，我们表明只有仅使用边界框标签训练密集的分割模型。在我们方法的核心上，\ name {}是一个深层模型，灵感来自经典的霍夫投票，直接投票赞成边界框参数，并且是专门针对边界盒票的专门定制的群集方法。这超出了常用的中心票，这不会完全利用边界框注释。在扫描仪测试中，我们弱监督的模型在其他弱监督的方法中获得了领先的性能（+18 MAP@50）。值得注意的是，它还达到了当前完全监督模型的50分数的地图的97％。为了进一步说明我们的工作的实用性，我们在最近发布的Arkitscenes数据集中训练Box2mask，该数据集仅使用3D边界框注释，并首次显示引人注目的3D实例细分掩码。

在域适应领域，模型性能与目标域注释的数量之间存在权衡。积极的学习，最大程度地提高了模型性能，几乎没有信息的标签数据，以方便这种情况。在这项工作中，我们提出了D2ADA，这是用于语义分割的一般活动域的适应框架。为了使模型使用最小查询标签调整到目标域，我们提出了在目标域中具有高概率密度的样品的获取标签，但源域中的概率密度较低，与现有源域标记的数据互补。为了进一步提高标签效率，我们设计了动态的调度策略，以调整域探索和模型不确定性之间的标签预算。广泛的实验表明，我们的方法的表现优于现有的活跃学习和域适应基线，这两个基准测试基准，GTA5-> CityScapes和Synthia-> CityScapes。对于目标域注释不到5％，我们的方法与完全监督的结果可比结果。我们的代码可在https://github.com/tsunghan-wu/d2ada上公开获取。

This paper presents the first attempt to learn semantic boundary detection using image-level class labels as supervision. Our method starts by estimating coarse areas of object classes through attentions drawn by an image classification network. Since boundaries will locate somewhere between such areas of different classes, our task is formulated as a multiple instance learning (MIL) problem, where pixels on a line segment connecting areas of two different classes are regarded as a bag of boundary candidates. Moreover, we design a new neural network architecture that can learn to estimate semantic boundaries reliably even with uncertain supervision given by the MIL strategy. Our network is used to generate pseudo semantic boundary labels of training images, which are in turn used to train fully supervised models. The final model trained with our pseudo labels achieves an outstanding performance on the SBD dataset, where it is as competitive as some of previous arts trained with stronger supervision.

必须在密集的注释图像上培训最先进的实例分段方法。虽然一般而言，这一要求对于生物医学图像尤其令人生畏，其中域专业知识通常需要注释，没有大的公共数据收集可用于预培训。我们建议通过基于非空间嵌入的非空间嵌入的联盟分割方法来解决密集的注释瓶颈，该方法利用所学习的嵌入空间的结构以可分散的方式提取单个实例。然后可以将分割损耗直接应用于实例，整体管道可以以完全或弱监督的方式培训，包括积极解贴的监管的具有挑战性的情况，其中为未标记的部分引入了一种新的自我监督的一致性损失训练数据。我们在不同显微镜模型以及城市景观和CVPPP实例分段基准中评估了对2D和3D分段问题的提出的方法，在后者上实现最先进的结果。该代码可用于：https://github.com/kreshuklab/spoco

知识蒸馏在分类中取得了巨大的成功，但是，仍然有挑战性。在用于检测的典型图像中，来自不同位置的表示可能对检测目标具有不同的贡献，使蒸馏难以平衡。在本文中，我们提出了一种有条件的蒸馏框架来蒸馏出所需的知识，即关于每个例子的分类和本地化有益的知识。该框架引入了一种可学习的条件解码模块，其将每个目标实例检索为查询的信息。具体而言，我们将条件信息编码为查询并使用教师的表示作为键。查询和键之间的注意用于测量不同特征的贡献，由本地化识别敏感辅助任务指导。广泛的实验表明了我们的方法的功效：我们在各种环境下观察到令人印象深刻的改进。值得注意的是，在1倍计划下，我们将通过37.4至40.7地图（+3.3）与Reset-50骨架的Restinetet提升。代码已在https://github.com/megvii-research/icd上发布。

从非结构化的3D点云学习密集点语义，虽然是一个逼真的问题，但在文献中探讨了逼真的问题。虽然现有的弱监督方法可以仅具有小数点的点级注释来有效地学习语义，但我们发现香草边界箱级注释也是大规模3D点云的语义分割信息。在本文中，我们介绍了一个神经结构，称为Box2Seg，以了解3D点云的点级语义，具有边界盒级监控。我们方法的关键是通过探索每个边界框内和外部的几何和拓扑结构来生成准确的伪标签。具体地，利用基于注意的自我训练（AST）技术和点类激活映射（PCAM）来估计伪标签。通过伪标签进行进一步培训并精制网络。在两个大型基准测试中的实验，包括S3DIS和Scannet，证明了该方法的竞争性能。特别是，所提出的网络可以培训，甚至是均匀的空缺边界箱级注释和子环级标签。

由于检测数据集的规模小，当前对象探测器的词汇量受到限制。另一方面，图像分类器的原因是大约更大的词汇表，因为他们的数据集更大，更容易收集。我们提出守则，只需在图像分类数据上培训检测器的分类器，从而扩展了探测器的词汇量到数万个概念。与现有工作不同，拒绝不会根据模型预测将图像标签分配给框，使其更容易实现和兼容一系列检测架构和骨架。我们的结果表明，即使没有箱子注释，否则差异也能产生出色的探测器。它优于开放词汇和长尾检测基准的事先工作。拒绝为所有类和8.3地图提供了2.4地图的增益，用于开放词汇LVIS基准测试中的新型类。在标准的LVIS基准测试中，守护者达到41.7地图所有课程和41.7地图以获得罕见课程。我们首次培训一个探测器，其中包含所有二十一千类的ImageNet数据集，并显示它在没有微调的情况下推广到新数据集。代码可在https://github.com/facebookresearch/dorm提供。

The semantic image segmentation task presents a trade-off between test time accuracy and training-time annotation cost. Detailed per-pixel annotations enable training accurate models but are very timeconsuming to obtain; image-level class labels are an order of magnitude cheaper but result in less accurate models. We take a natural step from image-level annotation towards stronger supervision: we ask annotators to point to an object if one exists. We incorporate this point supervision along with a novel objectness potential in the training loss function of a CNN model. Experimental results on the PASCAL VOC 2012 benchmark reveal that the combined effect of point-level supervision and objectness potential yields an improvement of 12.9% mIOU over image-level supervision. Further, we demonstrate that models trained with pointlevel supervision are more accurate than models trained with image-level, squiggle-level or full supervision given a fixed annotation budget.

Image instance segmentation is a fundamental research topic in autonomous driving, which is crucial for scene understanding and road safety. Advanced learning-based approaches often rely on the costly 2D mask annotations for training. In this paper, we present a more artful framework, LiDAR-guided Weakly Supervised Instance Segmentation (LWSIS), which leverages the off-the-shelf 3D data, i.e., Point Cloud, together with the 3D boxes, as natural weak supervisions for training the 2D image instance segmentation models. Our LWSIS not only exploits the complementary information in multimodal data during training, but also significantly reduces the annotation cost of the dense 2D masks. In detail, LWSIS consists of two crucial modules, Point Label Assignment (PLA) and Graph-based Consistency Regularization (GCR). The former module aims to automatically assign the 3D point cloud as 2D point-wise labels, while the latter further refines the predictions by enforcing geometry and appearance consistency of the multimodal data. Moreover, we conduct a secondary instance segmentation annotation on the nuScenes, named nuInsSeg, to encourage further research on multimodal perception tasks. Extensive experiments on the nuInsSeg, as well as the large-scale Waymo, show that LWSIS can substantially improve existing weakly supervised segmentation models by only involving 3D data during training. Additionally, LWSIS can also be incorporated into 3D object detectors like PointPainting to boost the 3D detection performance for free. The code and dataset are available at https://github.com/Serenos/LWSIS.

本文推动了在图像中分解伪装区域的信封，成了有意义的组件，即伪装的实例。为了促进伪装实例分割的新任务，我们将在数量和多样性方面引入DataSet被称为Camo ++，该数据集被称为Camo ++。新数据集基本上增加了具有分层像素 - 明智的地面真理的图像的数量。我们还为伪装实例分割任务提供了一个基准套件。特别是，我们在各种场景中对新构造的凸轮++数据集进行了广泛的评估。我们还提出了一种伪装融合学习（CFL）伪装实例分割框架，以进一步提高最先进的方法的性能。数据集，模型，评估套件和基准测试将在我们的项目页面上公开提供：https://sites.google.com/view/ltnghia/research/camo_plus_plus

大多数最先进的实例级人类解析模型都采用了两阶段的基于锚的探测器，因此无法避免启发式锚盒设计和像素级别缺乏分析。为了解决这两个问题，我们设计了一个实例级人类解析网络，该网络在像素级别上无锚固且可解决。它由两个简单的子网络组成：一个用于边界框预测的无锚检测头和一个用于人体分割的边缘引导解析头。无锚探测器的头继承了像素样的优点，并有效地避免了对象检测应用中证明的超参数的敏感性。通过引入部分感知的边界线索，边缘引导的解析头能够将相邻的人类部分与彼此区分开，最多可在一个人类实例中，甚至重叠的实例。同时，利用了精炼的头部整合盒子级别的分数和部分分析质量，以提高解析结果的质量。在两个多个人类解析数据集（即CIHP和LV-MHP-V2.0）和一个视频实例级人类解析数据集（即VIP）上进行实验，表明我们的方法实现了超过全球级别和实例级别的性能最新的一阶段自上而下的替代方案。

We present a new method for efficient high-quality image segmentation of objects and scenes. By analogizing classical computer graphics methods for efficient rendering with over-and undersampling challenges faced in pixel labeling tasks, we develop a unique perspective of image segmentation as a rendering problem. From this vantage, we present the PointRend (Point-based Rendering) neural network module: a module that performs point-based segmentation predictions at adaptively selected locations based on an iterative subdivision algorithm. PointRend can be flexibly applied to both instance and semantic segmentation tasks by building on top of existing state-ofthe-art models. While many concrete implementations of the general idea are possible, we show that a simple design already achieves excellent results. Qualitatively, PointRend outputs crisp object boundaries in regions that are oversmoothed by previous methods. Quantitatively, PointRend yields significant gains on COCO and Cityscapes, for both instance and semantic segmentation. PointRend's efficiency enables output resolutions that are otherwise impractical in terms of memory or computation compared to existing approaches. Code has been made available at https:// github.com/facebookresearch/detectron2/ tree/master/projects/PointRend.