弱监督的对象本地化（WSOL）旨在学习仅使用图像级类别标签编码对象位置的表示形式。但是，许多物体可以在不同水平的粒度标记。它是动物，鸟还是大角的猫头鹰？我们应该使用哪些图像级标签？在本文中，我们研究了标签粒度在WSOL中的作用。为了促进这项调查，我们引入了Inatloc500，这是一个新的用于WSOL的大规模细粒基准数据集。令人惊讶的是，我们发现选择正确的训练标签粒度比选择最佳的WSOL算法提供了更大的性能。我们还表明，更改标签粒度可以显着提高数据效率。

弱监督的对象本地化（WSOL）在过去几年中获得了普及，以便培训具有图像级标签的本地化模型。由于Soliminal WSOL类激活映射（CAM），该领域的重点是如何扩展注意区域更广泛地覆盖物体并更好地本地化。但是，这些策略依赖于验证超参数和模型选择的完全本地化监督，这是原则上禁止WSOL设置。在本文中，我们认为WSOL任务仅用图像级标签均不含糊，并提出了一种新的评估协议，其中全面监督仅限于仅与测试集没有重叠的小型举出的设置。我们观察到，根据我们的协议，五种最新的WSOL方法没有对CAM基线进行重大改进。此外，我们报告说，现有的WSOL方法尚未达到几次学习基准，其中验证时间的全面监督用于模型培训。根据我们的调查结果，我们讨论了WSOL的​​一些未来方向。

The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images. The challenge has been run annually from 2010 to present, attracting participation from more than fifty institutions. This paper describes the creation of this benchmark dataset and the advances in object recognition that have been possible as a result. We discuss the chal-

Existing image classification datasets used in computer vision tend to have a uniform distribution of images across object categories. In contrast, the natural world is heavily imbalanced, as some species are more abundant and easier to photograph than others. To encourage further progress in challenging real world conditions we present the iNaturalist species classification and detection dataset, consisting of 859,000 images from over 5,000 different species of plants and animals. It features visually similar species, captured in a wide variety of situations, from all over the world. Images were collected with different camera types, have varying image quality, feature a large class imbalance, and have been verified by multiple citizen scientists. We discuss the collection of the dataset and present extensive baseline experiments using state-of-the-art computer vision classification and detection models. Results show that current nonensemble based methods achieve only 67% top one classification accuracy, illustrating the difficulty of the dataset. Specifically, we observe poor results for classes with small numbers of training examples suggesting more attention is needed in low-shot learning.

在弱监督的本地化设置中，监督作为图像级标签。我们建议使用图像分类器$ F $，并培训发电网络$ G $，给定输入图像，指示图像内对象位置的每个像素权重映射。通过最大限度地减少原始图像上的分类器F $ F $的输出之间的差异来培训网络$ G $培训。该方案需要一个正常化术语，确保$ G $不提供统一的重量，以及提前停止标准，以防止超过段图像。我们的结果表明，该方法在充满挑战的细粒度分类数据集中的相当余量以及通用图像识别数据集中优于现有的本地化方法。另外，在细粒度分类数据集中的弱监督分割中，所获得的权重映射也是最新的。

Progress on object detection is enabled by datasets that focus the research community's attention on open challenges. This process led us from simple images to complex scenes and from bounding boxes to segmentation masks. In this work, we introduce LVIS (pronounced 'el-vis'): a new dataset for Large Vocabulary Instance Segmentation. We plan to collect ∼2 million high-quality instance segmentation masks for over 1000 entry-level object categories in 164k images. Due to the Zipfian distribution of categories in natural images, LVIS naturally has a long tail of categories with few training samples. Given that state-of-the-art deep learning methods for object detection perform poorly in the low-sample regime, we believe that our dataset poses an important and exciting new scientific challenge. LVIS is available at http://www.lvisdataset.org.

标记数据通常昂贵且耗时，特别是对于诸如对象检测和实例分割之类的任务，这需要对图像的密集标签进行密集的标签。虽然几张拍摄对象检测是关于培训小说中的模型（看不见的）对象类具有很少的数据，但它仍然需要在许多标记的基础（见）类的课程上进行训练。另一方面，自我监督的方法旨在从未标记数据学习的学习表示，该数据转移到诸如物体检测的下游任务。结合几次射击和自我监督的物体检测是一个有前途的研究方向。在本调查中，我们审查并表征了几次射击和自我监督对象检测的最新方法。然后，我们给我们的主要外卖，并讨论未来的研究方向。https://gabrielhuang.github.io/fsod-survey/的项目页面

细粒度的图像分析（FGIA）是计算机视觉和模式识别中的长期和基本问题，并为一组多种现实世界应用提供了基础。 FGIA的任务是从属类别分析视觉物体，例如汽车或汽车型号的种类。细粒度分析中固有的小阶级和阶级阶级内变异使其成为一个具有挑战性的问题。利用深度学习的进步，近年来，我们在深入学习动力的FGIA中见证了显着进展。在本文中，我们对这些进展的系统进行了系统的调查，我们试图通过巩固两个基本的细粒度研究领域 - 细粒度的图像识别和细粒度的图像检索来重新定义和扩大FGIA领域。此外，我们还审查了FGIA的其他关键问题，例如公开可用的基准数据集和相关域的特定于应用程序。我们通过突出几个研究方向和开放问题，从社区中突出了几个研究方向和开放问题。

通过使用图像级分类掩模监督其学习过程，弱监督对象本地化（WSOL）放宽对对象本地化的密度注释的要求。然而，当前的WSOL方法遭受背景位置的过度激活，并且需要后处理以获得定位掩模。本文将这些问题归因于背景提示的不明显，并提出了背景感知分类激活映射（B-CAM），以便仅使用图像级标签同时学习对象和背景的本地化分数。在我们的B-CAM中，两个图像级功能，由潜在背景和对象位置的像素级别功能聚合，用于从对象相关的背景中净化对象功能，并表示纯背景样本的功能，分别。然后基于这两个特征，学习对象分类器和背景分类器，以确定二进制对象本地化掩码。我们的B-CAM可以基于提出的错开分类损失以端到端的方式培训，这不仅可以改善对象本地化，而且还抑制了背景激活。实验表明，我们的B-CAM在Cub-200，OpenImages和VOC2012数据集上优于一级WSOL方法。

State-of-the-art visual perception models for a wide range of tasks rely on supervised pretraining. ImageNet classification is the de facto pretraining task for these models. Yet, ImageNet is now nearly ten years old and is by modern standards "small". Even so, relatively little is known about the behavior of pretraining with datasets that are multiple orders of magnitude larger. The reasons are obvious: such datasets are difficult to collect and annotate. In this paper, we present a unique study of transfer learning with large convolutional networks trained to predict hashtags on billions of social media images. Our experiments demonstrate that training for large-scale hashtag prediction leads to excellent results. We show improvements on several image classification and object detection tasks, and report the highest ImageNet-1k single-crop, top-1 accuracy to date: 85.4% (97.6% top-5). We also perform extensive experiments that provide novel empirical data on the relationship between large-scale pretraining and transfer learning performance. Name template Description train-IG-I-1.5k Instagram training set of I images and ∼1.5k hashtags from ImageNet-1k. train-IG-I-8.5k Instagram training set of I images and ∼8.5k hashtags from WordNet. train-IG-I-17k Instagram training set of I images and ∼17k hashtags from WordNet. train-IN-1M-1k The standard ImageNet-1k ILSVRC training set with 1.28M images. val-IN-50k-1k The standard ImageNet-1k ILSVRC validation set with 50k images. train-IN-I-L Extended ImageNet training set of I images and L ∈ {5k, 9k} labels. val-IN-I-L Extended ImageNet validation set of I images and L ∈ {5k, 9k} labels. train-CUB-6k-200 The Caltech-UCSD Birds-200-2011 training set. val-CUB-6k-200 The Caltech-UCSD Birds-200-2011 validation set. train-Places-1.8M-365 The Places365-Standard training set (high-resolution version). val-Places-37k-365 The Places365-Standard validation set (high-resolution version). train-COCO-135k-80 The standard COCO detection training set (2017 version). val-COCO-5k-80 The standard COCO detection validation set (2017 version). test-COCO-20k-80 The standard COCO detection test-dev set (2017 version).Table 1: Summary of image classification datasets. Each dataset is named with a template, role-source-I-L, that indicates its role (training, validation, testing), source, number of images I, and number of labels L.

Transfer of pre-trained representations improves sample efficiency and simplifies hyperparameter tuning when training deep neural networks for vision. We revisit the paradigm of pre-training on large supervised datasets and fine-tuning the model on a target task. We scale up pre-training, and propose a simple recipe that we call Big Transfer (BiT). By combining a few carefully selected components, and transferring using a simple heuristic, we achieve strong performance on over 20 datasets. BiT performs well across a surprisingly wide range of data regimes -from 1 example per class to 1 M total examples. BiT achieves 87.5% top-1 accuracy on ILSVRC-2012, 99.4% on CIFAR-10, and 76.3% on the 19 task Visual Task Adaptation Benchmark (VTAB). On small datasets, BiT attains 76.8% on ILSVRC-2012 with 10 examples per class, and 97.0% on CIFAR-10 with 10 examples per class. We conduct detailed analysis of the main components that lead to high transfer performance.

我们通过无监督学习的角度探索语义对应估计。我们使用标准化的评估协议彻底评估了最近提出的几种跨多个挑战数据集的无监督方法，在该协议中，我们会改变诸如骨干架构，预训练策略以及预训练和填充数据集等因素。为了更好地了解这些方法的故障模式，并为了提供更清晰的改进途径，我们提供了一个新的诊断框架以及一个新的性能指标，该指标更适合于语义匹配任务。最后，我们引入了一种新的无监督的对应方法，该方法利用了预训练的功能的强度，同时鼓励在训练过程中进行更好的比赛。与当前的最新方法相比，这会导致匹配性能明显更好。

弱监督对象本地化（WSOL）旨在仅使用图像级标签作为监控本地化对象区域。最近，通过生成前景预测映射（FPM）来实现新的范例来实现本地化任务。现有的基于FPM的方法使用跨熵（CE）来评估前景预测映射并引导发电机的学习。我们争辩使用激活值来实现更高效的学习。它基于实验观察，对于培训的网络，CE当前景掩模仅覆盖物体区域的一部分时，CE会聚到零。虽然激活值增加，直到掩码扩展到对象边界，这表明可以通过使用激活值来学习更多对象区域。在本文中，我们提出了背景激活抑制（BAS）方法。具体地，设计激活地图约束模块（AMC）以通过抑制背景激活值来促进生成器的学习。同时，通过使用前景区域指导和区域约束，BAS可以学习对象的整个区域。此外，在推理阶段，我们考虑不同类别的预测映射，以获得最终的本地化结果。广泛的实验表明，BAS通过CUB-200-2011和ILSVRC数据集的基线方法实现了显着和一致的改进。

In this work, we revisit the global average pooling layer proposed in [13], and shed light on how it explicitly enables the convolutional neural network to have remarkable localization ability despite being trained on image-level labels. While this technique was previously proposed as a means for regularizing training, we find that it actually builds a generic localizable deep representation that can be applied to a variety of tasks. Despite the apparent simplicity of global average pooling, we are able to achieve 37.1% top-5 error for object localization on ILSVRC 2014, which is remarkably close to the 34.2% top-5 error achieved by a fully supervised CNN approach. We demonstrate that our network is able to localize the discriminative image regions on a variety of tasks despite not being trained for them.

使用深度学习模型从组织学数据中诊断癌症提出了一些挑战。这些图像中关注区域（ROI）的癌症分级和定位通常依赖于图像和像素级标签，后者需要昂贵的注释过程。深度弱监督的对象定位（WSOL）方法为深度学习模型的低成本培训提供了不同的策略。仅使用图像级注释，可以训练这些方法以对图像进行分类，并为ROI定位进行分类类激活图（CAM）。本文综述了WSOL的​​最先进的DL方法。我们提出了一种分类法，根据模型中的信息流，将这些方法分为自下而上和自上而下的方法。尽管后者的进展有限，但最近的自下而上方法目前通过深层WSOL方法推动了很多进展。早期作品的重点是设计不同的空间合并功能。但是，这些方法达到了有限的定位准确性，并揭示了一个主要限制 - 凸轮的不足激活导致了高假阴性定位。随后的工作旨在减轻此问题并恢复完整的对象。评估和比较了两个具有挑战性的组织学数据集的分类和本地化准确性，对我们的分类学方法进行了评估和比较。总体而言，结果表明定位性能差，特别是对于最初设计用于处理自然图像的通用方法。旨在解决组织学数据挑战的方法产生了良好的结果。但是，所有方法都遭受高假阳性/阴性定位的影响。在组织学中应用深WSOL方法的应用是四个关键的挑战 - 凸轮的激活下/过度激活，对阈值的敏感性和模型选择。

由于检测数据集的规模小，当前对象探测器的词汇量受到限制。另一方面，图像分类器的原因是大约更大的词汇表，因为他们的数据集更大，更容易收集。我们提出守则，只需在图像分类数据上培训检测器的分类器，从而扩展了探测器的词汇量到数万个概念。与现有工作不同，拒绝不会根据模型预测将图像标签分配给框，使其更容易实现和兼容一系列检测架构和骨架。我们的结果表明，即使没有箱子注释，否则差异也能产生出色的探测器。它优于开放词汇和长尾检测基准的事先工作。拒绝为所有类和8.3地图提供了2.4地图的增益，用于开放词汇LVIS基准测试中的新型类。在标准的LVIS基准测试中，守护者达到41.7地图所有课程和41.7地图以获得罕见课程。我们首次培训一个探测器，其中包含所有二十一千类的ImageNet数据集，并显示它在没有微调的情况下推广到新数据集。代码可在https://github.com/facebookresearch/dorm提供。

将简单的体系结构与大规模预训练相结合已导致图像分类的大量改进。对于对象检测，预训练和缩放方法的确定性不佳，尤其是在长尾和开放式摄影的环境中，训练数据相对较少。在本文中，我们提出了一个强大的配方，用于将图像文本模型转移到开放式对象检测中。我们使用具有最小修改，对比度文本预训练和端到端检测微调的标准视觉变压器体系结构。我们对该设置的缩放属性的分析表明，增加图像级预训练和模型大小在下游检测任务上产生一致的改进。我们提供适应性策略和正规化，以实现零击文本条件和单次图像条件对象检测的非常强劲的性能。代码和型号可在GitHub上找到。

我们考虑临床应用异常定位问题。虽然深入学习推动了最近的医学成像进展，但许多临床挑战都没有完全解决，限制了其更广泛的使用。虽然最近的方法报告了高的诊断准确性，但医生因普遍缺乏算法决策和解释性而涉及诊断决策的这些算法，这是关注这些算法。解决这个问题的一种潜在方法是进一步培训这些模型，以便除了分类它们之外，除了分类。然而，准确地进行这一临床专家需要大量的疾病定位注释，这是对大多数应用程序来实现昂贵的任务。在这项工作中，我们通过一种新的注意力弱监督算法来解决这些问题，该弱势监督算法包括分层关注挖掘框架，可以以整体方式统一激活和基于梯度的视觉关注。我们的关键算法创新包括明确序号注意约束的设计，实现了以弱监督的方式实现了原则的模型培训，同时还通过本地化线索促进了产生视觉关注驱动的模型解释。在两个大型胸部X射线数据集（NIH Chescx-Ray14和Chexpert）上，我们展示了对现有技术的显着本地化性能，同时也实现了竞争的分类性能。我们的代码可在https://github.com/oyxhust/ham上找到。

It is desirable for detection and classification algorithms to generalize to unfamiliar environments, but suitable benchmarks for quantitatively studying this phenomenon are not yet available. We present a dataset designed to measure recognition generalization to novel environments. The images in our dataset are harvested from twenty camera traps deployed to monitor animal populations. Camera traps are fixed at one location, hence the background changes little across images; capture is triggered automatically, hence there is no human bias. The challenge is learning recognition in a handful of locations, and generalizing animal detection and classification to new locations where no training data is available. In our experiments state-of-the-art algorithms show excellent performance when tested at the same location where they were trained. However, we find that generalization to new locations is poor, especially for classification systems.

The explosion of image data on the Internet has the potential to foster more sophisticated and robust models and algorithms to index, retrieve, organize and interact with images and multimedia data. But exactly how such data can be harnessed and organized remains a critical problem. We introduce here a new database called "ImageNet", a largescale ontology of images built upon the backbone of the WordNet structure. ImageNet aims to populate the majority of the 80,000 synsets of WordNet with an average of 500-1000 clean and full resolution images. This will result in tens of millions of annotated images organized by the semantic hierarchy of WordNet. This paper offers a detailed analysis of ImageNet in its current state: 12 subtrees with 5247 synsets and 3.2 million images in total. We show that ImageNet is much larger in scale and diversity and much more accurate than the current image datasets. Constructing such a large-scale database is a challenging task. We describe the data collection scheme with Amazon Mechanical Turk. Lastly, we illustrate the usefulness of ImageNet through three simple applications in object recognition, image classification and automatic object clustering. We hope that the scale, accuracy, diversity and hierarchical structure of ImageNet can offer unparalleled opportunities to researchers in the computer vision community and beyond.