In this work, we revisit the global average pooling layer proposed in [13], and shed light on how it explicitly enables the convolutional neural network to have remarkable localization ability despite being trained on image-level labels. While this technique was previously proposed as a means for regularizing training, we find that it actually builds a generic localizable deep representation that can be applied to a variety of tasks. Despite the apparent simplicity of global average pooling, we are able to achieve 37.1% top-5 error for object localization on ILSVRC 2014, which is remarkably close to the 34.2% top-5 error achieved by a fully supervised CNN approach. We demonstrate that our network is able to localize the discriminative image regions on a variety of tasks despite not being trained for them.

We propose a technique for producing 'visual explanations' for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable.Our approach -Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say 'dog' in a classification network or a sequence of words in captioning network) flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept.Unlike previous approaches, Grad-CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fullyconnected layers (e.g. VGG), (2) CNNs used for structured outputs (e.g. captioning), (3) CNNs used in tasks with multimodal inputs (e.g. visual question answering) or reinforcement learning, all without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative vi-

Convolutional networks for image classification progressively reduce resolution until the image is represented by tiny feature maps in which the spatial structure of the scene is no longer discernible. Such loss of spatial acuity can limit image classification accuracy and complicate the transfer of the model to downstream applications that require detailed scene understanding. These problems can be alleviated by dilation, which increases the resolution of output feature maps without reducing the receptive field of individual neurons. We show that dilated residual networks (DRNs) outperform their non-dilated counterparts in image classification without increasing the model's depth or complexity. We then study gridding artifacts introduced by dilation, develop an approach to removing these artifacts ('degridding'), and show that this further increases the performance of DRNs. In addition, we show that the accuracy advantage of DRNs is further magnified in downstream applications such as object localization and semantic segmentation.

The rise of multi-million-item dataset initiatives has enabled data-hungry machine learning algorithms to reach nearhuman semantic classification performance at tasks such as visual object and scene recognition. Here we describe the Places Database, a repository of 10 million scene photographs, labeled with scene semantic categories, comprising a large and diverse list of the types of environments encountered in the world. Using the state-of-the-art Convolutional Neural Networks (CNNs), we provide scene classification CNNs (Places-CNNs) as baselines, that significantly outperform the previous approaches. Visualization of the CNNs trained on Places shows that object detectors emerge as an intermediate representation of scene classification. With its high-coverage and high-diversity of exemplars, the Places Database along with the Places-CNNs offer a novel resource to guide future progress on scene recognition problems.

在弱监督的本地化设置中，监督作为图像级标签。我们建议使用图像分类器$ F $，并培训发电网络$ G $，给定输入图像，指示图像内对象位置的每个像素权重映射。通过最大限度地减少原始图像上的分类器F $ F $的输出之间的差异来培训网络$ G $培训。该方案需要一个正常化术语，确保$ G $不提供统一的重量，以及提前停止标准，以防止超过段图像。我们的结果表明，该方法在充满挑战的细粒度分类数据集中的相当余量以及通用图像识别数据集中优于现有的本地化方法。另外，在细粒度分类数据集中的弱监督分割中，所获得的权重映射也是最新的。

We aim to model the top-down attention of a Convolutional Neural Network (CNN) classifier for generating task-specific attention maps. Inspired by a top-down human visual attention model, we propose a new backpropagation scheme, called Excitation Backprop, to pass along top-down signals downwards in the network hierarchy via a probabilistic Winner-Take-All process. Furthermore, we introduce the concept of contrastive attention to make the top-down attention maps more discriminative. In experiments, we demonstrate the accuracy and generalizability of our method in weakly supervised localization tasks on the MS COCO, PASCAL VOC07 and ImageNet datasets. The usefulness of our method is further validated in the text-to-region association task. On the Flickr30k Entities dataset, we achieve promising performance in phrase localization by leveraging the top-down attention of a CNN model that has been trained on weakly labeled web images.

Recently, increasing attention has been drawn to the internal mechanisms of convolutional neural networks, and the reason why the network makes specific decisions. In this paper, we develop a novel post-hoc visual explanation method called Score-CAM based on class activation mapping. Unlike previous class activation mapping based approaches, Score-CAM gets rid of the dependence on gradients by obtaining the weight of each activation map through its forward passing score on target class, the final result is obtained by a linear combination of weights and activation maps. We demonstrate that Score-CAM achieves better visual performance and fairness for interpreting the decision making process. Our approach outperforms previous methods on both recognition and localization tasks, it also passes the sanity check. We also indicate its application as debugging tools. The implementation is available 1 .

图像和视频中的消防本地化是自治系统对抗火事故的重要一步。基于深神经网络的最先进的图像分割方法需要大量的像素注释样本以以完全监督的方式训练卷积神经网络（CNNS）。在本文中，我们考虑了图像中的火灾的弱监管，其中仅使用图像标签来训练网络。我们示出在火灾分割的情况下，这是二进制分割问题，分类中的中层中的特征的平均值可以比传统的类激活映射（CAM）方法更好。我们还建议通过在上次卷积层的特征上添加旋转等值正则化损耗来进一步提高分割精度。我们的结果表明，对弱势监督的火灾细分的基线方法显着改善。

We propose a general framework called Network Dissection for quantifying the interpretability of latent representations of CNNs by evaluating the alignment between individual hidden units and a set of semantic concepts. Given any CNN model, the proposed method draws on a broad data set of visual concepts to score the semantics of hidden units at each intermediate convolutional layer. The units with semantics are given labels across a range of objects, parts, scenes, textures, materials, and colors. We use the proposed method to test the hypothesis that interpretability of units is equivalent to random linear combinations of units, then we apply our method to compare the latent representations of various networks when trained to solve different supervised and self-supervised training tasks. We further analyze the effect of training iterations, compare networks trained with different initializations, examine the impact of network depth and width, and measure the effect of dropout and batch normalization on the interpretability of deep visual representations. We demonstrate that the proposed method can shed light on characteristics of CNN models and training methods that go beyond measurements of their discriminative power.

弱监督的对象本地化（WSOL）旨在学习仅使用图像级类别标签编码对象位置的表示形式。但是，许多物体可以在不同水平的粒度标记。它是动物，鸟还是大角的猫头鹰？我们应该使用哪些图像级标签？在本文中，我们研究了标签粒度在WSOL中的作用。为了促进这项调查，我们引入了Inatloc500，这是一个新的用于WSOL的大规模细粒基准数据集。令人惊讶的是，我们发现选择正确的训练标签粒度比选择最佳的WSOL算法提供了更大的性能。我们还表明，更改标签粒度可以显着提高数据效率。

使用深度学习模型从组织学数据中诊断癌症提出了一些挑战。这些图像中关注区域（ROI）的癌症分级和定位通常依赖于图像和像素级标签，后者需要昂贵的注释过程。深度弱监督的对象定位（WSOL）方法为深度学习模型的低成本培训提供了不同的策略。仅使用图像级注释，可以训练这些方法以对图像进行分类，并为ROI定位进行分类类激活图（CAM）。本文综述了WSOL的​​最先进的DL方法。我们提出了一种分类法，根据模型中的信息流，将这些方法分为自下而上和自上而下的方法。尽管后者的进展有限，但最近的自下而上方法目前通过深层WSOL方法推动了很多进展。早期作品的重点是设计不同的空间合并功能。但是，这些方法达到了有限的定位准确性，并揭示了一个主要限制 - 凸轮的不足激活导致了高假阴性定位。随后的工作旨在减轻此问题并恢复完整的对象。评估和比较了两个具有挑战性的组织学数据集的分类和本地化准确性，对我们的分类学方法进行了评估和比较。总体而言，结果表明定位性能差，特别是对于最初设计用于处理自然图像的通用方法。旨在解决组织学数据挑战的方法产生了良好的结果。但是，所有方法都遭受高假阳性/阴性定位的影响。在组织学中应用深WSOL方法的应用是四个关键的挑战 - 凸轮的激活下/过度激活，对阈值的敏感性和模型选择。

Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that typically combine multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012-achieving a mAP of 53.3%. Our approach combines two key insights:(1) one can apply high-capacity convolutional neural networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost. Since we combine region proposals with CNNs, we call our method R-CNN: Regions with CNN features. We also compare R-CNN to OverFeat, a recently proposed sliding-window detector based on a similar CNN architecture. We find that R-CNN outperforms OverFeat by a large margin on the 200-class ILSVRC2013 detection dataset. Source code for the complete system is available at http://www.cs.berkeley.edu/ ˜rbg/rcnn.

弱监督对象本地化（WSOL）旨在仅通过使用图像级标签来学习对象本地化器。基于卷积神经网络（CNN）的技术通常导致突出显示物体的最辨别部分，同时忽略整个对象范围。最近，变压器架构已经部署到WSOL，以捕获具有自我关注机制和多层的Perceptron结构的远程特征依赖性。然而，变压器缺乏CNN所固有的局部感应偏差，因此可以恶化WSOL中的局部特征细节。在本文中，我们提出了一种基于变压器的新型框架，称为LCTR（局部连续性变压器），该框架被称为LCTR（局部连续性变压器），该框架在长期特征依赖项中提高全局特征的本地感知能力。为此，我们提出了一个关系的修补程序注意模块（RPAM），其考虑全球跨补丁信息。我们进一步设计了一个CUE挖掘模块（CDM），它利用本地特征来指导模型的学习趋势，以突出弱局部响应。最后，在两个广泛使用的数据集，即Cub-200-2011和ILSVRC上进行综合实验，以验证我们方法的有效性。

3D卷积神经网络（3D CNN）在诸如视频序列之类的3D数据中捕获空间和时间信息。然而，由于卷积和汇集机制，信息损失似乎是不可避免的。为了改善3D CNN的视觉解释和分类，我们提出了两种方法; i）使用培训的3dresnext网络聚合到本地（全局 - 本地）离散梯度的层面全局，II）实施注意门控网络以提高动作识别的准确性。拟议的方法打算通过视觉归因，弱监督行动本地化和行动识别，显示各层在3D CNN中被称为全球局部关注的有用性。首先，使用关于最大预测类的BackPropagation培训3dresnext培训并应用于动作分类。然后将每层的梯度和激活取样。稍后，聚合用于产生更细致的注意力，指出了预测类输入视频的最关键部分。我们使用最终关注的轮廓阈值为最终的本地化。我们使用3DCAM使用细粒度的视觉解释来评估修剪视频中的空间和时间动作定位。实验结果表明，该拟议方法产生了丰富的视觉解释和歧视性的关注。此外，通过每个层上的注意栅格的动作识别产生比基线模型更好的分类结果。

We present an integrated framework for using Convolutional Networks for classification, localization and detection. We show how a multiscale and sliding window approach can be efficiently implemented within a ConvNet. We also introduce a novel deep learning approach to localization by learning to predict object boundaries. Bounding boxes are then accumulated rather than suppressed in order to increase detection confidence. We show that different tasks can be learned simultaneously using a single shared network. This integrated framework is the winner of the localization task of the ImageNet Large Scale Visual Recognition Challenge 2013 (ILSVRC2013) and obtained very competitive results for the detection and classifications tasks. In post-competition work, we establish a new state of the art for the detection task. Finally, we release a feature extractor from our best model called OverFeat.

通过使用图像级分类掩模监督其学习过程，弱监督对象本地化（WSOL）放宽对对象本地化的密度注释的要求。然而，当前的WSOL方法遭受背景位置的过度激活，并且需要后处理以获得定位掩模。本文将这些问题归因于背景提示的不明显，并提出了背景感知分类激活映射（B-CAM），以便仅使用图像级标签同时学习对象和背景的本地化分数。在我们的B-CAM中，两个图像级功能，由潜在背景和对象位置的像素级别功能聚合，用于从对象相关的背景中净化对象功能，并表示纯背景样本的功能，分别。然后基于这两个特征，学习对象分类器和背景分类器，以确定二进制对象本地化掩码。我们的B-CAM可以基于提出的错开分类损失以端到端的方式培训，这不仅可以改善对象本地化，而且还抑制了背景激活。实验表明，我们的B-CAM在Cub-200，OpenImages和VOC2012数据集上优于一级WSOL方法。

细粒度视觉识别的挑战通常在于发现关键的歧视区域。虽然可以从大规模标记的数据集中自动识别此类区域，但是当仅提供少量注释时，类似的方法可能会降低效率。在低数据制度中，网络通常很难选择正确的区域以识别识别，并且倾向于从培训数据中过度拟合虚假的相关模式。为了解决这个问题，本文提出了一种自我提升的注意机制，这是一种新颖的方法，可以使网络正规化关注跨样本和类共享的关键区域。具体而言，提出的方法首先为每个训练图像生成一个注意图，突出显示用于识别地面真实对象类别的判别零件。然后将生成的注意图用作伪通量。该网络被执行以适合它们作为辅助任务。我们将这种方法称为自我增强注意机制（SAM）。我们还通过使用SAM创建多个注意地图来开发一个变体，以泳池卷积图的样式，以双线性合并，称为SAM双线性。通过广泛的实验研究，我们表明两种方法都可以显着提高低数据状态上的细粒度视觉识别性能，并可以纳入现有的网络体系结构中。源代码可公开可用：https：//github.com/ganperf/sam

基于弱监管的像素 - 明显的密集预测任务当前使用类注意映射（CAM）以产生伪掩模作为地面真理。然而，现有方法通常取决于诱人的训练模块，这可能会引入磨削计算开销和复杂的培训程序。在这项工作中，提出了语义结构知识推断（SSA）来探索隐藏在基于CNN的网络的不同阶段的语义结构信息，以在模型推断中产生高质量凸轮。具体地，首先提出语义结构建模模块（SSM）来生成类别不可知语义相关表示，其中每个项目表示一个类别对象和所有其他类别之间的亲和程度。然后，探索结构化特征表示通过点产品操作来抛光不成熟的凸轮。最后，来自不同骨架级的抛光凸轮融合为输出。所提出的方法具有没有参数的优点，不需要培训。因此，它可以应用于广泛的弱监管像素 - 明智的密集预测任务。对弱势监督对象本地化和弱监督语义分割任务的实验结果证明了该方法的效力，这使得新的最先进的结果实现了这两项任务。

We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be repurposed to novel generic tasks. Our generic tasks may differ significantly from the originally trained tasks and there may be insufficient labeled or unlabeled data to conventionally train or adapt a deep architecture to the new tasks. We investigate and visualize the semantic clustering of deep convolutional features with respect to a variety of such tasks, including scene recognition, domain adaptation, and fine-grained recognition challenges. We compare the efficacy of relying on various network levels to define a fixed feature, and report novel results that significantly outperform the state-of-the-art on several important vision challenges. We are releasing DeCAF, an open-source implementation of these deep convolutional activation features, along with all associated network parameters to enable vision researchers to be able to conduct experimentation with deep representations across a range of visual concept learning paradigms.

解释方法有助于开发学习有意义的概念的模型，并避免利用杂散相关性。我们说明了对流行神经网络的先前未被识别的限制解释方法GRAC-CAM：作为梯度平均步骤的副作用，Grad-Cam有时突出显示模型实际使用的位置。为了解决这个问题，我们提出了一种新颖的类特定的解释方法，保证只能突出显示用于制作每个预测的模型的位置。我们证明了亨克斯姆是凸轮的泛化，探索赫什罕与基于梯度的解释方法之间的关系。 Pascal VOC 2012的实验，包括人群源评级，阐述了虽然亨克斯姆的解释忠实地反映了该模型，但Grad-Cam往往会扩大注意力创造更大和更平滑的可视化。总体而言，这项工作进展了卷积神经网络的解释方法，可以帮助开发敏感应用的可靠性模型。