特征金字塔网络（FPN）已成为对象检测模型考虑对象的各种尺度的重要模块。但是，小物体上的平均精度（AP）相对低于中和大物体上的AP。原因是CNN较深层导致信息丢失作为特征提取水平的原因。我们提出了一个新的比例顺序（S^2）特征FPN的特征提取，以增强小物体的特征信息。我们将FPN结构视为尺度空间和提取尺度序列（s^2）特征，该特征是在FPN的水平轴上通过3D卷积。它基本上是扩展不变的功能，并建立在小物体的高分辨率金字塔功能图上。此外，建议的S^2功能可以扩展到基于FPN的大多数对象检测模型。我们证明所提出的S2功能可以提高COCO数据集中一阶段和两阶段探测器的性能。根据提出的S2功能，我们分别为Yolov4-P5和Yolov4-P6获得了高达1.3％和1.1％的AP改善。对于更快的RCNN和Mask R-CNN，我们分别观察到AP改进的2.0％和1.6％，分别具有建议的S^2功能。

特征金字塔网络（FPN）是对象检测器的关键组件之一。但是，对于研究人员来说，长期存在的难题是，引入FPN后通常会抑制大规模物体的检测性能。为此，本文首先在检测框架中重新审视FPN，并从优化的角度揭示了FPN成功的性质。然后，我们指出，大规模对象的性能退化是由于集成FPN后出现不当后传播路径所致。它使每个骨干网络的每个级别都只能查看一定尺度范围内的对象。基于这些分析，提出了两种可行的策略，以使每个级别的级别能够查看基于FPN的检测框架中的所有对象。具体而言，一个是引入辅助目标功能，以使每个骨干级在训练过程中直接接收各种尺度对象的后传播信号。另一个是以更合理的方式构建特征金字塔，以避免非理性的背部传播路径。对可可基准测试的广泛实验验证了我们的分析的健全性和方法的有效性。没有铃铛和口哨，我们证明了我们的方法在各种检测框架上实现了可靠的改进（超过2％）：一阶段，两阶段，基于锚的，无锚和变压器的检测器。

由于卷积在提取物体的局部上下文中，在过去十年中，对象检测在过去十年中取得了重大进展。但是，对象的尺度是多样的，当前卷积只能处理单尺度输入。因此，传统卷积具有固定接收场在处理这种规模差异问题方面的能力受到限制。多尺度功能表示已被证明是缓解规模差异问题的有效方法。最近的研究主要与某些量表或各个尺度的总体特征采用部分联系，并专注于整个量表的全球信息。但是，跨空间和深度维度的信息被忽略了。受此启发，我们提出了多尺度卷积（MSCONV）来解决此问题。同时考虑到量表，空间和深度信息，MSCONV能够更全面地处理多尺度输入。 MSCONV是有效的，并且在计算上是有效的，只有少量计算成本增加。对于大多数单阶段对象探测器，在检测头中用MSCONV代替传统的卷积可以带来AP的2.5 \％改进（在Coco 2017数据集上），只有3 \％的拖鞋增加了。 MSCONV对于两阶段对象探测器也具有灵活性和有效性。当扩展到主流两阶段对象检测器时，MSCONV的AP可以提高3.0 \％。我们在单尺度测试下的最佳模型在Coco 2017上实现了48.9 \％AP，\ textit {test-dev} Split，它超过了许多最新方法。

本文提出了平行残留的双融合特征金字塔网络（PRB-FPN），以快速准确地单光对象检测。特征金字塔（FP）在最近的视觉检测中被广泛使用，但是由于汇总转换，FP的自上而下的途径无法保留准确的定位。随着使用更多层的更深骨干，FP的优势被削弱了。此外，它不能同时准确地检测到小物体。为了解决这些问题，我们提出了一种新的并行FP结构，具有双向（自上而下和自下而上）的融合以及相关的改进，以保留高质量的特征以进行准确定位。我们提供以下设计改进：（1）具有自下而上的融合模块（BFM）的平行分歧FP结构，以高精度立即检测小物体和大对象。 （2）串联和重组（CORE）模块为特征融合提供了自下而上的途径，该途径导致双向融合FP，可以从低层特征图中恢复丢失的信息。 （3）进一步纯化核心功能以保留更丰富的上下文信息。自上而下和自下而上的途径中的这种核心净化只能在几次迭代中完成。 （4）将残留设计添加到核心中，导致了一个新的重核模块，该模块可以轻松训练和集成，并具有更深入或更轻的骨架。所提出的网络可在UAVDT17和MS COCO数据集上实现最新性能。代码可在https://github.com/pingyang1117/prbnet_pytorch上找到。

Feature pyramids are a basic component in recognition systems for detecting objects at different scales. But recent deep learning object detectors have avoided pyramid representations, in part because they are compute and memory intensive. In this paper, we exploit the inherent multi-scale, pyramidal hierarchy of deep convolutional networks to construct feature pyramids with marginal extra cost. A topdown architecture with lateral connections is developed for building high-level semantic feature maps at all scales. This architecture, called a Feature Pyramid Network (FPN), shows significant improvement as a generic feature extractor in several applications. Using FPN in a basic Faster R-CNN system, our method achieves state-of-the-art singlemodel results on the COCO detection benchmark without bells and whistles, surpassing all existing single-model entries including those from the COCO 2016 challenge winners. In addition, our method can run at 6 FPS on a GPU and thus is a practical and accurate solution to multi-scale object detection. Code will be made publicly available.

锥体网络是多尺度对象检测的标准方法。当前对特征金字塔网络的研究通常采用层连接来从特征层次结构的某些级别收集特征，并且不考虑它们之间的显着差异。我们提出了一个更好的特征金字塔网络的体系结构，称为选择性多尺度学习（SMSL），以解决此问题。SMSL高效且泛滥，可以将其集成到单阶段和两阶段检测器中以提高检测性能，几乎没有额外的推理成本。视网膜与SMSL的结合获得了可可数据集的AP（从39.1 \％到40.9 \％）的1.8 \％改进。与SMSL集成时，两阶段探测器的AP可以提高1.0 \％。

在传统的对象检测框架中，从图像识别模型继承的骨干体提取了深层特征，然后颈部模块融合了这些潜在特征，以在不同的尺度上捕获信息。由于对象检测的分辨率比图像识别大得多，因此骨干的计算成本通常主导了总推断成本。这种沉重的背部设计范式主要是由于历史遗产将图像识别模型传输到对象检测时，而不是端到端的优化设计以进行对象检测。在这项工作中，我们表明这种范式确实导致了亚最佳对象检测模型。为此，我们提出了一种新型的重颈范式，长颈鹿，这是一个类似长颈鹿的网络，用于有效的对象检测。长颈鹿使用极轻的骨干和非常深的颈部模块，可同时同时在不同的空间尺度以及不同级别的潜在语义之间进行密集的信息交换。该设计范式允许检测器即使在网络的早期阶段，也可以在相同的优先级处理高级语义信息和低级空间信息，从而使其在检测任务中更有效。对多个流行对象检测基准的数值评估表明，长颈鹿在广泛的资源约束中始终优于先前的SOTA模型。源代码可在https://github.com/jyqi/giraffedet上获得。

卷积神经网络（CNN）在许多计算机视觉任务（例如图像分类和对象检测）中取得了巨大的成功。但是，他们的性能在更艰巨的任务上迅速降低，因为图像是低分辨率或物体很小的。在本文中，我们指出，这根源于现有CNN体系结构中的有缺陷但常见的设计，即使用稳固的卷积和/或汇总层，这导致丢失细粒度的信息和学习较低有效的功能表示形式。为此，我们提出了一个新的CNN构建块，称为SPD-CONV，代替每个稳定的卷积层和每个池层（从而完全消除它们）。 SPD-CONV由一个对深度（SPD）层的组成，然后是非构造卷积（CORV）层，并且可以在大多数（如果不是全部）CNN体系结构中应用。我们在两个最具代表性的计算机视觉任务下解释了这种新设计：对象检测和图像分类。然后，我们通过将SPD-CONV应用于Yolov5和Resnet来创建新的CNN体​​系结构，并从经验上表明，我们的方法显着优于最先进的深度学习模型，尤其是在具有低分辨率图像和小物体的更艰巨的任务上。我们已经在https://github.com/labsaint/spd-conv上开源代码。

The way that information propagates in neural networks is of great importance. In this paper, we propose Path Aggregation Network (PANet) aiming at boosting information flow in proposal-based instance segmentation framework. Specifically, we enhance the entire feature hierarchy with accurate localization signals in lower layers by bottom-up path augmentation, which shortens the information path between lower layers and topmost feature. We present adaptive feature pooling, which links feature grid and all feature levels to make useful information in each feature level propagate directly to following proposal subnetworks. A complementary branch capturing different views for each proposal is created to further improve mask prediction.These improvements are simple to implement, with subtle extra computational overhead. Our PANet reaches the 1 st place in the COCO 2017 Challenge Instance Segmentation task and the 2 nd place in Object Detection task without large-batch training. It is also state-of-the-art on MVD and Cityscapes. Code is available at https://github. com/ShuLiu1993/PANet.

Modern object detectors have taken the advantages of backbone networks pre-trained on large scale datasets. Except for the backbone networks, however, other components such as the detector head and the feature pyramid network (FPN) remain trained from scratch, which hinders fully tapping the potential of representation models. In this study, we propose to integrally migrate pre-trained transformer encoder-decoders (imTED) to a detector, constructing a feature extraction path which is ``fully pre-trained" so that detectors' generalization capacity is maximized. The essential differences between imTED with the baseline detector are twofold: (1) migrating the pre-trained transformer decoder to the detector head while removing the randomly initialized FPN from the feature extraction path; and (2) defining a multi-scale feature modulator (MFM) to enhance scale adaptability. Such designs not only reduce randomly initialized parameters significantly but also unify detector training with representation learning intendedly. Experiments on the MS COCO object detection dataset show that imTED consistently outperforms its counterparts by $\sim$2.4 AP. Without bells and whistles, imTED improves the state-of-the-art of few-shot object detection by up to 7.6 AP. Code is available at https://github.com/LiewFeng/imTED.

现代物体检测网络追求一般物体检测数据集的更高精度，同时计算负担也随着精度的提高而越来越多。然而，推理时间和精度对于需要是实时的对象检测系统至关重要。没有额外的计算成本，有必要研究精度改进。在这项工作中，提出了两种模块以提高零成本的检测精度，这是一般对象检测网络的FPN和检测头改进。我们采用规模注意机制，以有效地保险熔断多级功能映射，参数较少，称为SA-FPN模块。考虑到分类头和回归头的相关性，我们使用顺序头取代广泛使用的并联头部，称为SEQ-Head模块。为了评估有效性，我们将这两个模块应用于一些现代最先进的对象检测网络，包括基于锚和无锚。 Coco DataSet上的实验结果表明，具有两个模块的网络可以将原始网络超越1.1 AP和0.8 AP，分别为锚的锚和无锚网络的零成本。代码将在https://git.io/jtfgl提供。

In recent years, object detection has achieved a very large performance improvement, but the detection result of small objects is still not very satisfactory. This work proposes a strategy based on feature fusion and dilated convolution that employs dilated convolution to broaden the receptive field of feature maps at various scales in order to address this issue. On the one hand, it can improve the detection accuracy of larger objects. On the other hand, it provides more contextual information for small objects, which is beneficial to improving the detection accuracy of small objects. The shallow semantic information of small objects is obtained by filtering out the noise in the feature map, and the feature information of more small objects is preserved by using multi-scale fusion feature module and attention mechanism. The fusion of these shallow feature information and deep semantic information can generate richer feature maps for small object detection. Experiments show that this method can have higher accuracy than the traditional YOLOv3 network in the detection of small objects and occluded objects. In addition, we achieve 32.8\% Mean Average Precision on the detection of small objects on MS COCO2017 test set. For 640*640 input, this method has 88.76\% mAP on the PASCAL VOC2012 dataset.

Cascade is a classic yet powerful architecture that has boosted performance on various tasks. However, how to introduce cascade to instance segmentation remains an open question. A simple combination of Cascade R-CNN and Mask R-CNN only brings limited gain. In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation. In this work, we propose a new framework, Hybrid Task Cascade (HTC), which differs in two important aspects: (1) instead of performing cascaded refinement on these two tasks separately, it interweaves them for a joint multi-stage processing; (2) it adopts a fully convolutional branch to provide spatial context, which can help distinguishing hard foreground from cluttered background. Overall, this framework can learn more discriminative features progressively while integrating complementary features together in each stage. Without bells and whistles, a single HTC obtains 38.4% and 1.5% improvement over a strong Cascade Mask R-CNN baseline on MSCOCO dataset. Moreover, our overall system achieves 48.6 mask AP on the test-challenge split, ranking 1st in the COCO 2018 Challenge Object Detection Task. Code is available at: https://github.com/ open-mmlab/mmdetection.

ous vision tasks without convolutions, where it can be used as a direct replacement for CNN backbones. (3) We validate PVT through extensive experiments, showing that it boosts the performance of many downstream tasks, including object detection, instance and semantic segmentation. For example, with a comparable number of parameters, PVT+RetinaNet achieves 40.4 AP on the COCO dataset, surpassing ResNet50+RetinNet (36.3 AP) by 4.1 absolute AP (see Figure 2). We hope that PVT could serve as an alternative and useful backbone for pixel-level predictions and facilitate future research.

Bottom-up human pose estimation methods have difficulties in predicting the correct pose for small persons due to challenges in scale variation. In this paper, we present HigherHRNet: a novel bottom-up human pose estimation method for learning scale-aware representations using high-resolution feature pyramids. Equipped with multi-resolution supervision for training and multiresolution aggregation for inference, the proposed approach is able to solve the scale variation challenge in bottom-up multi-person pose estimation and localize keypoints more precisely, especially for small person. The feature pyramid in HigherHRNet consists of feature map outputs from HRNet and upsampled higher-resolution outputs through a transposed convolution. HigherHR-Net outperforms the previous best bottom-up method by 2.5% AP for medium person on COCO test-dev, showing its effectiveness in handling scale variation. Furthermore, HigherHRNet achieves new state-of-the-art result on COCO test-dev (70.5% AP) without using refinement or other post-processing techniques, surpassing all existing bottom-up methods. HigherHRNet even surpasses all topdown methods on CrowdPose test (67.6% AP), suggesting its robustness in crowded scene. The code and models are available at https://github.com/HRNet/ Higher-HRNet-Human-Pose-Estimation.

两阶段和基于查询的实例分段方法取得了显着的结果。然而，他们的分段面具仍然非常粗糙。在本文中，我们呈现了用于高质量高效的实例分割的掩模转发器。我们的掩模转发器代替常规密集的张量，而不是在常规密集的张量上进行分解，并表示作为Quadtree的图像区域。我们基于变换器的方法仅处理检测到的错误易于树节点，并并行自我纠正其错误。虽然这些稀疏的像素仅构成总数的小比例，但它们对最终掩模质量至关重要。这允许掩模转换器以低计算成本预测高精度的实例掩模。广泛的实验表明，掩模转发器在三个流行的基准上优于当前实例分段方法，显着改善了COCO和BDD100K上的大型+3.0掩模AP的+3.0掩模AP的大余量和CityScapes上的+6.6边界AP。我们的代码和培训的型号将在http://vis.xyz/pub/transfiner提供。

本文介绍了Houghnet，这是一种单阶段，无锚，基于投票的，自下而上的对象检测方法。受到广义的霍夫变换的启发，霍尼特通过在该位置投票的总和确定了某个位置的物体的存在。投票是根据对数极极投票领域的近距离和长距离地点收集的。由于这种投票机制，Houghnet能够整合近距离和远程的班级条件证据以进行视觉识别，从而概括和增强当前的对象检测方法，这通常仅依赖于本地证据。在可可数据集中，Houghnet的最佳型号达到$ 46.4 $ $ $ ap $（和$ 65.1 $ $ $ ap_ {50} $），与自下而上的对象检测中的最先进的作品相同，超越了最重要的一项 - 阶段和两阶段方法。我们进一步验证了提案在其他视觉检测任务中的有效性，即视频对象检测，实例分割，3D对象检测和人为姿势估计的关键点检测以及其他“图像”图像生成任务的附加“标签”，其中集成的集成在所有情况下，我们的投票模块始终提高性能。代码可在https://github.com/nerminsamet/houghnet上找到。

Letting a deep network be aware of the quality of its own predictions is an interesting yet important problem. In the task of instance segmentation, the confidence of instance classification is used as mask quality score in most instance segmentation frameworks. However, the mask quality, quantified as the IoU between the instance mask and its ground truth, is usually not well correlated with classification score. In this paper, we study this problem and propose Mask Scoring R-CNN which contains a network block to learn the quality of the predicted instance masks. The proposed network block takes the instance feature and the corresponding predicted mask together to regress the mask IoU. The mask scoring strategy calibrates the misalignment between mask quality and mask score, and improves instance segmentation performance by prioritizing more accurate mask predictions during COCO AP evaluation. By extensive evaluations on the COCO dataset, Mask Scoring R-CNN brings consistent and noticeable gain with different models, and outperforms the state-of-the-art Mask R-CNN. We hope our simple and effective approach will provide a new direction for improving instance segmentation. The source code of our method is available at https:// github.com/zjhuang22/maskscoring_rcnn. * The work was done when Zhaojin Huang was an intern in Horizon Robotics Inc.

In object detection, the intersection over union (IoU) threshold is frequently used to define positives/negatives. The threshold used to train a detector defines its quality. While the commonly used threshold of 0.5 leads to noisy (low-quality) detections, detection performance frequently degrades for larger thresholds. This paradox of high-quality detection has two causes: 1) overfitting, due to vanishing positive samples for large thresholds, and 2) inference-time quality mismatch between detector and test hypotheses. A multi-stage object detection architecture, the Cascade R-CNN, composed of a sequence of detectors trained with increasing IoU thresholds, is proposed to address these problems. The detectors are trained sequentially, using the output of a detector as training set for the next. This resampling progressively improves hypotheses quality, guaranteeing a positive training set of equivalent size for all detectors and minimizing overfitting. The same cascade is applied at inference, to eliminate quality mismatches between hypotheses and detectors. An implementation of the Cascade R-CNN without bells or whistles achieves state-of-the-art performance on the COCO dataset, and significantly improves high-quality detection on generic and specific object detection datasets, including VOC, KITTI, CityPerson, and WiderFace. Finally, the Cascade R-CNN is generalized to instance segmentation, with nontrivial improvements over the Mask R-CNN. To facilitate future research, two implementations are made available at https://github.com/zhaoweicai/cascade-rcnn (Caffe) and https://github.com/zhaoweicai/Detectron-Cascade-RCNN (Detectron).

Compared with model architectures, the training process, which is also crucial to the success of detectors, has received relatively less attention in object detection. In this work, we carefully revisit the standard training practice of detectors, and find that the detection performance is often limited by the imbalance during the training process, which generally consists in three levels -sample level, feature level, and objective level. To mitigate the adverse effects caused thereby, we propose Libra R-CNN, a simple but effective framework towards balanced learning for object detection. It integrates three novel components: IoU-balanced sampling, balanced feature pyramid, and balanced L1 loss, respectively for reducing the imbalance at sample, feature, and objective level. Benefitted from the overall balanced design, Libra R-CNN significantly improves the detection performance. Without bells and whistles, it achieves 2.5 points and 2.0 points higher Average Precision (AP) than FPN Faster R-CNN and RetinaNet respectively on MSCOCO. 1