在对象检测模型中,检测骨干机消耗超过一半的整体推理成本。最近的研究试图通过在神经结构搜索(NAS)的帮助下优化骨干架构来降低这一成本。然而,对象检测的现有NAS方法需要数百至数千个GPU小时的搜索,使它们在快节奏的研究和开发中不切实际。在这项工作中,我们提出了一种新的零射NAS方法来解决这个问题。所提出的方法,命名为Zendet,在不训练网络参数的情况下自动设计有效的检测骨干网,从而降低了架构设计成本,几乎归零但提供了最先进的(SOTA)性能。在引擎盖下,Zendet最大化了检测骨干的差分熵,导致对象检测的更好的特征提取器,在相同的计算预算下。在仅为全自动设计的一个GPU日之后,Zendet在多个检测基准数据集上创新了SOTA检测骨干,具有很少的人为干预。与Reset-50个骨干相比,Zendet在Map中使用相同数量的拖波/参数时更好地+ 2.0%,并且在同一地图上的NVIDIA V100速度快1.54倍。稍后将发布代码和预先训练的型号。
translated by 谷歌翻译
最近,已经成功地应用于各种遥感图像(RSI)识别任务的大量基于深度学习的方法。然而,RSI字段中深度学习方法的大多数现有进步严重依赖于手动设计的骨干网络提取的特征,这严重阻碍了由于RSI的复杂性以及先前知识的限制而受到深度学习模型的潜力。在本文中,我们研究了RSI识别任务中的骨干架构的新设计范式,包括场景分类,陆地覆盖分类和对象检测。提出了一种基于权重共享策略和进化算法的一拍架构搜索框架,称为RSBNet,其中包括三个阶段:首先,在层面搜索空间中构造的超空网是在自组装的大型中预先磨削 - 基于集合单路径培训策略进行缩放RSI数据集。接下来,预先培训的SuperNet通过可切换识别模块配备不同的识别头,并分别在目标数据集上进行微调,以获取特定于任务特定的超网络。最后,我们根据没有任何网络训练的进化算法,搜索最佳骨干架构进行不同识别任务。对于不同识别任务的五个基准数据集进行了广泛的实验,结果显示了所提出的搜索范例的有效性,并证明搜索后的骨干能够灵活地调整不同的RSI识别任务并实现令人印象深刻的性能。
translated by 谷歌翻译
在传统的对象检测框架中,从图像识别模型继承的骨干体提取了深层特征,然后颈部模块融合了这些潜在特征,以在不同的尺度上捕获信息。由于对象检测的分辨率比图像识别大得多,因此骨干的计算成本通常主导了总推断成本。这种沉重的背部设计范式主要是由于历史遗产将图像识别模型传输到对象检测时,而不是端到端的优化设计以进行对象检测。在这项工作中,我们表明这种范式确实导致了亚最佳对象检测模型。为此,我们提出了一种新型的重颈范式,长颈鹿,这是一个类似长颈鹿的网络,用于有效的对象检测。长颈鹿使用极轻的骨干和非常深的颈部模块,可同时同时在不同的空间尺度以及不同级别的潜在语义之间进行密集的信息交换。该设计范式允许检测器即使在网络的早期阶段,也可以在相同的优先级处理高级语义信息和低级空间信息,从而使其在检测任务中更有效。对多个流行对象检测基准的数值评估表明,长颈鹿在广泛的资源约束中始终优于先前的SOTA模型。源代码可在https://github.com/jyqi/giraffedet上获得。
translated by 谷歌翻译
Recently, Neural architecture search has achieved great success on classification tasks for mobile devices. The backbone network for object detection is usually obtained on the image classification task. However, the architecture which is searched through the classification task is sub-optimal because of the gap between the task of image and object detection. As while work focuses on backbone network architecture search for mobile device object detection is limited, mainly because the backbone always requires expensive ImageNet pre-training. Accordingly, it is necessary to study the approach of network architecture search for mobile device object detection without expensive pre-training. In this work, we propose a mobile object detection backbone network architecture search algorithm which is a kind of evolutionary optimized method based on non-dominated sorting for NAS scenarios. It can quickly search to obtain the backbone network architecture within certain constraints. It better solves the problem of suboptimal linear combination accuracy and computational cost. The proposed approach can search the backbone networks with different depths, widths, or expansion sizes via a technique of weight mapping, making it possible to use NAS for mobile devices detection tasks a lot more efficiently. In our experiments, we verify the effectiveness of the proposed approach on YoloX-Lite, a lightweight version of the target detection framework. Under similar computational complexity, the accuracy of the backbone network architecture we search for is 2.0% mAP higher than MobileDet. Our improved backbone network can reduce the computational effort while improving the accuracy of the object detection network. To prove its effectiveness, a series of ablation studies have been carried out and the working mechanism has been analyzed in detail.
translated by 谷歌翻译
由于存储器和计算资源有限,部署在移动设备上的卷积神经网络(CNNS)是困难的。我们的目标是通过利用特征图中的冗余来设计包括CPU和GPU的异构设备的高效神经网络,这很少在神经结构设计中进行了研究。对于类似CPU的设备,我们提出了一种新颖的CPU高效的Ghost(C-Ghost)模块,以生成从廉价操作的更多特征映射。基于一组内在的特征映射,我们使用廉价的成本应用一系列线性变换,以生成许多幽灵特征图,可以完全揭示内在特征的信息。所提出的C-Ghost模块可以作为即插即用组件,以升级现有的卷积神经网络。 C-Ghost瓶颈旨在堆叠C-Ghost模块,然后可以轻松建立轻量级的C-Ghostnet。我们进一步考虑GPU设备的有效网络。在建筑阶段的情况下,不涉及太多的GPU效率(例如,深度明智的卷积),我们建议利用阶段明智的特征冗余来制定GPU高效的幽灵(G-GHOST)阶段结构。舞台中的特征被分成两个部分,其中使用具有较少输出通道的原始块处理第一部分,用于生成内在特征,另一个通过利用阶段明智的冗余来生成廉价的操作。在基准测试上进行的实验证明了所提出的C-Ghost模块和G-Ghost阶段的有效性。 C-Ghostnet和G-Ghostnet分别可以分别实现CPU和GPU的准确性和延迟的最佳权衡。代码可在https://github.com/huawei-noah/cv-backbones获得。
translated by 谷歌翻译
Adder神经网络(Addernets)在图像分类上表现出令人印象深刻的性能,只有加法操作,比使用乘法建立的传统卷积神经网络更节能。与分类相比,对通过Addernets降低现代对象探测器的能耗的强烈需求,例如自主驾驶和面部检测。在本文中,我们提出了对物体检测的addernets的实证研究。我们首先揭示了预先训练的加法器骨架中的批量归一化统计,不应冻结,因为Addernets的相对较大的特征方差。此外,我们在颈部中插入更多的快捷方式连接,并设计一个新的特征融合架构,以避免加法器层的稀疏功能。我们展示了广泛的消融研究,探讨了加法器探测器的几种设计选择。与最先进的比较在Coco和Pascal VOC基准上进行。具体而言,所提出的加法器FCOS在Coco Val集上实现了37.8 \%AP,展示了卷积对应物的相当性能,具有约1.4倍的能量减少。
translated by 谷歌翻译
神经结构搜索(NAS)已被广泛采用设计准确,高效的图像分类模型。但是,将NAS应用于新的计算机愿景任务仍然需要大量的努力。这是因为1)以前的NAS研究已经过度优先考虑图像分类,同时在很大程度上忽略了其他任务; 2)许多NAS工作侧重于优化特定于任务特定的组件,这些组件不能有利地转移到其他任务; 3)现有的NAS方法通常被设计为“Proxyless”,需要大量努力与每个新任务的培训管道集成。为了解决这些挑战,我们提出了FBNetv5,这是一个NAS框架,可以在各种视觉任务中寻找神经架构,以降低计算成本和人力努力。具体而言,我们设计1)一个简单但包容性和可转换的搜索空间; 2)用目标任务培训管道解开的多址搜索过程; 3)一种算法,用于同时搜索具有计算成本不可知的多个任务的架构到任务数。我们评估所提出的FBNetv5目标三个基本视觉任务 - 图像分类,对象检测和语义分割。 FBNETV5在单一搜索中搜索的模型在所有三个任务中都表现优于先前的议定书 - 现有技术:图像分类(例如,与FBNetv3相比,在与FBNetv3相比的同一拖鞋下的1 + 1.3%Imageet Top-1精度。 (例如,+ 1.8%较高的Ade20k Val。Miou比SegFormer为3.6倍的拖鞋),对象检测(例如,+ 1.1%Coco Val。与yolox相比,拖鞋的1.2倍的地图。
translated by 谷歌翻译
Model efficiency has become increasingly important in computer vision. In this paper, we systematically study neural network architecture design choices for object detection and propose several key optimizations to improve efficiency. First, we propose a weighted bi-directional feature pyramid network (BiFPN), which allows easy and fast multiscale feature fusion; Second, we propose a compound scaling method that uniformly scales the resolution, depth, and width for all backbone, feature network, and box/class prediction networks at the same time. Based on these optimizations and better backbones, we have developed a new family of object detectors, called EfficientDet, which consistently achieve much better efficiency than prior art across a wide spectrum of resource constraints. In particular, with singlemodel and single-scale, our EfficientDet-D7 achieves stateof-the-art 55.1 AP on COCO test-dev with 77M parameters and 410B FLOPs 1 , being 4x -9x smaller and using 13x -42x fewer FLOPs than previous detectors. Code is available at https://github.com/google/automl/tree/ master/efficientdet.
translated by 谷歌翻译
大多数对象检测框架都使用最初设计用于图像分类的主链体系结构,通常在Imagenet上具有预训练的参数。但是,图像分类和对象检测本质上是不同的任务,并且不能保证分类的最佳主链也适用于对象检测。最近的神经体系结构搜索(NAS)研究表明,自动设计专门用于对象检测的骨干有助于提高整体准确性。在本文中,我们引入了一种神经体系结构适应方法,该方法可以优化给定的主链以进行检测目的,同时仍允许使用预训练的参数。我们建议除了每个块的输出通道尺寸外,还通过搜索特定操作和层数来调整微体系结构。重要的是要找到最佳的通道深度,因为它极大地影响了特征表示功能和计算成本。我们使用搜索的主链进行对象检测进行实验,并证明我们的主链在可可数据集上的手动设计和搜索的最新骨干均优于手动设计和搜索的骨干。
translated by 谷歌翻译
In this paper, we aim to design an efficient real-time object detector that exceeds the YOLO series and is easily extensible for many object recognition tasks such as instance segmentation and rotated object detection. To obtain a more efficient model architecture, we explore an architecture that has compatible capacities in the backbone and neck, constructed by a basic building block that consists of large-kernel depth-wise convolutions. We further introduce soft labels when calculating matching costs in the dynamic label assignment to improve accuracy. Together with better training techniques, the resulting object detector, named RTMDet, achieves 52.8% AP on COCO with 300+ FPS on an NVIDIA 3090 GPU, outperforming the current mainstream industrial detectors. RTMDet achieves the best parameter-accuracy trade-off with tiny/small/medium/large/extra-large model sizes for various application scenarios, and obtains new state-of-the-art performance on real-time instance segmentation and rotated object detection. We hope the experimental results can provide new insights into designing versatile real-time object detectors for many object recognition tasks. Code and models are released at https://github.com/open-mmlab/mmdetection/tree/3.x/configs/rtmdet.
translated by 谷歌翻译
In this report, we present a fast and accurate object detection method dubbed DAMO-YOLO, which achieves higher performance than the state-of-the-art YOLO series. DAMO-YOLO is extended from YOLO with some new technologies, including Neural Architecture Search (NAS), efficient Reparameterized Generalized-FPN (RepGFPN), a lightweight head with AlignedOTA label assignment, and distillation enhancement. In particular, we use MAE-NAS, a method guided by the principle of maximum entropy, to search our detection backbone under the constraints of low latency and high performance, producing ResNet-like / CSP-like structures with spatial pyramid pooling and focus modules. In the design of necks and heads, we follow the rule of "large neck, small head". We import Generalized-FPN with accelerated queen-fusion to build the detector neck and upgrade its CSPNet with efficient layer aggregation networks (ELAN) and reparameterization. Then we investigate how detector head size affects detection performance and find that a heavy neck with only one task projection layer would yield better results. In addition, AlignedOTA is proposed to solve the misalignment problem in label assignment. And a distillation schema is introduced to improve performance to a higher level. Based on these new techs, we build a suite of models at various scales to meet the needs of different scenarios, i.e., DAMO-YOLO-Tiny/Small/Medium. They can achieve 43.0/46.8/50.0 mAPs on COCO with the latency of 2.78/3.83/5.62 ms on T4 GPUs respectively. The code is available at https://github.com/tinyvision/damo-yolo.
translated by 谷歌翻译
视觉变压器在识别和检测等实质性视野任务中显示了很大的视觉表示功率,从而在手动设计更有效的架构方面吸引了快速增长的努力。在本文中,我们建议使用神经架构搜索来自动化此过程,不仅可以搜索架构,还可以搜索搜索空间。中央观点是逐步发展使用权重共享超空网的E-T错误引导的不同搜索维度。此外,我们提供了一般视觉变压器的设计指南,根据空间搜索过程进行广泛的分析,这可以促进对视觉变压器的理解。值得注意的是,搜索空间的搜索模型,名为S3(用于搜索空间的短路),从搜索到的空间实现了卓越的性能,以最近提出的型号,例如在ImageNet上进行评估时的Swin,Deit和Vit。 S3的有效性也在对象检测,语义细分和视觉问题上说明,展示其泛度到下游视觉和视觉语言任务。代码和型号将在https://github.com/microsoft/cream中使用。
translated by 谷歌翻译
深度学习技术在各种任务中都表现出了出色的有效性,并且深度学习具有推进多种应用程序(包括在边缘计算中)的潜力,其中将深层模型部署在边缘设备上,以实现即时的数据处理和响应。一个关键的挑战是,虽然深层模型的应用通常会产生大量的内存和计算成本,但Edge设备通常只提供非常有限的存储和计算功能,这些功能可能会在各个设备之间差异很大。这些特征使得难以构建深度学习解决方案,以释放边缘设备的潜力,同时遵守其约束。应对这一挑战的一种有希望的方法是自动化有效的深度学习模型的设计,这些模型轻巧,仅需少量存储,并且仅产生低计算开销。该调查提供了针对边缘计算的深度学习模型设计自动化技术的全面覆盖。它提供了关键指标的概述和比较,这些指标通常用于量化模型在有效性,轻度和计算成本方面的水平。然后,该调查涵盖了深层设计自动化技术的三类最新技术:自动化神经体系结构搜索,自动化模型压缩以及联合自动化设计和压缩。最后,调查涵盖了未来研究的开放问题和方向。
translated by 谷歌翻译
Yolov7在5 fps到160 fps的速度和准确性上都超过了所有已知对象探测器,并且在GPU V100上具有30 fps或更高的所有已知实时对象探测器中,精度最高的56.8%AP。YOLOV7-E6对象检测器(56 fps v100,55.9%AP)优于两个基于变压器的检测器SWIN-L-CASCADE MAKS R-CNN(9.2 fps A100,53.9%AP)的速度和2%的准确性和2%基于卷积的检测器Convnext-XL级联膜面罩R-CNN(8.6 fps a100,55.2%AP)的速度为551%,精度为0.7%AP,Yolov7优于:Yolor,Yolox,Yolox,Scaled-Yolov4,Yolov4,Yolov5,Yolov5,Yolov5,Yolov5,Yolov5,Yolov5,Yolov5,Yolov5,Yolov5,DETR,可变形的DETR,DINO-5SCALE-R50,VIT-ADAPTER-B和许多其他对象探测器的速度和准确性。此外,我们仅在不使用任何其他数据集或预先训练的权重的情况下从头开始训练Yolov7。源代码在https://github.com/wongkinyiu/yolov7中发布。
translated by 谷歌翻译
ous vision tasks without convolutions, where it can be used as a direct replacement for CNN backbones. (3) We validate PVT through extensive experiments, showing that it boosts the performance of many downstream tasks, including object detection, instance and semantic segmentation. For example, with a comparable number of parameters, PVT+RetinaNet achieves 40.4 AP on the COCO dataset, surpassing ResNet50+RetinNet (36.3 AP) by 4.1 absolute AP (see Figure 2). We hope that PVT could serve as an alternative and useful backbone for pixel-level predictions and facilitate future research.
translated by 谷歌翻译
大多数现有的深神经网络都是静态的,这意味着它们只能以固定的复杂性推断。但资源预算可以大幅度不同。即使在一个设备上,实惠预算也可以用不同的场景改变,并且对每个所需预算的反复培训网络是非常昂贵的。因此,在这项工作中,我们提出了一种称为Mutualnet的一般方法,以训练可以以各种资源约束运行的单个网络。我们的方法列举了具有各种网络宽度和输入分辨率的模型配置队列。这种相互学习方案不仅允许模型以不同的宽度分辨率配置运行,而且还可以在这些配置之间传输独特的知识,帮助模型来学习更强大的表示。 Mutualnet是一般的培训方法,可以应用于各种网络结构(例如,2D网络:MobileNets,Reset,3D网络:速度,X3D)和各种任务(例如,图像分类,对象检测,分段和动作识别),并证明了实现各种数据集的一致性改进。由于我们只培训了这一模型,它对独立培训多种型号而言,它也大大降低了培训成本。令人惊讶的是,如果动态资源约束不是一个问题,则可以使用Mutualnet来显着提高单个网络的性能。总之,Mutualnet是静态和自适应,2D和3D网络的统一方法。代码和预先训练的模型可用于\ url {https://github.com/tayang1122/mutualnet}。
translated by 谷歌翻译
Evaluating neural network performance is critical to deep neural network design but a costly procedure. Neural predictors provide an efficient solution by treating architectures as samples and learning to estimate their performance on a given task. However, existing predictors are task-dependent, predominantly estimating neural network performance on image classification benchmarks. They are also search-space dependent; each predictor is designed to make predictions for a specific architecture search space with predefined topologies and set of operations. In this paper, we propose a novel All-in-One Predictor (AIO-P), which aims to pretrain neural predictors on architecture examples from multiple, separate computer vision (CV) task domains and multiple architecture spaces, and then transfer to unseen downstream CV tasks or neural architectures. We describe our proposed techniques for general graph representation, efficient predictor pretraining and knowledge infusion techniques, as well as methods to transfer to downstream tasks/spaces. Extensive experimental results show that AIO-P can achieve Mean Absolute Error (MAE) and Spearman's Rank Correlation (SRCC) below 1% and above 0.5, respectively, on a breadth of target downstream CV tasks with or without fine-tuning, outperforming a number of baselines. Moreover, AIO-P can directly transfer to new architectures not seen during training, accurately rank them and serve as an effective performance estimator when paired with an algorithm designed to preserve performance while reducing FLOPs.
translated by 谷歌翻译
Current state-of-the-art convolutional architectures for object detection are manually designed. Here we aim to learn a better architecture of feature pyramid network for object detection. We adopt Neural Architecture Search and discover a new feature pyramid architecture in a novel scalable search space covering all cross-scale connections. The discovered architecture, named NAS-FPN, consists of a combination of top-down and bottom-up connections to fuse features across scales. NAS-FPN, combined with various backbone models in the RetinaNet framework, achieves better accuracy and latency tradeoff compared to state-ofthe-art object detection models. NAS-FPN improves mobile detection accuracy by 2 AP compared to state-of-the-art SS-DLite with MobileNetV2 model in [32] and achieves 48.3 AP which surpasses Mask R-CNN [10] detection accuracy with less computation time.
translated by 谷歌翻译
我们探索普通的非层次视觉变压器(VIT)作为用于对象检测的骨干网络。该设计使原始的VIT体系结构可以进行微调以进行对象检测,而无需重新设计层次结构的主链以进行预训练。随着微调的最低适应性,我们的纯净背骨检测器可以取得竞争成果。令人惊讶的是,我们观察到:(i)足以从单尺度特征映射(没有常见的FPN设计)构建一个简单的特征金字塔,并且(ii)足以使用窗户注意力(无需转移),很少有帮助跨窗口传播块。凭借普通的VIT骨架作为掩盖自动编码器(MAE),我们的探测器(名为VITDET)可以与先前基于层次结构骨架的先前领先方法竞争,仅使用ImagEnet-1k Pre Pre pre to Coco Dataset上的61.3 ap_box竞争-训练。我们希望我们的研究能够引起人们对普通背骨检测器的研究。 VITDET的代码可在detectron2中获得。
translated by 谷歌翻译
我们从实际应用的角度重新审视了现有的出色变压器。他们中的大多数甚至不如基本的重新连接系列效率那么高,并且偏离了现实的部署方案。这可能是由于当前的标准测量计算效率,例如FLOPS或参数是单方面的,次优的和对硬件的不敏感的。因此,本文直接将特定硬件的紧张延迟视为效率指标,该指标提供了涉及计算能力,内存成本和带宽的更全面的反馈。基于一系列受控实验,这项工作为面向浓度和部署的网络设计提供了四个实用指南,例如,在阶段级别,早期的变压器和晚期CNN,在Block Level的早期CNN和Late Transformer。因此,提出了一个面向Tensortrt的变压器家族,缩写为TRT-VIT。广泛的实验表明,在不同的视觉任务(例如,图像分类,对象检测和语义细分)方面,TRT-VIT显着优于现有的Convnet和视觉变压器。例如,在82.7%的Imagenet-1k Top-1精度下,TRT-VIT比CSWIN快2.7 $ \ times $,比双胞胎快2.0 $ \ times $。在MS-COCO对象检测任务上,TRT-VIT与双胞胎达到可比的性能,而推理速度则增加了2.8 $ \ times $。
translated by 谷歌翻译