平均老师(MT)方案在半监督对象检测(SSOD)中被广泛采用。在MT中,通过手工制作的标签分配,采用了由教师的最终预测(例如,在无最大抑制(NMS)后处理之后)提供的稀疏伪标签(例如,在无最大抑制(NMS)后处理)。但是,稀疏到密集的范式使SSOD的管道复杂化,同时忽略了强大的直接,密集的教师监督。在本文中,我们试图直接利用教师的密集指导来监督学生培训,即密集至密集的范式。具体而言,我们建议逆NMS聚类(INC)和等级匹配(RM),以实例化密集的监督,而无需广泛使用的常规稀疏伪标签。 Inc带领学生像老师一样将候选箱子分组为NMS中的群集,这是通过学习在NMS过程中揭示的分组信息来实现的。在通过Inc获得了与教师相同的分组计划后,学生通过排名匹配进一步模仿了教师与聚类候选人的排名分配。借助拟议的Inc和RM,我们将密集的教师指导集成到半监督的对象检测(称为DTG-SSOD)中,成功地放弃了稀疏的伪标签,并在未标记的数据上提供了更有信息的学习。在可可基准上,我们的DTG-SSOD在各种标签率下实现了最先进的性能。例如,在10%的标签率下,DTG-SSOD将监督的基线从26.9提高到35.9地图,使以前的最佳方法软教师的表现优于1.9分。
translated by 谷歌翻译
联合超分辨率和反音调映射(联合SR-ITM)旨在增加低分辨率和标准动态范围图像的分辨率和动态范围。重点方法主要是诉诸图像分解技术,使用多支化的网络体系结构。 ,这些方法采用的刚性分解在很大程度上将其力量限制在各种图像上。为了利用其潜在能力,在本文中,我们将分解机制从图像域概括为更广泛的特征域。为此,我们提出了一个轻巧的特征分解聚合网络(FDAN)。特别是,我们设计了一个功能分解块(FDB),可以实现功能细节和对比度的可学习分离。通过级联FDB,我们可以建立一个用于强大的多级特征分解的分层功能分解组。联合SR-ITM,\ ie,SRITM-4K的新基准数据集,该数据集是大规模的,为足够的模型培训和评估提供了多功能方案。两个基准数据集的实验结果表明,我们的FDAN表明我们的FDAN有效,并且胜过了以前的方法sr-itm.ar代码和数据集将公开发布。
translated by 谷歌翻译
在本文中,我们在半监督对象检测(SSOD)中深入研究了两种关键技术,即伪标记和一致性训练。我们观察到,目前,这两种技术忽略了对象检测的一些重要特性,从而阻碍了对未标记数据的有效学习。具体而言,对于伪标记,现有作品仅关注分类得分,但不能保证伪框的本地化精度;为了保持一致性训练,广泛采用的随机训练只考虑了标签级的一致性,但错过了功能级别的训练,这在确保尺度不变性方面也起着重要作用。为了解决嘈杂的伪箱所产生的问题,我们设计了包括预测引导的标签分配(PLA)和正面验证一致性投票(PCV)的嘈杂伪盒学习(NPL)。 PLA依赖于模型预测来分配标签,并使甚至粗糙的伪框都具有鲁棒性。 PCV利用积极建议的回归一致性来反映伪盒的本地化质量。此外,在一致性训练中,我们提出了包括标签和特征水平一致性的机制的多视图尺度不变学习(MSL),其中通过将两个图像之间的移动特征金字塔对准具有相同内容但变化量表的变化来实现特征一致性。在可可基准测试上,我们的方法称为伪标签和一致性训练(PSECO),分别以2.0、1.8、2.0分的1%,5%和10%的标签比优于SOTA(软教师)。它还显着提高了SSOD的学习效率,例如,PSECO将SOTA方法的训练时间减半,但实现了更好的性能。代码可从https://github.com/ligang-cs/pseco获得。
translated by 谷歌翻译
在本文中,我们介绍了VCSL(视频复制段本地化),这是一种新的综合段级注释的视频复制数据集。与受视频级注释或小规模限制的现有复制检测数据集相比,VCSL不仅具有两个段级标签的数据级,其中有160k现实的视频副本对,其中包含超过280k的本地化copied seggment对,而且还包含超过280k涵盖各种视频类别和各种视频持续时间。每个收集的视频对中的所有复制段均经过手动提取,并伴随着精确注释的启动和结束时间戳。除了数据集外,我们还提出了一种新颖的评估协议,该协议可以更好地衡量视频对之间复制重叠段的预测准确性,并在不同情况下显示出改善的适应性。通过使用拟议的数据集和评估指标对几个基线和最先进的细分级视频副本检测方法进行基准测试,我们提供了一项全面的分析,可以揭示当前方法的优势和劣势作品。 VCSL数据集,公制和基准代码均在https://github.com/alipay/vcsl上公开获得。
translated by 谷歌翻译
知识蒸馏(KD)是一种广泛使用的技术,将繁琐的教师模型继承到紧凑的学生模型,从而实现模型压缩和加速度。与图像分类相比,对象检测是一个更复杂的任务,设计特定的KD方法用于对象检测是非微小的。在这项工作中,我们精心研究教师和学生检测模型之间的行为差​​异,并获得了两个有趣的观察:首先,教师和学生对其检测到的候选盒子相得益彰,这导致了它们的精确差异。其次,教师和学生之间的特征响应差异和预测差异之间存在相当大的差距,表明同样模仿老师的所有特征映射是提高学生准确性的次优选。基于这两个观察,我们提出了用于分别蒸馏单级探测器的测量模拟(RM)和预测引导的特征模仿(PFI)。 RM从教师那里夺取候选人盒的等级作为一种新的知识形式,蒸馏,这始终如一地优于传统的软标签蒸馏。 PFI试图将特征差异与预测差异相关,使特征模仿直接有助于提高学生的准确性。在MS Coco和Pascal VOC基准测试中,广泛的实验在不同骨干的各种探测器上进行,以验证我们方法的有效性。具体而言,具有Reset50的RetinAnet在MS Coco中实现了40.4%的图,比其基线高3.5%,并且还优于先前的KD方法。
translated by 谷歌翻译
目的:提出一种新的基于深度学习的方法,称为RG-NET(重建和生成网络),用于通过向下采样k空间高度加速的MR参数映射,并同时减少所获取的对比度。方法:所提出的框架包括重建模块和生成模块。在先前的帮助下,重建模块从所获取的少数下采样的k空间数据重建MR图像。然后,生成模块从重建的图像中综合剩余的多对比度图像,其中通过对完全采样标签的监督隐式模型被隐式地结合到图像生成中。在不同的加速率下对膝关节和大脑的映射数据进行评估RG-Net。 Cartilage和大脑的区域T1 \ R {HO}进行了分析,以获得RG-Net的性能。结果:RG-Net以高速加速度为17的高质量T1 \ R {Ho}地图。与仅借出k空间的竞争方法相比,我们的框架在T1 \ R {Ho}值中实现了更好的性能分析。我们的方法还提高了胶质瘤患者T1 \ R {Ho}的质量。结论:提出的RG-NET通过欠采样k空间采用新策略并同时减少快速先生参数映射的对比度,可以实现高加速率,同时保持良好的重建质量。我们的框架的生成模块也可以用作其他快速MR参数映射方法的插入模块。关键词:深度学习,卷积神经网络,快速先生参数映射
translated by 谷歌翻译
This paper focuses on designing efficient models with low parameters and FLOPs for dense predictions. Even though CNN-based lightweight methods have achieved stunning results after years of research, trading-off model accuracy and constrained resources still need further improvements. This work rethinks the essential unity of efficient Inverted Residual Block in MobileNetv2 and effective Transformer in ViT, inductively abstracting a general concept of Meta-Mobile Block, and we argue that the specific instantiation is very important to model performance though sharing the same framework. Motivated by this phenomenon, we deduce a simple yet efficient modern \textbf{I}nverted \textbf{R}esidual \textbf{M}obile \textbf{B}lock (iRMB) for mobile applications, which absorbs CNN-like efficiency to model short-distance dependency and Transformer-like dynamic modeling capability to learn long-distance interactions. Furthermore, we design a ResNet-like 4-phase \textbf{E}fficient \textbf{MO}del (EMO) based only on a series of iRMBs for dense applications. Massive experiments on ImageNet-1K, COCO2017, and ADE20K benchmarks demonstrate the superiority of our EMO over state-of-the-art methods, \eg, our EMO-1M/2M/5M achieve 71.5, 75.1, and 78.4 Top-1 that surpass \textbf{SoTA} CNN-/Transformer-based models, while trading-off the model accuracy and efficiency well.
translated by 谷歌翻译
Decompilation aims to transform a low-level program language (LPL) (eg., binary file) into its functionally-equivalent high-level program language (HPL) (e.g., C/C++). It is a core technology in software security, especially in vulnerability discovery and malware analysis. In recent years, with the successful application of neural machine translation (NMT) models in natural language processing (NLP), researchers have tried to build neural decompilers by borrowing the idea of NMT. They formulate the decompilation process as a translation problem between LPL and HPL, aiming to reduce the human cost required to develop decompilation tools and improve their generalizability. However, state-of-the-art learning-based decompilers do not cope well with compiler-optimized binaries. Since real-world binaries are mostly compiler-optimized, decompilers that do not consider optimized binaries have limited practical significance. In this paper, we propose a novel learning-based approach named NeurDP, that targets compiler-optimized binaries. NeurDP uses a graph neural network (GNN) model to convert LPL to an intermediate representation (IR), which bridges the gap between source code and optimized binary. We also design an Optimized Translation Unit (OTU) to split functions into smaller code fragments for better translation performance. Evaluation results on datasets containing various types of statements show that NeurDP can decompile optimized binaries with 45.21% higher accuracy than state-of-the-art neural decompilation frameworks.
translated by 谷歌翻译
Image Virtual try-on aims at replacing the cloth on a personal image with a garment image (in-shop clothes), which has attracted increasing attention from the multimedia and computer vision communities. Prior methods successfully preserve the character of clothing images, however, occlusion remains a pernicious effect for realistic virtual try-on. In this work, we first present a comprehensive analysis of the occlusions and categorize them into two aspects: i) Inherent-Occlusion: the ghost of the former cloth still exists in the try-on image; ii) Acquired-Occlusion: the target cloth warps to the unreasonable body part. Based on the in-depth analysis, we find that the occlusions can be simulated by a novel semantically-guided mixup module, which can generate semantic-specific occluded images that work together with the try-on images to facilitate training a de-occlusion try-on (DOC-VTON) framework. Specifically, DOC-VTON first conducts a sharpened semantic parsing on the try-on person. Aided by semantics guidance and pose prior, various complexities of texture are selectively blending with human parts in a copy-and-paste manner. Then, the Generative Module (GM) is utilized to take charge of synthesizing the final try-on image and learning to de-occlusion jointly. In comparison to the state-of-the-art methods, DOC-VTON achieves better perceptual quality by reducing occlusion effects.
translated by 谷歌翻译
In recent years, the Transformer architecture has shown its superiority in the video-based person re-identification task. Inspired by video representation learning, these methods mainly focus on designing modules to extract informative spatial and temporal features. However, they are still limited in extracting local attributes and global identity information, which are critical for the person re-identification task. In this paper, we propose a novel Multi-Stage Spatial-Temporal Aggregation Transformer (MSTAT) with two novel designed proxy embedding modules to address the above issue. Specifically, MSTAT consists of three stages to encode the attribute-associated, the identity-associated, and the attribute-identity-associated information from the video clips, respectively, achieving the holistic perception of the input person. We combine the outputs of all the stages for the final identification. In practice, to save the computational cost, the Spatial-Temporal Aggregation (STA) modules are first adopted in each stage to conduct the self-attention operations along the spatial and temporal dimensions separately. We further introduce the Attribute-Aware and Identity-Aware Proxy embedding modules (AAP and IAP) to extract the informative and discriminative feature representations at different stages. All of them are realized by employing newly designed self-attention operations with specific meanings. Moreover, temporal patch shuffling is also introduced to further improve the robustness of the model. Extensive experimental results demonstrate the effectiveness of the proposed modules in extracting the informative and discriminative information from the videos, and illustrate the MSTAT can achieve state-of-the-art accuracies on various standard benchmarks.
translated by 谷歌翻译