自我训练的人群计数尚未得到专心探索,尽管这是计算机视觉中的重要挑战之一。实际上,完全监督的方法通常需要大量的手动注释资源。为了应对这一挑战,这项工作引入了一种新的方法,以利用现有的数据集,以地面真理来在人群计数中对未标记的数据集(称为域名适应)产生更强大的预测。尽管网络接受了标记的数据训练,但培训过程中还添加了来自目标域的标签的样品。在此过程中,除了平行设计的对抗训练过程外,还计算和最小化熵图。在shanghaitech,UCF_CC_50和UCF-QNRF数据集上进行的实验证明,在跨域设置中,我们的方法对我们的方法进行了更广泛的改进。
translated by 谷歌翻译
Convolutional neural network-based approaches for semantic segmentation rely on supervision with pixel-level ground truth, but may not generalize well to unseen image domains. As the labeling process is tedious and labor intensive, developing algorithms that can adapt source ground truth labels to the target domain is of great interest. In this paper, we propose an adversarial learning method for domain adaptation in the context of semantic segmentation. Considering semantic segmentations as structured outputs that contain spatial similarities between the source and target domains, we adopt adversarial learning in the output space. To further enhance the adapted model, we construct a multi-level adversarial network to effectively perform output space domain adaptation at different feature levels. Extensive experiments and ablation study are conducted under various domain adaptation settings, including synthetic-to-real and cross-city scenarios. We show that the proposed method performs favorably against the stateof-the-art methods in terms of accuracy and visual quality.
translated by 谷歌翻译
Domain adaptation aims to bridge the domain shifts between the source and the target domain. These shifts may span different dimensions such as fog, rainfall, etc. However, recent methods typically do not consider explicit prior knowledge about the domain shifts on a specific dimension, thus leading to less desired adaptation performance. In this paper, we study a practical setting called Specific Domain Adaptation (SDA) that aligns the source and target domains in a demanded-specific dimension. Within this setting, we observe the intra-domain gap induced by different domainness (i.e., numerical magnitudes of domain shifts in this dimension) is crucial when adapting to a specific domain. To address the problem, we propose a novel Self-Adversarial Disentangling (SAD) framework. In particular, given a specific dimension, we first enrich the source domain by introducing a domainness creator with providing additional supervisory signals. Guided by the created domainness, we design a self-adversarial regularizer and two loss functions to jointly disentangle the latent representations into domainness-specific and domainness-invariant features, thus mitigating the intra-domain gap. Our method can be easily taken as a plug-and-play framework and does not introduce any extra costs in the inference time. We achieve consistent improvements over state-of-the-art methods in both object detection and semantic segmentation.
translated by 谷歌翻译
Semantic segmentation is a key problem for many computer vision tasks. While approaches based on convolutional neural networks constantly break new records on different benchmarks, generalizing well to diverse testing environments remains a major challenge. In numerous real world applications, there is indeed a large gap between data distributions in train and test domains, which results in severe performance loss at run-time. In this work, we address the task of unsupervised domain adaptation in semantic segmentation with losses based on the entropy of the pixel-wise predictions. To this end, we propose two novel, complementary methods using (i) an entropy loss and (ii) an adversarial loss respectively. We demonstrate state-of-theart performance in semantic segmentation on two challenging "synthetic-2-real" set-ups 1 and show that the approach can also be used for detection.
translated by 谷歌翻译
We consider the problem of unsupervised domain adaptation in semantic segmentation. A key in this campaign consists in reducing the domain shift, i.e., enforcing the data distributions of the two domains to be similar. One of the common strategies is to align the marginal distribution in the feature space through adversarial learning. However, this global alignment strategy does not consider the category-level joint distribution. A possible consequence of such global movement is that some categories which are originally well aligned between the source and target may be incorrectly mapped, thus leading to worse segmentation results in target domain. To address this problem, we introduce a category-level adversarial network, aiming to enforce local semantic consistency during the trend of global alignment. Our idea is to take a close look at the category-level joint distribution and align each class with an adaptive adversarial loss. Specifically, we reduce the weight of the adversarial loss for category-level aligned features while increasing the adversarial force for those poorly aligned. In this process, we decide how well a feature is category-level aligned between source and target by a co-training approach. In two domain adaptation tasks, i.e., GTA5 → Cityscapes and SYN-THIA → Cityscapes, we validate that the proposed method matches the state of the art in segmentation accuracy.
translated by 谷歌翻译
Although unsupervised domain adaptation methods have achieved remarkable performance in semantic scene segmentation in visual perception for self-driving cars, these approaches remain impractical in real-world use cases. In practice, the segmentation models may encounter new data that have not been seen yet. Also, the previous data training of segmentation models may be inaccessible due to privacy problems. Therefore, to address these problems, in this work, we propose a Continual Unsupervised Domain Adaptation (CONDA) approach that allows the model to continuously learn and adapt with respect to the presence of the new data. Moreover, our proposed approach is designed without the requirement of accessing previous training data. To avoid the catastrophic forgetting problem and maintain the performance of the segmentation models, we present a novel Bijective Maximum Likelihood loss to impose the constraint of predicted segmentation distribution shifts. The experimental results on the benchmark of continual unsupervised domain adaptation have shown the advanced performance of the proposed CONDA method.
translated by 谷歌翻译
Recent works on domain adaptation reveal the effectiveness of adversarial learning on filling the discrepancy between source and target domains. However, two common limitations exist in current adversarial-learning-based methods. First, samples from two domains alone are not sufficient to ensure domain-invariance at most part of latent space. Second, the domain discriminator involved in these methods can only judge real or fake with the guidance of hard label, while it is more reasonable to use soft scores to evaluate the generated images or features, i.e., to fully utilize the inter-domain information. In this paper, we present adversarial domain adaptation with domain mixup (DM-ADA), which guarantees domain-invariance in a more continuous latent space and guides the domain discriminator in judging samples' difference relative to source and target domains. Domain mixup is jointly conducted on pixel and feature level to improve the robustness of models. Extensive experiments prove that the proposed approach can achieve superior performance on tasks with various degrees of domain shift and data complexity.
translated by 谷歌翻译
Domain adaptive object detection (DAOD) aims to alleviate transfer performance degradation caused by the cross-domain discrepancy. However, most existing DAOD methods are dominated by computationally intensive two-stage detectors, which are not the first choice for industrial applications. In this paper, we propose a novel semi-supervised domain adaptive YOLO (SSDA-YOLO) based method to improve cross-domain detection performance by integrating the compact one-stage detector YOLOv5 with domain adaptation. Specifically, we adapt the knowledge distillation framework with the Mean Teacher model to assist the student model in obtaining instance-level features of the unlabeled target domain. We also utilize the scene style transfer to cross-generate pseudo images in different domains for remedying image-level differences. In addition, an intuitive consistency loss is proposed to further align cross-domain predictions. We evaluate our proposed SSDA-YOLO on public benchmarks including PascalVOC, Clipart1k, Cityscapes, and Foggy Cityscapes. Moreover, to verify its generalization, we conduct experiments on yawning detection datasets collected from various classrooms. The results show considerable improvements of our method in these DAOD tasks. Our code is available on \url{https://github.com/hnuzhy/SSDA-YOLO}.
translated by 谷歌翻译
无监督的域适应性(UDA)引起了相当大的关注,这将知识从富含标签的源域转移到相关但未标记的目标域。减少域间差异一直是提高UDA性能的关键因素,尤其是对于源域和目标域之间存在较大差距的任务。为此,我们提出了一种新颖的风格感知功能融合方法(SAFF),以弥合大域间隙和转移知识,同时减轻阶级歧视性信息的丧失。受到人类传递推理和学习能力的启发,研究了一种新颖的风格感知的自我互化领域(SSID),通过一系列中级辅助综合概念将两个看似无关的概念联系起来。具体而言,我们提出了一种新颖的SSID学习策略,该策略从源和目标域中选择样本作为锚点,然后随机融合这些锚的对象和样式特征,以生成具有标记和样式丰富的中级辅助功能以进行知识转移。此外,我们设计了一个外部存储库来存储和更新指定的标记功能,以获得稳定的类功能和班级样式功能。基于提议的内存库,内部和域间损耗功能旨在提高类识别能力和特征兼容性。同时,我们通过无限抽样模拟SSID的丰富潜在特征空间,并通过数学理论模拟损失函数的收敛性。最后,我们对常用的域自适应基准测试进行了全面的实验,以评估所提出的SAFF,并且实验结果表明,所提出的SAFF可以轻松地与不同的骨干网络结合在一起,并获得更好的性能作为插入插型模块。
translated by 谷歌翻译
为了将训练有素的模型直接概括为看不见的目标域,域概括(DG)是一种新提出的学习范式,引起了很大的关注。以前的DG模型通常需要在训练过程中观察到的源域中的足够数量的带注释的样品。在本文中,我们放宽了有关完全注释的要求,并研究了半监督域的概括(SSDG),在训练过程中,只有一个源域与其他完全未标记的域一起完全注释。由于要解决观察到的源域之间的域间隙和预测看不见的目标域之间的挑战,我们提出了一个通过关节域吸引的标签和双分类器的新型深框架,以产生高质量的伪标记。具体来说,为了预测域移位下的准确伪标记,开发了一个域吸引的伪标记模块。此外,考虑到概括和伪标记之间的目标不一致:前者防止在所有源域上过度拟合,而后者可能过分适合未标记的源域,以高精度,我们采用双分类器来独立执行伪标记和域名,并在训练过程中执行伪造域通用化。 。当为未标记的源域生成准确的伪标记时,将域混合操作应用于标记和未标记域之间的新域,这对于提高模型的通用能力是有益的。公开可用的DG基准数据集的广泛结果显示了我们提出的SSDG方法的功效。
translated by 谷歌翻译
语义细分是一种关键技术,涉及高分辨率遥感(HRS)图像的自动解释,并引起了遥感社区的广泛关注。由于其层次表示能力,深度卷积神经网络(DCNN)已成功应用于HRS图像语义分割任务。但是,对大量培训数据的严重依赖性以及对数据分布变化的敏感性严重限制了DCNNS在HRS图像的语义分割中的潜在应用。这项研究提出了一种新型的无监督域适应性语义分割网络(MemoryAdaptnet),用于HRS图像的语义分割。 MemoryAdaptnet构建了一种输出空间对抗学习方案,以弥合源域和目标域之间的域分布差异,并缩小域移位的影响。具体而言,我们嵌入了一个不变的特征内存模块来存储不变的域级上下文信息,因为从对抗学习获得的功能仅代表当前有限输入的变体特征。该模块由类别注意力驱动的不变域级上下文集合模块集成到当前伪不变功能,以进一步增强像素表示。基于熵的伪标签滤波策略用于更新当前目标图像的高额伪不变功能的内存模块。在三个跨域任务下进行的广泛实验表明,我们提出的记忆ADAPTNET非常优于最新方法。
translated by 谷歌翻译
用于图像分类的最可公开的数据集是单个标签,而图像在我们的日常生活中是固有的多标记。这种注释差距使得许多预先接受的单标准分类模型在实际情况下失败。该注释问题更加关注空中图像:从传感器收集的空中数据自然地覆盖具有多个标签的相对大的陆地面积,而被广泛可用的注释空中数据集(例如,UCM,AID)是单标记的。作为手动注释的多标签空中图像将是时间/劳动,我们提出了一种新的自我校正综合域适应(SCIDA)方法,用于自动多标签学习。 SCIDA是弱监督,即,自动学习多标签图像分类模型,从使用大量的公共可用的单一标签图像。为实现这一目标,我们提出了一种新颖的标签 - 明智的自我校正(LWC)模块,以更好地探索潜在的标签相关性。该模块还使无监督的域适配(UDA)从单个到多标签数据中可能。对于模型培训,所提出的型号仅使用单一标签信息,但不需要先验知识的多标记数据;它预测了多标签空中图像的标签。在我们的实验中,用单标签的MAI-AID-S和MAI-UCM-S数据集接受培训,所提出的模型直接在收集的多场景空中图像(MAI)数据集上进行测试。
translated by 谷歌翻译
对象检测的域适应性(DAOD)最近由于其检测目标对象而没有任何注释而引起了很多关注。为了解决该问题,以前的作品着重于通过对抗训练在两阶段检测器中从部分级别(例如图像级,实例级,RPN级)提取的对齐功能。但是,对象检测管道中的个体级别相互密切相关,并且尚未考虑此层次之间的关系。为此,我们为DAOD介绍了一个新的框架,该框架具有三个提出的组件:多尺度意识不确定性注意力(MUA),可转移的区域建议网络(TRPN)和动态实例采样(DIS)。使用这些模块,我们试图在训练过程中减少负转移效应,同时最大化可传递性以及两个领域的可区分性。最后,我们的框架隐含地学习了域不变区域,以通过利用可转移信息并通过协作利用其域信息来增强不同检测级别之间的互补性。通过消融研究和实验,我们表明所提出的模块以协同方式有助于性能提高,以证明我们方法的有效性。此外,我们的模型在各种基准测试方面达到了新的最新性能。
translated by 谷歌翻译
最近,无监督的人重新识别(RE-ID)引起了人们的关注,因为其开放世界情景设置有限,可用的带注释的数据有限。现有的监督方法通常无法很好地概括在看不见的域上,而无监督的方法(大多数缺乏多范围的信息),并且容易患有确认偏见。在本文中,我们旨在从两个方面从看不见的目标域上找到更好的特征表示形式,1)在标记的源域上进行无监督的域适应性和2)2)在未标记的目标域上挖掘潜在的相似性。此外,提出了一种协作伪标记策略,以减轻确认偏见的影响。首先,使用生成对抗网络将图像从源域转移到目标域。此外,引入了人身份和身份映射损失,以提高生成图像的质量。其次,我们提出了一个新颖的协作多元特征聚类框架(CMFC),以学习目标域的内部数据结构,包括全局特征和部分特征分支。全球特征分支(GB)在人体图像的全球特征上采用了无监督的聚类,而部分特征分支(PB)矿山在不同人体区域内的相似性。最后,在两个基准数据集上进行的广泛实验表明,在无监督的人重新设置下,我们的方法的竞争性能。
translated by 谷歌翻译
我们解决对象检测中的域适应问题,其中在源(带有监控)和目标域(没有监督的域的域名)之间存在显着的域移位。作为广泛采用的域适应方法,自培训教师学生框架(学生模型从教师模型生成的伪标签学习)在目标域中产生了显着的精度增益。然而,由于其偏向源域,它仍然存在从教师产生的大量低质量伪标签(例如,误报)。为了解决这个问题,我们提出了一种叫做自适应无偏见教师(AUT)的自我训练框架,利用对抗的对抗学习和弱强的数据增强来解决域名。具体而言,我们在学生模型中使用特征级的对抗性培训,确保从源和目标域中提取的功能共享类似的统计数据。这使学生模型能够捕获域不变的功能。此外,我们在目标领域的教师模型和两个域上的学生模型之间应用了弱强的增强和相互学习。这使得教师模型能够从学生模型中逐渐受益,而不会遭受域移位。我们展示了AUT通过大边距显示所有现有方法甚至Oracle(完全监督)模型的优势。例如,我们在有雾的城市景观(Clipart1k)上实现了50.9%(49.3%)地图,分别比以前的最先进和甲骨文高9.2%(5.2%)和8.2%(11.0%)
translated by 谷歌翻译
基于对抗性学习的现有无监督的域适应方法在多个医学成像任务中取得了良好的表现。但是,这些方法仅着眼于全局分布适应,而忽略了类别级别的分布约束,这将导致次级适应性的性能。本文基于类别级别的正则化提出了一个无监督的域适应框架,该框架从三个角度正规化了类别分布。具体而言,对于域间类别的正则化,提出了一个自适应原型比对模块,以使源和目标域中同一类别的特征原型对齐。此外,对于域内类别的正则化,我们分别针对源和目标域定制了正则化技术。在源域中,提出了原型引导的判别性损失,以通过执行阶层内紧凑性和类间的分离性来学习更多的判别特征表示,并作为对传统监督损失的补充。在目标域中,提出了增强的一致性类别的正则化损失,以迫使该模型为增强/未增强目标图像提供一致的预测,这鼓励在语义上相似的区域给予相同的标签。在两个公共底面数据集上进行的广泛实验表明,所提出的方法显着优于其他最先进的比较算法。
translated by 谷歌翻译
本文提出了一种新颖的像素级分布正则化方案(DRSL),用于自我监督的语义分割域的适应性。在典型的环境中,分类损失迫使语义分割模型贪婪地学习捕获类间变化的表示形式,以确定决策(类)边界。由于域的转移,该决策边界在目标域中未对齐,从而导致嘈杂的伪标签对自我监督域的适应性产生不利影响。为了克服这一限制,以及捕获阶层间变化,我们通过类感知的多模式分布学习(MMDL)捕获了像素级内的类内变化。因此,捕获阶层内变化所需的信息与阶层间歧视所需的信息明确分开。因此,捕获的功能更具信息性,导致伪噪声低的伪标记。这种分离使我们能够使用前者的基于跨凝结的自学习,在判别空间和多模式分布空间中进行单独的对齐。稍后,我们通过明确降低映射到同一模式的目标和源像素之间的距离来提出一种新型的随机模式比对方法。距离度量标签上计算出的距离度量损失,并从多模式建模头部反向传播,充当与分割头共享的基本网络上的正常化程序。关于合成到真实域的适应设置的全面实验的结果,即GTA-V/Synthia to CityScapes,表明DRSL的表现优于许多现有方法(MIOU的最小余量为2.3%和2.5%,用于MIOU,而合成的MIOU到CityScapes)。
translated by 谷歌翻译
This paper solves a generalized version of the problem of multi-source model adaptation for semantic segmentation. Model adaptation is proposed as a new domain adaptation problem which requires access to a pre-trained model instead of data for the source domain. A general multi-source setting of model adaptation assumes strictly that each source domain shares a common label space with the target domain. As a relaxation, we allow the label space of each source domain to be a subset of that of the target domain and require the union of the source-domain label spaces to be equal to the target-domain label space. For the new setting named union-set multi-source model adaptation, we propose a method with a novel learning strategy named model-invariant feature learning, which takes full advantage of the diverse characteristics of the source-domain models, thereby improving the generalization in the target domain. We conduct extensive experiments in various adaptation settings to show the superiority of our method. The code is available at https://github.com/lzy7976/union-set-model-adaptation.
translated by 谷歌翻译
人搜索是一项具有挑战性的任务,旨在实现共同的行人检测和人重新识别(REID)。以前的作品在完全和弱监督的设置下取得了重大进步。但是,现有方法忽略了人搜索模型的概括能力。在本文中,我们采取了进一步的步骤和现在的域自适应人员搜索(DAPS),该搜索旨在将模型从标记的源域概括为未标记的目标域。在这种新环境下出现了两个主要挑战:一个是如何同时解决检测和重新ID任务的域未对准问题,另一个是如何在目标域上训练REID子任务而不可靠的检测结果。为了应对这些挑战,我们提出了一个强大的基线框架,并使用两个专用设计。 1)我们设计一个域对齐模块,包括图像级和任务敏感的实例级别对齐,以最大程度地减少域差异。 2)我们通过动态聚类策略充分利用未标记的数据,并使用伪边界框来支持目标域上的REID和检测训练。通过上述设计,我们的框架在MAP中获得了34.7%的地图,而PRW数据集的TOP-1则达到80.6%,超过了直接转移基线的大幅度。令人惊讶的是,我们无监督的DAPS模型的性能甚至超过了一些完全和弱监督的方法。该代码可在https://github.com/caposerenity/daps上找到。
translated by 谷歌翻译
无监督域适应(UDA)已成功解决了可视应用程序的域移位问题。然而,由于以下原因,这些方法可能对时间序列数据的性能有限。首先,它们主要依赖于用于源预制的大规模数据集(即,ImageNet),这不适用于时间序列数据。其次,它们在域对齐步骤期间忽略源极限和目标域的特征空间上的时间维度。最后,最先前的UDA方法中的大多数只能对齐全局特征而不考虑目标域的细粒度分布。为了解决这些限制,我们提出了一个自我监督的自回归域适应(Slarda)框架。特别是,我们首先设计一个自我监督的学习模块,它利用预测作为辅助任务以提高源特征的可转换性。其次,我们提出了一种新的自回归域自适应技术,其包括在域对齐期间源和目标特征的时间依赖性。最后,我们开发了一个集合教师模型,通过自信的伪标记方法对准目标域中的类明智分发。已经在三个现实世界时间序列应用中进行了广泛的实验,具有30个跨域方案。结果表明,我们所提出的杆状方法明显优于时序序列域适应的最先进的方法。
translated by 谷歌翻译