已经提出了许多算法用于将知识从富含标签的域(源)转移到标签稀缺域(目标)。几乎所有这些都是针对封闭场景提出的,其中源和目标域完全共享其样本的类。我们称共享classt为\ doublequote {known class。}但是,实际上,当targetdomain中的样本没有标记时,我们无法知道域是否共享该类。 Atarget域可以包含sourcedomain不共享的类的样本。我们将这些类称为\ doublequote {unknown class},并且在开放设置情况下运行良好的算法非常实用。然而,用于域适应的mostexisting分布匹配方法在此设置中不能很好地工作,因为未知目标样本不应与源对齐。在本文中,我们提出了一种利用对抗性训练的开放式集合域自适应方法。训练分类器以在源和目标样本之间形成边界,而训练生成器以使得远离边界的目标样本。因此,我们为特征生成器分配两个选项:将它们与源已知样本对齐或拒绝它们作为未知目标样本。该方法允许提取将未知目标样本与已知目标样本分开的特征。我们的方法在领域适应设置中得到了广泛的评估,并且在大多数情况下表现优于其他方法。
translated by 谷歌翻译
提出了无监督域自适应的任务,以将标签丰富域(源域)的知识转移到标签稀缺域(目标域)。不同域之间的匹配特征分布是上述任务的广泛应用方法。但是,当两个域中的类不相同时,该方法不能很好地执行。具体地,当目标的类对应于源的类的子集时,目标样本可能与仅存在于源中的类不正确地对齐。这个问题设置被称为部分域自适应(PDA)。在这项研究中,我们提出了一种新的方法,称为PDA的两个加权不一致性减少网络(TWIN)。我们利用两个分类网络来估计每个类别中的目标样本的比例,其中加权分类损失被加权以适应目标域中存在的类别。此外,为了提取目标的判别特征,我们建议最小化由目标样本上的分类器不一致性测量的域之间的差异。我们凭经验证明,降低两个网络之间的不一致性对于PDA是有效的,并且我们的方法在几个数据集中的表现优于其他现有方法。
translated by 谷歌翻译
无监督域适配(UDA)将知识从标签丰富的源域转移到完全未标记的目标域。为了解决这个问题,最近的方法通过伪标签来诉诸于歧视性域名转移,以强制跨源和目标域的类级别分布对齐。然而,这些方法容易受到误差累积的影响,并且不能保持跨域类别的一致性,因为不能明确地保证伪标记的准确性。在本文中,我们提出了渐进特征对齐网络(PFAN),通过开发目标域内的类内变化,逐步有效地协调跨域的不一致特征。具体而言,我们首先开发一种易于转移的策略(EHTS)和一个AdaptivePrototype Alignment(APA)步骤,以迭代和替代方式训练我们的模型。此外,在观察到良好的域适应通常需要非饱和的源分类器时,我们考虑通过进一步将温度变量纳入soft-max函数来延迟源分类损失的收敛速度的简单而有效的途径。大量的实验结果表明,所提出的PFAN超过了三个UDA数据集的最新性能。
translated by 谷歌翻译
Deep-layered models trained on a large number of labeled samples boost the accuracy of many tasks. It is important to apply such models to different domains because collecting many labeled samples in various domains is expensive. In un-supervised domain adaptation, one needs to train a classifier that works well on a target domain when provided with labeled source samples and unlabeled target samples. Although many methods aim to match the distributions of source and target samples, simply matching the distribution cannot ensure accuracy on the target domain. To learn discriminative representations for the target domain, we assume that artificially labeling target samples can result in a good representation. Tri-training leverages three classifiers equally to give pseudo-labels to unlabeled samples, but the method does not assume labeling samples generated from a different domain. In this paper, we propose an asymmetric tri-training method for unsupervised domain adaptation, where we assign pseudo-labels to unlabeled samples and train neural networks as if they are true labels. In our work, we use three networks asymmetrically. By asymmetric, we mean that two networks are used to label unlabeled target samples and one network is trained by the samples to obtain target-discriminative representations. We evaluate our method on digit recognition and sentiment analysis datasets. Our proposed method achieves state-of-the-art performance on the benchmark digit recognition datasets of domain adaptation.
translated by 谷歌翻译
In this work, we present a method for unsupervised domain adaptation. Many adversarial learning methods train domain classifier networks to distinguish the features as either a source or target and train a feature generator network to mimic the discriminator. Two problems exist with these methods. First, the domain classifier only tries to distinguish the features as a source or target and thus does not consider task-specific decision boundaries between classes. Therefore, a trained generator can generate ambiguous features near class boundaries. Second, these methods aim to completely match the feature distributions between different domains, which is difficult because of each domain's characteristics. To solve these problems, we introduce a new approach that attempts to align distributions of source and target by utilizing the task-specific decision boundaries. We propose to maximize the discrepancy between two classifiers' outputs to detect target samples that are far from the support of the source. A feature generator learns to generate target features near the support to minimize the discrepancy. Our method outperforms other methods on several datasets of image classification and semantic segmen-tation. The codes are available at https://github. com/mil-tokyo/MCD_DA
translated by 谷歌翻译
We consider unsupervised domain adaptation: given labelled examples from a source domain and unlabelled examples from a related target domain, the goal is to infer the labels of target examples. Under the assumption that features from pre-trained deep neural networks are transferable across related domains, domain adaptation reduces to aligning source and target domain at class prediction uncertainty level. We tackle this problem by introducing a method based on adversarial learning which forces the label uncertainty predictions on the target domain to be indistinguishable from those on the source domain. Pre-trained deep neural networks are used to generate deep features having high transferability across related domains. We perform an extensive experimental analysis of the proposed method over a wide set of publicly available pre-trained deep neural networks. Results of our experiments on domain adaptation tasks for image classification show that class prediction uncertainty alignment with features extracted from pre-trained deep neural networks provides an efficient, robust and effective method for domain adaptation.
translated by 谷歌翻译
We present a method for transferring neural representations from label-rich source domains to unlabeled target domains. Recent adversarial methods proposed for this task learn to align features across domains by fooling a special domain critic network. However, a drawback of this approach is that the critic simply labels the generated features as in-domain or not, without considering the boundaries between classes. This can lead to ambiguous features being generated near class boundaries, reducing target classification accuracy. We propose a novel approach, Adversarial Dropout Regularization (ADR), to encourage the generator to output more discriminative features for the target domain. Our key idea is to replace the critic with one that detects non-discriminative features, using dropout on the clas-sifier network. The generator then learns to avoid these areas of the feature space and thus creates better features. We apply our ADR approach to the problem of unsupervised domain adaptation for image classification and semantic segmenta-tion tasks, and demonstrate significant improvement over the state of the art. We also show that our approach can be used to train Generative Adversarial Networks for semi-supervised learning.
translated by 谷歌翻译
用于图像分类任务的深度学习(例如,卷积神经网络)的准确性严格依赖于标记的训练数据的量。旨在解决缺少标记数据但获得对廉价可用未标记数据的访问的新域上的图像分类任务,通过假设来自不同域的图像具有不变特征,无监督域适应是提高性能而不产生额外标记成本的有前景的技术。在本文中,我们提出了一种新的无监督域自适应方法,称为深度神经网络的域 - 对抗残差传递(DART)学习,以解决跨域图像分类问题。与现有的无监督域自适应方法相比,所提出的DART不仅通过对抗训练来学习域不变特征,而且还通过遗传 - 转移策略实现了强大的域自适应分类,所有这些都在端到端的训练框架中。我们在几个众所周知的基准数据集上评估了所提出的跨域图像分类任务方法的性能,其中我们的方法明显优于最先进的方法。
translated by 谷歌翻译
The objective of unsupervised domain adaptation is to leverage features from a labeled source domain and learn a classifier for an unlabeled target domain, with a similar but different data distribution. Most deep learning approaches to domain adaptation consist of two steps: (i) learn features that preserve a low risk on labeled samples (source domain) and (ii) make the features from both domains to be as indistinguishable as possible, so that a clas-sifier trained on the source can also be applied on the target domain. In general, the classifiers in step (i) consist of fully-connected layers applied directly on the indistinguishable features learned in (ii). In this paper, we propose a different way to do the classification, using similarity learning. The proposed method learns a pairwise similarity function in which classification can be performed by computing similarity between prototype representations of each category. The domain-invariant features and the categorical prototype representations are learned jointly and in an end-to-end fashion. At inference time, images from the target domain are compared to the prototypes and the label associated with the one that best matches the image is out-puted. The approach is simple, scalable and effective. We show that our model achieves state-of-the-art performance in different unsupervised domain adaptation scenarios.
translated by 谷歌翻译
Classifiers trained on given databases perform poorly when tested on dataacquired in different settings. This is explained in domain adaptation througha shift among distributions of the source and target domains. Attempts to alignthem have traditionally resulted in works reducing the domain shift byintroducing appropriate loss terms, measuring the discrepancies between sourceand target distributions, in the objective function. Here we take a differentroute, proposing to align the learned representations by embedding in any givennetwork specific Domain Alignment Layers, designed to match the source andtarget feature distributions to a reference one. Opposite to previous workswhich define a priori in which layers adaptation should be performed, ourmethod is able to automatically learn the degree of feature alignment requiredat different levels of the deep network. Thorough experiments on differentpublic benchmarks, in the unsupervised setting, confirm the power of ourapproach.
translated by 谷歌翻译
It is important to transfer the knowledge from label-rich source domain to unlabeled target domain due to the expensive cost of manual labeling efforts. Prior domain adaptation methods address this problem through aligning the global distribution statistics between source domain and target domain, but a drawback of prior methods is that they ignore the semantic information contained in samples, e.g., features of backpack-s in target domain might be mapped near features of cars in source domain. In this paper, we present moving semantic transfer network, which learn semantic representations for unlabeled target samples by aligning labeled source centroid and pseudo-labeled target centroid. Features in same class but different domains are expected to be mapped nearby, resulting in an improved target classification accuracy. Moving average cen-troid alignment is cautiously designed to compensate the insufficient categorical information within each mini batch. Experiments testify that our model yields state of the art results on standard datasets.
translated by 谷歌翻译
深度学习方法已经在无监督域适应中显示出希望,其旨在利用标记的源域来学习具有不同分布的未标记目标域的分类器。但是,这种方法通常会学习一个域不变的表示空间来匹配源域和目标域的边缘分布,同时忽略它们的精细层次结构。在本文中,我们提出了与教师(CAT)的群集对齐,用于无监督的域适应,它可以有效地将两个域中的差异聚类结构合并到更好的适应中。技术上,CAT利用隐含的集合教师模型来可靠地发现类条件结构。未标记的目标域的特征空间。然后,CAT强制源和目标域的特征形成有区别的类条件集群,并跨域对齐相应的集群。实证结果表明,在几个无监督的领域适应情景中,CAT可以获得最先进的结果。
translated by 谷歌翻译
In domain adaptation, maximum mean discrepancy (M-MD) has been widely adopted as a discrepancy metric between the distributions of source and target domains. However , existing MMD-based domain adaptation methods generally ignore the changes of class prior distributions, i.e., class weight bias across domains. This remains an open problem but ubiquitous for domain adaptation, which can be caused by changes in sample selection criteria and application scenarios. We show that MMD cannot account for class weight bias and results in degraded domain adaptation performance. To address this issue, a weighted MMD model is proposed in this paper. Specifically, we introduce class-specific auxiliary weights into the original MMD for exploiting the class prior probability on source and target domains, whose challenge lies in the fact that the class label in target domain is unavailable. To account for it, our proposed weighted MMD model is defined by introducing an auxiliary weight for each class in the source domain, and a classification EM algorithm is suggested by alternating between assigning the pseudo-labels, estimating auxiliary weights and updating model parameters. Extensive experiments demonstrate the superiority of our weighted MMD over conventional MMD for domain adaptation.
translated by 谷歌翻译
This paper proposes an importance weighted adversar-ial nets-based method for unsupervised domain adaptation, specific for partial domain adaptation where the target domain has less number of classes compared to the source domain. Previous domain adaptation methods generally assume the identical label spaces, such that reducing the distribution divergence leads to feasible knowledge transfer. However, such an assumption is no longer valid in a more realistic scenario that requires adaptation from a larger and more diverse source domain to a smaller target domain with less number of classes. This paper extends the adversar-ial nets-based domain adaptation and proposes a novel ad-versarial nets-based partial domain adaptation method to identify the source samples that are potentially from the out-lier classes and, at the same time, reduce the shift of shared classes between domains.
translated by 谷歌翻译
We introduce a new representation learning approach for domain adaptation, in which data at training and test time come from similar but different distributions. Our approach is directly inspired by the theory on domain adaptation suggesting that, for effective domain transfer to be achieved, predictions must be made based on features that cannot discriminate between the training (source) and test (target) domains. The approach implements this idea in the context of neural network architectures that are trained on labeled data from the source domain and unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of features that are (i) discriminative for the main learning task on the source domain and (ii) indiscriminate with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation and stochastic gradient descent, and can thus be implemented with little effort using any of the deep learning packages. We demonstrate the success of our approach for two distinct classification problems (document sentiment analysis and image classification), where state-of-the-art domain adaptation performance on standard benchmarks is achieved. We also validate the approach for descriptor learning task in the context of person re-identification application.
translated by 谷歌翻译
域对抗性学习在双人迷你游戏中对齐源和目标域中的特征分布。现有的域对等网络通常假设跨不同域的相同标签空间。在存在大数据的情况下,有很强的动机将深层模型从现有的大域转移到未知的小域。本文将部分域自适应作为一种新的域适应场,将宽松的完全共享标签空间假设放宽到源标签空间包含目标标签空间。先前的方法通常将整个源域匹配到目标域,这是由于标签空间之间的大的不匹配而导致的部分域适应问题的易受调节的转移。我们提出了部分对抗域适应(PADA),它通过对异常源类的数据进行向下称量以同时训练源分类器和域对象来同时减轻负转移,并通过匹配共享标签空间中的特征分布来促进正转移。实验表明,PADA超过了几个数据集上部分域适应任务的最新结果。
translated by 谷歌翻译
我们研究了无监督域适应的问题,该问题旨在使在标记源域上训练的模型适应完全未标记的靶域。领域对抗训练是一种很有前途的方法,并且已成为许多最先进的无监督领域适应方法的基础。领域对抗训练的思想是通过对抗训练域分类器来对齐源域和目标域之间的特征空间。特征编码器。最近,聚类假设已经应用于无监督域适应并且实现了强大的性能。在本文中,我们提出了一种称为虚拟混合训练(VMT)的新正则化方法,它能够进一步约束聚类假设的假设.VMT的思想是通过平滑输出分布来对模型施加局部Lipschitz约束。训练样本之间的界限。与传统的混合模型不同,我们的方法构建没有标签信息的组合样本,允许它适用于受监督的域自适应。所提出的方法是通用的,并且可以使用域对抗训练与现有方法组合。我们将VMT与最近最先进的VADA模型结合起来,广泛的实验表明VMT显着提高了VADA在severaldomain自适应基准数据集上的性能。对于将MNIST适应SVHN的挑战性任务,当不使用实例规范化时,VMT将VADA的准确性提高了30%以上。当使用实例归一化时,我们的模型实现了96.4%的准确度,这非常接近于目标模型的准确度(96.5%)。代码将公开发布。
translated by 谷歌翻译
Current Domain Adaptation (DA) methods based on deep architectures assume that the source samples arise from a single distribution. However, in practice most datasets can be regarded as mixtures of multiple domains. In these cases exploiting single-source DA methods for learning target classifiers may lead to sub-optimal, if not poor, results. In addition, in many applications it is difficult to manually provide the domain labels for all source data points, i.e. latent domains should be automatically discovered. This paper introduces a novel Convolutional Neural Network (CNN) architecture which (i) automatically discovers latent domains in visual datasets and (ii) exploits this information to learn robust target classifiers. Our approach is based on the introduction of two main components, which can be embedded into any existing CNN architecture: (i) a side branch that automatically computes the assignment of a source sample to a latent domain and (ii) novel layers that exploit domain membership information to appropriately align the distribution of the CNN internal feature representations to a reference distribution. We test our approach on publicly-available datasets, showing that it outperforms state-of-the-art multi-source DA methods by a large margin.
translated by 谷歌翻译
无监督域适应旨在减少将知识从监督源域转移到无监督目标域的域转移。已经成功探索了对抗特征对齐,以最小化域差异。然而,当两个域不共享相同的标签空间时,现有方法通常难以优化混合学习目标并且易受负转移的影响。在本文中,我们凭经验揭示了目标域的不规则区分主要反映在其相对于源域的特征范数值低得多的特征范数值中。我们提出了一种非参数自适应特征范数AFN方法,它独立于两个域的标签空间之间的关联。我们证明,调整源和目标域的特征规范以在大范围的值上实现均衡可能导致不显着的域转移增益。没有花里胡哨,只有几行代码,我们的方法在很大程度上解除了对目标域的歧视(在VisDA2017中仅来自Source的23.7%)并实现了香草环境的新技术。此外,由于我们的方法不需要对特征分布进行有意识的对齐,因此对于负转换而言,它可以在部分设置下超出现有方法的极大幅度(Office-Home为9.8 \%,VisDA2017为14.1 \%)。代码可在https://github.com/jihanyang/AFN获得。我们对我们方法的可用性负责。
translated by 谷歌翻译
无监督域适应(UDA)对目标域进行预测,而手动注释仅在源域中可用。以前的方法可以最大限度地减少域差异,忽略类信息,这可能导致错位和泛化性能差。为了解决这个问题,本文提出了对比适应网络(CAN)优化新的度量,它明确地模拟了类内域差异和类间域差异。我们设计了一种以端到端方式加速CAN的交替更新策略。在两个真实世界的基准测试中的实验表明,CAN-31和VisDA-2017表明CAN对最先进的方法表现出色,并产生更多的辨别特征。我们很快就会发布代码。
translated by 谷歌翻译