In this paper, we investigate the problem of multi-domain translation: given an element $a$ of domain $A$, we would like to generate a corresponding $b$ sample in another domain $B$, and vice versa. Acquiring supervision in multiple domains can be a tedious task, also we propose to learn this translation from one domain to another when supervision is available as a pair $(a,b)\sim A\times B$ and leveraging possible unpaired data when only $a\sim A$ or only $b\sim B$ is available. We introduce a new unified framework called Latent Space Mapping (\model) that exploits the manifold assumption in order to learn, from each domain, a latent space. Unlike existing approaches, we propose to further regularize each latent space using available domains by learning each dependency between pairs of domains. We evaluate our approach in three tasks performing i) synthetic dataset with image translation, ii) real-world task of semantic segmentation for medical images, and iii) real-world task of facial landmark detection.

语义分割在广泛的计算机视觉应用中起着基本作用，提供了全球对图像​​的理解的关键信息。然而，最先进的模型依赖于大量的注释样本，其比在诸如图像分类的任务中获得更昂贵的昂贵的样本。由于未标记的数据替代地获得更便宜，因此无监督的域适应达到了语义分割社区的广泛成功并不令人惊讶。本调查致力于总结这一令人难以置信的快速增长的领域的五年，这包含了语义细分本身的重要性，以及将分段模型适应新环境的关键需求。我们提出了最重要的语义分割方法;我们对语义分割的域适应技术提供了全面的调查;我们揭示了多域学习，域泛化，测试时间适应或无源域适应等较新的趋势;我们通过描述在语义细分研究中最广泛使用的数据集和基准测试来结束本调查。我们希望本调查将在学术界和工业中提供具有全面参考指导的研究人员，并有助于他们培养现场的新研究方向。

Domain adaptation is critical for success in new, unseen environments. Adversarial adaptation models applied in feature spaces discover domain invariant representations, but are difficult to visualize and sometimes fail to capture pixel-level and low-level domain shifts. Recent work has shown that generative adversarial networks combined with cycle-consistency constraints are surprisingly effective at mapping images between domains, even without the use of aligned image pairs. We propose a novel discriminatively-trained Cycle-Consistent Adversarial Domain Adaptation model. CyCADA adapts representations at both the pixel-level and feature-level, enforces cycle-consistency while leveraging a task loss, and does not require aligned pairs. Our model can be applied in a variety of visual recognition and prediction settings. We show new state-of-the-art results across multiple adaptation tasks, including digit classification and semantic segmentation of road scenes demonstrating transfer from synthetic to real world domains.

Deep learning has produced state-of-the-art results for a variety of tasks. While such approaches for supervised learning have performed well, they assume that training and testing data are drawn from the same distribution, which may not always be the case. As a complement to this challenge, single-source unsupervised domain adaptation can handle situations where a network is trained on labeled data from a source domain and unlabeled data from a related but different target domain with the goal of performing well at test-time on the target domain. Many single-source and typically homogeneous unsupervised deep domain adaptation approaches have thus been developed, combining the powerful, hierarchical representations from deep learning with domain adaptation to reduce reliance on potentially-costly target data labels. This survey will compare these approaches by examining alternative methods, the unique and common elements, results, and theoretical insights. We follow this with a look at application areas and open research directions.

We propose a general framework for unsupervised domain adaptation, which allows deep neural networks trained on a source domain to be tested on a different target domain without requiring any training annotations in the target domain. This is achieved by adding extra networks and losses that help regularize the features extracted by the backbone encoder network. To this end we propose the novel use of the recently proposed unpaired image-toimage translation framework to constrain the features extracted by the encoder network. Specifically, we require that the features extracted are able to reconstruct the images in both domains. In addition we require that the distribution of features extracted from images in the two domains are indistinguishable. Many recent works can be seen as specific cases of our general framework. We apply our method for domain adaptation between MNIST, USPS, and SVHN datasets, and Amazon, Webcam and DSLR Office datasets in classification tasks, and also between GTA5 and Cityscapes datasets for a segmentation task. We demonstrate state of the art performance on each of these datasets.

这项工作提出了一个新颖的框架CISFA（对比图像合成和自我监督的特征适应），该框架建立在图像域翻译和无监督的特征适应性上，以进行跨模式生物医学图像分割。与现有作品不同，我们使用单方面的生成模型，并在输入图像的采样贴片和相应的合成图像之间添加加权贴片对比度损失，该图像用作形状约束。此外，我们注意到生成的图像和输入图像共享相似的结构信息，但具有不同的方式。因此，我们在生成的图像和输入图像上强制实施对比损失，以训练分割模型的编码器，以最大程度地减少学到的嵌入空间中成对图像之间的差异。与依靠对抗性学习进行特征适应的现有作品相比，这种方法使编码器能够以更明确的方式学习独立于域的功能。我们对包含腹腔和全心的CT和MRI图像的分割任务进行了广泛评估。实验结果表明，所提出的框架不仅输出了较小的器官形状变形的合成图像，而且还超过了最先进的域适应方法的较大边缘。

甚至在没有受限，监督的情况下，也提出了甚至在没有受限或有限的情况下学习普遍陈述的方法。使用适度数量的数据可以微调新的目标任务，或者直接在相应任务中实现显着性能的无奈域中使用的良好普遍表示。这种缓解数据和注释要求为计算机愿景和医疗保健的应用提供了诱人的前景。在本辅导纸上，我们激励了对解散的陈述，目前关键理论和详细的实际构建块和学习此类表示的标准的需求。我们讨论医学成像和计算机视觉中的应用，强调了在示例钥匙作品中进行的选择。我们通过呈现剩下的挑战和机会来结束。

Deep domain adaptation has emerged as a new learning technique to address the lack of massive amounts of labeled data. Compared to conventional methods, which learn shared feature subspaces or reuse important source instances with shallow representations, deep domain adaptation methods leverage deep networks to learn more transferable representations by embedding domain adaptation in the pipeline of deep learning. There have been comprehensive surveys for shallow domain adaptation, but few timely reviews the emerging deep learning based methods. In this paper, we provide a comprehensive survey of deep domain adaptation methods for computer vision applications with four major contributions. First, we present a taxonomy of different deep domain adaptation scenarios according to the properties of data that define how two domains are diverged. Second, we summarize deep domain adaptation approaches into several categories based on training loss, and analyze and compare briefly the state-of-the-art methods under these categories. Third, we overview the computer vision applications that go beyond image classification, such as face recognition, semantic segmentation and object detection. Fourth, some potential deficiencies of current methods and several future directions are highlighted.

对大脑的电子显微镜（EM）体积的精确分割对于表征细胞或细胞器水平的神经元结构至关重要。尽管有监督的深度学习方法在过去几年中导致了该方向的重大突破，但它们通常需要大量的带注释的数据才能接受培训，并且在类似的实验和成像条件下获得的其他数据上的表现不佳。这是一个称为域适应的问题，因为从样本分布（或源域）中学到的模型难以维持其对从不同分布或目标域提取的样品的性能。在这项工作中，我们解决了基于深度学习的域适应性的复杂案例，以跨不同组织和物种的EM数据集进行线粒体分割。我们提出了三种无监督的域适应策略，以根据（1）两个域之间的最新样式转移来改善目标域中的线粒体分割； （2）使用未标记的源和目标图像预先培训模型的自我监督学习，然后仅用源标签进行微调； （3）具有标记和未标记图像的端到端训练的多任务神经网络体系结构。此外，我们提出了基于在源域中仅获得的形态学先验的新训练停止标准。我们使用三个公开可用的EM数据集进行了所有可能的跨数据库实验。我们评估了目标数据集预测的线粒体语义标签的拟议策略。此处介绍的方法优于基线方法，并与最新的状态相比。在没有验证标签的情况下，监视我们提出的基于形态的度量是停止训练过程并在平均最佳模型中选择的直观有效的方法。

While deep learning methods hitherto have achieved considerable success in medical image segmentation, they are still hampered by two limitations: (i) reliance on large-scale well-labeled datasets, which are difficult to curate due to the expert-driven and time-consuming nature of pixel-level annotations in clinical practices, and (ii) failure to generalize from one domain to another, especially when the target domain is a different modality with severe domain shifts. Recent unsupervised domain adaptation~(UDA) techniques leverage abundant labeled source data together with unlabeled target data to reduce the domain gap, but these methods degrade significantly with limited source annotations. In this study, we address this underexplored UDA problem, investigating a challenging but valuable realistic scenario, where the source domain not only exhibits domain shift~w.r.t. the target domain but also suffers from label scarcity. In this regard, we propose a novel and generic framework called ``Label-Efficient Unsupervised Domain Adaptation"~(LE-UDA). In LE-UDA, we construct self-ensembling consistency for knowledge transfer between both domains, as well as a self-ensembling adversarial learning module to achieve better feature alignment for UDA. To assess the effectiveness of our method, we conduct extensive experiments on two different tasks for cross-modality segmentation between MRI and CT images. Experimental results demonstrate that the proposed LE-UDA can efficiently leverage limited source labels to improve cross-domain segmentation performance, outperforming state-of-the-art UDA approaches in the literature. Code is available at: https://github.com/jacobzhaoziyuan/LE-UDA.

Unsupervised image-to-image translation is an important and challenging problem in computer vision. Given an image in the source domain, the goal is to learn the conditional distribution of corresponding images in the target domain, without seeing any examples of corresponding image pairs. While this conditional distribution is inherently multimodal, existing approaches make an overly simplified assumption, modeling it as a deterministic one-to-one mapping. As a result, they fail to generate diverse outputs from a given source domain image. To address this limitation, we propose a Multimodal Unsupervised Image-to-image Translation (MUNIT) framework. We assume that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties. To translate an image to another domain, we recombine its content code with a random style code sampled from the style space of the target domain. We analyze the proposed framework and establish several theoretical results. Extensive experiments with comparisons to state-of-the-art approaches further demonstrate the advantage of the proposed framework. Moreover, our framework allows users to control the style of translation outputs by providing an example style image. Code and pretrained models are available at https://github.com/nvlabs/MUNIT.

无监督的域适应（UDA）旨在使源域上培训的模型适应到新的目标域，其中没有可用标记的数据。在这项工作中，我们调查从合成计算机生成的域的UDA的问题，以用于学习语义分割的类似但实际的域。我们提出了一种与UDA的一致性正则化方法结合的语义一致的图像到图像转换方法。我们克服了将合成图像转移到真实的图像的先前限制。我们利用伪标签来学习生成的图像到图像转换模型，该图像到图像转换模型从两个域上的语义标签接收额外的反馈。我们的方法优于最先进的方法，将图像到图像转换和半监督学习与相关域适应基准，即Citycapes和Synthia上的CutyCapes和Synthia进行了全面的学习。

生成的对抗网络（GANS）最近引入了执行图像到图像翻译的有效方法。这些模型可以应用于图像到图像到图像转换中的各种域而不改变任何参数。在本文中，我们调查并分析了八个图像到图像生成的对策网络：PIX2PX，Cyclegan，Cogan，Stargan，Munit，Stargan2，Da-Gan，以及自我关注GaN。这些模型中的每一个都呈现了最先进的结果，并引入了构建图像到图像的新技术。除了对模型的调查外，我们还调查了他们接受培训的18个数据集，并在其上进行了评估的9个指标。最后，我们在常见的一组指标和数据集中呈现6种这些模型的受控实验的结果。结果混合并显示，在某些数据集，任务和指标上，某些型号优于其他型号。本文的最后一部分讨论了这些结果并建立了未来研究领域。由于研究人员继续创新新的图像到图像GAN，因此他们非常重要地了解现有方法，数据集和指标。本文提供了全面的概述和讨论，以帮助构建此基础。

域的适应性引起了极大的兴趣，因为标签是一项昂贵且容易出错的任务，尤其是当像素级在语义分段中需要标签时。因此，人们希望能够在数据丰富并且标签精确的合成域上训练神经网络。但是，这些模型通常在室外图像上表现不佳。为了减轻输入的变化，可以使用图像到图像的方法。然而，使用合成训练域桥接部署领域的标准图像到图像方法并不关注下游任务，而仅关注视觉检查级别。因此，我们在图像到图像域的适应方法中提出了gan的“任务意识”版本。借助少量标记的地面真实数据，我们将图像到图像翻译指导为更合适的输入图像，用于培训合成数据（合成域专家）的语义分割网络。这项工作的主要贡献是1）一种模块化半监督域适应方法，通过训练下游任务Aware Cycean，同时避免适应合成语义分割专家2）该方法适用于复杂的域适应任务3）通过使用从头开始网络进行较不偏见的域间隙分析。我们在分类任务以及语义细分方面评估我们的方法。我们的实验表明，我们的方法比仅使用70（10％）地面真实图像的分类任务中的准确性优于标准图像到图像方法 - 准确性的准确性7％。对于语义细分，我们可以在训练过程中仅使用14个地面真相图像，在均值评估数据集上，平均交叉点比联合的平均交叉点约4％至7％。

Domain adaptation for semantic image segmentation is very necessary since manually labeling large datasets with pixel-level labels is expensive and time consuming. Existing domain adaptation techniques either work on limited datasets, or yield not so good performance compared with supervised learning. In this paper, we propose a novel bidirectional learning framework for domain adaptation of segmentation. Using the bidirectional learning, the image translation model and the segmentation adaptation model can be learned alternatively and promote to each other. Furthermore, we propose a self-supervised learning algorithm to learn a better segmentation adaptation model and in return improve the image translation model. Experiments show that our method is superior to the state-of-the-art methods in domain adaptation of segmentation with a big margin. The source code is available at https://github.com/liyunsheng13/BDL.

Unsupervised domain adaptation (UDA) for semantic segmentation is a promising task freeing people from heavy annotation work. However, domain discrepancies in low-level image statistics and high-level contexts compromise the segmentation performance over the target domain. A key idea to tackle this problem is to perform both image-level and feature-level adaptation jointly. Unfortunately, there is a lack of such unified approaches for UDA tasks in the existing literature. This paper proposes a novel UDA pipeline for semantic segmentation that unifies image-level and feature-level adaptation. Concretely, for image-level domain shifts, we propose a global photometric alignment module and a global texture alignment module that align images in the source and target domains in terms of image-level properties. For feature-level domain shifts, we perform global manifold alignment by projecting pixel features from both domains onto the feature manifold of the source domain; and we further regularize category centers in the source domain through a category-oriented triplet loss and perform target domain consistency regularization over augmented target domain images. Experimental results demonstrate that our pipeline significantly outperforms previous methods. In the commonly tested GTA5$\rightarrow$Cityscapes task, our proposed method using Deeplab V3+ as the backbone surpasses previous SOTA by 8%, achieving 58.2% in mIoU.

深度神经网络已经证明了他们可以从数据中自动提取有意义的功能的能力。但是，在监督学习中，特定于用于培训的数据集的信息，但与手头的任务无关，可以在提取的表示中仍然被编码。该剩余信息引入了特定于域的偏差，削弱了泛化性能。在这项工作中，我们建议将信息分成与任务相关的表示及其互补情境表示。我们提出了一种原始方法，结合对抗特征预测器和循环重建，以解开单域监督案例中的这两个表示。然后，我们将该方法适应无监督的域适应问题，包括训练能够在源域和目标域上执行的模型。特别是，尽管没有训练标签，我们的方法促进了目标领域的解剖学。这使得能够将特定于域的特定任务信息隔离为公共表示。任务特定的表示允许有效地将从源域获取的知识转移到目标域。在单一域案中，我们展示了我们关于信息检索任务的陈述的质量以及由锐化的任务特定陈述引起的泛化效益。然后，我们在几个古典域适应基准上验证所提出的方法，并说明了解除域适应的益处。

在本文中，我们展示了Facetunegan，一种新的3D面部模型表示分解和编码面部身份和面部表情。我们提出了对图像到图像翻译网络的第一次适应，该图像已经成功地用于2D域，到3D面几何。利用最近释放的大面扫描数据库，神经网络已经过培训，以便与面部更好的了解，使面部表情转移和中和富有效应面的变异因素。具体而言，我们设计了一种适应基础架构的对抗架构，并使用Spiralnet ++进行卷积和采样操作。使用两个公共数据集（FACESCAPE和COMA），Facetunegan具有比最先进的技术更好的身份分解和面部中和。它还通过预测较近地面真实数据的闪烁形状并且由于源极和目标之间的面部形态过于不同的面部形态而越来越多的不期望的伪像来优异。