Semantic segmentation is a key problem for many computer vision tasks. While approaches based on convolutional neural networks constantly break new records on different benchmarks, generalizing well to diverse testing environments remains a major challenge. In numerous real world applications, there is indeed a large gap between data distributions in train and test domains, which results in severe performance loss at run-time. In this work, we address the task of unsupervised domain adaptation in semantic segmentation with losses based on the entropy of the pixel-wise predictions. To this end, we propose two novel, complementary methods using (i) an entropy loss and (ii) an adversarial loss respectively. We demonstrate state-of-theart performance in semantic segmentation on two challenging "synthetic-2-real" set-ups 1 and show that the approach can also be used for detection.

Convolutional neural network-based approaches for semantic segmentation rely on supervision with pixel-level ground truth, but may not generalize well to unseen image domains. As the labeling process is tedious and labor intensive, developing algorithms that can adapt source ground truth labels to the target domain is of great interest. In this paper, we propose an adversarial learning method for domain adaptation in the context of semantic segmentation. Considering semantic segmentations as structured outputs that contain spatial similarities between the source and target domains, we adopt adversarial learning in the output space. To further enhance the adapted model, we construct a multi-level adversarial network to effectively perform output space domain adaptation at different feature levels. Extensive experiments and ablation study are conducted under various domain adaptation settings, including synthetic-to-real and cross-city scenarios. We show that the proposed method performs favorably against the stateof-the-art methods in terms of accuracy and visual quality.

We consider the problem of unsupervised domain adaptation in semantic segmentation. A key in this campaign consists in reducing the domain shift, i.e., enforcing the data distributions of the two domains to be similar. One of the common strategies is to align the marginal distribution in the feature space through adversarial learning. However, this global alignment strategy does not consider the category-level joint distribution. A possible consequence of such global movement is that some categories which are originally well aligned between the source and target may be incorrectly mapped, thus leading to worse segmentation results in target domain. To address this problem, we introduce a category-level adversarial network, aiming to enforce local semantic consistency during the trend of global alignment. Our idea is to take a close look at the category-level joint distribution and align each class with an adaptive adversarial loss. Specifically, we reduce the weight of the adversarial loss for category-level aligned features while increasing the adversarial force for those poorly aligned. In this process, we decide how well a feature is category-level aligned between source and target by a co-training approach. In two domain adaptation tasks, i.e., GTA5 → Cityscapes and SYN-THIA → Cityscapes, we validate that the proposed method matches the state of the art in segmentation accuracy.

本文提出FogAdapt，一种用于密集有雾场景的语义细分域的新方法。虽然已经针对显着的研究来减少语义分割中的域移位，但对具有恶劣天气条件的场景的适应仍然是一个开放的问题。由于天气状况，如雾，烟雾和雾度，加剧了域移位的场景的可见性，从而使得在这种情况下进行了无监督的适应性。我们提出了一种自熵和多尺度信息增强的自我监督域适应方法（FOGADAPT），以最大限度地减少有雾场景分割的域移位。由经验证据支持，雾密度的增加导致分割概率的高自熵性，我们引入了基于自熵的损耗功能来引导适应方法。此外，在不同的图像尺度上获得的推论由不确定性组合并加权，以生成目标域的尺度不变伪标签。这些规模不变的伪标签对可见性和比例变化具有鲁棒性。我们在真正的雾景场景中评估了真正的清晰天气场景模型，适应和综合非雾图像到真正的雾场景适应情景。我们的实验表明，FogAdapt在有雾图像的语义分割中的目前最先进的情况下显着优异。具体而言，通过考虑标准设置与最先进的（SOTA）方法相比，FogaDATK在Foggy苏黎世上获得3.8％，有雾的驾驶密集为6.0％，而在Miou的雾化驾驶的3.6％，在Miou，在MiOOP中改编为有雾的苏黎世。

Recent deep networks achieved state of the art performance on a variety of semantic segmentation tasks. Despite such progress, these models often face challenges in real world "wild tasks" where large difference between labeled training/source data and unseen test/target data exists. In particular, such difference is often referred to as "domain gap", and could cause significantly decreased performance which cannot be easily remedied by further increasing the representation power. Unsupervised domain adaptation (UDA) seeks to overcome such problem without target domain labels. In this paper, we propose a novel UDA framework based on an iterative self-training (ST) procedure, where the problem is formulated as latent variable loss minimization, and can be solved by alternatively generating pseudo labels on target data and re-training the model with these labels. On top of ST, we also propose a novel classbalanced self-training (CBST) framework to avoid the gradual dominance of large classes on pseudo-label generation, and introduce spatial priors to refine generated labels. Comprehensive experiments show that the proposed methods achieve state of the art semantic segmentation performance under multiple major UDA settings.⋆ indicates equal contribution.

Although unsupervised domain adaptation methods have achieved remarkable performance in semantic scene segmentation in visual perception for self-driving cars, these approaches remain impractical in real-world use cases. In practice, the segmentation models may encounter new data that have not been seen yet. Also, the previous data training of segmentation models may be inaccessible due to privacy problems. Therefore, to address these problems, in this work, we propose a Continual Unsupervised Domain Adaptation (CONDA) approach that allows the model to continuously learn and adapt with respect to the presence of the new data. Moreover, our proposed approach is designed without the requirement of accessing previous training data. To avoid the catastrophic forgetting problem and maintain the performance of the segmentation models, we present a novel Bijective Maximum Likelihood loss to impose the constraint of predicted segmentation distribution shifts. The experimental results on the benchmark of continual unsupervised domain adaptation have shown the advanced performance of the proposed CONDA method.

深度神经网络（DNN）极大地促进了语义分割中的性能增益。然而，训练DNN通常需要大量的像素级标记数据，这在实践中收集昂贵且耗时。为了减轻注释负担，本文提出了一种自组装的生成对抗网络（SE-GAN）利用语义分割的跨域数据。在SE-GaN中，教师网络和学生网络构成用于生成语义分割图的自组装模型，与鉴别器一起形成GaN。尽管它很简单，我们发现SE-GaN可以显着提高对抗性训练的性能，提高模型的稳定性，这是由大多数普遍培训的方法共享的常见障碍。我们理论上分析SE-GaN并提供$ \ Mathcal o（1 / \ sqrt {n}）$泛化绑定（$ n $是培训样本大小），这表明控制了鉴别者的假设复杂性，以提高概括性。因此，我们选择一个简单的网络作为鉴别器。两个标准设置中的广泛和系统实验表明，该方法显着优于最新的最先进的方法。我们模型的源代码即将推出。

语义分割在广泛的计算机视觉应用中起着基本作用，提供了全球对图像​​的理解的关键信息。然而，最先进的模型依赖于大量的注释样本，其比在诸如图像分类的任务中获得更昂贵的昂贵的样本。由于未标记的数据替代地获得更便宜，因此无监督的域适应达到了语义分割社区的广泛成功并不令人惊讶。本调查致力于总结这一令人难以置信的快速增长的领域的五年，这包含了语义细分本身的重要性，以及将分段模型适应新环境的关键需求。我们提出了最重要的语义分割方法;我们对语义分割的域适应技术提供了全面的调查;我们揭示了多域学习，域泛化，测试时间适应或无源域适应等较新的趋势;我们通过描述在语义细分研究中最广泛使用的数据集和基准测试来结束本调查。我们希望本调查将在学术界和工业中提供具有全面参考指导的研究人员，并有助于他们培养现场的新研究方向。

本文提出了一种新颖的像素级分布正则化方案（DRSL），用于自我监督的语义分割域的适应性。在典型的环境中，分类损失迫使语义分割模型贪婪地学习捕获类间变化的表示形式，以确定决策（类）边界。由于域的转移，该决策边界在目标域中未对齐，从而导致嘈杂的伪标签对自我监督域的适应性产生不利影响。为了克服这一限制，以及捕获阶层间变化，我们通过类感知的多模式分布学习（MMDL）捕获了像素级内的类内变化。因此，捕获阶层内变化所需的信息与阶层间歧视所需的信息明确分开。因此，捕获的功能更具信息性，导致伪噪声低的伪标记。这种分离使我们能够使用前者的基于跨凝结的自学习，在判别空间和多模式分布空间中进行单独的对齐。稍后，我们通过明确降低映射到同一模式的目标和源像素之间的距离来提出一种新型的随机模式比对方法。距离度量标签上计算出的距离度量损失，并从多模式建模头部反向传播，充当与分割头共享的基本网络上的正常化程序。关于合成到真实域的适应设置的全面实验的结果，即GTA-V/Synthia to CityScapes，表明DRSL的表现优于许多现有方法（MIOU的最小余量为2.3％和2.5％，用于MIOU，而合成的MIOU到CityScapes）。

我们专注于在不同情况下在车道检测中桥接域差异，以大大降低自动驾驶的额外注释和重新训练成本。关键因素阻碍了跨域车道检测的性能改善，即常规方法仅着眼于像素损失，同时忽略了泳道的形状和位置验证阶段。为了解决该问题，我们提出了多级域Adaptation（MLDA）框架，这是一种在三个互补语义级别的像素，实例和类别的互补语义级别处理跨域车道检测的新观点。具体而言，在像素级别上，我们建议在自我训练中应用跨级置信度限制，以应对车道和背景的不平衡置信分布。在实例层面上，我们超越像素，将分段车道视为实例，并通过三胞胎学习促进目标域中的判别特征，这有效地重建了车道的语义环境，并有助于减轻特征混乱。在类别级别，我们提出了一个自适应域间嵌入模块，以在自适应过程中利用泳道的先验位置。在两个具有挑战性的数据集（即Tusimple和Culane）中，我们的方法将车道检测性能提高了很大的利润率，与先进的领域适应算法相比，精度分别提高了8.8％和F1级的7.4％。

无监督的域适应（UDA）旨在使源域上培训的模型适应到新的目标域，其中没有可用标记的数据。在这项工作中，我们调查从合成计算机生成的域的UDA的问题，以用于学习语义分割的类似但实际的域。我们提出了一种与UDA的一致性正则化方法结合的语义一致的图像到图像转换方法。我们克服了将合成图像转移到真实的图像的先前限制。我们利用伪标签来学习生成的图像到图像转换模型，该图像到图像转换模型从两个域上的语义标签接收额外的反馈。我们的方法优于最先进的方法，将图像到图像转换和半监督学习与相关域适应基准，即Citycapes和Synthia上的CutyCapes和Synthia进行了全面的学习。

受益于从特定情况（源）收集的相当大的像素级注释，训练有素的语义分段模型表现得非常好，但由于大域移位而导致的新情况（目标）失败。为了缓解域间隙，先前的跨域语义分段方法始终在域对齐期间始终假设源数据和目标数据的共存。但是，在实际方案中访问源数据可能会引发隐私问题并违反知识产权。为了解决这个问题，我们专注于一个有趣和具有挑战性的跨域语义分割任务，其中仅向目标域提供训练源模型。具体地，我们提出了一种称为ATP的统一框架，其包括三种方案，即特征对准，双向教学和信息传播。首先，我们设计了课程熵最小化目标，以通过提供的源模型隐式对准目标功能与看不见的源特征。其次，除了vanilla自我训练中的正伪标签外，我们是第一个向该领域引入负伪标签的，并开发双向自我训练策略，以增强目标域中的表示学习。最后，采用信息传播方案来通过伪半监督学习进一步降低目标域内的域内差异。综合与跨城市驾驶数据集的广泛结果验证\ TextBF {ATP}产生最先进的性能，即使是需要访问源数据的方法。

由于难以获得地面真理标签，从虚拟世界数据集学习对于像语义分割等现实世界的应用非常关注。从域适应角度来看，关键挑战是学习输入的域名签名表示，以便从虚拟数据中受益。在本文中，我们提出了一种新颖的三叉戟架构，该架构强制执行共享特征编码器，同时满足对抗源和目标约束，从而学习域不变的特征空间。此外，我们还介绍了一种新颖的训练管道，在前向通过期间能够自我引起的跨域数据增强。这有助于进一步减少域间隙。结合自我培训过程，我们在基准数据集（例如GTA5或Synthia适应城市景观）上获得最先进的结果。Https://github.com/hmrc-ael/trideadapt提供了代码和预先训练的型号。

虽然监督语义分割存在重大进展，但由于领域偏差，将分段模型部署到解除域来仍然具有挑战性。域适应可以通过将知识从标记的源域传输到未标记的目标域来帮助。以前的方法通常尝试执行对全局特征的适应，然而，通常忽略要计入特征空间中的每个像素的本地语义附属机构，导致较少的可辨性。为解决这个问题，我们提出了一种用于细粒度阶级对齐的新型语义原型对比学习框架。具体地，语义原型提供了用于每个像素鉴别的表示学习的监控信号，并且需要在特征空间中的源极和目标域的每个像素来反映相应的语义原型的内容。通过这种方式，我们的框架能够明确地制作较近的类别的像素表示，并且进一步越来越多地分开，以改善分割模型的鲁棒性以及减轻域移位问题。与最先进的方法相比，我们的方法易于实施并达到优异的结果，如众多实验所展示的那样。代码在[此HTTPS URL]（https://github.com/binhuixie/spcl）上公开可用。

无监督的域适应性（UDA）旨在使在标记的源域上训练的模型适应未标记的目标域。在本文中，我们提出了典型的对比度适应（PROCA），这是一种无监督域自适应语义分割的简单有效的对比度学习方法。以前的域适应方法仅考虑跨各个域的阶级内表示分布的对齐，而阶层间结构关系的探索不足，从而导致目标域上的对齐表示可能不像在源上歧视的那样容易歧视。域了。取而代之的是，ProCA将类间信息纳入班级原型，并采用以班级为中心的分布对齐进行适应。通过将同一类原型与阳性和其他类原型视为实现以集体为中心的分配对齐方式的负面原型，Proca在经典领域适应任务上实现了最先进的性能，{\ em i.e. text {and} synthia $ \ to $ cityScapes}。代码可在\ href {https://github.com/jiangzhengkai/proca} {proca}获得代码

We describe a simple method for unsupervised domain adaptation, whereby the discrepancy between the source and target distributions is reduced by swapping the lowfrequency spectrum of one with the other. We illustrate the method in semantic segmentation, where densely annotated images are aplenty in one domain (e.g., synthetic data), but difficult to obtain in another (e.g., real images). Current state-of-the-art methods are complex, some requiring adversarial optimization to render the backbone of a neural network invariant to the discrete domain selection variable. Our method does not require any training to perform the domain alignment, just a simple Fourier Transform and its inverse. Despite its simplicity, it achieves state-of-the-art performance in the current benchmarks, when integrated into a relatively standard semantic segmentation model. Our results indicate that even simple procedures can discount nuisance variability in the data that more sophisticated methods struggle to learn away. 1

跨数据集的语义细分的域适应性，由相同类别组成，已经获得了一些最近的成功。但是，更一般的情况是源和目标数据集对应于非重叠标签空间时。例如，分割数据集中的类别根据环境或应用程序的类型发生了很大变化，但共享许多有价值的语义关系。基于特征对齐或差异最小化的现有方法不会考虑此类类别的转移。在这项工作中，我们提出了群集到适应（C2A），这是一种基于计算有效的聚类方法，用于跨分割数据集的域适应性，这些方法完全不同但可能相关类别。我们表明，在变换的特征空间中强制执行的这种聚类目标可以自动选择跨源和目标域的类别，这些类别可以对齐以改善目标性能，同时防止对无关类别的负转移。我们通过实验对室外的挑战性问题进行了实验，以少量拍摄和零拍设置来证明室内适应性的挑战性问题，在所有情况下，性能对现有方法和基准的绩效持续改善。

深度学习极大地提高了语义细分的性能，但是，它的成功依赖于大量注释的培训数据的可用性。因此，许多努力致力于域自适应语义分割，重点是将语义知识从标记的源域转移到未标记的目标域。现有的自我训练方法通常需要多轮训练，而基于对抗训练的另一个流行框架已知对超参数敏感。在本文中，我们提出了一个易于训练的框架，该框架学习了域自适应语义分割的域不变原型。特别是，我们表明域的适应性与很少的学习共享一个共同的角色，因为两者都旨在识别一些从大量可见数据中学到的知识的看不见的数据。因此，我们提出了一个统一的框架，用于域适应和很少的学习。核心思想是使用从几个镜头注释的目标图像中提取的类原型来对源图像和目标图像的像素进行分类。我们的方法仅涉及一个阶段训练，不需要对大规模的未经通知的目标图像进行培训。此外，我们的方法可以扩展到域适应性和几乎没有射击学习的变体。关于适应GTA5到CITYSCAPES和合成景观的实验表明，我们的方法实现了对最先进的竞争性能。

Domain adaptation is critical for success in new, unseen environments. Adversarial adaptation models applied in feature spaces discover domain invariant representations, but are difficult to visualize and sometimes fail to capture pixel-level and low-level domain shifts. Recent work has shown that generative adversarial networks combined with cycle-consistency constraints are surprisingly effective at mapping images between domains, even without the use of aligned image pairs. We propose a novel discriminatively-trained Cycle-Consistent Adversarial Domain Adaptation model. CyCADA adapts representations at both the pixel-level and feature-level, enforces cycle-consistency while leveraging a task loss, and does not require aligned pairs. Our model can be applied in a variety of visual recognition and prediction settings. We show new state-of-the-art results across multiple adaptation tasks, including digit classification and semantic segmentation of road scenes demonstrating transfer from synthetic to real world domains.

Domain adaptation for semantic image segmentation is very necessary since manually labeling large datasets with pixel-level labels is expensive and time consuming. Existing domain adaptation techniques either work on limited datasets, or yield not so good performance compared with supervised learning. In this paper, we propose a novel bidirectional learning framework for domain adaptation of segmentation. Using the bidirectional learning, the image translation model and the segmentation adaptation model can be learned alternatively and promote to each other. Furthermore, we propose a self-supervised learning algorithm to learn a better segmentation adaptation model and in return improve the image translation model. Experiments show that our method is superior to the state-of-the-art methods in domain adaptation of segmentation with a big margin. The source code is available at https://github.com/liyunsheng13/BDL.