无监督的域适应（UDA）是机器学习和模式识别领域的新兴的研究主题，其旨在通过从源域传输知识来帮助学习未标记的目标域。

鉴于机器学习环境快速变化和昂贵的数据标记，当来自源域的标记数据与目标域的部分标记的数据在统计上不同时，必须进行半监督域的适应（SSDA）。大多数先前的SSDA研究都在集中进行，需要访问源和目标数据。但是，如今许多字段中的数据是由分布式终端设备生成的。由于隐私问题，数据可能是本地存储的，无法共享，从而导致现有SSDA研究的无效性。本文提出了一种创新的方法，以通过联合半监督域适应（FSSDA）命名的多个分布式和机密数据集实现SSDA。 FSSDA基于战略设计的知识蒸馏技术将SSDA与联合学习集成在一起，通过并行执行源和目标培训来提高效率。此外，FSSDA通过正确选择关键参数（即模仿参数）来控制跨域传输的知识量。此外，建议的FSSDA可以有效地推广到多源域适应方案。进行了广泛的实验，以证明FSSDA设计的有效性和效率。

In this paper, we investigate a challenging unsupervised domain adaptation setting -unsupervised model adaptation. We aim to explore how to rely only on unlabeled target data to improve performance of an existing source prediction model on the target domain, since labeled source data may not be available in some real-world scenarios due to data privacy issues. For this purpose, we propose a new framework, which is referred to as collaborative class conditional generative adversarial net to bypass the dependence on the source data. Specifically, the prediction model is to be improved through generated target-style data, which provides more accurate guidance for the generator. As a result, the generator and the prediction model can collaborate with each other without source data. Furthermore, due to the lack of supervision from source data, we propose a weight constraint that encourages similarity to the source model. A clustering-based regularization is also introduced to produce more discriminative features in the target domain. Compared to conventional domain adaptation methods, our model achieves superior performance on multiple adaptation tasks with only unlabeled target data, which verifies its effectiveness in this challenging setting.

群集广泛用于文本分析，自然语言处理，图像分割和其他数据挖掘字段。作为一个有前途的聚类算法，通过允许对象属于几个类别的若干类来提供对数据的更深层次的识别，这延长了硬，模糊和可能性聚类。但是，由于需要估计比其他基于经典分区的算法更多的参数，因此当可用数据充足并且质量良好时，它才能很好地运行。为了克服这些缺点，本文通过引入转移学习策略来提出转移证据C-Mean（TECM）算法。 TECM的目标函数是通过基于ECM的目标函数在源域中引入源域中的重心而获得的目标函数，并且使用迭代优化策略来解决客观函数。另外，TECM可以适应源域和目标域中的簇数不同的情况。所提出的算法已在合成和现实世界数据集上验证。实验结果证明了与原始ECM的TECM的有效性以及其他代表多任务或转移聚类算法。

现代机器学习（ML）模型越来越流行，并广泛用于决策系统。但是，研究表明，ML歧视和不公平性的关键问题阻碍了他们对高级应用程序的采用。对公平分类器的最新研究引起了人们的重大关注，以开发有效的算法以实现公平性和良好的分类性能。尽管这些公平感知到的机器学习模型取得了巨大的成功，但大多数现有模型都需要敏感属性来预处理数据，将模型学习正规化或后处理预测以具有公平的预测。但是，由于隐私，法律或法规限制，敏感属性通常是不完整甚至不可用的。尽管我们缺乏训练目标域中公平模型的敏感属性，但可能存在具有敏感属性的类似域。因此，重要的是从类似域中利用辅助信息，以帮助改善目标域中的公平分类。因此，在本文中，我们研究了探索域适应以进行公平分类的新问题。我们提出了一个新框架，可以同时估算目标域中的公平分类器时，可以同时估算敏感属性。现实世界数据集的广泛实验说明了提出的公平分类模型的有效性，即使目标域中没有敏感属性。

We propose a novel unsupervised domain adaptation framework based on domain-specific batch normalization in deep neural networks. We aim to adapt to both domains by specializing batch normalization layers in convolutional neural networks while allowing them to share all other model parameters, which is realized by a twostage algorithm. In the first stage, we estimate pseudolabels for the examples in the target domain using an external unsupervised domain adaptation algorithm-for example, MSTN [27] or CPUA [14]-integrating the proposed domain-specific batch normalization. The second stage learns the final models using a multi-task classification loss for the source and target domains. Note that the two domains have separate batch normalization layers in both stages. Our framework can be easily incorporated into the domain adaptation techniques based on deep neural networks with batch normalization layers. We also present that our approach can be extended to the problem with multiple source domains. The proposed algorithm is evaluated on multiple benchmark datasets and achieves the state-of-theart accuracy in the standard setting and the multi-source domain adaption scenario.

Transfer learning aims at improving the performance of target learners on target domains by transferring the knowledge contained in different but related source domains. In this way, the dependence on a large number of target domain data can be reduced for constructing target learners. Due to the wide application prospects, transfer learning has become a popular and promising area in machine learning. Although there are already some valuable and impressive surveys on transfer learning, these surveys introduce approaches in a relatively isolated way and lack the recent advances in transfer learning. Due to the rapid expansion of the transfer learning area, it is both necessary and challenging to comprehensively review the relevant studies. This survey attempts to connect and systematize the existing transfer learning researches, as well as to summarize and interpret the mechanisms and the strategies of transfer learning in a comprehensive way, which may help readers have a better understanding of the current research status and ideas. Unlike previous surveys, this survey paper reviews more than forty representative transfer learning approaches, especially homogeneous transfer learning approaches, from the perspectives of data and model. The applications of transfer learning are also briefly introduced. In order to show the performance of different transfer learning models, over twenty representative transfer learning models are used for experiments. The models are performed on three different datasets, i.e., Amazon Reviews, Reuters-21578, and Office-31. And the experimental results demonstrate the importance of selecting appropriate transfer learning models for different applications in practice.

最近的智能故障诊断（IFD）的进展大大依赖于深度代表学习和大量标记数据。然而，机器通常以各种工作条件操作，或者目标任务具有不同的分布，其中包含用于训练的收集数据（域移位问题）。此外，目标域中的新收集的测试数据通常是未标记的，导致基于无监督的深度转移学习（基于UDTL为基础的）IFD问题。虽然它已经实现了巨大的发展，但标准和开放的源代码框架以及基于UDTL的IFD的比较研究尚未建立。在本文中，我们根据不同的任务，构建新的分类系统并对基于UDTL的IFD进行全面审查。对一些典型方法和数据集的比较分析显示了基于UDTL的IFD中的一些开放和基本问题，这很少研究，包括特征，骨干，负转移，物理前导等的可转移性，强调UDTL的重要性和再现性 - 基于IFD，整个测试框架将发布给研究界以促进未来的研究。总之，发布的框架和比较研究可以作为扩展界面和基本结果，以便对基于UDTL的IFD进行新的研究。代码框架可用于\ url {https:/github.com/zhaozhibin/udtl}。

无监督的域对点云语义分割的适应性引起了极大的关注，因为它在没有标记的数据中学习有效性。大多数现有方法都使用全局级特征对齐方式将知识从源域转移到目标域，这可能会导致特征空间的语义歧义。在本文中，我们提出了一个基于图形的框架，以探索两个域之间的局部特征对齐，可以在适应过程中保留语义歧视。具体而言，为了提取本地级特征，我们首先在两个域上动态构建本地特征图，并使用来自源域的图形构建存储库。特别是，我们使用最佳传输来生成图形匹配对。然后，基于分配矩阵，我们可以将两个域之间的特征分布与基于图的本地特征损失对齐。此外，我们考虑了不同类别的特征之间的相关性，并制定了类别引导的对比损失，以指导分割模型以学习目标域上的区分特征。对不同的合成到现实和真实域的适应情景进行了广泛的实验表明，我们的方法可以实现最先进的性能。

This paper solves a generalized version of the problem of multi-source model adaptation for semantic segmentation. Model adaptation is proposed as a new domain adaptation problem which requires access to a pre-trained model instead of data for the source domain. A general multi-source setting of model adaptation assumes strictly that each source domain shares a common label space with the target domain. As a relaxation, we allow the label space of each source domain to be a subset of that of the target domain and require the union of the source-domain label spaces to be equal to the target-domain label space. For the new setting named union-set multi-source model adaptation, we propose a method with a novel learning strategy named model-invariant feature learning, which takes full advantage of the diverse characteristics of the source-domain models, thereby improving the generalization in the target domain. We conduct extensive experiments in various adaptation settings to show the superiority of our method. The code is available at https://github.com/lzy7976/union-set-model-adaptation.

Recent work reported the label alignment property in a supervised learning setting: the vector of all labels in the dataset is mostly in the span of the top few singular vectors of the data matrix. Inspired by this observation, we derive a regularization method for unsupervised domain adaptation. Instead of regularizing representation learning as done by popular domain adaptation methods, we regularize the classifier so that the target domain predictions can to some extent ``align" with the top singular vectors of the unsupervised data matrix from the target domain. In a linear regression setting, we theoretically justify the label alignment property and characterize the optimality of the solution of our regularization by bounding its distance to the optimal solution. We conduct experiments to show that our method can work well on the label shift problems, where classic domain adaptation methods are known to fail. We also report mild improvement over domain adaptation baselines on a set of commonly seen MNIST-USPS domain adaptation tasks and on cross-lingual sentiment analysis tasks.

由于计算机软件的普遍性，软件漏洞（SVS）已成为普遍，严重和至关重要的问题。已经提出了许多基于机器学习的方法来解决软件漏洞检测（SVD）问题。但是，关于SVD仍然存在两个开放和重大问题，就i）学习自动表示以提高SVD的预测性能，ii）解决常规需要专家的标签漏洞数据集的稀缺性数据集。在本文中，我们提出了一种新颖的端到端方法来解决这两个关键问题。我们首先利用自动表示学习，并具有深层域的适应性，以进行软件漏洞检测。然后，我们提出了一个新型的跨域内核分类器，利用最大额度额定原则，以显着改善从标记项目到未标记的项目的软件漏洞的传输学习过程。现实世界软件数据集的实验结果表明，我们提出的方法优于最先进的基准。简而言之，与使用数据集中的第二高方法相比，我们的方法在SVD中获得了更高的F1量化性能，这是SVD中最重要的度量，从1.83％到6.25％。我们已发布的源代码样本可在https://github.com/vannguyennd/dam2p上公开获取

Deep neural networks are able to learn powerful representations from large quantities of labeled input data, however they cannot always generalize well across changes in input distributions. Domain adaptation algorithms have been proposed to compensate for the degradation in performance due to domain shift. In this paper, we address the case when the target domain is unlabeled, requiring unsupervised adaptation. CORAL[1] is a "frustratingly easy" unsupervised domain adaptation method that aligns the second-order statistics of the source and target distributions with a linear transformation. Here, we extend CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep neural networks (Deep CORAL). Experiments on standard benchmark datasets show state-of-the-art performance.

通过从完全标记的源域中利用数据，无监督域适应（UDA）通过显式差异最小化数据分布或对抗学习来提高未标记的目标域上的分类性能。作为增强，通过利用模型预测来加强目标特征识别期间涉及类别对齐。但是，在目标域上的错误类别预测中产生的伪标签不准确以及由源域的过度录制引起的分发偏差存在未探明的问题。在本文中，我们提出了一种模型 - 不可知的两阶段学习框架，这大大减少了使用软伪标签策略的缺陷模型预测，并避免了课程学习策略的源域上的过度拟合。从理论上讲，它成功降低了目标域上预期误差的上限的综合风险。在第一阶段，我们用分布对齐的UDA方法训练一个模型，以获得具有相当高的置位目标域上的软语义标签。为了避免在源域上的过度拟合，在第二阶段，我们提出了一种课程学习策略，以自适应地控制来自两个域的损失之间的加权，以便训练阶段的焦点从源分布逐渐移位到目标分布，以预测信心提升了目标分布在目标领域。对两个知名基准数据集的广泛实验验证了我们提出框架促进促进顶级UDA算法的性能的普遍效果，并展示其一致的卓越性能。

Unsupervised domain adaptation (UDA) aims to transfer knowledge from a well-labeled source domain to a different but related unlabeled target domain with identical label space. Currently, the main workhorse for solving UDA is domain alignment, which has proven successful. However, it is often difficult to find an appropriate source domain with identical label space. A more practical scenario is so-called partial domain adaptation (PDA) in which the source label set or space subsumes the target one. Unfortunately, in PDA, due to the existence of the irrelevant categories in the source domain, it is quite hard to obtain a perfect alignment, thus resulting in mode collapse and negative transfer. Although several efforts have been made by down-weighting the irrelevant source categories, the strategies used tend to be burdensome and risky since exactly which irrelevant categories are unknown. These challenges motivate us to find a relatively simpler alternative to solve PDA. To achieve this, we first provide a thorough theoretical analysis, which illustrates that the target risk is bounded by both model smoothness and between-domain discrepancy. Considering the difficulty of perfect alignment in solving PDA, we turn to focus on the model smoothness while discard the riskier domain alignment to enhance the adaptability of the model. Specifically, we instantiate the model smoothness as a quite simple intra-domain structure preserving (IDSP). To our best knowledge, this is the first naive attempt to address the PDA without domain alignment. Finally, our empirical results on multiple benchmark datasets demonstrate that IDSP is not only superior to the PDA SOTAs by a significant margin on some benchmarks (e.g., +10% on Cl->Rw and +8% on Ar->Rw ), but also complementary to domain alignment in the standard UDA

本文研究了一个新的，实用但具有挑战性的问题，称为类无监督的域名适应性（CI-UDA），其中标记的源域包含所有类别，但是未标记的目标域中的类别依次增加。由于两个困难，这个问题具有挑战性。首先，源和目标标签集在每个时间步骤都不一致，这使得很难进行准确的域对齐。其次，以前的目标类在当前步骤中不可用，从而忘记了先前的知识。为了解决这个问题，我们提出了一种新型的原型引导连续适应（PROCA）方法，由两种解决方案策略组成。 1）标签原型识别：我们通过检测具有目标样本的累积预测概率的共享类来识别目标标签原型。 2）基于原型的对齐和重播：基于确定的标签原型，我们对齐域并强制执行模型以保留先前的知识。有了这两种策略，ProCA能够有效地将源模型改编为类未标记的目标域。广泛的实验证明了Proca在解决CI-UDA方面的有效性和优势。源代码可从https://github.com/hongbin98/proca.git获得

在本文中，我们提出了一种对无监督域适应的新方法，与最佳运输，学习概率措施和无监督学习的概念相关。所提出的方法Hot-DA基于最佳运输的分层制定，其利用了由地面度量捕获的几何信息，源和目标域中的结构信息更丰富的结构信息。通过根据其类标签将样本分组到结构中，本质地形成标记的源域中的附加信息。在探索未标记的目标域中的隐藏结构的同时，通过Wassersein BaryCenter的学习概率措施的问题，我们证明是等同于光谱聚类。具有可控复杂性的玩具数据集的实验和两个具有挑战性的视觉适应数据集显示了所提出的方法的优越性。

域适应任务，如跨域情感分类的目标在于利用在目标域源域中现有数据标记和未标记的或标记的几个数据经由减少数据分布之间的偏移，以提高在目标域的性能。现有跨域情绪分类方法需要区分枢转，即，域共享情绪词语，和非枢转时，即，该特定域的情绪也就是说，对于优良的适应性能。在本文中，我们首先设计一个类别关注网络（CAN），然后提出一个名为CAN-CNN集成CAN和卷积神经网络（CNN）模型。在一方面，该模型的问候基点和非关键数据作为统一的类别属性的话，可以自动捕捉他们提高域自适应性能;在另一方面，该模式使得在解释性试图了解转移类别属性词。具体地，我们的模型的优化目标具有三个不同的部分：1）监督分类损失; 2）类别特征权重的分布丢失; 3）的域不变性损失。最后，该模型在三个民情分析数据集进行评估，结果表明，CAN-CNN能优于其他各种基线的方法。

The standard closed-set domain adaptation approaches seek to mitigate distribution discrepancies between two domains under the constraint of both sharing identical label sets. However, in realistic scenarios, finding an optimal source domain with identical label space is a challenging task. Partial domain adaptation alleviates this problem of procuring a labeled dataset with identical label space assumptions and addresses a more practical scenario where the source label set subsumes the target label set. This, however, presents a few additional obstacles during adaptation. Samples with categories private to the source domain thwart relevant knowledge transfer and degrade model performance. In this work, we try to address these issues by coupling variational information and adversarial learning with a pseudo-labeling technique to enforce class distribution alignment and minimize the transfer of superfluous information from the source samples. The experimental findings in numerous cross-domain classification tasks demonstrate that the proposed technique delivers superior and comparable accuracy to existing methods.

作为对数据有效使用的研究，多源无监督的域适应性将知识从带有标记数据的多个源域转移到了未标记的目标域。但是，目标域中不同域和嘈杂的伪标签之间的分布差异都导致多源无监督域适应方法的性能瓶颈。鉴于此，我们提出了一种将注意力驱动的领域融合和耐噪声学习（ADNT）整合到上述两个问题的方法。首先，我们建立了相反的注意结构，以在特征和诱导域运动之间执行信息。通过这种方法，当域差异降低时，特征的可区分性也可以显着提高。其次，基于无监督的域适应训练的特征，我们设计了自适应的反向横向熵损失，该损失可以直接对伪标签的产生施加约束。最后，结合了这两种方法，几个基准的实验结果进一步验证了我们提出的ADNT的有效性，并证明了优于最新方法的性能。