我们为半监督学习设置提供了一种新的数据增强技术，该技术强调从功能空间最具挑战性的地区学习。从完全监督的参考模型开始，我们首先确定较低的置信度预测。然后，这些样品用于训练变异自动编码器（VAE），该变量可以生成具有相似分布的无限额外图像。最后，使用最初标记的数据和合成生成的标记和未标记的数据，我们以半监视的方式重新训练了一个新模型。我们对两个基准RGB数据集进行实验：CIFAR-100和STL-10，并表明所提出的方案在准确性和鲁棒性方面提高了分类性能，同时就现有的完全监督的方法而产生可比或优越的结果。

Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets. In this work, we unify the current dominant approaches for semi-supervised learning to produce a new algorithm, MixMatch, that guesses low-entropy labels for data-augmented unlabeled examples and mixes labeled and unlabeled data using MixUp. MixMatch obtains state-of-the-art results by a large margin across many datasets and labeled data amounts. For example, on CIFAR-10 with 250 labels, we reduce error rate by a factor of 4 (from 38% to 11%) and by a factor of 2 on STL-10. We also demonstrate how MixMatch can help achieve a dramatically better accuracy-privacy trade-off for differential privacy. Finally, we perform an ablation study to tease apart which components of MixMatch are most important for its success. We release all code used in our experiments. 1

长期以来，半监督学习（SSL）已被证明是一种有限的标签模型的有效技术。在现有的文献中，基于一致性的基于正则化的方法，这些方法迫使扰动样本具有类似的预测，而原始的样本则引起了极大的关注。但是，我们观察到，当标签变得极为有限时，例如，每个类别的2或3标签时，此类方法的性能会大大降低。我们的实证研究发现，主要问题在于语义信息在数据增强过程中的漂移。当提供足够的监督时，可以缓解问题。但是，如果几乎没有指导，错误的正则化将误导网络并破坏算法的性能。为了解决该问题，我们（1）提出了一种基于插值的方法来构建更可靠的正样品对； （2）设计一种新颖的对比损失，以指导学习网络的嵌入以在样品之间进行线性更改，从而通过扩大保证金决策边界来提高网络的歧视能力。由于未引入破坏性正则化，因此我们提出的算法的性能在很大程度上得到了改善。具体而言，所提出的算法的表现优于第二好算法（COMATT），而当CIFAR-10数据集中的每个类只有两个标签可用时，可以实现88.73％的分类精度，占5.3％。此外，我们通过通过我们提出的策略大大改善现有最新算法的性能，进一步证明了所提出的方法的普遍性。

现有突出物体检测模型的成功依赖于大像素标记的训练数据集。如何，收集这样的数据集不仅耗时，而且非常昂贵。为了减少标签负担，我们研究半监督的突出物体检测，并通过识别具有较小自信预测的像素来将其作为未标记的数据集像素级置信度估计问题。具体地，我们在有效的潜在空间探索之前引入了一种新的潜在变量模型，以获得有效的潜伏空间探索，导致更可靠的置信度图。通过拟议的策略，未标记的图像可以有效地参与模型培训。实验结果表明，与原始培训数据集仅有1/16的注释，与最先进的完全监督模型相比，所提出的解决方案实现了竞争性能。

半监督学习方法已成为对打击获得大量注释数据的挑战的活跃研究领域。为了提高半监督学习方法表现的目标，我们提出了一种新颖的框架，Hiematch，一种半监督方法，利用分层信息来降低标签成本并表现以及vanilla半监督学习方法。分层信息通常是具有细粒标签的粗标签（例如，啄木鸟）的粗标签（例如，啄木鸟）的现有知识（例如，柔软的啄木鸟或金朝啄木鸟）。但是，尚未探讨使用使用粗类标签来改进半监督技术的监督。在没有细粒度的标签的情况下，Himatch利用标签层次结构，并使用粗级标签作为弱监控信号。此外，Himatch是一种改进任何半熟的学习框架的通用方法，我们使用我们的结果在最近的最先进的技术Mixmatch和Fixmatch上展示了这一点。我们评估了在两个基准数据集，即CiFar-100和Nabirds上的Himatch疗效。与MixMatch相比，HOMACHACT可以在CIFAR-100上减少50％的粒度标签50％的用量，仅在前1个精度的边缘下降0.59％。代码：https://github.com/07agarg/hiermatch.

Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, we argue that these benchmarks fail to address many issues that SSL algorithms would face in real-world applications. After creating a unified reimplementation of various widely-used SSL techniques, we test them in a suite of experiments designed to address these issues. We find that the performance of simple baselines which do not use unlabeled data is often underreported, SSL methods differ in sensitivity to the amount of labeled and unlabeled data, and performance can degrade substantially when the unlabeled dataset contains out-ofdistribution examples. To help guide SSL research towards real-world applicability, we make our unified reimplemention and evaluation platform publicly available. 2 * Equal contribution 2 https://github.com/brain-research/realistic-ssl-evaluation 32nd Conference on Neural Information Processing Systems (NeurIPS 2018),

这项工作的作者提出了最近半监督的学习方法和相关作品的概述。尽管神经网络在各种应用中取得了显着的成功，但很少有强大的约束，包括需要大量标记数据。因此，半监督的学习是一种学习方案，其中稀缺标签和大量未标记的数据被用于训练模型（例如，深度神经网络）变得越来越重要。基于半监督学习的关键假设，这是多种假设，集群假设和连续性假设，工作回顾了最近的半监督学习方法。特别是，主要讨论了在半监督学习环境中使用深神网络的方法。此外，现有的作品首先是根据基本思想进行了分类并解释的，然后详细介绍了统一上述思想的整体方法。

半监督学习（SSL）是规避建立高性能模型的昂贵标签成本的最有前途的范例之一。大多数现有的SSL方法常规假定标记和未标记的数据是从相同（类）分布中绘制的。但是，在实践中，未标记的数据可能包括课外样本；那些不能从标签数据中的封闭类中的单热编码标签，即未标记的数据是开放设置。在本文中，我们介绍了Opencos，这是一种基于最新的自我监督视觉表示学习框架来处理这种现实的半监督学习方案。具体而言，我们首先观察到，可以通过自我监督的对比度学习有效地识别开放式未标记数据集中的类外样本。然后，Opencos利用此信息来克服现有的最新半监督方法中的故障模式，通过利用一式旋转伪标签和软标签来为已识别的识别和外部未标记的标签数据分别。我们广泛的实验结果表明了Opencos的有效性，可以修复最新的半监督方法，适合涉及开放式无标记数据的各种情况。

最近，出现了许多新的半监督学习方法。随着时间的流逝，ImageNet和类似数据集的准确性提高，尚未探索自然图像分类以外的任务的性能。大多数半监督的学习方法都依赖于精心设计的数据增强管道，该数据无法转移，用于在其他域的图像上学习。在这项工作中，我们提出了一种半监督的学习方法，该方法自动为特定数据集选择了最有效的数据增强策略。我们以FixMatch方法为基础，并通过增强元学习扩展它。在分类培训之前，在额外的培训中学习了增强，并利用双层优化，以优化增强政策并最大程度地提高准确性。我们在两个特定领域的数据集上评估我们的方法，其中包含卫星图像和手绘草图，并获得最新的结果。我们在消融中进一步研究与学习增强策略相关的不同参数，并展示了如何使用策略学习将增强功能调整到ImageNet之外的数据集中。

This work tackles the problem of semi-supervised learning of image classifiers. Our main insight is that the field of semi-supervised learning can benefit from the quickly advancing field of self-supervised visual representation learning. Unifying these two approaches, we propose the framework of self-supervised semi-supervised learning (S 4 L) and use it to derive two novel semi-supervised image classification methods. We demonstrate the effectiveness of these methods in comparison to both carefully tuned baselines, and existing semi-supervised learning methods. We then show that S 4 L and existing semi-supervised methods can be jointly trained, yielding a new state-of-the-art result on semi-supervised ILSVRC-2012 with 10% of labels.

我们提出了Parse，这是一种新颖的半监督结构，用于学习强大的脑电图表现以进行情感识别。为了减少大量未标记数据与标记数据有限的潜在分布不匹配，Parse使用成对表示对准。首先，我们的模型执行数据增强，然后标签猜测大量原始和增强的未标记数据。然后将其锐化的标签和标记数据的凸组合锐化。最后，进行表示对准和情感分类。为了严格测试我们的模型，我们将解析与我们实施并适应脑电图学习的几种最先进的半监督方法进行了比较。我们对四个基于公共EEG的情绪识别数据集，种子，种子IV，种子V和Amigos（价和唤醒）进行这些实验。该实验表明，我们提出的框架在种子，种子-IV和Amigos（Valence）中的标记样品有限的情况下，取得了总体最佳效果，同时接近种子V和Amigos中的总体最佳结果（达到第二好） （唤醒）。分析表明，我们的成对表示对齐方式通过减少未标记数据和标记数据之间的分布比对来大大提高性能，尤其是当每类仅1个样本被标记时。

Semi-supervised learning (SSL) has achieved great success in leveraging a large amount of unlabeled data to learn a promising classifier. A popular approach is pseudo-labeling that generates pseudo labels only for those unlabeled data with high-confidence predictions. As for the low-confidence ones, existing methods often simply discard them because these unreliable pseudo labels may mislead the model. Nevertheless, we highlight that these data with low-confidence pseudo labels can be still beneficial to the training process. Specifically, although the class with the highest probability in the prediction is unreliable, we can assume that this sample is very unlikely to belong to the classes with the lowest probabilities. In this way, these data can be also very informative if we can effectively exploit these complementary labels, i.e., the classes that a sample does not belong to. Inspired by this, we propose a novel Contrastive Complementary Labeling (CCL) method that constructs a large number of reliable negative pairs based on the complementary labels and adopts contrastive learning to make use of all the unlabeled data. Extensive experiments demonstrate that CCL significantly improves the performance on top of existing methods. More critically, our CCL is particularly effective under the label-scarce settings. For example, we yield an improvement of 2.43% over FixMatch on CIFAR-10 only with 40 labeled data.

Semi-supervised learning (SSL) has made significant strides in the field of remote sensing. Finding a large number of labeled datasets for SSL methods is uncommon, and manually labeling datasets is expensive and time-consuming. Furthermore, accurately identifying remote sensing satellite images is more complicated than it is for conventional images. Class-imbalanced datasets are another prevalent phenomenon, and models trained on these become biased towards the majority classes. This becomes a critical issue with an SSL model's subpar performance. We aim to address the issue of labeling unlabeled data and also solve the model bias problem due to imbalanced datasets while achieving better accuracy. To accomplish this, we create "artificial" labels and train a model to have reasonable accuracy. We iteratively redistribute the classes through resampling using a distribution alignment technique. We use a variety of class imbalanced satellite image datasets: EuroSAT, UCM, and WHU-RS19. On UCM balanced dataset, our method outperforms previous methods MSMatch and FixMatch by 1.21% and 0.6%, respectively. For imbalanced EuroSAT, our method outperforms MSMatch and FixMatch by 1.08% and 1%, respectively. Our approach significantly lessens the requirement for labeled data, consistently outperforms alternative approaches, and resolves the issue of model bias caused by class imbalance in datasets.

机器学习模型通常会遇到与训练分布不同的样本。无法识别分布（OOD）样本，因此将该样本分配给课堂标签会显着损害模​​型的可靠性。由于其对在开放世界中的安全部署模型的重要性，该问题引起了重大关注。由于对所有可能的未知分布进行建模的棘手性，检测OOD样品是具有挑战性的。迄今为止，一些研究领域解决了检测陌生样本的问题，包括异常检测，新颖性检测，一级学习，开放式识别识别和分布外检测。尽管有相似和共同的概念，但分别分布，开放式检测和异常检测已被独立研究。因此，这些研究途径尚未交叉授粉，创造了研究障碍。尽管某些调查打算概述这些方法，但它们似乎仅关注特定领域，而无需检查不同领域之间的关系。这项调查旨在在确定其共同点的同时，对各个领域的众多著名作品进行跨域和全面的审查。研究人员可以从不同领域的研究进展概述中受益，并协同发展未来的方法。此外，据我们所知，虽然进行异常检测或单级学习进行了调查，但没有关于分布外检测的全面或最新的调查，我们的调查可广泛涵盖。最后，有了统一的跨域视角，我们讨论并阐明了未来的研究线，打算将这些领域更加紧密地融为一体。

Semi-supervised learning lately has shown much promise in improving deep learning models when labeled data is scarce. Common among recent approaches is the use of consistency training on a large amount of unlabeled data to constrain model predictions to be invariant to input noise. In this work, we present a new perspective on how to effectively noise unlabeled examples and argue that the quality of noising, specifically those produced by advanced data augmentation methods, plays a crucial role in semi-supervised learning. By substituting simple noising operations with advanced data augmentation methods such as RandAugment and back-translation, our method brings substantial improvements across six language and three vision tasks under the same consistency training framework. On the IMDb text classification dataset, with only 20 labeled examples, our method achieves an error rate of 4.20, outperforming the state-of-the-art model trained on 25,000 labeled examples. On a standard semi-supervised learning benchmark, CIFAR-10, our method outperforms all previous approaches and achieves an error rate of 5.43 with only 250 examples. Our method also combines well with transfer learning, e.g., when finetuning from BERT, and yields improvements in high-data regime, such as ImageNet, whether when there is only 10% labeled data or when a full labeled set with 1.3M extra unlabeled examples is used. 1

Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model's performance. This domain has seen fast progress recently, at the cost of requiring more complex methods. In this paper we propose FixMatch, an algorithm that is a significant simplification of existing SSL methods. FixMatch first generates pseudo-labels using the model's predictions on weaklyaugmented unlabeled images. For a given image, the pseudo-label is only retained if the model produces a high-confidence prediction. The model is then trained to predict the pseudo-label when fed a strongly-augmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94.93% accuracy on CIFAR-10 with 250 labels and 88.61% accuracy with 40 -just 4 labels per class. We carry out an extensive ablation study to tease apart the experimental factors that are most important to FixMatch's success. The code is available at https://github.com/google-research/fixmatch.

一个常见的分类任务情况是，有大量数据可用于培训，但只有一小部分用类标签注释。在这种情况下，半监督培训的目的是通过利用标记数据，而且从大量未标记的数据中提高分类准确性。最近的作品通过探索不同标记和未标记数据的不同增强性数据之间的一致性约束，从而取得了重大改进。遵循这条路径，我们提出了一个新颖的无监督目标，该目标侧重于彼此相似的高置信度未标记的数据之间所研究的关系较少。新提出的对损失最大程度地减少了高置信度伪伪标签之间的统计距离，其相似性高于一定阈值。我们提出的简单算法将对损失与MixMatch家族开发的技术结合在一起，显示出比以前在CIFAR-100和MINI-IMAGENET上的算法的显着性能增长，并且与CIFAR-的最先进方法相当。 10和SVHN。此外，简单还优于传输学习设置中最新方法，其中模型是由在ImainEnet或域内实现的权重初始化的。该代码可在github.com/zijian-hu/simple上获得。

Despite significant advances, the performance of state-of-the-art continual learning approaches hinges on the unrealistic scenario of fully labeled data. In this paper, we tackle this challenge and propose an approach for continual semi-supervised learning -- a setting where not all the data samples are labeled. An underlying issue in this scenario is the model forgetting representations of unlabeled data and overfitting the labeled ones. We leverage the power of nearest-neighbor classifiers to non-linearly partition the feature space and learn a strong representation for the current task, as well as distill relevant information from previous tasks. We perform a thorough experimental evaluation and show that our method outperforms all the existing approaches by large margins, setting a strong state of the art on the continual semi-supervised learning paradigm. For example, on CIFAR100 we surpass several others even when using at least 30 times less supervision (0.8% vs. 25% of annotations).

We present Self Meta Pseudo Labels, a novel semi-supervised learning method similar to Meta Pseudo Labels but without the teacher model. We introduce a novel way to use a single model for both generating pseudo labels and classification, allowing us to store only one model in memory instead of two. Our method attains similar performance to the Meta Pseudo Labels method while drastically reducing memory usage.

半监督学习（SSL）在稀缺标记的数据时取得了长足的进步，但未标记的数据丰富。至关重要的是，最近的工作假设这种未标记的数据是从与标记数据相同的分布中汲取的。在这项工作中，我们表明，在存在未标记的辅助数据的情况下，最先进的SSL算法在性能下遭受了降解，这些数据不一定具有与标签集相同的类别分布。我们将此问题称为辅助-SSL，并提出了AuxMix，这是一种利用自我监督的学习任务来学习通用功能，以掩盖与标记的集合在语义上相似的辅助数据。我们还建议通过最大化不同辅助样品的预测熵来正规化学习。当在CIFAR10数据集中培训带有4K标记的样品时，我们在Resnet-50型号上显示了5％的改善，并且从Tiny-ImageNet数据集中绘制所有未标记的数据。我们报告了几个数据集的竞争结果，并进行消融研究。