无法保证专家注释的培训数据的质量，甚至对于由分发样本组成的非IID数据（即，分布式和分布式样本都具有不同的分布），更是如此。 。专家可能会错误地注释与分布样本相同的分发样品，从而产生不可信的地面真相标签。学习这种非IID数据混合与不信任标签的分布样品混合在一起，既浅层和深度学习都有显着挑战，没有报告相关工作。可以识别样本的值得信赖的互补标签，指示其不属于哪些类，因为除分布外样品和分布外样品都不属于类别外，除了与地面真实标签相对应的类别。有了这个见解，我们提出了一种新颖的\ textit {灰色学习}方法，可以从非IID数据中学习具有分布式和分离外样品的非IID数据。由于训练样本的不确定分布，我们拒绝了低信心输入的互补标签，同时将高信心输入映射到培训中的地面真相标签。在统计学习理论的基础上，我们得出了概括误差，该误差表明灰色学习在非IID数据上实现了紧密的束缚。广泛的实验表明，我们的方法对可靠统计的替代方法提供了重大改进。

深度神经网络只会学会将分布输入映射到其在训练阶段的相应地面真实标签，而不会区分分配样本与分布情况。这是由于所有样品都是独立且分布相同而没有分布区别的假设。因此，从分布样品中学到的一个预处理的网络将分布的样本视为分布，并在测试阶段对它们进行高信心预测。为了解决这个问题，我们从培训分配样本附近分布中绘制出分布的样本，以学习拒绝对分数输入的预测。通过假设通过混合多个分发样品而生成的分布样本不会共享其组成部分相同的类别，从而引入了\ textit {跨级附近分布}。因此，我们通过从跨级附近分布中得出的分布样本对列表进行列表来提高预审慎的网络的可区分性，其中每个分布输入输入都对应于互补标签。各种内部/分布数据集的实验表明，所提出的方法在提高区分内部和分发样品的能力方面显着优于现有方法。

当分布（ID）样品与分布外（OOD）样本之间存在差异时，对ID样品进行训练的深神经网络遭受了OOD样品的高信心预测。这主要是由无法使用的OOD样品引起的，以限制培训过程中的网络。为了提高深网的OOD敏感性，几种最先进的方法将其他现实世界数据集的样本作为OOD样本引入训练过程，并将手动确定的标签分配给这些OOD样本。但是，他们牺牲了分类准确性，因为OOD样品的不可靠标记会破坏ID分类。为了平衡ID的概括和OOD检测，要解决的主要挑战是使OOD样本与ID兼容，这在本文中由我们提议的\ textit {监督适应}方法解决，以定义OOD样本的适应性监督信息。首先，通过通过共同信息来测量ID样本及其标签之间的依赖关系，我们根据所有类别的负概率揭示了监督信息的形式。其次，在通过解决多个二进制回归问题来探索ID和OOD样本之间的数据相关性之后，我们估算了监督信息以使ID类更可分离。我们使用两个ID数据集和11个OOD数据集对四个高级网络体系结构进行实验，以证明我们的监督适应方法在实现ID分类能力和OOD检测能力方面的平衡效果。

In deep neural learning, a discriminator trained on in-distribution (ID) samples may make high-confidence predictions on out-of-distribution (OOD) samples. This triggers a significant matter for robust, trustworthy and safe deep learning. The issue is primarily caused by the limited ID samples observable in training the discriminator when OOD samples are unavailable. We propose a general approach for \textit{fine-tuning discriminators by implicit generators} (FIG). FIG is grounded on information theory and applicable to standard discriminators without retraining. It improves the ability of a standard discriminator in distinguishing ID and OOD samples by generating and penalizing its specific OOD samples. According to the Shannon entropy, an energy-based implicit generator is inferred from a discriminator without extra training costs. Then, a Langevin dynamic sampler draws specific OOD samples for the implicit generator. Lastly, we design a regularizer fitting the design principle of the implicit generator to induce high entropy on those generated OOD samples. The experiments on different networks and datasets demonstrate that FIG achieves the state-of-the-art OOD detection performance.

为了对分布样本进行分类，深层神经网络学习标签 - 歧义表示表示，但是，根据信息瓶颈，这不一定是分布歧视性的。因此，训练有素的网络可以为与分布样本不同的分布样本分配出意外的高信任预测。具体而言，网络从分布样本中提取与标签相关的信息，以学习标签 - 歧义表示表示，但丢弃了与标签相关的弱信息。因此，网络用最小标签敏感的信息作为分布样本将分布式样品视为分布样品。根据分布样本的不同信息性属性，双重表示学习（DRL）方法学习与分配样本的标签无关的分布不同的表示形式，并结合了标签和分布歧义性歧视性。检测到分布样本的表示。对于标签 - 歧义表示形式，DRL通过隐式约束构建了互补分布不同的表示表示，即整合了不同的中间表示，其中中间表示与标签 - 歧义表示表示具有更高的权重。实验表明，DRL的表现优于分布外检测的最新方法。

最近关于使用嘈杂标签的学习的研究通过利用小型干净数据集来显示出色的性能。特别是，基于模型不可知的元学习的标签校正方法进一步提高了性能，通过纠正了嘈杂的标签。但是，标签错误矫予没有保障措施，导致不可避免的性能下降。此外，每个训练步骤都需要至少三个背部传播，显着减慢训练速度。为了缓解这些问题，我们提出了一种强大而有效的方法，可以在飞行中学习标签转换矩阵。采用转换矩阵使分类器对所有校正样本持怀疑态度，这减轻了错误的错误问题。我们还介绍了一个双头架构，以便在单个反向传播中有效地估计标签转换矩阵，使得估计的矩阵紧密地遵循由标签校正引起的移位噪声分布。广泛的实验表明，我们的方法在训练效率方面表现出比现有方法相当或更好的准确性。

Current deep neural networks (DNNs) can easily overfit to biased training data with corrupted labels or class imbalance. Sample re-weighting strategy is commonly used to alleviate this issue by designing a weighting function mapping from training loss to sample weight, and then iterating between weight recalculating and classifier updating. Current approaches, however, need manually pre-specify the weighting function as well as its additional hyper-parameters. It makes them fairly hard to be generally applied in practice due to the significant variation of proper weighting schemes relying on the investigated problem and training data. To address this issue, we propose a method capable of adaptively learning an explicit weighting function directly from data. The weighting function is an MLP with one hidden layer, constituting a universal approximator to almost any continuous functions, making the method able to fit a wide range of weighting functions including those assumed in conventional research. Guided by a small amount of unbiased meta-data, the parameters of the weighting function can be finely updated simultaneously with the learning process of the classifiers. Synthetic and real experiments substantiate the capability of our method for achieving proper weighting functions in class imbalance and noisy label cases, fully complying with the common settings in traditional methods, and more complicated scenarios beyond conventional cases. This naturally leads to its better accuracy than other state-of-the-art methods. Source code is available at https://github.com/xjtushujun/meta-weight-net. * Corresponding author. 1 We call the training data biased when they are generated from a joint sample-label distribution deviating from the distribution of evaluation/test set [1].

Recent deep networks are capable of memorizing the entire data even when the labels are completely random. To overcome the overfitting on corrupted labels, we propose a novel technique of learning another neural network, called Men-torNet, to supervise the training of the base deep networks, namely, StudentNet. During training, MentorNet provides a curriculum (sample weighting scheme) for StudentNet to focus on the sample the label of which is probably correct. Unlike the existing curriculum that is usually predefined by human experts, MentorNet learns a data-driven curriculum dynamically with StudentNet. Experimental results demonstrate that our approach can significantly improve the generalization performance of deep networks trained on corrupted training data. Notably, to the best of our knowledge, we achieve the best-published result on We-bVision, a large benchmark containing 2.2 million images of real-world noisy labels. The code are at https://github.com/google/mentornet.

Deep neural networks (DNNs) have achieved tremendous success in a variety of applications across many disciplines. Yet, their superior performance comes with the expensive cost of requiring correctly annotated large-scale datasets. Moreover, due to DNNs' rich capacity, errors in training labels can hamper performance. To combat this problem, mean absolute error (MAE) has recently been proposed as a noise-robust alternative to the commonly-used categorical cross entropy (CCE) loss. However, as we show in this paper, MAE can perform poorly with DNNs and challenging datasets. Here, we present a theoretically grounded set of noise-robust loss functions that can be seen as a generalization of MAE and CCE. Proposed loss functions can be readily applied with any existing DNN architecture and algorithm, while yielding good performance in a wide range of noisy label scenarios. We report results from experiments conducted with CIFAR-10, CIFAR-100 and FASHION-MNIST datasets and synthetically generated noisy labels.

近年来，已取得了巨大进展，以通过半监督学习（SSL）来纳入未标记的数据来克服效率低下的监督问题。大多数最先进的模型是基于对未标记的数据追求一致的模型预测的想法，该模型被称为输入噪声，这称为一致性正则化。尽管如此，对其成功的原因缺乏理论上的见解。为了弥合理论和实际结果之间的差距，我们在本文中提出了SSL的最坏情况一致性正则化技术。具体而言，我们首先提出了针对SSL的概括，该概括由分别在标记和未标记的训练数据上观察到的经验损失项组成。在这种界限的激励下，我们得出了一个SSL目标，该目标可最大程度地减少原始未标记的样本与其多重增强变体之间最大的不一致性。然后，我们提供了一种简单但有效的算法来解决提出的最小问题，从理论上证明它会收敛到固定点。五个流行基准数据集的实验验证了我们提出的方法的有效性。

监督学习的关键假设是培训和测试数据遵循相同的概率分布。然而，这种基本假设在实践中并不总是满足，例如，由于不断变化的环境，样本选择偏差，隐私问题或高标签成本。转移学习（TL）放松这种假设，并允许我们在分销班次下学习。通常依赖于重要性加权的经典TL方法 - 基于根据重要性（即测试过度训练密度比率）的训练损失培训预测器。然而，由于现实世界机器学习任务变得越来越复杂，高维和动态，探讨了新的新方法，以应对这些挑战最近。在本文中，在介绍基于重要性加权的TL基础之后，我们根据关节和动态重要预测估计审查最近的进步。此外，我们介绍一种因果机制转移方法，该方法包含T1中的因果结构。最后，我们讨论了TL研究的未来观点。

深度学习在大量大数据的帮助下取得了众多域中的显着成功。然而，由于许多真实情景中缺乏高质量标签，数据标签的质量是一个问题。由于嘈杂的标签严重降低了深度神经网络的泛化表现，从嘈杂的标签（强大的培训）学习是在现代深度学习应用中成为一项重要任务。在本调查中，我们首先从监督的学习角度描述了与标签噪声学习的问题。接下来，我们提供62项最先进的培训方法的全面审查，所有这些培训方法都按照其方法论差异分为五个群体，其次是用于评估其优越性的六种性质的系统比较。随后，我们对噪声速率估计进行深入分析，并总结了通常使用的评估方法，包括公共噪声数据集和评估度量。最后，我们提出了几个有前途的研究方向，可以作为未来研究的指导。所有内容将在https://github.com/songhwanjun/awesome-noisy-labels提供。

深度神经网络的成功在很大程度上取决于大量高质量注释的数据的可用性，但是这些数据很难或昂贵。由此产生的标签可能是类别不平衡，嘈杂或人类偏见。从不完美注释的数据集中学习无偏分类模型是一项挑战，我们通常会遭受过度拟合或不足的折磨。在这项工作中，我们彻底研究了流行的软马克斯损失和基于保证金的损失，并提供了一种可行的方法来加强通过最大化最小样本余量来限制的概括误差。我们为此目的进一步得出了最佳条件，该条件指示了类原型应锚定的方式。通过理论分析的激励，我们提出了一种简单但有效的方法，即原型锚定学习（PAL），可以轻松地将其纳入各种基于学习的分类方案中以处理不完美的注释。我们通过对合成和现实世界数据集进行广泛的实验来验证PAL对班级不平衡学习和降低噪声学习的有效性。

In the presence of noisy labels, designing robust loss functions is critical for securing the generalization performance of deep neural networks. Cross Entropy (CE) loss has been shown to be not robust to noisy labels due to its unboundedness. To alleviate this issue, existing works typically design specialized robust losses with the symmetric condition, which usually lead to the underfitting issue. In this paper, our key idea is to induce a loss bound at the logit level, thus universally enhancing the noise robustness of existing losses. Specifically, we propose logit clipping (LogitClip), which clamps the norm of the logit vector to ensure that it is upper bounded by a constant. In this manner, CE loss equipped with our LogitClip method is effectively bounded, mitigating the overfitting to examples with noisy labels. Moreover, we present theoretical analyses to certify the noise-tolerant ability of LogitClip. Extensive experiments show that LogitClip not only significantly improves the noise robustness of CE loss, but also broadly enhances the generalization performance of popular robust losses.

Training accurate deep neural networks (DNNs) in the presence of noisy labels is an important and challenging task. Though a number of approaches have been proposed for learning with noisy labels, many open issues remain. In this paper, we show that DNN learning with Cross Entropy (CE) exhibits overfitting to noisy labels on some classes ("easy" classes), but more surprisingly, it also suffers from significant under learning on some other classes ("hard" classes). Intuitively, CE requires an extra term to facilitate learning of hard classes, and more importantly, this term should be noise tolerant, so as to avoid overfitting to noisy labels. Inspired by the symmetric KL-divergence, we propose the approach of Symmetric cross entropy Learning (SL), boosting CE symmetrically with a noise robust counterpart Reverse Cross Entropy (RCE). Our proposed SL approach simultaneously addresses both the under learning and overfitting problem of CE in the presence of noisy labels. We provide a theoretical analysis of SL and also empirically show, on a range of benchmark and real-world datasets, that SL outperforms state-of-the-art methods. We also show that SL can be easily incorporated into existing methods in order to further enhance their performance.

Deep neural networks may easily memorize noisy labels present in real-world data, which degrades their ability to generalize. It is therefore important to track and evaluate the robustness of models against noisy label memorization. We propose a metric, called susceptibility, to gauge such memorization for neural networks. Susceptibility is simple and easy to compute during training. Moreover, it does not require access to ground-truth labels and it only uses unlabeled data. We empirically show the effectiveness of our metric in tracking memorization on various architectures and datasets and provide theoretical insights into the design of the susceptibility metric. Finally, we show through extensive experiments on datasets with synthetic and real-world label noise that one can utilize susceptibility and the overall training accuracy to distinguish models that maintain a low memorization on the training set and generalize well to unseen clean data.

深入学习在现代分类任务中取得了许多突破。已经提出了众多架构用于不同的数据结构，但是当涉及丢失功能时，跨熵损失是主要的选择。最近，若干替代损失已经看到了深度分类器的恢复利益。特别是，经验证据似乎促进了方形损失，但仍然缺乏理论效果。在这项工作中，我们通过系统地研究了在神经切线内核（NTK）制度中的过度分化的神经网络的表现方式来促进对分类方面损失的理论理解。揭示了关于泛化误差，鲁棒性和校准错误的有趣特性。根据课程是否可分离，我们考虑两种情况。在一般的不可分类案例中，为错误分类率和校准误差建立快速收敛速率。当类是可分离的时，错误分类率改善了速度快。此外，经过证明得到的余量被证明是低于零的较低，提供了鲁棒性的理论保证。我们希望我们的调查结果超出NTK制度并转化为实际设置。为此，我们对实际神经网络进行广泛的实证研究，展示了合成低维数据和真实图像数据中方损的有效性。与跨熵相比，方形损耗具有可比的概括误差，但具有明显的鲁棒性和模型校准的优点。

部分标签学习（PLL）是一个典型的弱监督学习框架，每个培训实例都与候选标签集相关联，其中只有一个标签是有效的。为了解决PLL问题，通常方法试图通过使用先验知识（例如培训数据的结构信息）或以自训练方式提炼模型输出来对候选人集进行歧义。不幸的是，由于在模型训练的早期阶段缺乏先前的信息或不可靠的预测，这些方法通常无法获得有利的性能。在本文中，我们提出了一个新的针对部分标签学习的框架，该框架具有元客观指导性的歧义（MOGD），该框架旨在通过在小验证集中求解元目标来从设置的候选标签中恢复地面真相标签。具体而言，为了减轻假阳性标签的负面影响，我们根据验证集的元损失重新权重。然后，分类器通过最大程度地减少加权交叉熵损失来训练。通过使用普通SGD优化器的各种深网络可以轻松实现所提出的方法。从理论上讲，我们证明了元目标的收敛属性，并得出了所提出方法的估计误差界限。在各种基准数据集和实际PLL数据集上进行的广泛实验表明，与最先进的方法相比，所提出的方法可以实现合理的性能。

Partial label learning (PLL) is a typical weakly supervised learning, where each sample is associated with a set of candidate labels. The basic assumption of PLL is that the ground-truth label must reside in the candidate set. However, this assumption may not be satisfied due to the unprofessional judgment of the annotators, thus limiting the practical application of PLL. In this paper, we relax this assumption and focus on a more general problem, noisy PLL, where the ground-truth label may not exist in the candidate set. To address this challenging problem, we further propose a novel framework called "Automatic Refinement Network (ARNet)". Our method consists of multiple rounds. In each round, we purify the noisy samples through two key modules, i.e., noisy sample detection and label correction. To guarantee the performance of these modules, we start with warm-up training and automatically select the appropriate correction epoch. Meanwhile, we exploit data augmentation to further reduce prediction errors in ARNet. Through theoretical analysis, we prove that our method is able to reduce the noise level of the dataset and eventually approximate the Bayes optimal classifier. To verify the effectiveness of ARNet, we conduct experiments on multiple benchmark datasets. Experimental results demonstrate that our ARNet is superior to existing state-of-the-art approaches in noisy PLL. Our code will be made public soon.

The standard empirical risk minimization (ERM) can underperform on certain minority groups (i.e., waterbirds in lands or landbirds in water) due to the spurious correlation between the input and its label. Several studies have improved the worst-group accuracy by focusing on the high-loss samples. The hypothesis behind this is that such high-loss samples are \textit{spurious-cue-free} (SCF) samples. However, these approaches can be problematic since the high-loss samples may also be samples with noisy labels in the real-world scenarios. To resolve this issue, we utilize the predictive uncertainty of a model to improve the worst-group accuracy under noisy labels. To motivate this, we theoretically show that the high-uncertainty samples are the SCF samples in the binary classification problem. This theoretical result implies that the predictive uncertainty is an adequate indicator to identify SCF samples in a noisy label setting. Motivated from this, we propose a novel ENtropy based Debiasing (END) framework that prevents models from learning the spurious cues while being robust to the noisy labels. In the END framework, we first train the \textit{identification model} to obtain the SCF samples from a training set using its predictive uncertainty. Then, another model is trained on the dataset augmented with an oversampled SCF set. The experimental results show that our END framework outperforms other strong baselines on several real-world benchmarks that consider both the noisy labels and the spurious-cues.