Data in vision domain often exhibit highly-skewed class distribution, i.e., most data belong to a few majority classes, while the minority classes only contain a scarce amount of instances. To mitigate this issue, contemporary classification methods based on deep convolutional neural network (CNN) typically follow classic strategies such as class re-sampling or cost-sensitive training. In this paper, we conduct extensive and systematic experiments to validate the effectiveness of these classic schemes for representation learning on class-imbalanced data. We further demonstrate that more discriminative deep representation can be learned by enforcing a deep network to maintain both intercluster and inter-class margins. This tighter constraint effectively reduces the class imbalance inherent in the local data neighborhood. We show that the margins can be easily deployed in standard deep learning framework through quintuplet instance sampling and the associated triple-header hinge loss. The representation learned by our approach, when combined with a simple k-nearest neighbor (kNN) algorithm, shows significant improvements over existing methods on both high-and low-level vision classification tasks that exhibit imbalanced class distribution.

Recent years witnessed the breakthrough of face recognition with deep convolutional neural networks. Dozens of papers in the field of FR are published every year. Some of them were applied in the industrial community and played an important role in human life such as device unlock, mobile payment, and so on. This paper provides an introduction to face recognition, including its history, pipeline, algorithms based on conventional manually designed features or deep learning, mainstream training, evaluation datasets, and related applications. We have analyzed and compared state-of-the-art works as many as possible, and also carefully designed a set of experiments to find the effect of backbone size and data distribution. This survey is a material of the tutorial named The Practical Face Recognition Technology in the Industrial World in the FG2023.

The accuracy of k-nearest neighbor (kNN) classification depends significantly on the metric used to compute distances between different examples. In this paper, we show how to learn a Mahalanobis distance metric for kNN classification from labeled examples. The Mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes kNN classification using Euclidean distances. In our approach, the metric is trained with the goal that the k-nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. As in support vector machines (SVMs), the margin criterion leads to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our approach requires no modification or extension for problems in multiway (as opposed to binary) classification. In our framework, the Mahalanobis distance metric is obtained as the solution to a semidefinite program. On several data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification. Sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster. We show how to learn and combine these local metrics in a globally integrated manner.

数据不平衡，即来自不同课程的培训观测数量之间的歧视，仍然是影响当代机器学习的最重要挑战之一。数据预处理技术可以减少数据不平衡对传统分类算法的负面影响，可以减少操纵训练数据以人为地降低不平衡程度的方法。然而，现有的数据预处理技术，特别是粉迹及其衍生物构成最普遍的数据预处理的范式，往往易于各种数据难度因素。这部分是由于原始粉碎算法不利用有关多数类观察的信息的事实。本文的重点是利用少数群体和多数阶级的分布的信息，自然地发展新的数据重采样策略。本文总结了12个研究论文的内容，专注于所提出的二进制数据重采采样策略，它们与多级环境的翻译，以及对组织病理数据分类问题的实际应用。

大坝水库在实现可持续发展目标和全球气候目标方面发挥着重要作用。但是，特别是对于小型水坝水库，其地理位置缺乏一致的数据。为了解决此数据差距，一种有前途的方法是根据全球可用的遥感图像进行自动水坝水库提取。它可以被认为是水体提取的精细颗粒任务，涉及在图像中提取水区，然后将水坝储层与天然水体分开。我们提出了一种基于新型的深神经网络（DNN）管道，该管道将大坝水库提取到水体分割和大坝储层识别中。首先将水体与分割模型中的背景土地分开，然后将每个水体预测为大坝储层或分类模型中的天然水体。对于以前的一步，将跨图像的点级度量学习注入分段模型，以解决水域和土地区域之间的轮廓模棱两可。对于后一个步骤，将带有簇的三重态的先前引导的度量学习注入到分类模型中，以根据储层簇在细粒度中优化图像嵌入空间。为了促进未来的研究，我们建立了一个带有地球图像数据的基准数据集，并从西非和印度的河流盆地标记为人类标记的水库。在水体分割任务，水坝水库识别任务和关节坝储层提取任务中，对这个基准进行了广泛的实验。将我们的方法与艺术方法的方法进行比较时，已经在各自的任务中观察到了卓越的性能。

深度学习模型记住培训数据，这损害了他们推广到代表性不足的课程的能力。我们从经验上研究了卷积神经网络对图像数据不平衡数据的内部表示，并测量了训练和测试集中模型特征嵌入之间的概括差距，这表明该差距对于少数类别的差异更大。这个洞察力使我们能够为不平衡数据设计有效的三相CNN培训框架。该框架涉及训练网络端到端的数据不平衡数据以学习准确的功能嵌入，在学习的嵌入式空间中执行数据增强以平衡火车分布，并在嵌入式平衡的培训数据上微调分类器头。我们建议在培训框架中使用广泛的过采样（EOS）作为数据增强技术。 EOS形成合成训练实例，作为少数族类样本与其最近的敌人之间的凸组合，以减少概括差距。提出的框架提高了与不平衡学习中常用的领先成本敏感和重新采样方法的准确性。此外，它比标准数据预处理方法（例如SMOTE和基于GAN的过采样）更有效，因为它需要更少的参数和更少的训练时间。

学习遥感（RS）图像之间的相似性形成基于内容的RS图像检索（CBIR）的基础。最近，将图像的语义相似性映射到嵌入（度量标准）空间的深度度量学习方法已经发现非常流行。学习公制空间的常见方法依赖于将与作为锚称为锚的参考图像的类似（正）和不同（负）图像的三胞胎的选择。选择三胞胎是一个难以为多标签RS CBIR的困难任务，其中每个训练图像由多个类标签注释。为了解决这个问题，在本文中，我们提出了一种在为多标签RS CBIR问题定义的深神经网络（DNN）的框架中提出了一种新颖的三联样品采样方法。该方法基于两个主要步骤选择一小部分最多代表性和信息性三元组。在第一步中，使用迭代算法从当前迷你批量选择在嵌入空间中彼此多样化的一组锚。在第二步中，通过基于新颖的策略评估彼此之间的图像的相关性，硬度和多样性来选择不同的正面和负图像。在两个多标签基准档案上获得的实验结果表明，在DNN的上下文中选择最具信息丰富和代表性的三胞胎，导致：i）降低DNN训练阶段的计算复杂性，而性能没有任何显着损失; ii）由于信息性三元组允许快速收敛，因此学习速度的增加。所提出的方法的代码在https://git.tu-berlin.de/rsim/image-reetrieval-from-tropls上公开使用。

In recent years, benefiting from the expressive power of Graph Convolutional Networks (GCNs), significant breakthroughs have been made in face clustering area. However, rare attention has been paid to GCN-based clustering on imbalanced data. Although imbalance problem has been extensively studied, the impact of imbalanced data on GCN- based linkage prediction task is quite different, which would cause problems in two aspects: imbalanced linkage labels and biased graph representations. The former is similar to that in classic image classification task, but the latter is a particular problem in GCN-based clustering via linkage prediction. Significantly biased graph representations in training can cause catastrophic over-fitting of a GCN model. To tackle these challenges, we propose a linkage-based doubly imbalanced graph learning framework for face clustering. In this framework, we evaluate the feasibility of those existing methods for imbalanced image classification problem on GCNs, and present a new method to alleviate the imbalanced labels and also augment graph representations using a Reverse-Imbalance Weighted Sampling (RIWS) strategy. With the RIWS strategy, probability-based class balancing weights could ensure the overall distribution of positive and negative samples; in addition, weighted random sampling provides diverse subgraph structures, which effectively alleviates the over-fitting problem and improves the representation ability of GCNs. Extensive experiments on series of imbalanced benchmark datasets synthesized from MS-Celeb-1M and DeepFashion demonstrate the effectiveness and generality of our proposed method. Our implementation and the synthesized datasets will be openly available on https://github.com/espectre/GCNs_on_imbalanced_datasets.

近年来，已经产生了大量的视觉内容，并从许多领域共享，例如社交媒体平台，医学成像和机器人。这种丰富的内容创建和共享引入了新的挑战，特别是在寻找类似内容内容的图像检索（CBIR）-A的数据库中，即长期建立的研究区域，其中需要改进的效率和准确性来实时检索。人工智能在CBIR中取得了进展，并大大促进了实例搜索过程。在本调查中，我们审查了最近基于深度学习算法和技术开发的实例检索工作，通过深网络架构类型，深度功能，功能嵌入方法以及网络微调策略组织了调查。我们的调查考虑了各种各样的最新方法，在那里，我们识别里程碑工作，揭示各种方法之间的联系，并呈现常用的基准，评估结果，共同挑战，并提出未来的未来方向。

对比度学习是视觉表示学习最成功的方法之一，可以通过在学习的表示上共同执行聚类来进一步提高其性能。但是，现有的联合聚类和对比度学习的方法在长尾数据分布上表现不佳，因为多数班级压倒了少数群体的损失，从而阻止了学习有意义的表示形式。由此激励，我们通过适应偏见的对比损失，以避免群集中的少数群体类别的不平衡数据集来开发一种新颖的联合聚类和对比度学习框架。我们表明，我们提出的修改后的对比损失和分歧聚类损失可改善多个数据集和学习任务的性能。源代码可从https://anonymon.4open.science/r/ssl-debiased-clustering获得

This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spreadout features. To achieve this goal, we propose a novel instance based softmax embedding method, which directly optimizes the 'real' instance features on top of the softmax function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.

从不平衡数据中学习是一项具有挑战性的任务。在进行不平衡数据训练时，标准分类算法的性能往往差。需要通过修改数据分布或重新设计基础分类算法以实现理想的性能来采用一些特殊的策略。现实世界数据集中不平衡的流行率导致为班级不平衡问题创造了多种策略。但是，并非所有策略在不同的失衡情况下都有用或提供良好的性能。处理不平衡的数据有许多方法，但是尚未进行此类技术的功效或这些技术之间的实验比较。在这项研究中，我们对26种流行抽样技术进行了全面分析，以了解它们在处理不平衡数据方面的有效性。在50个数据集上进行了严格的实验，具有不同程度的不平衡，以彻底研究这些技术的性能。已经提出了对技术的优势和局限性的详细讨论，以及如何克服此类局限性。我们确定了影响采样策略的一些关键因素，并提供有关如何为特定应用选择合适的采样技术的建议。

不平衡的数据（ID）是阻止机器学习（ML）模型以实现令人满意的结果的问题。 ID是一种情况，即属于一个类别的样本的数量超过另一个类别的情况，这使此类模型学习过程偏向多数类。近年来，为了解决这个问题，已经提出了几种解决方案，该解决方案选择合成为少数族裔类生成新数据，或者减少平衡数据的多数类的数量。因此，在本文中，我们研究了基于深神经网络（DNN）和卷积神经网络（CNN）的方法的有效性，并与各种众所周知的不平衡数据解决方案混合，这意味着过采样和降采样。为了评估我们的方法，我们使用了龙骨，乳腺癌和Z-Alizadeh Sani数据集。为了获得可靠的结果，我们通过随机洗牌的数据分布进行了100次实验。分类结果表明，混合的合成少数族裔过采样技术（SMOTE） - 正态化-CNN优于在24个不平衡数据集上达到99.08％精度的不同方法。因此，提出的混合模型可以应用于其他实际数据集上的不平衡算法分类问题。

类不平衡是分类任务中经常发生的情况。从不平衡数据中学习提出了一个重大挑战，这在该领域引起了很多研究。使用采样技术进行数据预处理是处理数据中存在的不平衡的标准方法。由于标准分类算法在不平衡数据上的性能不佳，因此在培训之前，数据集需要足够平衡。这可以通过过度采样少数族裔级别或对多数级别的采样来实现。在这项研究中，已经提出了一种新型的混合采样算法。为了克服采样技术的局限性，同时确保保留采样数据集的质量，已经开发了一个复杂的框架来正确结合三种不同的采样技术。首先应用邻里清洁规则以减少失衡。然后从策略上与SMOTE算法策略性地采样，以在数据集中获得最佳平衡。该提出的混合方法学称为“ smote-rus-nc”，已与其他最先进的采样技术进行了比较。该策略进一步合并到集合学习框架中，以获得更健壮的分类算法，称为“ SRN-BRF”。对26个不平衡数据集进行了严格的实验，并具有不同程度的失衡。在几乎所有数据集中，提出的两种算法在许多情况下都超过了现有的采样策略，其差额很大。尤其是在流行抽样技术完全失败的高度不平衡数据集中，他们实现了无与伦比的性能。获得的优越结果证明了所提出的模型的功效及其在不平衡域中具有强大采样算法的潜力。

Person re-identification (Re-ID) aims at retrieving a person of interest across multiple non-overlapping cameras. With the advancement of deep neural networks and increasing demand of intelligent video surveillance, it has gained significantly increased interest in the computer vision community. By dissecting the involved components in developing a person Re-ID system, we categorize it into the closed-world and open-world settings. The widely studied closed-world setting is usually applied under various research-oriented assumptions, and has achieved inspiring success using deep learning techniques on a number of datasets. We first conduct a comprehensive overview with in-depth analysis for closed-world person Re-ID from three different perspectives, including deep feature representation learning, deep metric learning and ranking optimization. With the performance saturation under closed-world setting, the research focus for person Re-ID has recently shifted to the open-world setting, facing more challenging issues. This setting is closer to practical applications under specific scenarios. We summarize the open-world Re-ID in terms of five different aspects. By analyzing the advantages of existing methods, we design a powerful AGW baseline, achieving state-of-the-art or at least comparable performance on twelve datasets for FOUR different Re-ID tasks. Meanwhile, we introduce a new evaluation metric (mINP) for person Re-ID, indicating the cost for finding all the correct matches, which provides an additional criteria to evaluate the Re-ID system for real applications. Finally, some important yet under-investigated open issues are discussed.

Deep embeddings answer one simple question: How similar are two images? Learning these embeddings is the bedrock of verification, zero-shot learning, and visual search. The most prominent approaches optimize a deep convolutional network with a suitable loss function, such as contrastive loss or triplet loss. While a rich line of work focuses solely on the loss functions, we show in this paper that selecting training examples plays an equally important role. We propose distance weighted sampling, which selects more informative and stable examples than traditional approaches. In addition, we show that a simple margin based loss is sufficient to outperform all other loss functions. We evaluate our approach on the Stanford Online Products, CAR196, and the CUB200-2011 datasets for image retrieval and clustering, and on the LFW dataset for face verification. Our method achieves state-of-the-art performance on all of them.

对比度学习最近在无监督的视觉表示学习中显示出巨大的潜力。在此轨道中的现有研究主要集中于图像内不变性学习。学习通常使用丰富的图像内变换来构建正对，然后使用对比度损失最大化一致性。相反，相互影响不变性的优点仍然少得多。利用图像间不变性的一个主要障碍是，尚不清楚如何可靠地构建图像间的正对，并进一步从它们中获得有效的监督，因为没有配对注释可用。在这项工作中，我们提出了一项全面的实证研究，以更好地了解从三个主要组成部分的形象间不变性学习的作用：伪标签维护，采样策略和决策边界设计。为了促进这项研究，我们引入了一个统一的通用框架，该框架支持无监督的内部和间形内不变性学习的整合。通过精心设计的比较和分析，揭示了多个有价值的观察结果：1）在线标签收敛速度比离线标签更快； 2）半硬性样品比硬否定样品更可靠和公正； 3）一个不太严格的决策边界更有利于形象间的不变性学习。借助所有获得的食谱，我们的最终模型（即InterCLR）对多个标准基准测试的最先进的内图内不变性学习方法表现出一致的改进。我们希望这项工作将为设计有效的无监督间歇性不变性学习提供有用的经验。代码：https：//github.com/open-mmlab/mmselfsup。

在癌症诊断和病理研究中，组织病理学图像的分类均具有巨大的价值。但是，多种原因（例如由放大因素和阶级失衡引起的变化）使其成为一项艰巨的任务，在许多情况下，从图像标签数据集中学习的常规方法在许多情况下都无法令人满意。我们观察到同一类的肿瘤通常具有共同的形态学模式。为了利用这一事实，我们提出了一种方法，该方法可以学习基于相似性的多尺度嵌入（SMSE），以实现非放大依赖性的组织病理学图像分类。特别是，利用了一对损失和三胞胎损失，以从图像对或图像三联体中学习基于相似性的嵌入。学到的嵌入提供了对图像之间相似性的准确测量，这被认为是组织病理学形态比正常图像特征更有效的表示形式。此外，为了确保生成的模型独立于放大，以不同放大因素获取的图像在学习多尺度嵌入过程中同时被馈送到网络中。除了SMSE之外，我们还消除了类不平衡的影响，而不是使用凭直觉丢弃一些简单样品的硬采矿策略，我们引入了新的增强局灶性损失，以同时惩罚硬误分类的样品，同时抑制了容易分类良好的样品。实验结果表明，与以前的方法相比，SMSE改善了乳腺癌和肝癌的组织病理图像分类任务的性能。特别是，与使用传统功能相比，SMSE在Breakhis基准测试中取得了最佳性能，其改善范围从5％到18％。

In this study, we systematically investigate the impact of class imbalance on classification performance of convolutional neural networks (CNNs) and compare frequently used methods to address the issue. Class imbalance is a common problem that has been comprehensively studied in classical machine learning, yet very limited systematic research is available in the context of deep learning. In our study, we use three benchmark datasets of increasing complexity, MNIST, CIFAR-10 and ImageNet, to investigate the effects of imbalance on classification and perform an extensive comparison of several methods to address the issue: oversampling, undersampling, two-phase training, and thresholding that compensates for prior class probabilities. Our main evaluation metric is area under the receiver operating characteristic curve (ROC AUC) adjusted to multi-class tasks since overall accuracy metric is associated with notable difficulties in the context of imbalanced data. Based on results from our experiments we conclude that (i) the effect of class imbalance on classification performance is detrimental; (ii) the method of addressing class imbalance that emerged as dominant in almost all analyzed scenarios was oversampling; (iii) oversampling should be applied to the level that completely eliminates the imbalance, whereas the optimal undersampling ratio depends on the extent of imbalance; (iv) as opposed to some classical machine learning models, oversampling does not cause overfitting of CNNs; (v) thresholding should be applied to compensate for prior class probabilities when overall number of properly classified cases is of interest.

由于机器学习和数据挖掘领域的不平衡数据集的分类问题，但学习的不平衡学习是重要的并且具有挑战性。提出采样方法来解决这个问题，而基于群集的过采样方法表现出很大的潜力，因为它们的目标是同时解决课堂和级别的不平衡问题。但是，所有现有的聚类方法都基于一次性方法。由于缺乏先验知识，通常存在的群集数量不当设置，这导致集群性能不佳。此外，现有方法可能会产生嘈杂的情况。为了解决这些问题，本文提出了一种基于模糊C-MATION（MLFCM）的基于深度外观信封网络的不平衡学习算法，以及基于最大均值（MINMD）的最小中间层间差异机制。在没有先前知识的情况下，该算法可以使用深度实例包络网络来保证高质量的平衡实例。在实验部分中，三十三个流行的公共数据集用于验证，并且超过十个代表性算法用于比较。实验结果表明，该方法显着优于其他流行的方法。