考虑到数据注释的成本以及几乎没有标记的样本所提供的准确性提高，几乎没有射击的成本，几乎没有射击的转移学习越来越多。尤其是在少量分类（FSC）中，最近的作品探索了旨在最大程度地相对于未知参数的可能性或后二阶段的特征分布。遵循这种静脉，并考虑到FSC和聚类之间的平行，我们寻求更好地考虑到由于缺乏数据而导致的估计不确定性，以及与每个类相关的群集的统计属性更好。因此，在本文中，我们提出了一种基于变异贝叶斯推论的新聚类方法，基于概率线性判别分析，自适应维度降低进一步改善。当应用于先前研究中使用的功能时，我们提出的方法可显着提高在各种少量基准测试的现实不平衡转导设置中的准确性，其准确性高达$ 6 \％$。此外，当应用于平衡设置时，我们将获得非常有竞争力的结果，而无需使用对实际用例的级别平衡伪像。我们还提供了方法的性能，以高性能的主链链链，其报告的结果进一步超过了当前的最新准确性，这表明该方法的通用性。

我们考虑了一个新颖的表述，即主动射击分类（AFSC）的问题，其目的是对标签预算非常限制的小规定，最初未标记的数据集进行分类。这个问题可以看作是与经典的跨托管少数射击分类（TFSC）的竞争对手范式，因为这两种方法都适用于相似的条件。我们首先提出了一种结合统计推断的方法，以及一种非常适合该框架的原始两级积极学习策略。然后，我们从TFSC领域调整了几个标准视觉基准。我们的实验表明，AFSC的潜在优势可能是很大的，与最先进的TFSC方法相比，对于同一标签预算，平均加权准确性高达10％。我们认为，这种新的范式可能会导致数据筛选学习设置的新发展和标准。

现代深度学习需要大规模广泛标记的数据集进行培训。少量学习旨在通过有效地从少数标记的例子中学习来缓解这个问题。在先前提出的少量视觉分类器中，假设对分类器决定的特征歧管具有不相关的特征尺寸和均匀特征方差。在这项工作中，我们专注于通过提出以低标签制度运行的差异敏感的模型来解决这一假设引起的限制。第一种方法简单的CNAP，采用基于分层正规的Mahalanobis距离基于距离的分类器，与现有神经自适应特征提取器的状态相结合，以在元数据集，迷你成像和分层图像基准基准上实现强大性能。我们进一步将这种方法扩展到转换学习设置，提出转导压盖。这种转换方法将软k-means参数细化过程与两步任务编码器相结合，以实现使用未标记数据的改进的测试时间分类精度。转导CNAP在元数据集上实现了最先进的性能。最后，我们探讨了我们的方法（简单和转换）的使用“开箱即用”持续和积极的学习。大规模基准的广泛实验表明了这一点的鲁棒性和多功能性，相对说话，简单的模型。所有培训的模型检查点和相应的源代码都已公开可用。

我们探索了深度神经网络的软磁预测的聚类，并引入了一种新型的概率聚类方法，称为k-sbetas。在聚类分布的一般环境中，现有方法着重于探索针对单纯形数据（例如KL Divergence）量身定制的失真度量，作为标准欧几里得距离的替代方法。我们提供了聚类分布的一般观点，该观点强调，基于失真的方法的统计模型可能不够描述。取而代之的是，我们优化了一个可混合变量的目标，该目标测量了每个集群中数据的一致性与引入的SBETA密度函数，其参数受到约束并与二进制分配变量共同估​​算。我们的多功能公式近似于用于建模群集数据的各种参数密度，并使能够控制群集平衡偏置。这会产生高度竞争性的性能，以在各种情况下进行有效无监督的黑盒预测调整，包括一声分类和实时的无监督域适应道路，以进行道路分割。实施可在https://github.com/fchiaroni/clustering_softmax_predictions上获得。

We introduce an information-maximization approach for the Generalized Category Discovery (GCD) problem. Specifically, we explore a parametric family of loss functions evaluating the mutual information between the features and the labels, and find automatically the one that maximizes the predictive performances. Furthermore, we introduce the Elbow Maximum Centroid-Shift (EMaCS) technique, which estimates the number of classes in the unlabeled set. We report comprehensive experiments, which show that our mutual information-based approach (MIB) is both versatile and highly competitive under various GCD scenarios. The gap between the proposed approach and the existing methods is significant, more so when dealing with fine-grained classification problems. Our code: \url{https://github.com/fchiaroni/Mutual-Information-Based-GCD}.

从少数样本中学习的多功能性是人类智能的标志。很少有学习能力超越机器的努力。受概率深度学习的承诺和力量的启发，我们提出了一个新颖的变异推理网络，用于几个射击分类（被构成三叉戟），将图像的表示形式分离为语义和标记潜在变量，并以相互交织的方式推断它们。为了诱导任务意识，作为三叉戟推理机制的一部分，我们使用一种新型的基于内置的基于注意的转导功能提取模块（我们致电ATTFEX）在查询和支持图像上借鉴了几次任务的图像。我们广泛的实验结果证实了三叉戟的功效，并证明，使用最简单的骨架，它在最常见的数据集Miniimagenet和Tieredimagenet中设置了新的最新时间（最多可提高4％和5％，，高达4％和5％分别是）以及最近具有挑战性的跨域迷你膜 - > CUB场景，其范围超出了最佳现有跨域基线的显着利润率（最高20％）。可以在我们的GitHub存储库中找到代码和实验：https：//github.com/anujinho/trident

很少的识别涉及训练图像分类器，以使用几个示例（Shot）在测试时间区分新颖概念。现有方法通常假定测试时间的射击号是事先知道的。这是不现实的，当火车和测试射击不匹配时，流行和基础方法的性能已被证明会受到影响。我们对该现象进行了系统的经验研究。与先前的工作一致，我们发现射击灵敏度在基于度量的几个学习者中广泛存在，但是与先前的工作相反，较大的神经体系结构为变化的测试拍摄提供了一定程度的内置鲁棒性。更重要的是，通过消除对样品噪声的敏感性，一种基于余弦距离的简单，以前已知但非常忽略了一类方法，可以极大地改善对射击变化的鲁​​棒性。我们为流行和最近的几个弹药分类器提供了余弦替代品，从而扩大了它们对现实环境的适用性。这些余弦模型一致地提高了射击力，超越先前的射击状态，并在一系列基准和架构上提供竞争精度，包括在非常低的射击方案中取得的显着增长。

现代深度学习方法构成了令人难以置信的强大工具，以解决无数的挑战问题。然而，由于深度学习方法作为黑匣子运作，因此与其预测相关的不确定性往往是挑战量化。贝叶斯统计数据提供了一种形式主义来理解和量化与深度神经网络预测相关的不确定性。本教程概述了相关文献和完整的工具集，用于设计，实施，列车，使用和评估贝叶斯神经网络，即使用贝叶斯方法培训的随机人工神经网络。

The estimation of the generalization error of classifiers often relies on a validation set. Such a set is hardly available in few-shot learning scenarios, a highly disregarded shortcoming in the field. In these scenarios, it is common to rely on features extracted from pre-trained neural networks combined with distance-based classifiers such as nearest class mean. In this work, we introduce a Gaussian model of the feature distribution. By estimating the parameters of this model, we are able to predict the generalization error on new classification tasks with few samples. We observe that accurate distance estimates between class-conditional densities are the key to accurate estimates of the generalization performance. Therefore, we propose an unbiased estimator for these distances and integrate it in our numerical analysis. We show that our approach outperforms alternatives such as the leave-one-out cross-validation strategy in few-shot settings.

我们解决了几个射击开放式识别（FSOSR）问题，即在我们只有很少的标签样本的一组类中分类的实例，同时检测不属于任何已知类别的实例。偏离现有文献，我们专注于开发模型不足的推理方法，这些方法可以插入任何现有模型，无论其架构或培训程序如何。通过评估嵌入的各种模型的质量，我们量化了模型 - 敏捷FSOSR的内在难度。此外，公平的经验评估表明，在FSOSR的电感环境中，KNN检测器和原型分类器的天真组合在专业或复杂方法之前。这些观察结果促使我们诉诸于转导，这是对标准的几次学习问题的流行而实用的放松。我们介绍了一个开放的设置转导信息最大化方法OSTIM，该方法幻觉了异常原型，同时最大程度地提高了提取的特征和作业之间的相互信息。通过跨越5个数据集的广泛实验，我们表明OSTIM在检测开放式实例的同时，在与最强的托管方法竞争时，在检测开放式实例时都超过了电感和现有的转导方法。我们进一步表明，OSTIM的模型不可知论使其能够成功利用最新体系结构和培训策略的强大表现能力而没有任何超参数修改，这是一个有希望的信号，即将来临的建筑进步将继续积极影响Ostim的表现。

Few-shot learning (FSL) is a central problem in meta-learning, where learners must efficiently learn from few labeled examples. Within FSL, feature pre-training has recently become an increasingly popular strategy to significantly improve generalization performance. However, the contribution of pre-training is often overlooked and understudied, with limited theoretical understanding of its impact on meta-learning performance. Further, pre-training requires a consistent set of global labels shared across training tasks, which may be unavailable in practice. In this work, we address the above issues by first showing the connection between pre-training and meta-learning. We discuss why pre-training yields more robust meta-representation and connect the theoretical analysis to existing works and empirical results. Secondly, we introduce Meta Label Learning (MeLa), a novel meta-learning algorithm that learns task relations by inferring global labels across tasks. This allows us to exploit pre-training for FSL even when global labels are unavailable or ill-defined. Lastly, we introduce an augmented pre-training procedure that further improves the learned meta-representation. Empirically, MeLa outperforms existing methods across a diverse range of benchmarks, in particular under a more challenging setting where the number of training tasks is limited and labels are task-specific. We also provide extensive ablation study to highlight its key properties.

We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning. Stochastic Weight Averaging (SWA), which computes the first moment of stochastic gradient descent (SGD) iterates with a modified learning rate schedule, has recently been shown to improve generalization in deep learning. With SWAG, we fit a Gaussian using the SWA solution as the first moment and a low rank plus diagonal covariance also derived from the SGD iterates, forming an approximate posterior distribution over neural network weights; we then sample from this Gaussian distribution to perform Bayesian model averaging. We empirically find that SWAG approximates the shape of the true posterior, in accordance with results describing the stationary distribution of SGD iterates. Moreover, we demonstrate that SWAG performs well on a wide variety of tasks, including out of sample detection, calibration, and transfer learning, in comparison to many popular alternatives including MC dropout, KFAC Laplace, SGLD, and temperature scaling.

Few-shot learning is a rapidly evolving area of research in machine learning where the goal is to classify unlabeled data with only one or "a few" labeled exemplary samples. Neural networks are typically trained to minimize a distance metric between labeled exemplary samples and a query set. Early few-shot approaches use an episodic training process to sub-sample the training data into few-shot batches. This training process matches the sub-sampling done on evaluation. Recently, conventional supervised training coupled with a cosine distance has achieved superior performance for few-shot. Despite the diversity of few-shot approaches over the past decade, most methods still rely on the cosine or Euclidean distance layer between the latent features of the trained network. In this work, we investigate the distributions of trained few-shot features and demonstrate that they can be roughly approximated as exponential distributions. Under this assumption of an exponential distribution, we propose a new maximum log-likelihood metric for few-shot architectures. We demonstrate that the proposed metric achieves superior performance accuracy w.r.t. conventional similarity metrics (e.g., cosine, Euclidean, etc.), and achieve state-of-the-art inductive few-shot performance. Further, additional gains can be achieved by carefully combining multiple metrics and neither of our methods require post-processing feature transformations, which are common to many algorithms. Finally, we demonstrate a novel iterative algorithm designed around our maximum log-likelihood approach that achieves state-of-the-art transductive few-shot performance when the evaluation data is imbalanced. We have made our code publicly available at https://github.com/samuelhess/MLL_FSL/.

模型不合时宜的元学习（MAML）可以说是当今最流行的元学习算法之一。然而，它在几次分类上的性能远远远远远远远远远远远远远远落在许多致力于该问题的算法。在本文中，我们指出了如何训练MAML以进行几次分类的几个关键方面。首先，我们发现MAML在其内部循环更新中需要大量的梯度步骤，这与其常见的用法相矛盾。其次，我们发现MAML对元测试过程中的类标签分配敏感。具体而言，MAML Meta-Trains $ n$道分类器的初始化。这些$ n $方式，在元测试期间，然后具有“ $ n！$”的“ $ n！$”排列，并与$ n $新颖的课程配对。我们发现这些排列会导致巨大的准确性差异，从而使MAML不稳定。第三，我们研究了几种使MAML置换不变的方法，其中元训练单个向量以初始化分类头中的所有$ n $重量矢量的初始化。在Miniimagenet和Tieredimagenet等基准数据集上，我们命名Unicorn-MAML的方法在不牺牲MAML的简单性的情况下以与许多最近的几杆分类算法相同甚至优于许多近期的几个次数分类算法。

深度学习正在推动许多计算机视觉应用中的最新技术。但是，它依赖于大量注释的数据存储库，并且捕获现实世界数据的不受约束性质尚未解决。半监督学习（SSL）用大量未标记的数据来补充带注释的培训数据，以降低注释成本。标准SSL方法假设未标记的数据来自与注释数据相同的分布。最近，Orca [9]引入了一个更现实的SSL问题，称为开放世界SSL，假设未注释的数据可能包含来自未知类别的样本。这项工作提出了一种在开放世界中解决SSL的新方法，我们同时学习对已知和未知类别进行分类。在我们方法的核心方面，我们利用样本不确定性，并将有关类分布的先验知识纳入，以生成可靠的伪标记，以适用于已知和未知类别的未标记数据。我们广泛的实验在几个基准数据集上展示了我们的方法的有效性，在该数据集上，它在其中的七个不同数据集（包括CIFAR-100（17.6％）（17.6％），Imagenet-100（5.7％）（5.7％）和微小成像网（9.9％）。

我们研究了用于半监控学习（SSL）的无监督数据选择，其中可以提供大规模的未标记数据集，并且为标签采集预算小额数据子集。现有的SSL方法专注于学习一个有效地集成了来自给定小标记数据和大型未标记数据的信息的模型，而我们专注于选择正确的数据以用于SSL的注释，而无需任何标签或任务信息。直观地，要标记的实例应统称为下游任务的最大多样性和覆盖范围，并且单独具有用于SSL的最大信息传播实用程序。我们以三步数据为中心的SSL方法形式化这些概念，使稳定性和精度的纤维液改善8％的CiFar-10（标记为0.08％）和14％的Imagenet -1k（标记为0.2％）。它也是一种具有各种SSL方法的通用框架，提供一致的性能增益。我们的工作表明，在仔细选择注释数据上花费的小计算带来了大注释效率和模型性能增益，而无需改变学习管道。我们完全无监督的数据选择可以轻松扩展到其他弱监督的学习设置。

我们提出了一种新的非参数混合物模型，用于多变量回归问题，灵感来自概率K-Nearthimest邻居算法。使用有条件指定的模型，对样本外输入的预测基于与每个观察到的数据点的相似性，从而产生高斯混合物表示的预测分布。在混合物组件的参数以及距离度量标准的参数上，使用平均场变化贝叶斯算法进行后推断，并具有基于随机梯度的优化过程。在与数据大小相比，输入 - 输出关系很复杂，预测分布可能偏向或多模式的情况下，输入相对较高的尺寸，该方法尤其有利。对五个数据集进行的计算研究，其中两个是合成生成的，这说明了我们的高维输入的专家混合物方法的明显优势，在验证指标和视觉检查方面都优于竞争者模型。

在这项工作中，我们建议使用分布式样本，即来自目标类别外部的未标记样本，以改善几乎没有记录的学习。具体而言，我们利用易于可用的分布样品来驱动分类器，以避免通过最大化原型到分布样品的距离，同时最大程度地减少分布样品的距离（即支持，查询数据），以避免使用分类器。。我们的方法易于实施，不可知论的是提取器，轻量级，而没有任何额外的预训练费用，并且适用于归纳和跨传输设置。对各种标准基准测试的广泛实验表明，所提出的方法始终提高具有不同架构的预审计网络的性能。

Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set, which well suits many real-world data annotation scenarios with label ambiguity. Despite the promise, the performance of PLL often lags behind the supervised counterpart. In this work, we bridge the gap by addressing two key research challenges in PLL -- representation learning and label disambiguation -- in one coherent framework. Specifically, our proposed framework PiCO consists of a contrastive learning module along with a novel class prototype-based label disambiguation algorithm. PiCO produces closely aligned representations for examples from the same classes and facilitates label disambiguation. Theoretically, we show that these two components are mutually beneficial, and can be rigorously justified from an expectation-maximization (EM) algorithm perspective. Moreover, we study a challenging yet practical noisy partial label learning setup, where the ground-truth may not be included in the candidate set. To remedy this problem, we present an extension PiCO+ that performs distance-based clean sample selection and learns robust classifiers by a semi-supervised contrastive learning algorithm. Extensive experiments demonstrate that our proposed methods significantly outperform the current state-of-the-art approaches in standard and noisy PLL tasks and even achieve comparable results to fully supervised learning.

Nearest-Neighbor (NN) classification has been proven as a simple and effective approach for few-shot learning. The query data can be classified efficiently by finding the nearest support class based on features extracted by pretrained deep models. However, NN-based methods are sensitive to the data distribution and may produce false prediction if the samples in the support set happen to lie around the distribution boundary of different classes. To solve this issue, we present P3DC-Shot, an improved nearest-neighbor based few-shot classification method empowered by prior-driven data calibration. Inspired by the distribution calibration technique which utilizes the distribution or statistics of the base classes to calibrate the data for few-shot tasks, we propose a novel discrete data calibration operation which is more suitable for NN-based few-shot classification. Specifically, we treat the prototypes representing each base class as priors and calibrate each support data based on its similarity to different base prototypes. Then, we perform NN classification using these discretely calibrated support data. Results from extensive experiments on various datasets show our efficient non-learning based method can outperform or at least comparable to SOTA methods which need additional learning steps.