计算机视觉中的当前预训练方法专注于日常生活中的自然图像。但是,诸如图标和符号之类的抽象图在现实世界中是常见的,很重要。这项工作受到坦格图的启发,这是一种需要从七个解剖形状复制抽象模式的游戏。通过录制人类在解决坦文图谜题方面的体验,我们展示了Tangram DataSet,并显示Tangram上的预先训练的神经模型有助于解决一些基于低分辨率视觉的迷你视觉任务。广泛的实验表明,我们所提出的方法为折叠衣服和评估室布局等审美任务产生智能解决方案。预训练的特征提取器可以促进人类手写的几秒钟学习任务的收敛性,并提高轮廓识别图标的准确性。Tangram DataSet可在https://github.com/yizhouzhao/tangram上获得。
translated by 谷歌翻译
Few-shot learning aims to fast adapt a deep model from a few examples. While pre-training and meta-training can create deep models powerful for few-shot generalization, we find that pre-training and meta-training focuses respectively on cross-domain transferability and cross-task transferability, which restricts their data efficiency in the entangled settings of domain shift and task shift. We thus propose the Omni-Training framework to seamlessly bridge pre-training and meta-training for data-efficient few-shot learning. Our first contribution is a tri-flow Omni-Net architecture. Besides the joint representation flow, Omni-Net introduces two parallel flows for pre-training and meta-training, responsible for improving domain transferability and task transferability respectively. Omni-Net further coordinates the parallel flows by routing their representations via the joint-flow, enabling knowledge transfer across flows. Our second contribution is the Omni-Loss, which introduces a self-distillation strategy separately on the pre-training and meta-training objectives for boosting knowledge transfer throughout different training stages. Omni-Training is a general framework to accommodate many existing algorithms. Evaluations justify that our single framework consistently and clearly outperforms the individual state-of-the-art methods on both cross-task and cross-domain settings in a variety of classification, regression and reinforcement learning problems.
translated by 谷歌翻译
The focus of recent meta-learning research has been on the development of learning algorithms that can quickly adapt to test time tasks with limited data and low computational cost. Few-shot learning is widely used as one of the standard benchmarks in meta-learning. In this work, we show that a simple baseline: learning a supervised or selfsupervised representation on the meta-training set, followed by training a linear classifier on top of this representation, outperforms state-of-the-art few-shot learning methods. An additional boost can be achieved through the use of selfdistillation. This demonstrates that using a good learned embedding model can be more effective than sophisticated meta-learning algorithms. We believe that our findings motivate a rethinking of few-shot image classification benchmarks and the associated role of meta-learning algorithms.
translated by 谷歌翻译
尽管深度神经网络能够在各种任务上实现优于人类的表现,但他们臭名昭著,因为他们需要大量的数据和计算资源,将其成功限制在可用的这些资源的领域。金属学习方法可以通过从相关任务中转移知识来解决此问题,从而减少学习新任务所需的数据和计算资源的数量。我们组织了元数据竞赛系列,该系列为世界各地的研究小组提供了创建和实验评估实际问题的新元学习解决方案的机会。在本文中,我们在竞争组织者和排名最高的参与者之间进行了合作,我们描述了竞争的设计,数据集,最佳实验结果以及Neurips 2021挑战中最高的方法,这些方法吸引了15进入最后阶段的活跃团队(通过表现优于基线),在反馈阶段进行了100多次代码提交。顶级参与者的解决方案是开源的。汲取的经验教训包括学习良好的表示对于有效的转移学习至关重要。
translated by 谷歌翻译
很少有射击分类旨在仅使用几个标签示例就可以很好地学习新对象类别。从其他模型转移功能表示是一种流行的方法,用于解决几乎没有射击的分类问题。在这项工作中,我们对各种功能表示形式进行了系统的研究,以进行几次射击分类,包括从MAML中学到的表示,监督分类和几个常见的自我监督任务。我们发现,从更复杂的任务中学习倾向于为几个射击分类提供更好的表示,因此我们建议使用从多个任务中学到的表示形式进行几次分类。加上功能选择和投票以处理小样本量的新技巧,我们的直接转移学习方法提供的性能可与几个基准数据集上的最先进相提并论。
translated by 谷歌翻译
近年来,预先培训的表述的出现是计算机视觉,自然语言和语音中AI应用的强大抽象。但是,控制策略学习仍然由Tabula-Rasa学习范式主导,而Visuo-Motor策略经常使用部署环境中的数据进行培训。在这种情况下,我们重新审视并研究了预训练的视觉表示对控制的作用,以及在大规模计算机视觉数据集中训练的特定表示。通过对不同控制域(栖息地,深态控制,Adroit,Franka Kitchen)的广泛经验评估,我们隔离和研究了不同表示培训方法,数据增强和功能层次结构的重要性。总体而言,我们发现,预先训练的视觉表示可以比培训控制政策的基本真实状态表示能力更具竞争力甚至更好。尽管仅使用来自标准视觉数据集中的室外数据,但这是没有部署环境中的任何域内数据。源代码以及更多信息,请访问https://sites.google.com/view/pvr-control。
translated by 谷歌翻译
细粒度的图像分析(FGIA)是计算机视觉和模式识别中的长期和基本问题,并为一组多种现实世界应用提供了基础。 FGIA的任务是从属类别分析视觉物体,例如汽车或汽车型号的种类。细粒度分析中固有的小阶级和阶级阶级内变异使其成为一个具有挑战性的问题。利用深度学习的进步,近年来,我们在深入学习动力的FGIA中见证了显着进展。在本文中,我们对这些进展的系统进行了系统的调查,我们试图通过巩固两个基本的细粒度研究领域 - 细粒度的图像识别和细粒度的图像检索来重新定义和扩大FGIA领域。此外,我们还审查了FGIA的其他关键问题,例如公开可用的基准数据集和相关域的特定于应用程序。我们通过突出几个研究方向和开放问题,从社区中突出了几个研究方向和开放问题。
translated by 谷歌翻译
State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories. This restricted form of supervision limits their generality and usability since additional labeled data is needed to specify any other visual concept. Learning directly from raw text about images is a promising alternative which leverages a much broader source of supervision. We demonstrate that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet. After pre-training, natural language is used to reference learned visual concepts (or describe new ones) enabling zero-shot transfer of the model to downstream tasks. We study the performance of this approach by benchmarking on over 30 different existing computer vision datasets, spanning tasks such as OCR, action recognition in videos, geo-localization, and many types of fine-grained object classification. The model transfers non-trivially to most tasks and is often competitive with a fully supervised baseline without the need for any dataset specific training. For instance, we match the accuracy of the original ResNet-50 on ImageNet zero-shot without needing to use any of the 1.28 million training examples it was trained on. We release our code and pre-trained model weights at https://github.com/OpenAI/CLIP.
translated by 谷歌翻译
Recent progress in artificial intelligence (AI) has renewed interest in building systems that learn and think like people. Many advances have come from using deep neural networks trained end-to-end in tasks such as object recognition, video games, and board games, achieving performance that equals or even beats humans in some respects. Despite their biological inspiration and performance achievements, these systems differ from human intelligence in crucial ways. We review progress in cognitive science suggesting that truly human-like learning and thinking machines will have to reach beyond current engineering trends in both what they learn, and how they learn it. Specifically, we argue that these machines should (a) build causal models of the world that support explanation and understanding, rather than merely solving pattern recognition problems; (b) ground learning in intuitive theories of physics and psychology, to support and enrich the knowledge that is learned; and (c) harness compositionality and learning-to-learn to rapidly acquire and generalize knowledge to new tasks and situations. We suggest concrete challenges and promising routes towards these goals that can combine the strengths of recent neural network advances with more structured cognitive models.
translated by 谷歌翻译
Imitation learning techniques aim to mimic human behavior in a given task. An agent (a learning machine) is trained to perform a task from demonstrations by learning a mapping between observations and actions. The idea of teaching by imitation has been around for many years, however, the field is gaining attention recently due to advances in computing and sensing as well as rising demand for intelligent applications. The paradigm of learning by imitation is gaining popularity because it facilitates teaching complex tasks with minimal expert knowledge of the tasks. Generic imitation learning methods could potentially reduce the problem of teaching a task to that of providing demonstrations; without the need for explicit programming or designing reward functions specific to the task. Modern sensors are able to collect and transmit high volumes of data rapidly, and processors with high computational power allow fast processing that maps the sensory data to actions in a timely manner. This opens the door for many potential AI applications that require real-time perception and reaction such as humanoid robots, self-driving vehicles, human computer interaction and computer games to name a few. However, specialized algorithms are needed to effectively and robustly learn models as learning by imitation poses its own set of challenges. In this paper, we survey imitation learning methods and present design options in different steps of the learning process. We introduce a background and motivation for the field as well as highlight challenges specific to the imitation problem. Methods for designing and evaluating imitation learning tasks are categorized and reviewed. Special attention is given to learning methods in robotics and games as these domains are the most popular in the literature and provide a wide array of problems and methodologies. We extensively discuss combining imitation learning approaches using different sources and methods, as well as incorporating other motion learning methods to enhance imitation. We also discuss the potential impact on industry, present major applications and highlight current and future research directions.
translated by 谷歌翻译
Continual Learning (CL) is a field dedicated to devise algorithms able to achieve lifelong learning. Overcoming the knowledge disruption of previously acquired concepts, a drawback affecting deep learning models and that goes by the name of catastrophic forgetting, is a hard challenge. Currently, deep learning methods can attain impressive results when the data modeled does not undergo a considerable distributional shift in subsequent learning sessions, but whenever we expose such systems to this incremental setting, performance drop very quickly. Overcoming this limitation is fundamental as it would allow us to build truly intelligent systems showing stability and plasticity. Secondly, it would allow us to overcome the onerous limitation of retraining these architectures from scratch with the new updated data. In this thesis, we tackle the problem from multiple directions. In a first study, we show that in rehearsal-based techniques (systems that use memory buffer), the quantity of data stored in the rehearsal buffer is a more important factor over the quality of the data. Secondly, we propose one of the early works of incremental learning on ViTs architectures, comparing functional, weight and attention regularization approaches and propose effective novel a novel asymmetric loss. At the end we conclude with a study on pretraining and how it affects the performance in Continual Learning, raising some questions about the effective progression of the field. We then conclude with some future directions and closing remarks.
translated by 谷歌翻译
Jitendra Malik once said, "Supervision is the opium of the AI researcher". Most deep learning techniques heavily rely on extreme amounts of human labels to work effectively. In today's world, the rate of data creation greatly surpasses the rate of data annotation. Full reliance on human annotations is just a temporary means to solve current closed problems in AI. In reality, only a tiny fraction of data is annotated. Annotation Efficient Learning (AEL) is a study of algorithms to train models effectively with fewer annotations. To thrive in AEL environments, we need deep learning techniques that rely less on manual annotations (e.g., image, bounding-box, and per-pixel labels), but learn useful information from unlabeled data. In this thesis, we explore five different techniques for handling AEL.
translated by 谷歌翻译
很少有视觉识别是指从一些标记实例中识别新颖的视觉概念。通过将查询表示形式与类表征进行比较以预测查询实例的类别,许多少数射击的视觉识别方法采用了基于公制的元学习范式。但是,当前基于度量的方法通常平等地对待所有实例,因此通常会获得有偏见的类表示,考虑到并非所有实例在总结了类级表示的实例级表示时都同样重要。例如,某些实例可能包含无代表性的信息,例如过多的背景和无关概念的信息,这使结果偏差。为了解决上述问题,我们提出了一个新型的基于公制的元学习框架,称为实例自适应类别表示网络(ICRL-net),以进行几次视觉识别。具体而言,我们开发了一个自适应实例重新平衡网络,具有在生成班级表示,通过学习和分配自适应权重的不同实例中的自适应权重时,根据其在相应类的支持集中的相对意义来解决偏见的表示问题。此外,我们设计了改进的双线性实例表示,并结合了两个新型的结构损失,即,阶层内实例聚类损失和阶层间表示区分损失,以进一步调节实例重估过程并完善类表示。我们对四个通常采用的几个基准测试:Miniimagenet,Tieredimagenet,Cifar-FS和FC100数据集进行了广泛的实验。与最先进的方法相比,实验结果证明了我们的ICRL-NET的优势。
translated by 谷歌翻译
We present a retrospective on the state of Embodied AI research. Our analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR. These challenges are grouped into three themes: (1) visual navigation, (2) rearrangement, and (3) embodied vision-and-language. We discuss the dominant datasets within each theme, evaluation metrics for the challenges, and the performance of state-of-the-art models. We highlight commonalities between top approaches to the challenges and identify potential future directions for Embodied AI research.
translated by 谷歌翻译
我们介绍了几个新的数据集即想象的A / O和Imagenet-R以及合成环境和测试套件,我们称为CAOS。 Imagenet-A / O允许研究人员专注于想象成剩余的盲点。由于追踪稳健的表示,以特殊创建了ImageNet-R,因为表示不再简单地自然,而是包括艺术和其他演绎。 Caos Suite由Carla Simulator构建,允许包含异常物体,可以创建可重复的合成环境和用于测试稳健性的场景。所有数据集都是为测试鲁棒性和衡量鲁棒性的衡量进展而创建的。数据集已用于各种其他作品中,以衡量其具有鲁棒性的自身进步,并允许切向进展,这些进展不会完全关注自然准确性。鉴于这些数据集,我们创建了几种旨在推进鲁棒性研究的新方法。我们以最大Logit的形式和典型程度的形式构建简单的基线,并以深度的形式创建新的数据增强方法,从而提高上述基准。最大Logit考虑Logit值而不是SoftMax操作后的值,而微小的变化会产生明显的改进。典型程分将输出分布与类的后部分布进行比较。我们表明,除了分段任务之外,这将提高对基线的性能。猜测可能在像素级别,像素的语义信息比类级信息的语义信息不太有意义。最后,新的Deepaulment的新增强技术利用神经网络在彻底不同于先前使用的传统几何和相机的转换的图像上创建增强。
translated by 谷歌翻译
本文简要介绍了元学习与自我监督学习之间的联系。可以应用元学习来提高模型泛化能力并构建一般AI算法。自我监督的学习利用原始数据的自我监督,并通过无监督的预训练或优化对比损失目标提取更高级别的更广泛的功能。在自我监督的学习中,数据增强技术是广泛应用的,并且不需要数据标签,因为可以从类似任务的训练模型估计伪标签。元学习旨在调整训练有素的深度模型来解决各种任务,并开发普通AI算法。我们审查了元学习的协会与生成和对比自我监督的学习模式。即使数据源差异很差,也可以共同考虑来自多个来源的未标记数据。我们表明,元学习和自我监督的学习模型的整合可以最适合提高模型泛化能力。由Meta-Learner和普通META学习算法引导的自我监督学习是可能组合的两个例子。
translated by 谷歌翻译
本文回顾了概念,建模方法和最新发现,沿着不同级别的神经网络模型的抽象范围,包括跨(1)样本跨(2)分布,(3)域,(4)任务,(5)模态的概括,(2) ,和(6)范围。 (1)样品概括的结果表明,对于ImageNet而言,几乎所有最近的改进都减少了训练误差,而过度拟合则保持平坦。几乎消除了所有训练错误,未来的进度将需要专注于减少过度拟合。统计数据的观点突出显示了(2)分布概括如何交替地视为样本权重的变化或输入输出关系的变化。总结了(3)域概括的转移学习方法,以及最新的进步和域适应性基准数据集的财富。在(4)任务概括中调查的最新突破包括很少的元学习方法和BERT NLP引擎以及最近(5)个模态概括研究,这些研究整合了图像和文本数据,并应用了跨嗅觉的生物学启发的网络,视觉和听觉方式。回顾了最近(6)个范围泛化结果,将知识图嵌入深度NLP方法中。此外,讨论了关于大脑的模块化结构以及多巴胺驱动的条件导致抽象思维的步骤。
translated by 谷歌翻译
众所周知,深度学习方法是渴望数据的,它需要大量标记的样本。不幸的是,大量的交互式样品标记工作极大地阻碍了深度学习方法的应用,尤其是对于需要异质样本的3D建模任务。为了减轻对FA \ c {C} ADS的3D建模的数据注释的工作,本文提出了一种半监督的对抗识别策略,该策略嵌入了逆程序建模中。从纹理LOD-2(详细级别)模型开始,我们使用经典的卷积神经网络来识别来自图像补丁的类型并估算Windows的参数。然后将窗口类型和参数组装到程序语法中。一个简单的程序引擎是在现有的3D建模软件中构建的,产生了细粒的窗户几何形状。为了从一些标记的样品中获得有用的模型,我们利用生成对抗网络以半监督的方式训练特征提取器。对抗训练策略还可以利用未标记的数据,使训练阶段更加稳定。使用公开可用的FA \ c {C} ADE图像数据集的实验表明,在同一网络结构下,提出的培训策略可以提高分类精度的提高约10%,参数估计提高了50%。此外,在针对具有不同fa \ c {c} ADE样式的不同数据测试时,性能提高更为明显。
translated by 谷歌翻译
随着深度学习技术的快速发展和计算能力的提高,深度学习已广泛应用于高光谱图像(HSI)分类领域。通常,深度学习模型通常包含许多可训练参数,并且需要大量标记的样品来实现最佳性能。然而,关于HSI分类,由于手动标记的难度和耗时的性质,大量标记的样本通常难以获取。因此,许多研究工作侧重于建立一个少数标记样本的HSI分类的深层学习模型。在本文中,我们专注于这一主题,并对相关文献提供系统审查。具体而言,本文的贡献是双重的。首先,相关方法的研究进展根据学习范式分类,包括转移学习,积极学习和少量学习。其次,已经进行了许多具有各种最先进的方法的实验,总结了结果以揭示潜在的研究方向。更重要的是,虽然深度学习模型(通常需要足够的标记样本)和具有少量标记样本的HSI场景之间存在巨大差距,但是通过深度学习融合,可以很好地表征小样本集的问题方法和相关技术,如转移学习和轻量级模型。为了再现性,可以在HTTPS://github.com/shuguoj/hsi-classification中找到纸张中评估的方法的源代码.git。
translated by 谷歌翻译
最近,已经观察到,转移学习解决方案可能是我们解决许多少量学习基准的全部 - 因此提出了有关何时以及如何部署元学习算法的重要问题。在本文中,我们试图通过1.提出一个新颖的指标(多样性系数)来阐明这些问题,以测量几次学习基准和2.的任务多样性。 )并在公平条件下进行学习(相同的体系结构,相同的优化器和所有经过培训的模型)。使用多样性系数,我们表明流行的迷你胶原和Cifar-fs几乎没有学习基准的多样性低。这种新颖的洞察力将转移学习解决方案比在公平比较的低多样性方面的元学习解决方案更好。具体而言,我们从经验上发现,低多样性系数与转移学习和MAML学习解决方案之间的高相似性在元测试时间和分类层相似性方面(使用基于特征的距离指标,例如SVCCA,PWCCA,CKA和OPD) )。为了进一步支持我们的主张,我们发现这种元测试的准确性仍然存在,即使模型大小变化也是如此。因此,我们得出的结论是,在低多样性制度中,MAML和转移学习在公平比较时具有等效的元检验性能。我们也希望我们的工作激发了对元学习基准测试基准的更周到的结构和定量评估。
translated by 谷歌翻译