Our education system comprises a series of curricula. For example, when we learn mathematics at school, we learn in order from addition, to multiplication, and later to integration. Delineating a curriculum for teaching either a human or a machine shares the underlying goal of maximizing the positive knowledge transfer from early to later tasks and minimizing forgetting of the early tasks. Here, we exhaustively surveyed the effect of curricula on existing continual learning algorithms in the class-incremental setting, where algorithms must learn classes one at a time from a continuous stream of data. We observed that across a breadth of possible class orders (curricula), curricula influence the retention of information and that this effect is not just a product of stochasticity. Further, as a primary effort toward automated curriculum design, we proposed a method capable of designing and ranking effective curricula based on inter-class feature similarities. We compared the predicted curricula against empirically determined effectual curricula and observed significant overlaps between the two. To support the study of a curriculum designer, we conducted a series of human psychophysics experiments and contributed a new Continual Learning benchmark in object recognition. We assessed the degree of agreement in effective curricula between humans and machines. Surprisingly, our curriculum designer successfully predicts an optimal set of curricula that is effective for human learning. There are many considerations in curriculum design, such as timely student feedback and learning with multiple modalities. Our study is the first attempt to set a standard framework for the community to tackle the problem of teaching humans and machines to learn to learn continuously.

Artificial neural networks thrive in solving the classification problem for a particular rigid task, acquiring knowledge through generalized learning behaviour from a distinct training phase. The resulting network resembles a static entity of knowledge, with endeavours to extend this knowledge without targeting the original task resulting in a catastrophic forgetting. Continual learning shifts this paradigm towards networks that can continually accumulate knowledge over different tasks without the need to retrain from scratch. We focus on task incremental classification, where tasks arrive sequentially and are delineated by clear boundaries. Our main contributions concern (1) a taxonomy and extensive overview of the state-of-the-art; (2) a novel framework to continually determine the stability-plasticity trade-off of the continual learner; (3) a comprehensive experimental comparison of 11 state-of-the-art continual learning methods and 4 baselines. We empirically scrutinize method strengths and weaknesses on three benchmarks, considering Tiny Imagenet and large-scale unbalanced iNaturalist and a sequence of recognition datasets. We study the influence of model capacity, weight decay and dropout regularization, and the order in which the tasks are presented, and qualitatively compare methods in terms of required memory, computation time and storage.

我们研究深度神经网络中不同的输出层如何学习并忘记在持续的学习环境中。以下三个因素可能会影响输出层中的灾难性忘记：（1）权重修改，（2）干扰和（3）投影漂移。在本文中，我们的目标是提供更多关于如何改变输出层可以解决（1）和（2）的洞察。在几个连续学习情景中提出并评估了这些问题的一些潜在解决方案。我们表明，最佳执行类型的输出层取决于数据分布漂移和/或可用数据量。特别地，在某些情况下，在某些情况下，标准线性层将失败，结果改变参数化是足够的，以便实现显着更好的性能，从而引入持续学习算法，而是使用标准SGD训练模型。我们的分析和结果在连续学习场景中输出层动态的阐明，并表明了一种选择给定场景的最佳输出层的方法。

Continual Learning (CL) is a field dedicated to devise algorithms able to achieve lifelong learning. Overcoming the knowledge disruption of previously acquired concepts, a drawback affecting deep learning models and that goes by the name of catastrophic forgetting, is a hard challenge. Currently, deep learning methods can attain impressive results when the data modeled does not undergo a considerable distributional shift in subsequent learning sessions, but whenever we expose such systems to this incremental setting, performance drop very quickly. Overcoming this limitation is fundamental as it would allow us to build truly intelligent systems showing stability and plasticity. Secondly, it would allow us to overcome the onerous limitation of retraining these architectures from scratch with the new updated data. In this thesis, we tackle the problem from multiple directions. In a first study, we show that in rehearsal-based techniques (systems that use memory buffer), the quantity of data stored in the rehearsal buffer is a more important factor over the quality of the data. Secondly, we propose one of the early works of incremental learning on ViTs architectures, comparing functional, weight and attention regularization approaches and propose effective novel a novel asymmetric loss. At the end we conclude with a study on pretraining and how it affects the performance in Continual Learning, raising some questions about the effective progression of the field. We then conclude with some future directions and closing remarks.

在不忘记以前的任务的情况下不断获得新知识的能力仍然是计算机视觉系统的具有挑战性问题。标准的持续学习基准专注于在离线设置中从静态IID图像学习。在这里，我们研究了一个更具挑战性和现实的在线持续学习问题，称为在线流学习。像人类一样，一些AI代理必须从连续的不重复数据流逐步学习。我们提出了一种新颖的模型，假设驱动的增强存储器网络（HAMN），其有效地使用“假设”的增强内存矩阵来巩固先前的知识，并重播重建的图像特征以避免灾难性的遗忘。与像素级和生成的重播方法相比，Hamn的优点是两倍。首先，基于假设的知识合并避免了图像像素空间中的冗余信息，并使内存使用更有效。其次，增强记忆中的假设可以重新用于学习新任务，提高泛化和转移学习能力。鉴于视频流缺乏在线增量类学习数据集，我们介绍并调整两个额外的视频数据集，Toybox和Ilab，用于在线流学习。我们还在Core50和在线CIFAR100数据集上评估我们的方法。我们的方法显着优于所有最先进的方法，同时提供更有效的内存使用情况。所有源代码和数据都在https://github.com/kreimanlab/augmem公开使用

持续的学习方法努力减轻灾难性遗忘（CF），在学习新任务时，从以前学习的任务中丢失了知识。在这些算法中，有些在训练时维护以前任务中的样本子集。这些样本称为内存。这些方法表现出出色的性能，同时在概念上简单易于实现。然而，尽管它们很受欢迎，但几乎没有做任何事情来理解要包含在记忆中的元素。当前，这种记忆通常是通过随机抽样填充的，没有指导原则可以有助于保留以前的知识。在这项工作中，我们提出了一个基于称为一致性意识采样（CAWS）的样本的学习一致性的标准。该标准优先考虑通过深网更容易学习的样本。我们对三种不同的基于内存的方法进行研究：AGEM，GDUMB和经验重播，在MNIST，CIFAR-10和CIFAR-100数据集上。我们表明，使用最一致的元素在受到计算预算的约束时会产生性能提高；如果在没有这种约束的情况下，随机抽样是一个强大的基线。但是，在经验重播上使用CAWS可以改善随机基线的性能。最后，我们表明CAWS取得了与流行的内存选择方法相似的结果，同时需要大大减少计算资源。

人类智慧的主食是以不断的方式获取知识的能力。在Stark对比度下，深网络忘记灾难性，而且为此原因，类增量连续学习促进方法的子字段逐步学习一系列任务，将顺序获得的知识混合成综合预测。这项工作旨在评估和克服我们以前提案黑暗体验重播（Der）的陷阱，这是一种简单有效的方法，将排练和知识蒸馏结合在一起。灵感来自于我们的思想不断重写过去的回忆和对未来的期望，我们赋予了我的能力，即我的能力来修改其重播记忆，以欢迎有关过去数据II的新信息II）为学习尚未公开的课程铺平了道路。我们表明，这些策略的应用导致了显着的改进;实际上，得到的方法 - 被称为扩展-DAR（X-DER） - 优于标准基准（如CiFar-100和MiniimAgeNet）的技术状态，并且这里引入了一个新颖的。为了更好地了解，我们进一步提供了广泛的消融研究，以证实并扩展了我们以前研究的结果（例如，在持续学习设置中知识蒸馏和漂流最小值的价值）。

Many real-world learning scenarios face the challenge of slow concept drift, where data distributions change gradually over time. In this setting, we pose the problem of learning temporally sensitive importance weights for training data, in order to optimize predictive accuracy. We propose a class of temporal reweighting functions that can capture multiple timescales of change in the data, as well as instance-specific characteristics. We formulate a bi-level optimization criterion, and an associated meta-learning algorithm, by which these weights can be learned. In particular, our formulation trains an auxiliary network to output weights as a function of training instances, thereby compactly representing the instance weights. We validate our temporal reweighting scheme on a large real-world dataset of 39M images spread over a 9 year period. Our extensive experiments demonstrate the necessity of instance-based temporal reweighting in the dataset, and achieve significant improvements to classical batch-learning approaches. Further, our proposal easily generalizes to a streaming setting and shows significant gains compared to recent continual learning methods.

人类的持续学习（CL）能力与稳定性与可塑性困境密切相关，描述了人类如何实现持续的学习能力和保存的学习信息。自发育以来，CL的概念始终存在于人工智能（AI）中。本文提出了对CL的全面审查。与之前的评论不同，主要关注CL中的灾难性遗忘现象，本文根据稳定性与可塑性机制的宏观视角来调查CL。类似于生物对应物，“智能”AI代理商应该是I）记住以前学到的信息（信息回流）; ii）不断推断新信息（信息浏览:); iii）转移有用的信息（信息转移），以实现高级CL。根据分类学，评估度量，算法，应用以及一些打开问题。我们的主要贡献涉及I）从人工综合情报层面重新检查CL; ii）在CL主题提供详细和广泛的概述; iii）提出一些关于CL潜在发展的新颖思路。

持续的学习是遭受灾难性的遗忘，这是一个早期学识渊博的概念被遗忘的现象，以牺牲更新的样本。在这项工作中，我们挑战持续学习不可避免地与灾难性忘记相关的假设，通过展示一系列令人惊讶的是在不断学习时令人惊讶地没有灾难性的遗忘遗忘。我们提供了证据表明，这些重建类型任务表现出正向转移，并且单视网型重建随着时间的推移提高了学习和新型类别的性能。通过查看顺序学习任务的产出分配转移，我们提供了对知识转移能力的新颖分析。最后，我们表明这些任务的稳健性导致具有用于连续分类的代理代表学习任务的可能性。可以在https://github.com/rehg-lab/lrorec中找到与本文发布的CodeBase，DataSet和预训练模型。

Despite significant advances, the performance of state-of-the-art continual learning approaches hinges on the unrealistic scenario of fully labeled data. In this paper, we tackle this challenge and propose an approach for continual semi-supervised learning -- a setting where not all the data samples are labeled. An underlying issue in this scenario is the model forgetting representations of unlabeled data and overfitting the labeled ones. We leverage the power of nearest-neighbor classifiers to non-linearly partition the feature space and learn a strong representation for the current task, as well as distill relevant information from previous tasks. We perform a thorough experimental evaluation and show that our method outperforms all the existing approaches by large margins, setting a strong state of the art on the continual semi-supervised learning paradigm. For example, on CIFAR100 we surpass several others even when using at least 30 times less supervision (0.8% vs. 25% of annotations).

已知应用于任务序列的标准梯度下降算法可在深层神经网络中产生灾难性遗忘。当对序列中的新任务进行培训时，该模型会在当前任务上更新其参数，从而忘记过去的知识。本文探讨了我们在有限环境中扩展任务数量的方案。这些方案由与重复数据的长期任务组成。我们表明，在这种情况下，随机梯度下降可以学习，进步并融合到根据现有文献需要持续学习算法的解决方案。换句话说，我们表明该模型在没有特定的记忆机制的情况下执行知识保留和积累。我们提出了一个新的实验框架，即Scole（缩放量表），以研究在潜在无限序列中的知识保留和算法的积累。为了探索此设置，我们对1,000个任务的序列进行了大量实验，以更好地了解这种新的设置家庭。我们还提出了对香草随机梯度下降的轻微修改，以促进这种情况下的持续学习。 SCOLE框架代表了对实用训练环境的良好模拟，并允许长序列研究收敛行为。我们的实验表明，在短方案上以前的结果不能总是推断为更长的场景。

尽管深度神经网络（DNNS）在封闭世界的学习方案中取得了令人印象深刻的分类性能，但它们通常无法概括地在动态的开放世界环境中看不见的类别，在这种环境中，概念数量无界的数量。相反，人类和动物学习者具有通过识别和适应新颖观察结果来逐步更新知识的能力。特别是，人类通过独家（唯一）基本特征集来表征概念，这些特征既用于识别已知类别和识别新颖性。受到自然学习者的启发，我们引入了稀疏的高级独特，低水平共享的特征表示（Shels），同时鼓励学习独家的高级功能和必不可少的，共享的低级功能。高级功能的排他性使DNN能够自动检测到分布（OOD）数据，而通过稀疏的低级功能可以有效利用容量，可以容纳新知识。最终的方法使用OOD检测来执行班级持续学习，而没有已知的类边界。我们表明，使用木材进行新颖性检测导致对各种基准数据集的最新OOD检测方法的统计显着改善。此外，我们证明了木木模型在课堂学习环境中减轻灾难性的遗忘，从而实现了一个组合的新颖性检测和住宿框架，该框架支持在开放世界中学习

Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan. This ability, referred to as lifelong learning, is mediated by a rich set of neurocognitive mechanisms that together contribute to the development and specialization of our sensorimotor skills as well as to long-term memory consolidation and retrieval. Consequently, lifelong learning capabilities are crucial for computational systems and autonomous agents interacting in the real world and processing continuous streams of information. However, lifelong learning remains a long-standing challenge for machine learning and neural network models since the continual acquisition of incrementally available information from non-stationary data distributions generally leads to catastrophic forgetting or interference. This limitation represents a major drawback for state-of-the-art deep neural network models that typically learn representations from stationary batches of training data, thus without accounting for situations in which information becomes incrementally available over time. In this review, we critically summarize the main challenges linked to lifelong learning for artificial learning systems and compare existing neural network approaches that alleviate, to different extents, catastrophic forgetting. Although significant advances have been made in domain-specific learning with neural networks, extensive research efforts are required for the development of robust lifelong learning on autonomous agents and robots. We discuss well-established and emerging research motivated by lifelong learning factors in biological systems such as structural plasticity, memory replay, curriculum and transfer learning, intrinsic motivation, and multisensory integration.

新颖的检测方法识别不代表模型训练集的样本，从而标记误导性预测并在部署时间带来更大的灵活性和透明度。但是，该领域的研究仅考虑了离线环境中的新颖性检测。最近，在计算机视觉社区中，应用程序越来越多，应用程序需要更灵活的框架 - 持续学习 - 在该框架中，代表新域，新类或新任务的新数据在不同的时间点可用。在这种情况下，新颖性检测变得越来越重要，有趣且具有挑战性。这项工作确定了这两个问题之间的关键联系，并研究了持续学习环境下的新颖性检测问题。我们制定了持续的新颖性检测问题，并提出了基准，在该基准中，我们比较了不同持续学习设置下的几种新颖性检测方法。我们表明，持续学习会影响新颖性检测算法的行为，而新颖性检测可以确定持续学习者的行为的见解。我们进一步提出了基准并讨论可能的研究方向。我们认为，这两个问题的耦合是将视觉模型付诸实践的有前途的方向。

持续学习研究的主要重点领域是通过设计新算法对分布变化更强大的新算法来减轻神经网络中的“灾难性遗忘”问题。尽管持续学习文献的最新进展令人鼓舞，但我们对神经网络的特性有助于灾难性遗忘的理解仍然有限。为了解决这个问题，我们不关注持续的学习算法，而是在这项工作中专注于模型本身，并研究神经网络体系结构对灾难性遗忘的“宽度”的影响，并表明宽度在遗忘遗产方面具有出人意料的显着影响。为了解释这种效果，我们从各个角度研究网络的学习动力学，例如梯度正交性，稀疏性和懒惰的培训制度。我们提供了与不同架构和持续学习基准之间的经验结果一致的潜在解释。

Continual Learning, also known as Lifelong or Incremental Learning, has recently gained renewed interest among the Artificial Intelligence research community. Recent research efforts have quickly led to the design of novel algorithms able to reduce the impact of the catastrophic forgetting phenomenon in deep neural networks. Due to this surge of interest in the field, many competitions have been held in recent years, as they are an excellent opportunity to stimulate research in promising directions. This paper summarizes the ideas, design choices, rules, and results of the challenge held at the 3rd Continual Learning in Computer Vision (CLVision) Workshop at CVPR 2022. The focus of this competition is the complex continual object detection task, which is still underexplored in literature compared to classification tasks. The challenge is based on the challenge version of the novel EgoObjects dataset, a large-scale egocentric object dataset explicitly designed to benchmark continual learning algorithms for egocentric category-/instance-level object understanding, which covers more than 1k unique main objects and 250+ categories in around 100k video frames.

本文认为，连续学习方法可以通过分割多种模型的学习者的容量来利益。我们使用统计学习理论和实验分析来展示多种任务在单个型号培训时以非琐碎的方式互相交互。特定任务上的泛化误差可以随着协同任务培训，但在竞争任务训练时也可以恶化。该理论激励了我们名为Model动物园的方法，这是从升压文献的启发，增长了小型型号的集合，每个集中都在持续学习的一集中训练。我们展示了模型动物园的准确性提高了各种持续学习基准问题。

Lack of performance when it comes to continual learning over non-stationary distributions of data remains a major challenge in scaling neural network learning to more human realistic settings. In this work we propose a new conceptualization of the continual learning problem in terms of a temporally symmetric trade-off between transfer and interference that can be optimized by enforcing gradient alignment across examples. We then propose a new algorithm, Meta-Experience Replay (MER), that directly exploits this view by combining experience replay with optimization based meta-learning. This method learns parameters that make interference based on future gradients less likely and transfer based on future gradients more likely. 1 We conduct experiments across continual lifelong supervised learning benchmarks and non-stationary reinforcement learning environments demonstrating that our approach consistently outperforms recently proposed baselines for continual learning. Our experiments show that the gap between the performance of MER and baseline algorithms grows both as the environment gets more non-stationary and as the fraction of the total experiences stored gets smaller.

持续学习（CL）旨在制定模仿人类能力顺序学习新任务的能力，同时能够保留从过去经验获得的知识。在本文中，我们介绍了内存约束在线连续学习（MC-OCL）的新问题，这对存储器开销对可能算法可以用于避免灾难性遗忘的记忆开销。最多，如果不是全部，之前的CL方法违反了这些约束，我们向MC-OCL提出了一种算法解决方案：批量蒸馏（BLD），基于正则化的CL方法，有效地平衡了稳定性和可塑性，以便学习数据流，同时保留通过蒸馏解决旧任务的能力。我们在三个公开的基准测试中进行了广泛的实验评估，经验证明我们的方法成功地解决了MC-OCL问题，并实现了需要更高内存开销的先前蒸馏方法的可比准确性。