The GLOM architecture proposed by Hinton [2021] is a recurrent neural network for parsing an image into a hierarchy of wholes and parts. When a part is ambiguous, GLOM assumes that the ambiguity can be resolved by allowing the part to make multi-modal predictions for the pose and identity of the whole to which it belongs and then using attention to similar predictions coming from other possibly ambiguous parts to settle on a common mode that is predicted by several different parts. In this study, we describe a highly simplified version of GLOM that allows us to assess the effectiveness of this way of dealing with ambiguity. Our results show that, with supervised training, GLOM is able to successfully form islands of very similar embedding vectors for all of the locations occupied by the same object and it is also robust to strong noise injections in the input and to out-of-distribution input transformations.

A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule becomes active. We show that a discrimininatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher level capsules whose activity vectors have a big scalar product with the prediction coming from the lower-level capsule.

Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features. Yet, most deep learning approaches learn distributed representations that do not capture the compositional properties of natural scenes. In this paper, we present the Slot Attention module, an architectural component that interfaces with perceptual representations such as the output of a convolutional neural network and produces a set of task-dependent abstract representations which we call slots. These slots are exchangeable and can bind to any object in the input by specializing through a competitive procedure over multiple rounds of attention. We empirically demonstrate that Slot Attention can extract object-centric representations that enable generalization to unseen compositions when trained on unsupervised object discovery and supervised property prediction tasks.

The artificial neural networks that are used to recognize shapes typically use one or more layers of learned feature detectors that produce scalar outputs. By contrast, the computer vision community uses complicated, hand-engineered features, like SIFT [6], that produce a whole vector of outputs including an explicit representation of the pose of the feature. We show how neural networks can be used to learn features that output a whole vector of instantiation parameters and we argue that this is a much more promising way of dealing with variations in position, orientation, scale and lighting than the methods currently employed in the neural networks community. It is also more promising than the handengineered features currently used in computer vision because it provides an efficient way of adapting the features to the domain.

最近，数据驱动的单视图重建方法在建模3D穿着人类中表现出很大的进展。然而，这种方法严重影响了单视图输入所固有的深度模糊和闭塞。在本文中，我们通过考虑一小部分输入视图并调查从这些视图中适当利用信息的最佳策略来解决这个问题。我们提出了一种数据驱动的端到端方法，其从稀疏相机视图重建穿着人的人类的隐式3D表示。具体而言，我们介绍了三个关键组件：首先是使用透视相机模型的空间一致的重建，允许使用人员在输入视图中的任意放置;第二个基于关注的融合层，用于从多个观点来看聚合视觉信息;第三种机制在多视图上下文下编码本地3D模式。在实验中，我们展示了所提出的方法优于定量和定性地在标准数据上表达现有技术。为了展示空间一致的重建，我们将我们的方法应用于动态场景。此外，我们在使用多摄像头平台获取的真实数据上应用我们的方法，并证明我们的方法可以获得与多视图立体声相当的结果，从而迅速更少的视图。

在本文中，我们提出了TAC2POSE，这是一种特定于对象的触觉方法，从首次触摸已知对象的触觉估计。鉴于对象几何形状，我们在模拟中学习了一个量身定制的感知模型，该模型估计了给定触觉观察的可能对象姿势的概率分布。为此，我们模拟了一个密集的物体姿势将在传感器上产生的密集对象姿势的接触形状。然后，鉴于从传感器获得的新接触形状，我们使用使用对比度学习学习的对象特定于对象的嵌入式将其与预计集合进行了匹配。我们从传感器中获得接触形状，并具有对象不足的校准步骤，该步骤将RGB触觉观测值映射到二进制接触形状。该映射可以在对象和传感器实例上重复使用，是唯一接受真实传感器数据训练的步骤。这导致了一种感知模型，该模型从第一个真实的触觉观察中定位对象。重要的是，它产生姿势分布，并可以纳入来自其他感知系统，联系人或先验的其他姿势限制。我们为20个对象提供定量结果。 TAC2POSE从独特的触觉观测中提供了高精度的姿势估计，同时回归有意义的姿势分布，以说明可能由不同对象姿势产生的接触形状。我们还测试了从3D扫描仪重建的对象模型上的TAC2POSE，以评估对象模型中不确定性的鲁棒性。最后，我们证明了TAC2POSE的优势与三种基线方法进行触觉姿势估计：直接使用神经网络回归对象姿势，将观察到的接触与使用标准分类神经网络的一组可能的接触匹配，并直接的像素比较比较观察到的一组可能的接触接触。网站：http：//mcube.mit.edu/research/tac2pose.html

The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, auto-encoders, manifold learning, and deep networks. This motivates longer-term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation and manifold learning.

以对象为中心的表示是通过提供柔性抽象可以在可以建立的灵活性抽象来实现更系统的推广的有希望的途径。最近的简单2D和3D数据集的工作表明，具有对象的归纳偏差的模型可以学习段，并代表单独的数据的统计结构中的有意义对象，而无需任何监督。然而，尽管使用越来越复杂的感应偏差（例如，用于场景的尺寸或3D几何形状），但这种完全无监督的方法仍然无法扩展到不同的现实数据。在本文中，我们采取了弱监督的方法，并专注于如何使用光流的形式的视频数据的时间动态，2）调节在简单的对象位置上的模型可以用于启用分段和跟踪对象在明显更现实的合成数据中。我们介绍了一个顺序扩展，以便引入我们训练的推出，我们训练用于预测现实看的合成场景的光流，并显示调节该模型的初始状态在一小组提示，例如第一帧中的物体的质量中心，是足以显着改善实例分割。这些福利超出了新型对象，新颖背景和更长的视频序列的培训分配。我们还发现，在推论期间可以使用这种初始状态调节作为对特定物体或物体部分的型号查询模型，这可能会为一系列弱监管方法铺平，并允许更有效的互动训练有素的型号。

胶囊网络（参见例如Hinton等，2018）旨在编码有关对象及其部分之间关系的知识和理由。在本文中，我们为此类数据指定了一个生成模型，并得出了一种用于推断场景中每个模型对象转换的变异算法以及观察到的部分对对象的分配。我们基于变异期望最大化来得出对象模型的学习算法（Jordan等，1999）。我们还根据Fischler和Bolles（1981）的RANSAC方法研究了一种替代推理算法。我们将这些推理方法应用于（i）从正方形和三角形（“星座”）等多个几何对象生成的数据，以及（ii）基于零件的面部模型的数据。 Kosiorek等人的最新工作。 （2019年）通过堆叠的胶囊自动编码器（SCAE）使用摊销推理来解决此问题 - 我们的结果表明，我们在可以进行比较的地方（在星座数据上）大大优于它们。

以对象表示的学习背后的想法是，自然场景可以更好地建模为对象的组成及其关系，而不是分布式表示形式。可以将这种归纳偏置注入神经网络中，以可能改善具有多个对象的场景中下游任务的系统概括和性能。在本文中，我们在五个常见的多对象数据集上训练最先进的无监督模型，并评估细分指标和下游对象属性预测。此外，我们通过调查单个对象不超出分布的设置（例如，具有看不见的颜色，质地或形状或场景的全局属性）来研究概括和鲁棒性，例如，通过闭塞来改变，裁剪或增加对象的数量。从我们的实验研究中，我们发现以对象为中心的表示对下游任务很有用，并且通常对影响对象的大多数分布转移有用。但是，当分布转移以较低结构化的方式影响输入时，在模型和分布转移的情况下，分割和下游任务性能的鲁棒性可能会有很大差异。

Multilayer Neural Networks trained with the backpropagation algorithm constitute the best example of a successful Gradient-Based Learning technique. Given an appropriate network architecture, Gradient-Based Learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques.Real-life document recognition systems are composed of multiple modules including eld extraction, segmentation, recognition, and language modeling. A new learning paradigm, called Graph Transformer Networks (GTN), allows such multi-module systems to be trained globally using Gradient-Based methods so as to minimize an overall performance measure.Two systems for on-line handwriting recognition are described. Experiments demonstrate the advantage of global training, and the exibility of Graph Transformer Networks.A Graph Transformer Network for reading bank check is also described. It uses Convolutional Neural Network character recognizers combined with global training techniques to provides record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.

模棱两可的神经网络，其隐藏的特征根据G组作用于数据的表示，表现出训练效率和提高的概括性能。在这项工作中，我们将群体不变和模棱两可的表示学习扩展到无监督的深度学习领域。我们根据编码器框架提出了一种通用学习策略，其中潜在表示以不变的术语和模棱两可的组动作组件分开。关键的想法是，网络学会通过学习预测适当的小组操作来对齐输入和输出姿势以解决重建任务的适当组动作来编码和从组不变表示形式进行编码和解码数据。我们在Equivariant编码器上得出必要的条件，并提出了对任何G（离散且连续的）有效的构造。我们明确描述了我们的旋转，翻译和排列的构造。我们在采用不同网络体系结构的各种数据类型的各种实验中测试了方法的有效性和鲁棒性。

The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth further investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities, including minus the sum of the squared activities. If the positive and negative passes could be separated in time, the negative passes could be done offline, which would make the learning much simpler in the positive pass and allow video to be pipelined through the network without ever storing activities or stopping to propagate derivatives.

The International Workshop on Reading Music Systems (WoRMS) is a workshop that tries to connect researchers who develop systems for reading music, such as in the field of Optical Music Recognition, with other researchers and practitioners that could benefit from such systems, like librarians or musicologists. The relevant topics of interest for the workshop include, but are not limited to: Music reading systems; Optical music recognition; Datasets and performance evaluation; Image processing on music scores; Writer identification; Authoring, editing, storing and presentation systems for music scores; Multi-modal systems; Novel input-methods for music to produce written music; Web-based Music Information Retrieval services; Applications and projects; Use-cases related to written music. These are the proceedings of the 3rd International Workshop on Reading Music Systems, held in Alicante on the 23rd of July 2021.

机器人操纵可以配制成诱导一系列空间位移：其中移动的空间可以包括物体，物体的一部分或末端执行器。在这项工作中，我们提出了一个简单的模型架构，它重新排列了深度功能，以从视觉输入推断出可视输入的空间位移 - 这可以参数化机器人操作。它没有对象的假设（例如规范姿势，模型或关键点），它利用空间对称性，并且比我们学习基于视觉的操纵任务的基准替代方案更高的样本效率，并且依赖于堆叠的金字塔用看不见的物体组装套件;从操纵可变形的绳索，以将堆积的小物体推动，具有闭环反馈。我们的方法可以表示复杂的多模态策略分布，并推广到多步顺序任务，以及6dof拾取器。 10个模拟任务的实验表明，它比各种端到端基线更快地学习并概括，包括使用地面真实对象姿势的政策。我们在现实世界中使用硬件验证我们的方法。实验视频和代码可在https://transporternets.github.io获得

Understanding the 3D world without supervision is currently a major challenge in computer vision as the annotations required to supervise deep networks for tasks in this domain are expensive to obtain on a large scale. In this paper, we address the problem of unsupervised viewpoint estimation. We formulate this as a self-supervised learning task, where image reconstruction provides the supervision needed to predict the camera viewpoint. Specifically, we make use of pairs of images of the same object at training time, from unknown viewpoints, to self-supervise training by combining the viewpoint information from one image with the appearance information from the other. We demonstrate that using a perspective spatial transformer allows efficient viewpoint learning, outperforming existing unsupervised approaches on synthetic data, and obtains competitive results on the challenging PASCAL3D+ dataset.

我们的工作重点是开发人类姿势的可学习神经代表，用于先进的AI辅助动画工具。具体而言，我们解决了基于稀疏和可变的用户输入（例如，身体关节子集的位置和/或方向）构建完整静态人姿势的问题。为了解决这个问题，我们提出了一种新型的神经结构，将残留连接与部分指定姿势编码的原型结合在一起，以从学习的潜在空间中创建一个新的完整姿势。我们表明，在准确性和计算效率方面，我们的体系结构的表现优于基准基线。此外，我们开发了一个用户界面，以将我们的神经模型集成到Unity，这是一个实时3D开发平台。此外，我们基于高质量的人类运动捕获数据，介绍了代表静态人类姿势建模问题的两个新数据集，该数据将与模型代码一起公开发布。

这本数字本书包含在物理模拟的背景下与深度学习相关的一切实际和全面的一切。尽可能多，所有主题都带有Jupyter笔记本的形式的动手代码示例，以便快速入门。除了标准的受监督学习的数据中，我们将看看物理丢失约束，更紧密耦合的学习算法，具有可微分的模拟，以及加强学习和不确定性建模。我们生活在令人兴奋的时期：这些方法具有从根本上改变计算机模拟可以实现的巨大潜力。

单图像姿势估计是许多视觉和机器人任务中的一个基本问题，并且现有的深度学习方法不会完全建模和处理来遭受：i）关于预测的不确定性，ii）具有多个（有时是无限）正确姿势的对称对象。为此，我们引入了一种在SO（3）上估算任意非参数分布的方法。我们的关键思想是通过神经网络隐含地表示分布，该神经网络估计给定输入图像和候选姿势的概率。网格采样或梯度上升可用于找到最有可能的姿势，但也可以评估任何姿势的概率，从而实现关于对称性和不确定性的推理。这是代表流形分布的最通用方法，为了展示丰富的表现力，我们介绍了一个具有挑战性的对称和几乎对称对象的数据集。我们不需要对姿势不确定性的监督 - 模型仅以一个示例训练单个姿势。但是，我们的隐式模型具有高度表达能力在3D姿势上处理复杂的分布，同时仍然在标准的非歧义环境上获得准确的姿势估计，从而在Pascal3d+和ModelNet10-SO-SO（3）基准方面实现了最先进的性能。

以对象为中心的表示是人类感知的基础，并使我们能够对世界进行推理，并系统地推广到新的环境。当前，大多数在无监督的对象发现上的作品集中在基于插槽的方法上，这些方法明确将单个对象的潜在表示分开。尽管结果很容易解释，但通常需要设计相关建筑的设计。与此相反，我们提出了一种相对简单的方法 - 复杂的自动编码器（CAE） - 创建分布式以对象为中心的表示。遵循对生物神经元中对象表示为基础的编码方案，其复杂值激活表示两个消息：它们的幅度表达了特征的存在，而神经元之间的相对相位差异应绑定在一起以创建关节对象表示。 。与以前使用复杂值激活进行对象发现的方法相反，我们提出了一种完全无监督的方法，该方法是端到端训练的 - 导致了性能和效率的显着提高。此外，我们表明，与最新的基于最新的插槽方法相比，CAE在简单的多对象数据集上实现了竞争性或更好的无监督对象发现性能，同时训练的速度要快100倍。