We introduce MegaPose, a method to estimate the 6D pose of novel objects, that is, objects unseen during training. At inference time, the method only assumes knowledge of (i) a region of interest displaying the object in the image and (ii) a CAD model of the observed object. The contributions of this work are threefold. First, we present a 6D pose refiner based on a render&compare strategy which can be applied to novel objects. The shape and coordinate system of the novel object are provided as inputs to the network by rendering multiple synthetic views of the object's CAD model. Second, we introduce a novel approach for coarse pose estimation which leverages a network trained to classify whether the pose error between a synthetic rendering and an observed image of the same object can be corrected by the refiner. Third, we introduce a large-scale synthetic dataset of photorealistic images of thousands of objects with diverse visual and shape properties and show that this diversity is crucial to obtain good generalization performance on novel objects. We train our approach on this large synthetic dataset and apply it without retraining to hundreds of novel objects in real images from several pose estimation benchmarks. Our approach achieves state-of-the-art performance on the ModelNet and YCB-Video datasets. An extensive evaluation on the 7 core datasets of the BOP challenge demonstrates that our approach achieves performance competitive with existing approaches that require access to the target objects during training. Code, dataset and trained models are available on the project page: https://megapose6d.github.io/.

在从机器人控制到仿真的各种机器人应用中，碰撞检测似乎是规范操作，包括运动计划和估计。尽管该主题的开创性工作可以追溯到80年代，但直到最近，正确区分碰撞检测的问题才成为一个中心问题，尤其要归功于科学界围绕该主题所做的持续和各种努力物理。然而，到目前为止，很少有人提出过解决方案，并且只有对所涉及形状的性质的强烈假设。在这项工作中，我们引入了一种通用和高效的方法，以计算任何一对凸形的碰撞检测的导数，这是通过尤其利用随机平滑技术而显示的，这些技术特别适合于捕获非平滑问题的衍生物。这种方法是在HPP-FCL和Pinocchio生态系统中实现的，并在机器人文献的经典数据集和问题上进行了评估，显示了很少的微秒时间来计算许多真实的机器人应用程序直接利用的信息衍生物，包括许多真实的机器人应用程序，包括可不同的模拟。

强化学习（RL）和轨迹优化（TO）具有强大的互补优势。一方面，RL方法能够直接从数据中学习全球控制策略，但通常需要大型样本量以正确地收敛于可行的策略。另一方面，对方法能够利用从模拟器提取的基于梯度的信息，以快速收敛到局部最佳控制轨迹，该轨迹仅在解决方案附近有效。在过去的十年中，几种方法旨在充分结合两类方法，以获得两全其美的最佳选择。从这一研究开始，我们提出了这些方法的一些改进，以更快地学习全球控制政策，尤其是通过通过Sobolev学习来利用敏感性信息，并增强了Lagrangian技术来实施与政策学习之间的共识。我们通过与文献中的现有方法进行比较，评估了这些改进对机器人技术各种经典任务的好处。

在过去的几年中，按照可区分的编程范式，人们对计算物理过程的梯度信息（例如，物理模拟，图像渲染）的梯度越来越兴趣。但是，此类过程可能是不可差异的，也可能产生非信息性梯度（I.D.几乎到处都是无效的）。当面对以前的陷阱时，通过分析表达或数值技术（例如自动分化和有限差异）估算的梯度使经典优化方案融合到质量较差的解决方案中。因此，仅依靠这些梯度提供的本地信息通常不足以解决涉及此类物理过程的高级优化问题，尤其是当它们受到非平滑度和不稳定性问题的影响。零订单优化，我们通过估计邻域中的梯度来利用随机平滑来增强可微分的物理。我们的实验表明，在优化算法中整合这种方法可能对像网格重建的任务相似，从图像或对机器人系统的最佳控制也有所不同。

机器人将机器人的无缝集成到人类环境需要机器人来学习如何使用现有的人类工具。学习工具操纵技能的目前方法主要依赖于目标机器人环境中提供的专家演示，例如，通过手动引导机器人操纵器或通过远程操作。在这项工作中，我们介绍了一种自动化方法，取代了一个专家演示，用YouTube视频来学习工具操纵策略。主要贡献是双重的。首先，我们设计一个对齐过程，使模拟环境与视频中观察到的真实世界。这是作为优化问题，找到刀具轨迹的空间对齐，以最大化环境给出的稀疏目标奖励。其次，我们描述了一种专注于工具的轨迹而不是人类的运动的模仿学习方法。为此，我们将加强学习与优化过程相结合，以基于对准环境中的工具运动来找到控制策略和机器人的放置。我们展示了仿真中的铲子，镰刀和锤子工具的建议方法，并展示了训练有素的政策对真正的弗兰卡·埃米卡熊猫机器人示范的卫生政策的有效性。

在本文中，我们介绍一种方法来自动重建与来自单个RGB视频相互作用的人的3D运动。我们的方法估计人的3D与物体姿势，接触位置和施加在人体上的接触力的姿势。这项工作的主要贡献是三倍。首先，我们介绍一种通过建模触点和相互作用的动态来联合估计人与人的运动和致动力的方法。这是一个大规模的轨迹优化问题。其次，我们开发一种方法来从输入视频自动识别，从输入视频中识别人和物体或地面之间的2D位置和时序，从而显着简化了优化的复杂性。第三，我们在最近的视频+ Mocap数据集上验证了捕获典型的Parkour行动的方法，并在互联网视频的新数据集上展示其表现，显示人们在不受约束的环境中操纵各种工具。

We introduce an approach for recovering the 6D pose of multiple known objects in a scene captured by a set of input images with unknown camera viewpoints. First, we present a single-view single-object 6D pose estimation method, which we use to generate 6D object pose hypotheses. Second, we develop a robust method for matching individual 6D object pose hypotheses across different input images in order to jointly estimate camera viewpoints and 6D poses of all objects in a single consistent scene. Our approach explicitly handles object symmetries, does not require depth measurements, is robust to missing or incorrect object hypotheses, and automatically recovers the number of objects in the scene. Third, we develop a method for global scene refinement given multiple object hypotheses and their correspondences across views. This is achieved by solving an object-level bundle adjustment problem that refines the poses of cameras and objects to minimize the reprojection error in all views. We demonstrate that the proposed method, dubbed Cosy-Pose, outperforms current state-of-the-art results for single-view and multi-view 6D object pose estimation by a large margin on two challenging benchmarks: the YCB-Video and T-LESS datasets. Code and pre-trained models are available on the project webpage. 5

Participants in political discourse employ rhetorical strategies -- such as hedging, attributions, or denials -- to display varying degrees of belief commitments to claims proposed by themselves or others. Traditionally, political scientists have studied these epistemic phenomena through labor-intensive manual content analysis. We propose to help automate such work through epistemic stance prediction, drawn from research in computational semantics, to distinguish at the clausal level what is asserted, denied, or only ambivalently suggested by the author or other mentioned entities (belief holders). We first develop a simple RoBERTa-based model for multi-source stance predictions that outperforms more complex state-of-the-art modeling. Then we demonstrate its novel application to political science by conducting a large-scale analysis of the Mass Market Manifestos corpus of U.S. political opinion books, where we characterize trends in cited belief holders -- respected allies and opposed bogeymen -- across U.S. political ideologies.

3D shapes have complementary abstractions from low-level geometry to part-based hierarchies to languages, which convey different levels of information. This paper presents a unified framework to translate between pairs of shape abstractions: $\textit{Text}$ $\Longleftrightarrow$ $\textit{Point Cloud}$ $\Longleftrightarrow$ $\textit{Program}$. We propose $\textbf{Neural Shape Compiler}$ to model the abstraction transformation as a conditional generation process. It converts 3D shapes of three abstract types into unified discrete shape code, transforms each shape code into code of other abstract types through the proposed $\textit{ShapeCode Transformer}$, and decodes them to output the target shape abstraction. Point Cloud code is obtained in a class-agnostic way by the proposed $\textit{Point}$VQVAE. On Text2Shape, ShapeGlot, ABO, Genre, and Program Synthetic datasets, Neural Shape Compiler shows strengths in $\textit{Text}$ $\Longrightarrow$ $\textit{Point Cloud}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Text}$, $\textit{Point Cloud}$ $\Longrightarrow$ $\textit{Program}$, and Point Cloud Completion tasks. Additionally, Neural Shape Compiler benefits from jointly training on all heterogeneous data and tasks.

In order for artificial neural networks to begin accurately mimicking biological ones, they must be able to adapt to new exigencies without forgetting what they have learned from previous training. Lifelong learning approaches to artificial neural networks attempt to strive towards this goal, yet have not progressed far enough to be realistically deployed for natural language processing tasks. The proverbial roadblock of catastrophic forgetting still gate-keeps researchers from an adequate lifelong learning model. While efforts are being made to quell catastrophic forgetting, there is a lack of research that looks into the importance of class ordering when training on new classes for incremental learning. This is surprising as the ordering of "classes" that humans learn is heavily monitored and incredibly important. While heuristics to develop an ideal class order have been researched, this paper examines class ordering as it relates to priming as a scheme for incremental class learning. By examining the connections between various methods of priming found in humans and how those are mimicked yet remain unexplained in life-long machine learning, this paper provides a better understanding of the similarities between our biological systems and the synthetic systems while simultaneously improving current practices to combat catastrophic forgetting. Through the merging of psychological priming practices with class ordering, this paper is able to identify a generalizable method for class ordering in NLP incremental learning tasks that consistently outperforms random class ordering.