By transferring knowledge from large, diverse, task-agnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small task-specific datasets to a high level of performance. While this capability has been demonstrated in other fields such as computer vision, natural language processing or speech recognition, it remains to be shown in robotics, where the generalization capabilities of the models are particularly critical due to the difficulty of collecting real-world robotic data. We argue that one of the keys to the success of such general robotic models lies with open-ended task-agnostic training, combined with high-capacity architectures that can absorb all of the diverse, robotic data. In this paper, we present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks. The project's website and videos can be found at robotics-transformer.github.io
translated by 谷歌翻译
大型语言模型可以编码有关世界的大量语义知识。这种知识对于旨在采取自然语言表达的高级,时间扩展的指示的机器人可能非常有用。但是,语言模型的一个重大弱点是,它们缺乏现实世界的经验,这使得很难利用它们在给定的体现中进行决策。例如,要求语言模型描述如何清洁溢出物可能会导致合理的叙述,但是它可能不适用于需要在特定环境中执行此任务的特定代理商(例如机器人)。我们建议通过预处理的技能来提供现实世界的基础,这些技能用于限制模型以提出可行且在上下文上适当的自然语言动作。机器人可以充当语​​言模型的“手和眼睛”,而语言模型可以提供有关任务的高级语义知识。我们展示了如何将低级技能与大语言模型结合在一起,以便语言模型提供有关执行复杂和时间扩展说明的过程的高级知识,而与这些技能相关的价值功能则提供了连接必要的基础了解特定的物理环境。我们在许多现实世界的机器人任务上评估了我们的方法,我们表明了对现实世界接地的需求,并且这种方法能够在移动操纵器上完成长远,抽象的自然语言指令。该项目的网站和视频可以在https://say-can.github.io/上找到。
translated by 谷歌翻译
Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato.
translated by 谷歌翻译
最近的作品表明,如何将大语言模型(LLM)的推理能力应用于自然语言处理以外的领域,例如机器人的计划和互动。这些具体的问题要求代理商了解世界上许多语义方面:可用技能的曲目,这些技能如何影响世界以及对世界的变化如何映射回该语言。在体现环境中规划的LLMS不仅需要考虑要做什么技能,还需要考虑如何以及何时进行操作 - 答案随着时间的推移而变化,以响应代理商自己的选择。在这项工作中,我们调查了在这种体现的环境中使用的LLM在多大程度上可以推论通过自然语言提供的反馈来源,而无需任何其他培训。我们建议,通过利用环境反馈,LLM能够形成内部独白,使他们能够在机器人控制方案中进行更丰富的处理和计划。我们研究了各种反馈来源,例如成功检测,场景描述和人类互动。我们发现,闭环语言反馈显着改善了三个领域的高级指导完成,包括模拟和真实的桌面顶部重新排列任务以及现实世界中厨房环境中的长途移动操作任务。
translated by 谷歌翻译
We present a retrospective on the state of Embodied AI research. Our analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR. These challenges are grouped into three themes: (1) visual navigation, (2) rearrangement, and (3) embodied vision-and-language. We discuss the dominant datasets within each theme, evaluation metrics for the challenges, and the performance of state-of-the-art models. We highlight commonalities between top approaches to the challenges and identify potential future directions for Embodied AI research.
translated by 谷歌翻译
与人类在环境中共存的通用机器人必须学会将人类语言与其在一系列日常任务中有用的看法和行动联系起来。此外,他们需要获取各种曲目的一般专用技能,允许通过遵循无约束语言指示来组成长地平任务。在本文中,我们呈现了凯文(从语言和愿景撰写的行动),是一个露天模拟基准,用于学习Long-Horizo​​ n语言条件的任务。我们的目的是使可以开发能够通过船上传感器解决许多机器人操纵任务的代理商,并且仅通过人类语言指定。 Calvin任务在序列长度,动作空间和语言方面更复杂,而不是现有的视觉和语言任务数据集,并支持灵活的传感器套件规范。我们评估零拍摄的代理商以新颖的语言指示以及新的环境和对象。我们表明,基于多语境模仿学习的基线模型在凯文中表现不佳,表明有很大的空间,用于开发创新代理,了解学习将人类语言与这款基准相关的世界模型。
translated by 谷歌翻译
我们研究了复杂几何物体的机器人堆叠问题。我们提出了一个挑战和多样化的这些物体,这些物体被精心设计,以便要求超出简单的“拾取”解决方案之外的策略。我们的方法是加强学习(RL)方法与基于视觉的互动政策蒸馏和模拟到现实转移相结合。我们的学习政策可以有效地处理现实世界中的多个对象组合,并展示各种各样的堆叠技能。在一个大型的实验研究中,我们调查在模拟中学习这种基于视觉的基于视觉的代理的选择,以及对真实机器人的最佳转移产生了什么影响。然后,我们利用这些策略收集的数据并通过离线RL改善它们。我们工作的视频和博客文章作为补充材料提供。
translated by 谷歌翻译
Developing robots that are capable of many skills and generalization to unseen scenarios requires progress on two fronts: efficient collection of large and diverse datasets, and training of high-capacity policies on the collected data. While large datasets have propelled progress in other fields like computer vision and natural language processing, collecting data of comparable scale is particularly challenging for physical systems like robotics. In this work, we propose a framework to bridge this gap and better scale up robot learning, under the lens of multi-task, multi-scene robot manipulation in kitchen environments. Our framework, named CACTI, has four stages that separately handle data collection, data augmentation, visual representation learning, and imitation policy training. In the CACTI framework, we highlight the benefit of adapting state-of-the-art models for image generation as part of the augmentation stage, and the significant improvement of training efficiency by using pretrained out-of-domain visual representations at the compression stage. Experimentally, we demonstrate that 1) on a real robot setup, CACTI enables efficient training of a single policy capable of 10 manipulation tasks involving kitchen objects, and robust to varying layouts of distractor objects; 2) in a simulated kitchen environment, CACTI trains a single policy on 18 semantic tasks across up to 50 layout variations per task. The simulation task benchmark and augmented datasets in both real and simulated environments will be released to facilitate future research.
translated by 谷歌翻译
我们研究了从机器人交互的大型离线数据集学习一系列基于视觉的操纵任务的问题。为了实现这一目标,人类需要简单有效地将任务指定给机器人。目标图像是一种流行的任务规范形式,因为它们已经在机器人的观察空间接地。然而,目标图像也有许多缺点:它们对人类提供的不方便,它们可以通过提供导致稀疏奖励信号的所需行为,或者在非目标达到任务的情况下指定任务信息。自然语言为任务规范提供了一种方便而灵活的替代方案,而是随着机器人观察空间的接地语言挑战。为了可扩展地学习此基础,我们建议利用具有人群源语言标签的离线机器人数据集(包括高度最佳,自主收集的数据)。使用此数据,我们学习一个简单的分类器,该分类器预测状态的更改是否完成了语言指令。这提供了一种语言调节奖励函数,然后可以用于离线多任务RL。在我们的实验中,我们发现,在语言条件的操作任务中,我们的方法优于目标 - 图像规格和语言条件仿制技术超过25%,并且能够从自然语言中执行Visuomotor任务,例如“打开右抽屉“和”移动订书机“,在弗兰卡·埃米卡熊猫机器人上。
translated by 谷歌翻译
变形金刚用大型数据集的扩展能力彻底改变了视力和自然语言处理。但是在机器人的操作中,数据既有限又昂贵。我们仍然可以从具有正确的问题制定的变压器中受益吗?我们用Peract进行了调查,这是一种用于多任务6 DOF操纵的语言条件的行为结合剂。 Peract用感知器变压器编码语言目标和RGB-D Voxel观测值,并通过“检测下一个最佳素素动作”来输出离散的动作。与在2D图像上运行的框架不同,体素化的观察和动作空间为有效学习的6-DOF策略提供了强大的结构性先验。通过此公式,我们训练一个单个多任务变压器,用于18个RLBench任务(具有249个变体)和7个现实世界任务(具有18个变体),从每个任务仅几个演示。我们的结果表明,针对各种桌面任务,佩内的磨损明显优于非结构化图像到作用剂和3D Convnet基准。
translated by 谷歌翻译
Humans are excellent at understanding language and vision to accomplish a wide range of tasks. In contrast, creating general instruction-following embodied agents remains a difficult challenge. Prior work that uses pure language-only models lack visual grounding, making it difficult to connect language instructions with visual observations. On the other hand, methods that use pre-trained vision-language models typically come with divided language and visual representations, requiring designing specialized network architecture to fuse them together. We propose a simple yet effective model for robots to solve instruction-following tasks in vision-based environments. Our \ours method consists of a multimodal transformer that encodes visual observations and language instructions, and a policy transformer that predicts actions based on encoded representations. The multimodal transformer is pre-trained on millions of image-text pairs and natural language text, thereby producing generic cross-modal representations of observations and instructions. The policy transformer keeps track of the full history of observations and actions, and predicts actions autoregressively. We show that this unified transformer model outperforms all state-of-the-art pre-trained or trained-from-scratch methods in both single-task and multi-task settings. Our model also shows better model scalability and generalization ability than prior work.
translated by 谷歌翻译
长摩根和包括一系列隐性子任务的日常任务仍然在离线机器人控制中构成了重大挑战。尽管许多先前的方法旨在通过模仿和离线增强学习的变体来解决这种设置,但学习的行为通常是狭窄的,并且经常努力实现可配置的长匹配目标。由于这两个范式都具有互补的优势和劣势,因此我们提出了一种新型的层次结构方法,结合了两种方法的优势,以从高维相机观察中学习任务无关的长胜压策略。具体而言,我们结合了一项低级政策,该政策通过模仿学习和从离线强化学习中学到的高级政策学习潜在的技能,以促进潜在的行为先验。各种模拟和真实机器人控制任务的实验表明,我们的配方使以前看不见的技能组合能够通过“缝制”潜在技能通过目标链条,并在绩效上提高绩效的顺序,从而实现潜在的目标。艺术基线。我们甚至还学习了一个多任务视觉运动策略,用于现实世界中25个不同的操纵任务,这既优于模仿学习和离线强化学习技术。
translated by 谷歌翻译
Learning policies that effectively utilize language instructions in complex, multi-task environments is an important problem in sequential decision-making. While it is possible to condition on the entire language instruction directly, such an approach could suffer from generalization issues. In our work, we propose \emph{Learning Interpretable Skill Abstractions (LISA)}, a hierarchical imitation learning framework that can learn diverse, interpretable primitive behaviors or skills from language-conditioned demonstrations to better generalize to unseen instructions. LISA uses vector quantization to learn discrete skill codes that are highly correlated with language instructions and the behavior of the learned policy. In navigation and robotic manipulation environments, LISA outperforms a strong non-hierarchical Decision Transformer baseline in the low data regime and is able to compose learned skills to solve tasks containing unseen long-range instructions. Our method demonstrates a more natural way to condition on language in sequential decision-making problems and achieve interpretable and controllable behavior with the learned skills.
translated by 谷歌翻译
通过模仿学习(IL)使用用户提供的演示,或者通过使用大量的自主收集的体验来学习机器人技能。方法具有互补的经验和缺点:RL可以达到高度的性能,但需要缺陷,但是需要缺乏要求,但是需要达到高水平的性能,但需要达到高度的性能这可能非常耗时和不安全; IL不要求Xploration,但只学习与所提供的示范一样好的技能。一种方法将两种方法的优势结合在一起?一系列的方法旨在解决这个问题,提出了整合IL和RL的元素的各种技术。然而,扩大了这种方法,这些方法复杂的机器人技能,整合了不同的离线数据,概括到现实世界的情景仍然存在重大挑战。在本文中,USAIM是测试先前IL + RL算法的可扩展性,并设计了一种系统的详细实验实验,这些实验结合了现有的组件,其具有效果有效和可扩展的方式。为此,我们展示了一系列关于了解每个设计决定的影响的一系列实验,以便开发可以利用示范和异构的先前数据在一系列现实世界和现实的模拟问题上获得最佳表现的批准方法。我们通过致电Wap-opt的完整方法将优势加权回归[1,2]和QT-opt [3]结合在一起,提供了一个UnifiedAgveach,用于集成机器人操作的演示和离线数据。请参阅HTTPS: //awopt.github.io有关更多详细信息。
translated by 谷歌翻译
强化学习(RL)算法有望为机器人系统实现自主技能获取。但是,实际上,现实世界中的机器人RL通常需要耗时的数据收集和频繁的人类干预来重置环境。此外,当部署超出知识的设置超出其学习的设置时,使用RL学到的机器人政策通常会失败。在这项工作中,我们研究了如何通过从先前看到的任务中收集的各种离线数据集的有效利用来应对这些挑战。当面对一项新任务时,我们的系统会适应以前学习的技能,以快速学习执行新任务并将环境返回到初始状态,从而有效地执行自己的环境重置。我们的经验结果表明,将先前的数据纳入机器人增强学习中可以实现自主学习,从而大大提高了学习的样本效率,并可以更好地概括。
translated by 谷歌翻译
在移动操作(MM)中,机器人可以在内部导航并与其环境进行交互,因此能够完成比仅能够导航或操纵的机器人的更多任务。在这项工作中,我们探讨如何应用模仿学习(IL)来学习MM任务的连续Visuo-Motor策略。许多事先工作表明,IL可以为操作或导航域训练Visuo-Motor策略,但很少有效应用IL到MM域。这样做是挑战的两个原因:在数据方面,当前的接口使得收集高质量的人类示范困难,在学习方面,有限数据培训的政策可能会在部署时遭受协变速转变。为了解决这些问题,我们首先提出了移动操作Roboturk(Momart),这是一种新颖的遥控框架,允许同时导航和操纵移动操纵器,并在现实的模拟厨房设置中收集一类大规模的大规模数据集。然后,我们提出了一个学习错误检测系统来解决通过检测代理处于潜在故障状态时的协变量转变。我们从该数据中培训表演者的IL政策和错误探测器,在专家数据培训时,在多个多级任务中达到超过45%的任务成功率和85%的错误检测成功率。 CodeBase,DataSets,Visualization,以及更多可用的https://sites.google.com/view/il-for-mm/home。
translated by 谷歌翻译
在人类环境中,预计在简单的自然语言指导下,机器人将完成各种操纵任务。然而,机器人的操纵极具挑战性,因为它需要精细颗粒的运动控制,长期记忆以及对以前看不见的任务和环境的概括。为了应对这些挑战,我们提出了一种基于统一的变压器方法,该方法考虑了多个输入。特别是,我们的变压器体系结构集成了(i)自然语言指示和(ii)多视图场景观察,而(iii)跟踪观察和动作的完整历史。这种方法使历史和指示之间的学习依赖性可以使用多个视图提高操纵精度。我们评估我们的方法在具有挑战性的RLBench基准和现实世界机器人方面。值得注意的是,我们的方法扩展到74个不同的RLBench任务,并超越了最新的现状。我们还解决了指导条件的任务,并证明了对以前看不见的变化的出色概括。
translated by 谷歌翻译
在工厂或房屋等环境中协助我们的机器人必须学会使用对象作为执行任务的工具,例如使用托盘携带对象。我们考虑了学习常识性知识何时可能有用的问题,以及如何与其他工具一起使用其使用以完成由人类指示的高级任务。具体而言,我们引入了一种新型的神经模型,称为Tooltango,该模型首先预测要使用的下一个工具,然后使用此信息来预测下一项动作。我们表明,该联合模型可以告知学习精细的策略,从而使机器人可以顺序使用特定工具,并在使模型更加准确的情况下增加了重要价值。 Tooltango使用图神经网络编码世界状态,包括对象和它们之间的符号关系,并使用人类教师的演示进行了培训,这些演示是指导物理模拟器中的虚拟机器人的演示。该模型学会了使用目标和动作历史的知识来参加场景,最终将符号动作解码为执行。至关重要的是,我们解决了缺少一些已知工具的看不见的环境的概括,但是存在其他看不见的工具。我们表明,通过通过从知识库中得出的预训练的嵌入来增强环境的表示,该模型可以有效地将其推广到新的环境中。实验结果表明,在预测具有看不见对象的新型环境中模拟移动操纵器的成功符号计划时,至少48.8-58.1%的绝对改善对基准的绝对改善。这项工作朝着使机器人能够快速合成复杂任务的强大计划的方向,尤其是在新颖的环境中
translated by 谷歌翻译
Transformer, originally devised for natural language processing, has also attested significant success in computer vision. Thanks to its super expressive power, researchers are investigating ways to deploy transformers to reinforcement learning (RL) and the transformer-based models have manifested their potential in representative RL benchmarks. In this paper, we collect and dissect recent advances on transforming RL by transformer (transformer-based RL or TRL), in order to explore its development trajectory and future trend. We group existing developments in two categories: architecture enhancement and trajectory optimization, and examine the main applications of TRL in robotic manipulation, text-based games, navigation and autonomous driving. For architecture enhancement, these methods consider how to apply the powerful transformer structure to RL problems under the traditional RL framework, which model agents and environments much more precisely than deep RL methods, but they are still limited by the inherent defects of traditional RL algorithms, such as bootstrapping and "deadly triad". For trajectory optimization, these methods treat RL problems as sequence modeling and train a joint state-action model over entire trajectories under the behavior cloning framework, which are able to extract policies from static datasets and fully use the long-sequence modeling capability of the transformer. Given these advancements, extensions and challenges in TRL are reviewed and proposals about future direction are discussed. We hope that this survey can provide a detailed introduction to TRL and motivate future research in this rapidly developing field.
translated by 谷歌翻译
While large-scale sequence modeling from offline data has led to impressive performance gains in natural language and image generation, directly translating such ideas to robotics has been challenging. One critical reason for this is that uncurated robot demonstration data, i.e. play data, collected from non-expert human demonstrators are often noisy, diverse, and distributionally multi-modal. This makes extracting useful, task-centric behaviors from such data a difficult generative modeling problem. In this work, we present Conditional Behavior Transformers (C-BeT), a method that combines the multi-modal generation ability of Behavior Transformer with future-conditioned goal specification. On a suite of simulated benchmark tasks, we find that C-BeT improves upon prior state-of-the-art work in learning from play data by an average of 45.7%. Further, we demonstrate for the first time that useful task-centric behaviors can be learned on a real-world robot purely from play data without any task labels or reward information. Robot videos are best viewed on our project website: https://play-to-policy.github.io
translated by 谷歌翻译