Vascular shunt insertion is a fundamental surgical procedure used to temporarily restore blood flow to tissues. It is often performed in the field after major trauma. We formulate a problem of automated vascular shunt insertion and propose a pipeline to perform Automated Vascular Shunt Insertion (AVSI) using a da Vinci Research Kit. The pipeline uses a learned visual model to estimate the locus of the vessel rim, plans a grasp on the rim, and moves to grasp at that point. The first robot gripper then pulls the rim to stretch open the vessel with a dilation motion. The second robot gripper then proceeds to insert a shunt into the vessel phantom (a model of the blood vessel) with a chamfer tilt followed by a screw motion. Results suggest that AVSI achieves a high success rate even with tight tolerances and varying vessel orientations up to 30{\deg}. Supplementary material, dataset, videos, and visualizations can be found at https://sites.google.com/berkeley.edu/autolab-avsi.
translated by 谷歌翻译
机器人外科助理(RSAs)通常用于通过专家外科医生进行微创手术。然而,长期以来充满了乏味和重复的任务,如缝合可以导致外科医生疲劳,激励缝合的自动化。随着薄反射针的视觉跟踪极具挑战性,在未反射对比涂料的情况下修改了针。作为朝向无修改针的缝合子任务自动化的步骤,我们提出了休斯顿:切换未经修改,外科手术,工具障碍针,一个问题和算法,它使用学习的主动传感策略与立体声相机本地化并对齐针头进入另一臂的可见和可访问的姿势。为了补偿机器人定位和针头感知误差,然后算法执行使用多个摄像机的高精度抓握运动。在使用Da Vinci研究套件(DVRK)的物理实验中,休斯顿成功通过了96.7%的成功率,并且能够在故障前平均地在臂32.4倍之间顺序地执行切换。在培训中看不见的针头,休斯顿实现了75-92.9%的成功率。据我们所知,这项工作是第一个研究未修改的手术针的切换。查看https://tinyurl.com/huston-surgery用于额外​​的材料。
translated by 谷歌翻译
电缆在许多环境中无处不在,但容易出现自我闭合和结,使它们难以感知和操纵。挑战通常会随着电缆长度而增加:长电缆需要更复杂的松弛管理和策略,以促进可观察性和可及性。在本文中,我们专注于使用双边机器人自动弄清长达3米的电缆。我们开发了新的运动原语,以有效地解开长电缆和专门用于此任务的新型Gripper Jaws。我们提出了缠结操作(SGTM)的滑动和抓握,该算法将这些原始物与RGBD视觉构成迭代性毫无障碍。SGTM在隔离的外手上取消了67%的成功率,图8节和更复杂的配置上的50%。可以在https://sites.google.com/view/rss-2022-untangling/home上找到补充材料,可视化和视频。
translated by 谷歌翻译
我们探索一种新的方法来感知和操纵3D铰接式物体,该物体可以概括地使机器人阐明看不见的对象。我们提出了一个基于视觉的系统,该系统学会预测各种铰接物体的各个部分的潜在运动,以指导系统的下游运动计划以表达对象。为了预测对象运动,我们训练一个神经网络,以输出一个密集的向量场,代表点云中点云中点的点运动方向。然后,我们根据该向量领域部署一个分析运动计划者,以实现产生最大发音的政策。我们完全在模拟中训练视觉系统,并演示了系统在模拟和现实世界中概括的对象实例和新颖类别的能力,并将我们的政策部署在没有任何填充的锯耶机器人上。结果表明,我们的系统在模拟和现实世界实验中都达到了最先进的性能。
translated by 谷歌翻译
本文介绍了DGBench,这是一种完全可重现的开源测试系统,可在机器人和对象之间具有不可预测的相对运动的环境中对动态抓握进行基准测试。我们使用拟议的基准比较几种视觉感知布置。由于传感器的最小范围,遮挡和有限的视野,用于静态抓握的传统感知系统无法在掌握的最后阶段提供反馈。提出了一个多摄像机的眼睛感知系统,该系统具有比常用的相机配置具有优势。我们用基于图像的视觉宣传控制器进行定量评估真实机器人的性能,并在动态掌握任务上显示出明显提高的成功率。
translated by 谷歌翻译
The accurate detection and grasping of transparent objects are challenging but of significance to robots. Here, a visual-tactile fusion framework for transparent object grasping under complex backgrounds and variant light conditions is proposed, including the grasping position detection, tactile calibration, and visual-tactile fusion based classification. First, a multi-scene synthetic grasping dataset generation method with a Gaussian distribution based data annotation is proposed. Besides, a novel grasping network named TGCNN is proposed for grasping position detection, showing good results in both synthetic and real scenes. In tactile calibration, inspired by human grasping, a fully convolutional network based tactile feature extraction method and a central location based adaptive grasping strategy are designed, improving the success rate by 36.7% compared to direct grasping. Furthermore, a visual-tactile fusion method is proposed for transparent objects classification, which improves the classification accuracy by 34%. The proposed framework synergizes the advantages of vision and touch, and greatly improves the grasping efficiency of transparent objects.
translated by 谷歌翻译
Cloth in the real world is often crumpled, self-occluded, or folded in on itself such that key regions, such as corners, are not directly graspable, making manipulation difficult. We propose a system that leverages visual and tactile perception to unfold the cloth via grasping and sliding on edges. By doing so, the robot is able to grasp two adjacent corners, enabling subsequent manipulation tasks like folding or hanging. As components of this system, we develop tactile perception networks that classify whether an edge is grasped and estimate the pose of the edge. We use the edge classification network to supervise a visuotactile edge grasp affordance network that can grasp edges with a 90% success rate. Once an edge is grasped, we demonstrate that the robot can slide along the cloth to the adjacent corner using tactile pose estimation/control in real time. See http://nehasunil.com/visuotactile/visuotactile.html for videos.
translated by 谷歌翻译
We describe a learning-based approach to handeye coordination for robotic grasping from monocular images. To learn hand-eye coordination for grasping, we trained a large convolutional neural network to predict the probability that task-space motion of the gripper will result in successful grasps, using only monocular camera images and independently of camera calibration or the current robot pose. This requires the network to observe the spatial relationship between the gripper and objects in the scene, thus learning hand-eye coordination. We then use this network to servo the gripper in real time to achieve successful grasps. To train our network, we collected over 800,000 grasp attempts over the course of two months, using between 6 and 14 robotic manipulators at any given time, with differences in camera placement and hardware. Our experimental evaluation demonstrates that our method achieves effective real-time control, can successfully grasp novel objects, and corrects mistakes by continuous servoing.
translated by 谷歌翻译
我们介绍了一个机器人组装系统,该系统简化了从产品组件的CAD模型到完整编程和自适应组装过程的设计对制造工作流程。我们的系统(在CAD工具中)捕获了特定机器人工作电脑组装过程的意图,并生成了任务级指令的配方。通过将视觉传感与深度学习的感知模型相结合,机器人推断出从生成的配方中组装设计的必要动作。感知模型是直接从模拟训练的,从而使系统可以根据CAD信息识别各个部分。我们用两个机器人的工作栏演示了系统,以组装互锁的3D零件设计。我们首先在模拟中构建和调整组装过程,并验证生成的食谱。最后,真正的机器人工作电池使用相同的行为组装了设计。
translated by 谷歌翻译
在机器人操作中,以前未见的新物体的自主抓住是一个持续的挑战。在过去的几十年中,已经提出了许多方法来解决特定机器人手的问题。最近引入的Unigrasp框架具有推广到不同类型的机器人抓手的能力。但是,此方法不适用于具有闭环约束的抓手,并且当应用于具有MultiGRASP配置的机器人手时,具有数据范围。在本文中,我们提出了有效绘制的,这是一种独立于抓手模型规范的广义掌握合成和抓地力控制方法。有效绘制利用抓地力工作空间功能,而不是Unigrasp的抓属属性输入。这在训练过程中将记忆使用量减少了81.7%,并可以推广到更多类型的抓地力,例如具有闭环约束的抓手。通过在仿真和现实世界中进行对象抓住实验来评估有效绘制的有效性;结果表明,所提出的方法在仅考虑没有闭环约束的抓手时也胜过Unigrasp。在这些情况下,有效抓取在产生接触点的精度高9.85%,模拟中的握把成功率提高了3.10%。现实世界实验是用带有闭环约束的抓地力进行的,而Unigrasp无法处理,而有效绘制的成功率达到了83.3%。分析了该方法的抓地力故障的主要原因,突出了增强掌握性能的方法。
translated by 谷歌翻译
抓握是通过在一组触点上施加力和扭矩来挑选对象的过程。深度学习方法的最新进展允许在机器人对象抓地力方面快速进步。我们在过去十年中系统地调查了出版物,特别感兴趣使用最终效果姿势的所有6度自由度抓住对象。我们的综述发现了四种用于机器人抓钩的常见方法:基于抽样的方法,直接回归,强化学习和示例方法。此外,我们发现了围绕抓握的两种“支持方法”,这些方法使用深入学习来支持抓握过程,形状近似和负担能力。我们已经将本系统评论(85篇论文)中发现的出版物提炼为十个关键要点,我们认为对未来的机器人抓握和操纵研究至关重要。该调查的在线版本可从https://rhys-newbury.github.io/projects/6dof/获得
translated by 谷歌翻译
Current learning-based robot grasping approaches exploit human-labeled datasets for training the models. However, there are two problems with such a methodology: (a) since each object can be grasped in multiple ways, manually labeling grasp locations is not a trivial task; (b) human labeling is biased by semantics. While there have been attempts to train robots using trial-and-error experiments, the amount of data used in such experiments remains substantially low and hence makes the learner prone to over-fitting. In this paper, we take the leap of increasing the available training data to 40 times more than prior work, leading to a dataset size of 50K data points collected over 700 hours of robot grasping attempts. This allows us to train a Convolutional Neural Network (CNN) for the task of predicting grasp locations without severe overfitting. In our formulation, we recast the regression problem to an 18way binary classification over image patches. We also present a multi-stage learning approach where a CNN trained in one stage is used to collect hard negatives in subsequent stages. Our experiments clearly show the benefit of using large-scale datasets (and multi-stage training) for the task of grasping. We also compare to several baselines and show state-of-the-art performance on generalization to unseen objects for grasping.
translated by 谷歌翻译
操纵铰接对象通常需要多个机器人臂。使多个机器人武器能够在铰接物体上协作地完成操纵任务是一项挑战性。在本文中,我们呈现$ \ textbf {v-mao} $,这是一个学习铰接物体的多臂操纵的框架。我们的框架包括一个变分生成模型,可以为每个机器人臂的物体刚性零件学习接触点分布。从与模拟环境的交互获得训练信号,该模拟环境是通过规划和用于铰接对象的对象控制的新颖制定的新颖制定。我们在定制的Mujoco仿真环境中部署了我们的框架,并证明我们的框架在六种不同的对象和两个不同的机器人上实现了高成功率。我们还表明,生成建模可以有效地学习铰接物体上的接触点分布。
translated by 谷歌翻译
As the basis for prehensile manipulation, it is vital to enable robots to grasp as robustly as humans. In daily manipulation, our grasping system is prompt, accurate, flexible and continuous across spatial and temporal domains. Few existing methods cover all these properties for robot grasping. In this paper, we propose a new methodology for grasp perception to enable robots these abilities. Specifically, we develop a dense supervision strategy with real perception and analytic labels in the spatial-temporal domain. Additional awareness of objects' center-of-mass is incorporated into the learning process to help improve grasping stability. Utilization of grasp correspondence across observations enables dynamic grasp tracking. Our model, AnyGrasp, can generate accurate, full-DoF, dense and temporally-smooth grasp poses efficiently, and works robustly against large depth sensing noise. Embedded with AnyGrasp, we achieve a 93.3% success rate when clearing bins with over 300 unseen objects, which is comparable with human subjects under controlled conditions. Over 900 MPPH is reported on a single-arm system. For dynamic grasping, we demonstrate catching swimming robot fish in the water.
translated by 谷歌翻译
形状通知如何将对象掌握,无论是如何以及如何。因此,本文介绍了一种基于分割的架构,用于将用深度摄像机进行分解为多个基本形状的对象,以及用于机器人抓握的后处理管道。分段采用深度网络,称为PS-CNN,在具有6个类的原始形状和使用模拟引擎生成的合成数据上培训。每个原始形状都设计有参数化掌握家族,允许管道识别每个形状区域的多个掌握候选者。掌握是排序的排名,选择用于执行的第一个可行的。对于无任务掌握单个对象,该方法达到94.2%的成功率将其放置在顶部执行掌握方法中,与自上而下和SE(3)基础相比。涉及变量观点和杂波的其他测试展示了设置的鲁棒性。对于面向任务的掌握,PS-CNN实现了93.0%的成功率。总体而言,结果支持该假设,即在抓地管道内明确地编码形状原语应该提高掌握性能,包括无任务和任务相关的掌握预测。
translated by 谷歌翻译
人类和许多动物都表现出稳健的能力来操纵不同的物体,通常与他们的身体直接和有时与工具间接地进行操作。这种灵活性可能是由物理处理的基本一致性,例如接触和力闭合。通过将工具视为我们的机构的扩展来启发,我们提出了工具 - 作为实施例(TAE),用于处理同一表示空间中的手动对象和工具对象交互的基于工具的操作策略的参数化。结果是单一策略,可以在机器人上递归地应用于使用结束效果来操纵对象,并使用对象作为工具,即新的最终效果,以操纵其他对象。通过对不同实施例的共享经验进行掌握或推动,我们的政策表现出比训练单独的政策更高的性能。我们的框架可以利用将对启用工具的实施例的不同分辨率的所有经验用于每个操纵技能的单个通用策略。 https://sites.google.com/view/recursivemanipulation的视频
translated by 谷歌翻译
食品包装行业通常使用工厂工人手动包装的季节性成分。对于由体积或重量挑选的小型食物,倾向于使缠绕,棒或聚集在一起,很难预测他们从视觉检查中有多么交流,使其成为准确掌握必要目标大量的挑战。工人依赖于称重鳞片的组合和一系列复杂的操作,以分离食物并达到目标质量。这使得过程自动化是非琐碎的事件。在这项研究中,我们提出了一种结合1)预先抓住以降低缠结程度的方法,2)在掌握量大于掌握量时仔细丢弃多余的食物以调整抓住质量的缠绕。目标质量和3)选择抓取点以抓住可能合理地高于目标抓地质量的量。我们评估了各种食品的方法,缠绕,粘和丛的各种食物,每个食物具有不同的尺寸,形状和材料特性,例如体积质量密度。我们使用我们所提出的方法表现出掌握用户指定目标群众的准确性的显着改进。
translated by 谷歌翻译
成功掌握对象的能力在机器人中是至关重要的,因为它可以实现多个交互式下游应用程序。为此,大多数方法要么计算兴趣对象的完整6D姿势,要么学习预测一组掌握点。虽然前一种方法对多个对象实例或类没有很好地扩展,但后者需要大的注释数据集,并且受到新几何形状的普遍性能力差的阻碍。为了克服这些缺点,我们建议教授一个机器人如何用简单而简短的人类示范掌握一个物体。因此,我们的方法既不需要许多注释图像,也不限于特定的几何形状。我们首先介绍了一个小型RGB-D图像,显示人对象交互。然后利用该序列来构建表示所描绘的交互的相关手和对象网格。随后,我们完成重建对象形状的缺失部分,并估计了场景中的重建和可见对象之间的相对变换。最后,我们从物体和人手之间的相对姿势转移a-prioriz知识,随着当前对象在场景中的估计到机器人的必要抓握指令。与丰田的人类支持机器人(HSR)在真实和合成环境中的详尽评估证明了我们所提出的方法的适用性及其优势与以前的方法相比。
translated by 谷歌翻译
软机器人抓手有助于富含接触的操作,包括对各种物体的强大抓握。然而,软抓手的有益依从性也会导致重大变形,从而使精确的操纵具有挑战性。我们提出视觉压力估计与控制(VPEC),这种方法可以使用外部摄像头的RGB图像施加的软握力施加的压力。当气动抓地力和肌腱握力与平坦的表面接触时,我们为视觉压力推断提供了结果。我们还表明,VPEC可以通过对推断压力图像的闭环控制进行精确操作。在我们的评估中,移动操纵器(来自Hello Robot的拉伸RE1)使用Visual Servoing在所需的压力下进行接触;遵循空间压力轨迹;并掌握小型低调的物体,包括microSD卡,一分钱和药丸。总体而言,我们的结果表明,对施加压力的视觉估计可以使软抓手能够执行精确操作。
translated by 谷歌翻译
We present a generalised architecture for reactive mobile manipulation while a robot's base is in motion toward the next objective in a high-level task. By performing tasks on-the-move, overall cycle time is reduced compared to methods where the base pauses during manipulation. Reactive control of the manipulator enables grasping objects with unpredictable motion while improving robustness against perception errors, environmental disturbances, and inaccurate robot control compared to open-loop, trajectory-based planning approaches. We present an example implementation of the architecture and investigate the performance on a series of pick and place tasks with both static and dynamic objects and compare the performance to baseline methods. Our method demonstrated a real-world success rate of over 99%, failing in only a single trial from 120 attempts with a physical robot system. The architecture is further demonstrated on other mobile manipulator platforms in simulation. Our approach reduces task time by up to 48%, while also improving reliability, gracefulness, and predictability compared to existing architectures for mobile manipulation. See https://benburgesslimerick.github.io/ManipulationOnTheMove for supplementary materials.
translated by 谷歌翻译