Robot-Assisted handling of Limp and Deformable Objects

This project is part of the Collaborative Research Centre 453 "Wirklichkeitsnahe Telepräsenz und Teleaktion", which is funded by the German Research Foundation DFG.

Like all other transfer projects, T4 is conducted in cooperation with an industrial partner. We were able to acquire Mitsubishi Electric as a partner for this project, a company with outstanding experience on the field of industrial robotics. The primary objective of this project is the transfer and application of expertise, which was obtained in project I4: i.e. techniques for the recognition, tracking, modelling and prediction of deformable objects (particularly thread material) in surgical environments will be transferred to industrial applications. Analogies are manifold: surgical threads show high felxibility, deformability and limpness. These features can also be found in assembly material like cables, wire straps, tubes and sealings. Due to the mentioned material properties, handling of such structures is extremely diffcult.

First results for tackling the challenges that arise with limp materials have already been developed in project I4. Surgical threads have been successfully segmented and their geometrical features have been extracted. In addtion, the corresponding operation environment has been reconstructed from stereo images. For further processing, the structures are modelled as splines and free-form surfaces.

Autonomous Execution of a Limp Object Handling Task

This video shows completion of the full autonomous task, which consists of several independent steps:

• Visual segmentation of the workpiece from the table,
• Analysis of the object-structure (find knots),
• Performing unknotting,
• Determination of a possible grasping point,
• Handover the workpiece to a statically mounted gripper,
• Avoid twists of the workpiece by pulling it through a guide,
• Grasp an end and determine the orientation of it,
• Correct the orientations and translation-offsets of the ends
• Finally insert both ends into the assembly-guide to combine them.

Control Framework for the Experimental Setup

Autonomous Execution of a Limp Object Handling Task (First steps)

People

• Dr. Thomas Müller (Robotics, Automation)

Former Members

• Dipl.-Inf. Istvan Nagy (Stereo-Vision, 3D Reconstruction)

Student Projects

Publications