Bi-Manual Inverted Robotic Manipulation

To develop and evaluate safe skills for autonomous robotic manipulation by two cooperating inverted (ceiling mounted) UR10 robots.
Description of the Project: 


In real world applications, a large variety of jobs involve handling various objects as the core process. However, most of these jobs nowadays are still performed by people. One obvious fact of human manipulation is the use of two hands that cooperate to achieve some task. But not too much research has been done involving coordination and optimal use of two hands. At the system level, having two hands naturally introduces physical redundancy that provides extra layer of safety and reliability against malfunctions such as object slippage.

Project description

This project aims at advancing the technology needed for using two interacting hands, such as are used when cooking a meal or assembly. This includes high level elements such as task planning and execution, dual arm collision avoidance, and grasp planning, as well as low level skills such as identifying grasping points, coordinating the execution of handing an object from one hand to another, coordination of tasks that require two hands to complete, getting the manipulators to the identified contact points, etc.

The main data expected to be used here is a 3D point cloud acquired from, e.g. an Intel RealSense sensor mounted on the robot arm, as well as global 3D sensors (e.g. Kinect). One potential application area is domestic robotics in a kitchen setting. For example, a robot taking dirty dishes from a table, washing them, placing them in a cupboard. Another possible application is simple cooking tasks requiring two hands, e.g. one holding a pan and another stirring. A third possible application area is in a surgical operating theatre, where robotic arms are of critical assistance for safe surgeries, i.e. collecting or holding surgical components, that are known to be helpful to reduce cognitive load, mental fatigue, and sustain safe surgeries especially during long operation time.

Resources required: 
Dual UR10 robot arms with robotic hands/grippers, range sensor
Project number: 
First Supervisor: 
University of Edinburgh
Second Supervisor(s): 
First supervisor university: 
University of Edinburgh
Essential skills and knowledge: 
Linux, (C++, Python or Matlab), some computer vision and robotics experience, machine learning experience
Desirable skills and knowledge: 
ROS, experience with Physics engines (eg. ODE, Bullet, Mujoco, Physx3.4) and Physics simulators (eg. Gazebo, Pybullet, VRep, Unity)

[1] A Projected Inverse Dynamics Approach for Multi-Arm Cartesian Impedance Control; Lin, Hsiu-Chin, Smith, Joshua, Babarahmati, Keyhan Kouhkiloui, Dehio, Niels, Mistry, Michael; Proc ICRA 2018

[2] Fast Object Learning and Dual-Arm Coordination for Cluttered Stowing, Picking, and Packing; Schwarz, Max, Lenz, Christian, Martin Garcia, Koo, Seongyong,Periyasamy, Arul Selvam, Schreiber, Michael,  Behnke, Sven Bonn