From Visual to Haptic: How do Laparoscopy Experts Estimate Haptics from Visual Cues in Robotic Surgery?

To investigate whether there is a skill of transformation of visual feedback to haptic information in robotic surgery, to find out the best modality feedback to acquire such skill, to investigate the visual features that are used with such skill to estimate object stiffness.
Description of the Project: 

One of the major drawbacks of robotic surgery is the loss of haptic feedback. The only feedback that surgeons receive is simply the visual cues. Expert laparoscopic surgeons can perform better than the novice surgeons with the loss of haptics. We hypothesise that expert surgeons achieve this by transforming the visual cues into haptic information using their memory of past experiences. The transformation from visual to haptic should be an experience or a skill that experts build in time. In this project we aim at 1) proving the hypothesis that there is a skill of transformation of visual feedback to haptic information, and 2) investigating the conditions of how this skill is best formed. We will conduct the study using a tele-operated robotic system with a standard laparoscopy training box. We will experiment with different groups of novice subjects through interaction with various tissue mimicking materials, while allowing the subjects to receive different combination/amount of feedback (haptic, audio, visual). We will analyse the visual cues in order find out the visual features that allow the subjects estimate the stiffness.

Resources required: 
Touch Haptic Device Consumables and equipment repair costs (to be budgeted to the project). Two robot arms, two force/torques sensors, a laparoscopy training box, laparoscopy training games.
Project number: 
First Supervisor: 
Heriot-Watt University
Second Supervisor(s): 
First supervisor university: 
Heriot-Watt University
Essential skills and knowledge: 
Desirable skills and knowledge: 
Medical robotics, machine learning, image processing.
  • Tugal, H., Gautier, B., Kircicek, M., and Erden, M.S., 2018, "Hand-Impedance Measurement During Laparoscopic Training Coupled with Robotic Manipulators", Proc. of  IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018), October 1-5, Madrid, accepted.
  • Erden, M.S., Rosa, B., Szewczyk, J. and Morel, G., 2013, “Understanding Soft Tissue Behavior for Application to Microlaparoscopic Surface Scan”, IEEE Transactions on Biomedical Engineering, vol. 60 (4): 1059-1068.
  • El Boghdady, M., Alijani, A., 2018, "The Application of a Performance-Based Self-Administered Intra-procedural Checklist on Surgical Trainees During Laparoscopic Cholecystectomy", World J Surg., Jun, 42 (6): 1695-1700.
  • El Boghdady, M., Ramakrishnan, G., Tang, B., and Alijani, A., 2018, "A Comparative Study of Generic Visual Components of Two-Dimensional Versus Three-Dimensional Laparoscopic Images", World J Surg., Mar, 42 (3): 688-694.
  • Alam, M., Wilson, MSJ., Tang, B., Tait, I.S., and Alijani, A., 2017, "A training tool to assess laparoscopic image navigation task performance in novice camera assistants", J Surg Res., Nov, 219:232-237.