Tactile sensing for surgery and grasping
A surgeon working through a robot or a long grasper sees the tissue but barely feels it. A robot hand that cannot feel an object slip will drop it. The work here adds the missing touch, with sensors on grasper jaws, feedback on the fingertip and a reflex for slipping objects, and it steers a capsule camera inside the stomach.
Seeing a squeeze, or feeling it
In robotic surgery the surgeon gets video but no sense of grip force. A sensorised grasper measured the force and a small indenter pressed it back onto the surgeon's fingertip. The question was how small a change in force people can pick up by sight, by touch, or both.
Sixty people compared two grasps at a time and said which was firmer, until the gap was too small to tell. Feeling the force on the fingertip helped more than seeing it, and the two together did best. The combined result came within 13.1% of what a sensory integration model predicts.
Sources: IEEE Robotics and Automation Letters, 2021, and the dataset with every participant's answers. The grasper and fingertip are drawn for illustration. The grasper sensor itself is a US patent.
A reflex for a slipping object
When a cup you are holding starts to slide, your grip tightens before you think about it. A bio-inspired controller gives robot fingers the same reflex. Add weight to the object and watch the spikes, the delay and the grip.
Skin has nerve endings that fire when something slides across it. The method copies them: a slip sensor turns motion into spikes, and each spike tells the fingers to squeeze harder. In the lab, slip was stopped before the object fell in 80% of trials, with a response under 85 ms, about as fast as a person adjusts their grip.
Source: IEEE Sensors Journal, 2019. This is a simple simulation of the idea. The 85 ms delay comes from the paper; the forces, friction and spike rate are made up for the demonstration. A 2017 paper tuned the same trade-off between slip and squeeze with a monotonic PI controller.
Steering a camera in the stomach
A tethered capsule endoscope is swallowed like a pill and stays on a thin, flexible cord, with no anaesthetic needed. The cord is too floppy to aim the camera. Three linear actuators pull tendons through a short bending section, so the capsule can be pointed anywhere in the stomach.
Above 20° of bend, the 2015 prototype hit its target direction to within 1° on average.
Sources: IEEE CIS 2015 and URAI 2017. With the capsule deployed 40 mm, it covered 360° around and up to 110° of bend. Tendon pulls are worked out from the direction you pick.
Spikes, gloves and gas
Touch in spikes
A fabric sensor array turned contact into spike patterns like those of skin receptors. A spike-timing classifier told object curvatures apart with more than 99% accuracy.
Shapes from a glove
Finger joint angles from a sensor glove identified boxes, cylinders and spheres with 100% accuracy. When each shape was held in more than one way, accuracy fell to 71%.
Sniffing for H. pylori
A dual optical fibre coated with dyes senses carbon dioxide and ammonia, two markers of the stomach bacterium linked to cancer, at ppm levels. It is meant to ride on the capsule.
Graphene in the jaws
A film of reduced graphene oxide in PDMS, cast to any shape, measured forces from 0.5 to 20 N on a grasper tip.
Papers and grants
13 papers
- 2022Dataset for influence of visual and haptic feedback on the detection of threshold forces in a surgical grasping task
QUData in Brief, 2 citations - 2021Influence of Visual and Haptic Feedback on the Detection of Threshold Forces in a Surgical Grasping Task
QUIEEE Robotics and Automation Letters, 19 citations - 2019A Bio-Inspired Slip Detection and Reflex-Like Suppression Method for Robotic Manipulators
QUIEEE Sensors Journal, 23 citations - 2018Experimental characterization of a tactile sensor for surgical applications
QU2018 IEEE International Conference on Innovative Research and Development (ICIRD) - 2018Graphene-filled PDMS Composite for Tactile Sensing of Surgical Graspers
QU2018 IEEE 18th International Conference on Nanotechnology (IEEE-NANO) - 2017Design and verification of a flexible device for steering a tethered capsule endoscope in the stomach
QU2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), 4 citations - 2017Pareto-front analysis of a monotonie PI control law for slip suppression in a robotic manipulator
QU2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2 citations - 2016A Flexible Gastric Gas Sensor Based on Functionalized Optical Fiber
QUIEEE Sensors Journal, 8 citations - 2016Reflex System for Intelligent Robotics
QUQatar Foundation Annual Research Conference Proceedings Volume 2016 Issue 1, 2 citations - 2015Design of a steering mechanism for a Tethered Capsule Endoscope
QU2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), 4 citations - 2013Bio-mimetic strategies for tactile sensing
NUS2013 ieee sensors, 30 citations - 2013Tactile sensing in an object passing task
NUS2013 IEEE Conference on Cybernetics and Intelligent Systems (CIS), 3 citations - 2013Object shape discrimination using sensorized glove
NUS2013 10th IEEE International Conference on Control and Automation (ICCA), 2 citations
Grants
- 2015Neuromorphic Tactile Sensing: A Paradigm Shift for Prosthetics and Robotic Surgery
US$823,782 · Qatar National Research Fund · Lead Principal Investigator - 2012Integrated Bio-Sensors and Automated Instrumentation for Early Stomach Cancer Detection Using Flexible Capsule Endoscope
US$1,035,523 · Qatar National Research Fund · Co-Lead Principal Investigator - 2012Development of Smart Minimum Invasive Surgery Tools with Tactile Sensing Capabilities for Telerobot Surgery System
US$1,018,554 · Qatar National Research Fund · Co-Lead Principal Investigator