Multi-modal and Cross-modal Visual-Tactile Perception for Robot Dexterous Manipulation

Add to your list(s) Download to your calendar using vCal

• Shan Luo, Lecturer at University of Liverpool
• Friday 30 July 2021, 10:00-11:00
• https://bham-ac-uk.zoom.us/j/81432789784?pwd=dzV3N1lKM2prRTVXN1N4aVlITGxUQT09.

If you have a question about this talk, please contact Martin Rudorfer.

In our daily life, both vision and touch are indispensable sensing methods for us to interact with the surrounding environment, and the two often assist each other to complete some tasks, such as grasping objects. Similarly, visual and tactile perception can also assist each other as the main channel for robots to understand the surrounding environment. Inspired by this, we developed a new type of touch sensor GelTip, which is shaped like a human finger. It uses a vision camera at the bottom to capture the contact between the sensor surface elastomers and the contacting object, and extracts information about the contacting object based on the obtained tactile image. We have also designed a series of learning algorithms for extracting information from both visual and tactile images. Through the integration of vision and touch (both multi- and cross-modal), it can effectively improve the robot’s object perception ability, so that the robot can effectively improve its capability of object recognition and localisation. Recently, we have also developed a simulation model of high-resolution tactile sensors that can be applied in the widely used ROS simulation environments and enables the Sim2Real learning with tactile sensing. Our research can be used to improve the dexterous manipulation of robots, suitable for warehouse robotics, inspection, medical robotics and other fields.

Dr. Shan Luo is a Lecturer (Assistant Professor) in Robotics at the Department of Computer Science at the University of Liverpool. He leads the advanced robotics laboratory smARTlab at the Department. His main research strengths are in visuo-tactile robotics, including the development of robot visuo-tactile sensors, object recognition and localisation using vision and touch, and multimodal perception, etc. Dr. Luo received his Ph.D. in Robotics from King’s College London in 2016. He visited the MIT Computer and Artificial Intelligence Laboratory (CSAIL) in 2016. After graduating from his Ph.D., he worked as a Postdoctoral Research Fellow at the University of Leeds and Harvard University, and joined the University of Liverpool in 2018. He has more than 30 publications in high-impact robotics journals and international conferences (including Autonomous Robots, ICRA , IROS, ICML . AAMAS, etc.). He has been a Principal Investigator for a number of major projects, of more than £800k funding from the EPSRC , Innovate UK and other funding bodies. He has also been a Co-Investigator for multiple other projects of over £13m funded by the EPSRC , AHRC, and Royal Society.

This talk is part of the IRLab Seminars: Robotics, Computer Vision & AI series.

This talk is included in these lists:

• IRLab Seminars: Robotics, Computer Vision & AI
• https://bham-ac-uk.zoom.us/j/81432789784?pwd=dzV3N1lKM2prRTVXN1N4aVlITGxUQT09

Note that ex-directory lists are not shown.