Latest in Tomorrow

Image credit: MIT CSAIL

MIT's new robot can identify things by sight and by touch

Touchy-feely robots could be the future.
740 Shares
Share
Tweet
Share

Sponsored Links

For humans, it's easy to predict how an object will feel by looking at it or tell what an object looks like by touching it, but this can be a big challenge for machines. Now, a new robot developed by MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) is attempting to do just that.

The team took a KUKA robot arm and added a tactile sensor called GelSight, which was created by Ted Adelson's group at CSAIL. The information collected by GelSight was then fed to an AI so it could learn the relationship between visual and tactile information.

To teach the AI how to identify objects by touch, the team recorded 12,000 videos of 200 objects like fabrics, tools and household objects being touched. The videos were broken down into still images and the AI used this dataset to connect tactile and visual data.

"By looking at the scene, our model can imagine the feeling of touching a flat surface or a sharp edge", says Yunzhu Li, CSAIL PhD student and lead author on a new paper about the system. "By blindly touching around, our model can predict the interaction with the environment purely from tactile feelings. Bringing these two senses together could empower the robot and reduce the data we might need for tasks involving manipulating and grasping objects."

For now, the robot can only identify objects in a controlled environment. The next step is to build a larger data set so the robot can work in more diverse settings.

"Methods like this have the potential to be very useful for robotics, where you need to answer questions like 'is this object hard or soft?', or 'if I lift this mug by its handle, how good will my grip be?'," says Andrew Owens, a postdoctoral researcher at the University of California at Berkeley. "This is a very challenging problem, since the signals are so different, and this model has demonstrated great capability."


Video
Presenter: Terrence O'Brien
Script: Terrence O'Brien
Script Editor: Dana Wollman
Editor: Kyle Maack
Producer/Camera: Michael Morris

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
740 Shares
Share
Tweet
Share

Popular on Engadget

Google could add call recording to its Android Phone app

Google could add call recording to its Android Phone app

View
'Pokémon Go' online battle feature starts rolling out

'Pokémon Go' online battle feature starts rolling out

View
Lime knows when you're riding its scooters on the sidewalk

Lime knows when you're riding its scooters on the sidewalk

View
Google Translate is getting a transcribe mode for lectures, speeches

Google Translate is getting a transcribe mode for lectures, speeches

View
Google's latest stab at messaging may be an all-in-one business app

Google's latest stab at messaging may be an all-in-one business app

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr