For over a decade, researchers at MIT have been trying to figure out how to help robots find things they can’t see. A team of researchers led by Associate Professor Fadel Adib figured out how to use millimeter wave (mmWave) signals to see through obstacles like plastic, cardboard, and wood.

These waves bounce off hidden objects and travel back to a sensor, but there’s a catch. Because of how the waves reflect, the robot usually only “sees” the very top of an object. The sides and bottom remain a mystery, leaving the robot with a blurry, partial picture.

To fix this, the researchers started using generative AI. Instead of just guessing what the rest of an object looks like, the AI fills in the blanks. It’s a bit like a digital artist completing a sketch based on just a few lines.

“What we’ve done now is develop generative AI models that help us understand wireless reflections. This opens up a lot of interesting new applications, but technically it is also a qualitative leap in capabilities, from being able to fill in gaps we were not able to see before to being able to interpret reflections and reconstruct entire scenes,” said Adib. “We are using AI to finally unlock wireless vision.”

Better Vision For Robots

The team’s new system, called Wave-Former, is surprisingly good at this. It can reconstruct the shapes of about 70 different everyday items, like fruit, tools, and boxes, even when they are tucked away behind a wall.

A second system, known as RISE, can actually map out an entire room by watching how wireless signals bounce off people as they move around. Normally, these extra reflections are considered “noise” or “ghost signals” and are thrown away. But the MIT team realized these ghosts actually carry info about where the walls and furniture are.

This tech could help warehouse robots double-check that a box is packed correctly before it ships, which saves money on returns. It could also make smart homes safer by helping robots know exactly where people are without using cameras.