Our hands are incredibly complex. For example, a simple scroll on your phone requires 34 muscles and over 100 ligaments working in perfect harmony. This is exactly why it’s been so hard for scientists to get robots or virtual avatars to mimic them accurately.

Now, engineers at MIT have developed an ultrasound wristband that can “see” what your hand is doing by looking at the movement inside your wrist. By pairing these images with an AI algorithm, the device can track the position of your fingers and palm in real time.

How the Wearable Band Sees

Current tech usually relies on cameras or bulky gloves covered in sensors. Cameras can be blocked by objects, and gloves can feel stiff or unnatural. This new wristband takes a different path by imaging the tendons and muscles directly.

“The tendons and muscles in your wrist are like strings pulling on puppets, which are your fingers,” said Gengxi Lu, one of the researchers. “So the idea is, each time you take a picture of the state of the strings, you’ll know the state of the hand.”

The team used AI to translate these “pictures” into 22 different types of movement. To test it out, volunteers wore the band while performing various tasks—like signing the alphabet in ASL or holding a tennis ball. The system was able to predict their hand positions with impressive precision.

A Range of Uses

In demonstrations, a person wearing the band controlled a robotic hand wirelessly. As the person moved their fingers, the robot mimicked them, playing a simple tune on a piano and even shooting a tiny basketball into a hoop. It also worked for virtual reality, allowing users to pinch their fingers to zoom in on digital objects.

The goal is to eventually use this data to help humanoid robots handle delicate tasks, like surgery. “We think this work has immediate impact in potentially replacing hand tracking techniques with wearable ultrasound bands in virtual and augmented reality,” said Xuanhe Zhao, a professor at MIT. “It could also provide huge amounts of training data for dexterous humanoid robots.”