Voice tech has become incredibly advanced at recognizing people’s voices over the years. We can dictate text and talk to our phones with little effort. But that hasn’t been the case for the roughly 70 million Deaf and hard-of-hearing people who use over 200 different sign languages.

Google has introduced a new sign language-to-text model, called SL2T, to bridge the gap. It is launching first on the Pixel 11 through Gboard and Live Transcribe. Right now, it starts by translating American Sign Language (ASL) into English.

Instead of typing, Deaf users can just sign to their phone anywhere they would normally type. A user can draft messages, search the web, or ask Gemini to do tasks. In Live Transcribe, users can sign your responses instead of typing them out back and forth. Google stated in a press release, “According to our testers, signing in ASL is faster, more natural, and more delightful than typing in English.”

Texting with Sign Language

A demonstration of the sign language-to-text model; Photo: Google

Translating sign language is difficult because they’re full, independent languages with their own distinct grammar. Earlier tech tried using things like special gloves, but the problem with that approach is that meaning comes from the whole body.

Advertisement

Google noted, “sign languages aren’t simply ‘English on the hands.’ They require complex visual perception of fine-grained whole-body movements and full-fledged language translation.”

To make this work, the model tracks your body’s movements, including the hands, arms, torso, head, and face. According to the tech giant, it keeps your privacy by not keeping raw video of you. It only tracks the geometric points of your movement on the device, sends those basic coordinates to a server to be translated, and immediately deletes the original video.

Only ASL… for Now

This might work well for real users because the team actually focused on daily habits. The model was trained on over 100,000 hours of data across 50 sign languages. Developers ensure it works fairly for the 10% of signers who are left-handed. They even made sure it handles one-handed signing, which you naturally have to do when holding a phone in your other hand.

Right now, there are limitations. It is only on the Pixel 11, and it only works for ASL. Google stated that more devices and languages are coming soon at no extra cost. Creators added, “Google’s mission is to organize the world’s information and make it universally accessible and useful.”