Google Sign Language AI Ships in Gboard on Pixel 11

Research That Finally Left the Lab

Sign language translation has been an active research area for years, and it has spent almost all of that time being a demo. On August 12, 2026, alongside the Pixel 11 launch, Google DeepMind moved it into shipping consumer software: a model called SL2T, short for sign-language-to-text, now powers sign-to-text dictation in Gboard and Live Transcribe on the Pixel 11.

• Trained on more than 100,000 hours of data spanning over 50 sign languages, about a quarter of it American Sign Language
• Ships first as American Sign Language to English, with more languages and devices planned
• An on-device model converts video into pose coordinates, and the raw footage is discarded rather than uploaded
• Scores 70 BLEURT zero-shot on the FLEURS-ASL benchmark, ahead of prior published results

The gap between a research result and a keyboard you can actually type with is enormous, and most accessibility AI never crosses it. This one did.

How Does Sign-Language-to-Text Work on a Phone?

The pipeline is more interesting than "point a camera at a person." A separate model running on the phone uses MediaPipe Holistic to track pose landmarks — the coordinates of hands, arms, and face — and reduces the camera feed to a moving wireframe. Those coordinates, not the video frames, are what gets sent onward for translation. The raw footage is discarded on the device.

That design choice does two jobs at once. It is a meaningful privacy property, because a conversation in sign language is a conversation, and video of one leaving the phone is a very different thing from a stream of anonymous joint positions. It also makes the problem tractable at conversational latency, since a stream of landmark coordinates is far smaller than video.

The second technical decision is that SL2T translates from coordinates straight to text, skipping the intermediate "gloss" annotation step that most earlier systems relied on. Gloss is a written shorthand for signs, and building datasets of it is slow, expensive, and lossy. Removing it from the loop is what let the training set scale to six figures of hours.

What Makes This Model Different From Earlier Attempts?

Mostly, it is that the team optimized for how people actually sign rather than for how benchmark datasets are recorded.

Benchmark video is typically two-handed signing, filmed straight-on, by someone standing still. Real signing at a phone means one hand, because the other hand is holding the phone. The model is explicitly optimized for one-handed signing under those conditions. It also accounts for left-handed signers, who make up roughly 10% of users and who get quietly broken by systems trained on right-handed data alone.

There is also deliberate work on minimizing streaming latency and on preventing hallucinations — the failure mode where a translation model produces a fluent, confident sentence that the person never signed. In an accessibility tool, a plausible wrong answer is worse than no answer, because the person relying on it has no way to catch the error.

Who Does This Actually Help?

Roughly 70 million Deaf and hard-of-hearing people use more than 200 sign languages worldwide. For many of them, a phone keyboard is a second-language interface — written English is not the language they think in, and typing it is slower and more effortful than signing.

Sign-to-text dictation in Gboard means signing works anywhere text input works: searching the web, drafting a message or a document, or handing a request to Gemini. Live Transcribe support extends it to in-person conversation, where a Deaf user can sign a reply instead of typing one back. Neither of those is a special-purpose accessibility app you have to remember to open; they are the ordinary input surfaces of the phone.

That is the part worth noting for anyone building assistive technology. The win here is not a higher benchmark score. It is that the capability landed inside the software people already use, which is the step that research demos almost never take. It follows the same on-device direction as Google's earlier work putting Gemma 4 into Android's AICore for offline AI, and it rhymes with the multilingual speech work happening elsewhere in the field, like NVIDIA's open-weight Magpie TTS across 12 languages.

What Comes Next for Sign Language AI?

The honest answer is: more languages, and that is genuinely hard. ASL is one of more than 200 sign languages, and they are not dialects of each other — British Sign Language and American Sign Language are mutually unintelligible despite both serving English-speaking countries. The 50-plus sign languages in the training data are a foundation for expansion rather than 50 shipping products.

Google says more devices are coming. The pose-landmark approach helps here, because it does not require a specific camera stack, just a camera good enough to track hands. Whether that translates into broad Android availability is the thing to watch over the next year.

For now, the milestone stands on its own: an accessibility capability that spent a decade in papers is in a keyboard. More coverage of on-device models in our AI section, including our look at running local models on Apple Silicon.

Sources: Google DeepMind — August 12, 2026; Engadget — August 12, 2026; SiliconANGLE — August 12, 2026.