Google has included an AI model in the Pixel 11 that converts sign language into text.
Google DeepMind's sign language team has announced a 'sign-language-to-text (SL2T) translation model' that converts sign language video into text. It is expected that users will be able to input text using sign language on
Putting sign language AI into users' hands — Google DeepMind
https://deepmind.google/blog/putting-sign-language-ai-into-users-hands/
SL2T is a model trained on over 100,000 hours of data across more than 50 sign languages, and can read the body movements of sign language users, convert them into coordinates, and translate them.
SL2T is state-of-the-art on academic benchmarks, plus it's optimized for real-world use like one-handed signing while holding a phone.
— Google DeepMind (@GoogleDeepMind) August 12, 2026
To protect privacy, it tracks body poses on-device, while our servers translate them into text. pic.twitter.com/6AkxT6mO3D
Google DeepMind explains that translating sign language presents challenges such as 'each sign language having its own unique grammar and vocabulary' and 'the AI model having to learn to understand body movements by observing them,' and that accurately tracking these at high frame rates has been difficult, making development challenging until now. SL2T also focuses on practical challenges such as minimizing latency during streaming, preventing hallucination of non-sign language inputs, and improving performance when using one hand for sign language.
SL2T will initially be available on the Pixel 11, and can be experienced with the text input app 'Gboard' and the transcription app 'Live Transcribe.' Initially, it will only support conversion from American Sign Language to English, with plans to add more compatible devices over time. The sign language input function will also be available at no additional cost.
SL2T is our breakthrough sign language-to-text model powering new features for Deaf and hard of hearing users on @Android .
— Google DeepMind (@GoogleDeepMind) August 12, 2026
Starting with American Sign Language-to-English on Pixel 11, people can sign directly into Gboard and Live Transcribe instead of typing. pic.twitter.com/p9Vx7tJtLT
Google DeepMind stated, 'AI's ability to process speech and language has advanced rapidly in recent decades, enabling features that feel natural to users, such as automatic translation and conversational interfaces. However, this technology still lags behind the more than 200 sign languages in the world and the estimated 70 million deaf and hard-of-hearing people who use them. With SL2T, deaf people can use sign language to input text wherever they need to. They can search the web in sign language, compose messages and documents, have Gemini answer questions, and perform tasks. Test participants have reported that using sign language is faster, more natural, and more comfortable than typing.'
Related Posts:
in AI, Posted by log1p_kr