Developed an Android app that allows Google to 'talk with eyes'
To help people who have difficulty speaking by voice due to language or movement disorders, Google has developed an Android app ' Look to Speak ' that allows you to 'talk with your eyes.' By using this app, users who have difficulty speaking can select pre-registered phrases with their eyes and have a smooth conversation with others.
Look to Speak helps people communicate with their eyes
Google Look to Speak lets you use your eyes to select and speak phrases --The Verge
https://www.theverge.com/2020/12/8/22160011/google-look-to-speak-eye-tracking-android-disability
'Look to Speak' announced by Google on December 8, 2020 is an app developed to facilitate communication for people with disabilities in voice conversation. Richard Cave, a speech therapist who has been working with Google's research team since early 2020, has been involved with people in need of communication assistance for over a decade. In a Google blog about 'Look to Speak,' Cave said, 'Every day, I work to find easier and more accessible ways for people to express their daily needs, opinions, emotions, and identities. I've been there. '
'If you have a device that you can operate with your eyes, you can help people with disabilities in voice communication,' Cave said. As ubiquitous mobile devices have evolved through machine learning, etc., he has come to think that it may be possible to support voice communication using the line of sight using smartphones. After learning that some people were experimenting with similar ideas on Google, Cave joined a research team to develop 'Look to Speak,' which enables eye-catching conversations.
You can see what kind of app 'Look to Speak' actually is by watching the following movie.
Getting started with Look to Speak --YouTube
When using 'Look to Speak', fix your smartphone slightly below your face and turn your gaze to the right ...
Turn to the left
You can operate it by pointing it up.
On the main screen of the app, various pre-registered phrases are displayed separately on the left and right sides of the screen. If you want to select the phrase 'How are you?' From this ...
First, select your line of sight toward the left where 'How are you?' Is displayed.
Then, the phrases displayed on the left side were displayed on the left and right sides of the screen in half.
This time, 'How are you?' Is displayed on the right side, so select with your eyes facing the right.
By repeating this, the phrases are narrowed down ...
The voice of the finally selected phrase will be played.
By raising your eyes ...
You can put the app in a standby state.
It seems that the screen of the smartphone should be fixed slightly below the face.
The phrases to be registered can be set up freely ...
You can set the phrases you use most often to your liking. In addition, the first displayed phrase is 8 on each side, 16 in total, so you can narrow down the phrase by operating the line of sight four times. The content of the phrase is not shared with the outside or a third party, and is saved only in the smartphone.
The setting screen cannot be operated by the line of sight, so you or your supporter must set it manually.
You can see how people who support Mr. Cave actually use 'Look to Speak' by watching the following movie.
The man on the right side of the screen is Mr. Cave, and the one sitting on the chair on the left side is Mr. Sarah who has a movement disorder.
Sara can't speak, but ...
He has been successful as an artist and has given lectures.
Use your eyes when Sarah talks ...
Select a letter on the digital keyboard and enter a word.
When I was able to write a sentence, I heard the voice 'Hello, Richard'. Assistive devices that use your line of sight to create text are useful, but they are relatively large and not always portable.
So since 2020, Cave has been working with the Google team to develop an app that helps people talk on their smartphones.
If you use the face recognition function of your smartphone, you can get a rough idea of the direction of your line of sight ...
It is possible to smoothly select pre-registered phrases.
Being able to easily carry and support conversations with smartphones owned by many people would be of great benefit to people in a variety of social and economic situations.
Mr. Cave said that Sarah also used 'Look to Speak' to get feedback from people who actually need the app.
It seems that human hands are required because the initial setup and phrase registration cannot be completed only by looking at the eyes.
Sarah operates 'Look to Speak' with her gaze to the left.
Then, the voice 'My name is Sarah' was heard. In this way, it seems that 'Look to Speak' was developed with the feedback of people with disabilities.
Cave doesn't think 'Look to Speak' will completely replace communication, as traditional assistive devices are so versatile that you can freely enter words other than registered phrases.
When Mr. Cave actually asked some people to use 'Look to Speak', he said that communication using smartphones became easier in places where support devices could not be carried, such as outdoors, on the move, and in the shower. I heard that I got feedback. “Conversations are now easier to do where there may have been silence before,” said Cave.
Katie Siever of the Google development team points out that communication support tools like 'Look to Speak' aren't just for people with disabilities at this point. He argued that such support tools would be useful to a wide range of people, as everyone is at risk of having some disability due to an accident or illness and may be temporarily unable to speak due to surgery or the like.
'Look to Speak' can be downloaded from Google Play and is compatible with devices running Android 9.0 or higher.
Look to Speak-Google Play app
https://play.google.com/store/apps/details?id=com.androidexperiments.looktospeak
Related Posts:
