Jun 12, 2019 14:00:00

AI is developed to generate a speaker's face image from 'voice'

Information such as gender, age, and sometimes hometown can be determined from just the ' voice ' of the person speaking. ' Speech2Face ' is an AI that generates images by predicting the speaker's face from human voice and speaking style, and is developed to derive human physical features from speech.

Speech2Face: Learning the Face Behind a Voice

https://speech2face.github.io/

[1905.09773] Speech2Face: Learning the Face Behind a Voice
(PDF) https://arxiv.org/pdf/1905.09773.pdf

AI Listening to People's Voices. Then It Generated Their Faces.
https://www.livescience.com/65689-ai-human-voice-face.html

Speech2Face predicts the image of the speaker's face from the voice by machine learning about the relationship between the speaker's 'age', 'sex', 'race', 'speaking style' and 'voice' from the movie posted on YouTube To generate it. Millions of movies were used for learning, and Speech2Face learned more than 100,000 voices and faces.

This is the 'face image' that Speech2Face actually generated from the voice. The photo in the left column is the original face, the middle column is the original image processed with the face facing front and glasses removed, and the right column is the photo of the face generated by Speech2Face. Although the detail formation differs between the actual face and the face by Speech2Face, it seems that race, gender, age, etc. match. In addition, all face images generated by Speech2Face will be expressionless.

According to research, the face image generated by Speech2Face corrects the age, race, and gender in most cases, and the accuracy increases as the input voice gets longer, but the accuracy is not 'perfect'. . Even with the same person, if you generate face images from 'speech talking Chinese' and 'speech talking English' respectively, the white face image (image left) when speaking English, Chinese When speaking a language, it is also possible to generate an Asian face image (image right). You can actually listen to the voice you entered at the destination you clicked on the image below.

In addition, it seems that there is a tendency that the voice of the bass produces a male face image and the voice of a high tone produces a female face image.