Google is developing automatic movie dubbing & lip sync AI, it is possible to dub without changing the voice tone, it is too high performance and it is abusive attention
At the keynote speech of '
Google I/O 2023
https://io.google/2023/intl/en/
The announcement of Universal Translator can be confirmed from around 1 hour 14 minutes 38 seconds in the following movie.
Google Keynote (Google I / O '23)-YouTube
Universal Translator is an AI that recognizes the voice in the movie and translates it into another language without changing the tone of voice. Google has released a demo that automatically dubs university online lectures using Universal Translator.
The original movie will be lectured in English.
In the dubbed version of the movie, the lecture proceeds in a different language while keeping the same voice as the original movie. Also, the movement of the mouth changes according to the voice.
The Universal Translator realizes the dubbing function through the steps of 'translation', 'adjustment of intonation', 'generation of voice that reproduces the voice of the speaker', and 'adjustment of mouth movement'.
Universal Translator's dubbing feature is very powerful, but it can also be abused for things like creating deepfakes. For this reason, Google will only offer Universal Translator to certified partners.
Related Posts:
