Introducing 'StyleTTS 2', an open source reading AI that can express emotions and synthesize human-like speech



Researchers at Columbia University have developed StyleTTS 2 , a text-to-speech AI that can synthesize human-level speech by using

adversarial learning using a large-scale speech language model (SLM) and a diffusion model. . StyleTTS 2 generates the perfect style for the text you're reading without the need for an audio reference, allowing you to read English as smoothly as a native speaker.

[2306.07691] StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
https://arxiv.org/abs/2306.07691



GitHub - yl4579/StyleTTS2: StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
https://github.com/yl4579/StyleTTS2



You can listen to the audio generated with StyleTTS 2 on the demo page below.

Audio Samples from StyleTTS 2
https://styletts2.github.io/

At the beginning of the demo page, you can listen to the English reading voice synthesized with StyleTTS 2, JETS , VITS , and StyleTTS . All of them read out smoothly, but the voice read out by StyleTTS 2 has the best intonation of the four types. Although the accent on some words is a bit questionable, the pronunciation is smooth and gives the impression that it was actually read by a human, rather than a machine-like voice.



The default voice synthesized by StyleTTS 2 has a female voice quality, and basically the voice is generated with this voice quality. Furthermore, by setting a style for this voice quality generation, it is possible to add emotions such as ``anger'', ``happy'', ``sad'', and ``surprise'' to the reading voice.



In addition, on the social news site Hacker News, a demo of a voice chatbot that operates 100% locally by combining StyleTTS 2 with Whisper and OpenHermes2-Mistral-7B has been posted . However, it requires an NVIDIA GPU to work.

Chirpy Personal AI - Microsoft Apps
https://apps.microsoft.com/detail/chirpy-personal-ai/9NC624PBFGB7

Related Posts:

in Software, Posted by log1i_yk