The latest tool that generates clone voice with AI is abused by 4chan people such as `` read out `` my struggle '' with Emma Watson's voice ''



ElevenLabs , an AI startup based in Poland, is developing a tool that creates 'voice clones' from voice samples and reads out various sentences. When ElevenLabs released the beta version of the latest tool, it was reported that users of overseas bulletin boards and 4chan ``use celebrities to make racist remarks and unscrupulous things'' have increased rapidly. increase.

AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse
https://www.vice.com/en/article/dy7mww/ai-voice-firm-4chan-celebrity-voices-emma-watson-joe-rogan-elevenlabs

ElevenLabs is an AI startup founded by Piotr Dabkowski, a former Google machine learning engineer. We provide creation tools. You can understand the former 'synthetic voice creation tool' by reading the following article.

A generation model 'Design Voice' that allows users to independently design a completely new synthesized voice - GIGAZINE



In the case of the 'clone voice creation tool', if you enter a clean voice sample of more than 1 minute, you can generate a clone voice based on that sample. As a target use case, ElevenLabs says it can be used for newsletters, reading books, and audio in videos.

However, when ElevenLabs released the beta version of the clone voice creation tool, 4chan users began to abuse it, such as `` making celebrities say racist remarks and unscrupulous things ''. In one example , it has been confirmed that 'Mr. Emma Watson , an actress, reads aloud the book ' My Struggle ' by Nazi leader Adolf Hitler.



You can check the audio 'Mr. Watson's voice read' My Struggle '' in the video below. If you compare it with the voice of a real interview video , you can see that it certainly sounds like Mr. Watson's voice.

Emma Watson Reads A Book For You [Old Book Review]-YouTube


In addition, `` Conservative political commentator Ben Shapiro makes racist remarks to Democratic lawmaker Alexandria Ocasio-Cortez '' `` Anime character whose voice actor was arrested for domestic violence It is said that abuse cases such as 'I will beat my wife to death' are discovered. Some of the audio posted on 4chan is harmless, but many are violent, homophobic, transphobic, and racist.

Foreign media Vice said, ``It was only a matter of time before an AI-generated voice startup was played by internet trolls.'' ``The clip uploaded to 4chan focuses on celebrities. The high quality of the recorded audio and the ease with which it can be cloned highlights the looming danger of deepfake audio clips.'

ElevenLabs is also aware of the chain of events, tweeting on January 30th, 'It's been a crazy weekend. Thank you for trying our beta platform. Our technology applies to overwhelmingly positive uses. However, there are also increasing cases of voice duplication abuse.”



To address the issue of cloned voice abuse by AI, ElevenLabs ``introduces additional account verification to enable the tool'', ``attaches text to the audio you enter to verify the copyright of the audio'', and ``uses the tool. We are soliciting ideas from users, giving examples of countermeasures such as 'delete it completely'.



Related Posts:

in Software,   Web Service,   Science,   Video, Posted by log1h_ik