China and the Islamic State are reporting AI-generated propaganda disguised as news

In recent years, the performance and ease of use of generative AI has improved dramatically, but the proliferation of AI-generated propaganda and fake news has become an issue. Overseas media has reported on the reality that China and

https://www.theguardian.com/technology/article/2024/may/18/how-china-is-using-ai-news-anchors-to-deliver-its-propaganda

How ISIS allies are using AI fakes to spread propaganda quickly - The Washington Post
https://www.washingtonpost.com/technology/2024/05/17/ai-isis-propaganda/

According to The Guardian, a British daily newspaper, a video that appears to be a news program has been spreading on the Internet, criticizing Tsai Ing-wen , who stepped down as President of Taiwan (Republic of China) on May 20, 2024, by comparing her to 'Yosai'. This is a pun on the Chinese word 'Yosai' (water spinach) and 'Tsai (Chinese: empty)'.

The Guardian points out that the news content in this video is not read by a human, but by a 'fake news anchor generated by AI.' The origin and creator of the video are unknown, but it is likely created by someone close to China, as it aims to sow doubts among those who support Taiwanese independence.

Propaganda and fake news using these AI-generated avatars are already spreading on social media. Taylor Williams, director of disinformation research firm Graphika, said, 'AI-generated propaganda doesn't need to be perfect. If a user is just scrolling through X (formerly Twitter) or TikTok, they won't be able to pick up the little nuances on a small screen.'

China is at the forefront of AI-generated fake news, with the Microsoft Threat Analysis Center (MTAC) warning in an April 2024 report that 'China plans to use AI-generated content to disrupt elections in the U.S., South Korea and India.' Clint Watts of MTAC noted that media companies in China officially use synthetic news anchors, which has allowed them to hone their synthetic news anchor format. 'China is trying to incorporate AI into their systems, including propaganda and fake news. China is trying everything, and AI is not particularly effective,' Watts said.

Microsoft warns that China will disrupt elections in the US, South Korea and India with AI-generated content; has it already been used in Taiwan's presidential election?

The Washington Post also reported that the Islamic extremist group ISIS is also using AI for propaganda. In March 2024, a terrorist attack occurred at a concert venue in Moscow, Russia, killing more than 100 people. Just four days later, a video was released on a private platform linked to a terrorist organization in which a news anchor wearing a helmet and military uniform claimed that 'this attack is not a terrorist attack, but a normal situation seen in the fierce war between ISIS and countries in conflict.'

According to the SITE Intelligence Group , which tracks online activity by terrorists and extremists, the news anchor in the video is an AI-generated avatar created by ISIS supporters as part of an AI-generated media program called News Harvest.

ISIS is characterized by an active online propaganda strategy. It is natural that ISIS, which has long used social media and video sharing sites to gather fighters and supporters from the Middle East and around the world, would turn its attention to generative AI. 'AI is a game changer for ISIS,' said Rita Katz, co-founder of SITE Intelligence Group. 'It will give them a quick way to spread the results of their bloody attacks to every corner of the world.'

The image below is a screenshot of a news video created by an ISIS supporter. The Washington Post has labeled the AI-generated news anchor as 'AI-GENERATED,' but it is quite difficult to tell from the still image that it is AI-generated.

ISIS supporters are enthusiastic about using AI for propaganda. In March 2024, an ISIS supporter named 'Al Kurdi 500' commented on a private messaging server for ISIS supporters, 'It would be great if my brothers could make videos about the daily news instead of reading the news in text and watching images like on news channels like Al Jazeera. Technology has evolved a lot and it would be easy to do today, especially with the use of AI.'

Since the Russian terrorist attack, six 'News Harvest' videos have been posted, in which ISIS supporters explain the activities of ISIS-related organizations around the world. Stephen Stalinsky, executive director of the Middle East Media Research Institute (MEMRI) , said that since these videos were not produced by the official ISIS department, it is unclear whether they match ISIS's overall strategy, but since ISIS has been proactive in adopting the latest technology for some time, it is not surprising that an AI media department has been established within ISIS.

Indeed, an ISIS message obtained by MEMRI on April 23 stated that the group's media wing was seeking 'experts in poster creation, article writing and video editing,' specifically those familiar with 'Adobe Photoshop and Premiere, and AI.' ' Mujahideen of the media, the media awaits your attack,' the post read.

Other Islamic extremist groups are also beginning to look at the potential of AI, with an al-Qaeda-affiliated group hosting an online AI workshop in February 2024.

Aaron Zelin, a senior fellow at the Washington Institute for Near East Policy , said AI could lead to the rise of extremism abroad by exposing people around the world to propaganda and extremist groups. 'If you throw enough spaghetti against a wall, eventually some of it will stick,' he said. 'AI could provide a path for this kind of content to go viral.'