China's sophisticated AI censorship system revealed in leaked data set

A leaked dataset online has revealed that the Chinese government is developing an advanced censorship system based on large-scale language models (LLMs). The targets of the censorship are wide-ranging, going beyond the traditional taboos of 'Tiananmen Square' and 'Taiwan,' to include rural poverty, corrupt police, and Communist Party corruption.

Prompts in the open: China rules for LLMs - NetAskari

https://netaskari.substack.com/p/llms-and-china-rules

Leaked data exposes a Chinese AI censorship machine | TechCrunch
https://techcrunch.com/2025/03/26/leaked-data-exposes-a-chinese-ai-censorship-machine/

Security researcher NetAskari discovered a roughly 300GB dataset on how a Chinese large-scale language model (LLM) classifies data. The dataset was stored in an unprotected Elasticsearch database on the servers of Chinese IT company Baidu, with the latest entry dating back to December 2024.

The dataset contained around 133,000 entries and had references to 'eb35' and 'eb_speedpro,' suggesting it was a training set for Baidu's AI chatbot, Ernie Bot , and NetAskari believes the dataset is being used to train 'advanced AI systems' designed to automatically flag sensitive content for the Chinese government.

The content targeted for censorship included complaints about rural poverty, news reports about Communist Party corruption, and posts about corrupt police officers extorting money from entrepreneurs. Political, social, and military-related content was given 'high priority' in this dataset, meaning it needed to be flagged immediately.

The system also found explicit references to Taiwan in the political activity category, with the word 'Taiwan' mentioned more than 15,000 times, reflecting China's high interest in Taiwan's political situation, NetAskari said.

The dataset is described as being for 'public opinion manipulation,' which refers to government censorship and propaganda activities overseen by the Cyberspace Administration of China (CAC). Chinese General Secretary Xi Jinping has positioned the Internet as the 'front line' for the Chinese Communist Party's public opinion manipulation.

Xiao Zhang, a security researcher at the University of California, Berkeley, points out that this dataset is 'clear evidence that the Chinese government or its affiliates want to use LLM to improve their repression.' China's previous censorship methods relied on basic algorithms that automatically blocked banned words such as 'Tiananmen Massacre' and 'Xi Jinping,' but LLM can detect criticism that is difficult to identify with conventional algorithms on a large scale, making censorship more efficient.

'At a time when Chinese AI models like DeepSeek-R1 are making waves, I think it's very important to highlight how AI-driven censorship is evolving and state control over public speech is becoming more sophisticated,' Zhang said.