Google Threat Intelligence Group reports that China, Iran, North Korea, Russia, and others are using Google's Gemini AI to carry out cyber attacks

The Google Threat Intelligence Group (GTIG) has reported that it has found dozens of instances in which hacking groups backed by Iran, North Korea, China, Russia and other countries are using Google's multimodal AI, Gemini, for malicious purposes, such as creating malware, translating content and improving phishing attacks.

Adversarial Misuse of Generative AI | Google Cloud Blog
https://cloud.google.com/blog/topics/threat-intelligence/adversarial-misuse-generative-ai/?hl=en

According to the GTIG investigation, the hacking group did not attempt to develop its own prompt attacks or other AI-specific threats, but instead used publicly available jailbreak prompts in a basic way, suggesting that attackers are using Gemini to increase productivity but not develop new capabilities.

Iranian hacking groups were the most active users of Gemini, using it for a wide range of purposes, including spying on international defense and government organizations, vulnerability research, and creating content for their campaigns. In particular, the group known as APT42 focused on creating phishing campaigns and conducting reconnaissance against defense professionals and organizations.

Chinese hacking groups primarily used Gemini to support their activities after infiltrating target networks, performing activities such as privilege escalation, data theft, and detection evasion. The usage patterns were similar to those used by IT administrators to streamline their work, but it has been discovered that they were actually used for malicious purposes.

North Korean hacking groups have used Gemini in multiple phases of their attack lifecycle, including infrastructure surveys, reconnaissance of target organizations, payload development, assisting in the creation of malicious scripts, and even for creating resumes and job applications as part of attempts to place undercover operatives at Western companies.

The Russian hacking group's use of Gemini has been relatively limited, which could be due to an attempt to avoid Western platforms to avoid monitoring of their activities, or the use of Russian-made AI tools, GTIG said.

In terms of information operations, hacking groups from Iran, China and Russia used Gemini primarily for research, content generation, translation and reach expansion, with Iranian hacking groups being the most active, accounting for three-quarters of all input and output prompts.

GTIG is continuously working to ensure strong security measures and safety for the responsible development of AI. Specifically, it is introducing the Secure AI Framework (SAIF) , providing comprehensive toolkits for developers, implementing safeguards, evaluating model safety, and testing and protecting AI systems through red teaming. GITG said that these efforts aim to minimize the risk of misuse while leveraging the great potential of generative AI.