Crimes exploiting the AI boom are surging; Microsoft analyzes methods used to steal authentication credentials using ChatGPT and Claude names.
Microsoft has warned of an increase in phishing attacks impersonating AI brands such as ChatGPT, Claude, DeepSeek, and Microsoft Copilot, as well as malvertising, which involves using fraudulent advertisements to lure attackers.
AI brands as bait: How threat actors are using the AI hype in social engineering | Microsoft Security Blog
On May 5, 2026, Microsoft detected phishing emails impersonating ChatGPT. These emails used 'ChatGPT' as the sender name and had a subject line requesting recipients to update their ChatGPT Plus payment method. The email body warned that recipients would be downgraded to the free plan if they did not register a valid payment method within 7 days, and a large ChatGPT logo was displayed at the top. According to Microsoft, 4,500 of these emails were sent to South Africa, and a broader campaign using the same theme and infrastructure has been confirmed to have sent up to 100,000 emails in a single day to Switzerland, Austria, and South Africa.
Clicking the 'Update payment method' button in the email reportedly triggers numerous redirects, first to the legitimate customer management service, then to Amazon's email open and click tracking domain, and finally to the URL shortening service Rebrandly. These redirects are believed to be a tactic to bypass email filters and make users believe the email is legitimate.
Ultimately, users are redirected to a folder called '/ChatGPT/' on an unofficial domain. Instead of immediately showing the card information input screen, the system first displays a simple CAPTCHA-like screen with only an 'Update payment' button. After the user clicks the button, they proceed to a page where they enter their name and address, and finally, their cardholder name, card number, expiration date, and verification code. This step-by-step process appears to verify that the information is being entered by a human.
The attacks impersonating Claude used the phrase 'Dispute Terms of Service Violation' rather than 'Payment Renewal.' According to Microsoft, between April 20th and 22nd, 2026, emails impersonating Anthropic-related services were sent to more than 2,000 organizations, primarily in the United States, the United Kingdom, and India. Information technology, business-related, and financial services sectors were prominently targeted.
The email body was in HTML format using the Anthropic and Claude brands and explained that the account had violated the Terms of Use (AUP) and that an appeal process had been initiated.
Furthermore, they attached a PDF titled 'Fill and Sign Claude Appeal Form.pdf' (Claude Appeal Form.pdf), which instructed users to copy the appeal ID and click the 'Claude Appeal' link.
Clicking a link within the PDF will redirect you to a domain controlled by the attacker, where a screen disguised as Cloudflare authentication will be displayed.
Users who passed authentication were directed to a Claude-like 'Account Security & Compliance' page, where they were asked to copy an access code and continue. Microsoft explains that this process was likely used as a gateway to prevent automated analysis.
Although the final page was no longer available at the time of Microsoft's analysis, the source code still contained conditional branching that differentiated the redirection destination between mobile devices and desktop environments.
The overall picture of the attack is as follows: Microsoft says it is highly likely that the shared intermediate domains and similar redirect processes ultimately led to a page impersonating the Microsoft sign-in screen, resulting in a man-in-the-middle (AiTM) attack that stole authentication tokens.
The misuse of the AI brand isn't limited to email; it also involves a technique called 'malvertising,' which uses advertising space to redirect users to malware distribution pages. Microsoft has confirmed attacks where users who click on video players on free movie streaming sites or try to close pop-up ads are redirected to the 'Awesome AI Windows Plugin' download page, and they've linked this attack to Storm-3075, a criminal group known as an 'early access broker' that creates and sells entry points for intrusions. A single campaign on March 13, 2026, reportedly targeted over 66,000 devices.
'Awesome AI Windows Plugin' is not a real product; it's a name used to make it appear as if it offers free access to high-definition videos. The downloaded executable file is 'ProFluxeFlowAi-win-Setup.exe' and was located in an AI-themed folder on GitHub. Furthermore, the executable file was signed with a fraudulently obtained Microsoft-issued code signing certificate, adding to the impression that it was 'trustworthy' to Windows and users. Microsoft has already revoked the associated signing certificate, and GitHub has removed the related repository.
The malware doesn't start stealing information immediately after execution; instead, it displays a checkbox labeled 'Continue' and refrains from performing any suspicious actions until the user clicks it. Microsoft believes this waiting period is a mechanism to avoid sandboxing and automated analysis. After clicking, a Python-based downloader is deployed, ultimately distributing the information-stealing malware, Vidar.
In attacks impersonating DeepSeek, the search results themselves served as the entry point. According to Microsoft, in April 2026, a fake GitHub organization and repository were created that exploited interest in DeepSeek V4, and used GitHub's release function to distribute information-stealing malware such as Vidar. The fact that DeepSeek had not yet published an official V4 repository on GitHub also contributed to the visibility of the fake repository. It is believed that the attackers prepared new GitHub organizations, repositories, and release tags within hours of the DeepSeek V4 preview announcement, and were ready to distribute the malware in less than an hour.
The repository contained the DeepSeek logo, actual benchmark data, searchable tags, and an llms.txt file that appeared to be for AI search. It garnered 91 stars and 27 forks in four days, but Microsoft stated that it could not determine the proportion of genuine engagement and inflated numbers.
However, a closer look revealed some inconsistencies. The repository contained no genuine model code, only a README, LICENSE, llms.txt, and a nearly empty folder. Nine commits were made in a short period of time by a single author on April 24, 2026, and while the README claimed an MIT license, the repository metadata indicated an Apache 2.0 license. A fake release page distributed a 102MB 7z archive, and it was reported that the hash was changed at least three times over three days while maintaining the same filename.
To avoid these attacks, you should check the domain of the page you're opening, the presence of unexpected PDFs or executables, and whether the repository in the search results contains genuine code, rather than relying on logos or sender names. For organizations, it is recommended to require multi-factor authentication for all accounts, use passkeys or phishing-resistant authentication, and evaluate sign-in requests from additional signals using conditional access. Email countermeasures include enabling zero-hour auto-purge to quarantine emails found to be dangerous after delivery, and Safe Links to re-evaluate links when clicked.
Microsoft explains that attackers are not only using AI to accelerate their attacks, but also leveraging interest in AI itself as material for long-term social engineering. However, Microsoft emphasizes that 'this case is merely an example of misuse of an AI brand name and does not indicate that any of the AI service providers mentioned have been compromised.' Microsoft states that because the name of AI continues to be used not only as a 'useful tool' but also as 'bait to get clicks,' it is necessary to strengthen detection and response across email, IDs, and devices.
Related Posts:
