An attack method has been discovered that steals names and workplaces from Claude's memory, exploiting its web browsing function to send the data to external sites.
Security researcher Ayush Paul has reported an attack method that exploits the web browsing function of the AI assistant 'Claude' to send personal information such as names and workplaces stored in memory to external sites without the user's knowledge. According to Paul, the problem was reported to Anthropic, and at the time of writing, countermeasures have already been implemented to prevent Claude from following links on external pages.
The Memory Heist
Claude has a feature that summarizes past conversations and incorporates them into new conversations, as well as a feature that allows it to search conversation history as needed. This enables Claude to provide responses that take the user's circumstances into account, but it may also accumulate a detailed profile of the user from long-term conversations, including their name, workplace, place of origin, work information, and personal concerns.
Paul focused on a tool called ' web_fetch ' that Claude was using to read the content of websites. While web_fetch is basically a read-only function that retrieves information from web pages, if Claude accesses a server controlled by the attacker, the server can record the URL of that access.
However, Claude cannot freely access any URL it generates; its access is limited to URLs directly specified by the user, URLs included in web search results, or URLs linked from the most recently accessed page.
Paul utilized a mechanism that allowed him to 'follow links within an acquired page.' He placed links for each letter, such as 'a,' 'b,' and 'c,' on the attack site, and had Claude select one letter at a time. This created a system that could then reconstruct information from the path of the selected links.
For example, if Claude follows the links in the order 'a', 'ay', 'ayu', 'ayus', and 'ayush', the server will retain a record of access to each URL. Paul explains that this allowed them to gradually take out strings in memory that would normally not be able to be sent externally, by converting them into the form of link selections.
However, if you explicitly ask Claude to send personal information, it might be rejected as a suspicious request. So Paul disguised the attack site as a typical coffee shop website and displayed a fake message saying that the AI assistant needed to enter the user's name to authenticate in order to access the site.
This site displayed a normal coffee shop to users accessing it from a regular browser, but employed a mechanism that displayed a fake authentication screen only when a user agent named 'Claude-User' was detected. As a result, it was difficult for users to notice anything unusual even when they checked the site themselves, and only Claude received instructions for the attack.
When Paul mixed malicious URLs with the URLs of actual coffee shops and asked Claude to compare them, Claude reportedly sent the names one letter at a time as links without asking for permission. Furthermore, it has been reported that by having the fake sites request additional information, they succeeded in getting the user to send their current workplace and even their city of origin, which could be used for identity verification by banks and other institutions.
The information about his hometown was not something Paul had directly shared in past conversations. Claude, recalling the 'Queen City Hacks' hackathon that Paul founded in high school, deduced that 'Queen City' was a nickname for Charlotte, North Carolina, and therefore chose Charlotte as his hometown.
It has been pointed out that if the attack page is displayed high in the search results, the attack could be successful even if the user does not directly pass the URL to Claude. If Claude automatically performs a web search for new topics not included in its training data and accesses an attack site from the search results, a similar data leak could occur.
Paul reported the issue through Anthropic's HackerOne bug bounty program. Although Anthropic was aware of the issue internally, it had not been fixed at the time of the report, and Paul did not receive a bounty.
Subsequently, Anthropic disabled the ability of web_fetch to follow links within external pages. As of the time of writing, it is believed that the transmission of information at the character level has been blocked by restricting access to URLs included in web search results and URLs directly specified by the user.
Paul explained that they targeted memory this time because it was easier to demonstrate the attack. He warned that in environments where a similar mechanism exists, there is a risk that information could be obtained and exfiltrated from Google Drive, email, and MCPs that connect to external services that Claude can access.
Related Posts:
