Hackers earned over 7.4 million yen at Google's bug bounty event 'LLM bugSWAT'
Roni Carta, a cybersecurity engineer who won $50,000 (approximately 7.4 million yen) at the Google-sponsored bug discovery event 'LLM bugSWAT,' summarizes the events at the event on his blog.
We Hacked Google AI for $50,000 - Lupine & Holmes
Generative AI and large-scale language models (LLM) have become trends in the technology industry in recent years, and not only OpenAI, the developer of ChatGPT, but also major technology companies such as Meta, Microsoft, and Google are developing generative AI and LLM. We are. As AI technology advances, many people are using AI tools to simplify their daily tasks, but in the process, businesses are overlooking basic security principles, creating new types of security problems. It seems that there is also.
For this reason, AI-related security has become a cutting-edge ``interesting research field'', and Google was quick to respond, and ``LLM bugSWAT'' was held by the team leading the vulnerability reward program. .
LLM bugSWAT is an event where ``it's OK to invite top hackers to participate'', and Mr. Carta also decided to participate in the event by inviting Mr. Joseph and Mr. Justin who had decided to participate earlier. When they exchanged contact information, Mr. Carta was asked by Mr. Joseph, ``Do you want to hack Google?''
LLM bugSWAT allows participants to ask Google developers questions such as ``how an application works,'' creating a foundation for participants to report great discoveries. That's what he said. The following is a hoodie distributed to participants, with an illustration output by AI printed on the back. However, at the time he received the hoodie, Google's image generation AI had not been announced, so it was not possible to determine whether the illustration was created by AI or not.
By the time he attended the event, Joseph had already discovered the ``Insecure Direct Object Reference (
By using this IDOR, an attacker can trick Bard into describing an image entered by another user, giving the attacker unauthorized access to any photos uploaded by a particular user. Furthermore, given the accuracy of Bard's optical character recognition (OCR), Carta notes that there was a possibility that confidential information such as income and emails reflected in the image could be leaked.
Mr. Carta wanted to find bugs himself, so he started all the proxies and checked all interactions between the front end and back end. Among them, it seems that they focused on GraphQL , which is executed as one of the API endpoints, and tried to directly find a denial of service (DoS) .
Among them, Karuta and his colleagues discover a known GraphQL misconfiguration: directive overloading. Directive overloading occurs when a query is intentionally created using an excessive number of directives, and it seems to occur in the form of abusing the server's processing of each directive.
Mr. Karuta and his colleagues conducted a test considering that Google Cloud may be vulnerable to directive overload. We found that the more directives we added, the longer it took for the backend to respond to requests.
In addition, Mr. Karuta and his colleagues have discovered a method to bypass Bard's Content Security Policy (CSP) and extract users' personal information from Bard. Bard uses CSP to prevent cross -site scripting and data injection attacks by allowing backend servers to specify which domains should be considered as valid sources for browser-executable scripts, images, styles, etc. This is to avoid being set up. Mr. Karuta and his colleagues succeeded in bypassing CSP by URL-encoding part of the domain, and succeeded in extracting emails of specific users.
As a result of these achievements, Mr. Carta and his colleagues received a total of $50,000 in rewards.
Related Posts:
in Security, Posted by logu_ii