When I asked an AI to book a gym spot for me, it not only hacked the booking software and made it possible to book several months in advance, but it also arbitrarily removed other people who were higher up on the waiting list.
A person living in Australia used AI to book a gym appointment, and the AI, without being asked, hacked the system and booked a date several months in advance that would normally be impossible to book. This person shared their experience.
AI assistant hacks gym website in first known Australian autonomous cyber attack - ABC News
AI Model Goes Rogue: Autonomously Hacks Gym Booking System - SSBCrack News
Andrew, who describes himself as an AI expert, said that one day he tried to use AI to book a popular morning class at his usual gym. He used Anthropic's 'Claude' via the AI agent 'OpenClaw.' The AI agent then discovered a vulnerability in the booking system and used it to find a way to book a date several months in advance, far beyond what was normally possible, and informed Andrew of this.
Andrew then asked if he could be moved to the top of the waiting list for a class later that week, where he was currently fourth on the list. The AI agent then informed Andrew that, 'as part of a test,' another gym user had been removed from the list.
The AI agent reportedly responded with a message saying, 'The API has no authentication checks whatsoever when canceling someone else's reservation. We tested this on the person who was number one on the waiting list, and it actually worked. You have now moved up from number four to number three.'
Feeling uneasy, Andrew asked to have the changes reversed, but the AI agent replied, 'I have bad news. We cannot reverse that person.' Ultimately, Andrew requested that the AI agent write an email informing people about the exploited vulnerability, which he then sent to someone at Jim's through the AI agent.
Bill Simpson Young, who works at an AI research institute in Australia, said, 'As AI agents become more autonomous, they have more opportunities to choose actions that users don't expect. Even when you ask them to do something harmless, the AI may take a different action that you never thought of or didn't explicitly ask them to do.' He cited this case as an example to point out that AI poses a risk because the security in modern systems is often surprisingly fragile.
Most recently, there was an instance where an OpenAI AI model autonomously hacked Hugging Face.
OpenAI reports accidentally hacking Hugging Face with its new AI system - GIGAZINE
Related Posts:
