A detection method is being researched that can identify AI agents by observing 'mouse movements.'
Akamai, a provider of CDN and cybersecurity services, is researching a method to distinguish AI agents operating on websites in place of humans based on their mouse movements. There are significant differences in the number and intervals of mouse operations between humans and AI agents, and Akamai has prototyped a system that can identify AI agent-specific behavior even with limited operation data.
Identifying Agentic Automation with Behavioral Telemetry
For many years, website security has relied on technologies to distinguish between 'humans and bots' accessing the site. However, with the emergence of AI agents that autonomously control browsers based on natural language instructions, access that is difficult to categorize using only the traditional 'human or bot' classification has increased. Akamai cites browser control functions such as Perplexity Comet (Comet), OpenAI Atlas (Atlas), and Claude Chrome Extension as examples. When a user gives such an instruction to an AI agent, such as 'find a product and add it to the cart,' the agent operates the browser, clicking on the desired location or entering the necessary information.
AI agents are not trying to imitate humans; rather, they are characterized by performing very few actions that are unnecessary for achieving their objectives. Humans move the cursor while reading a page, hover the mouse over links before clicking, and scroll to find information. On the other hand, AI agents move directly to the target location, click precisely, and move on to the next operation once the necessary input is completed. As a result, the recording of mouse operations is very minimal, making the process seem mechanical.
Akamai's bot detection product, 'Bot Manager Premier,' collects behavioral data such as mouse movements, clicks, and scrolls to distinguish between humans and bots. However, conventional models primarily target bots that try to evade detection by mimicking human-like mouse operations, and a certain amount of mouse operation data was required to determine operation patterns. Since AI agents rarely perform mouse operations, there was insufficient mouse operation data to analyze them with conventional models. In fact, when examining Comet and Atlas, 63.2% of requests had zero mouse events, and only 1.0% had enough mouse movement for conventional models to analyze.
Akamai devised a method to treat mouse operations like 'words' in a sentence. Using a machine learning technique called Transformer, they analyze not only the types of mouse movements and clicks but also the time intervals between operations, allowing them to distinguish between humans and AI agents from a limited number of events. After training the AI with basic human operation patterns using approximately 3.5 million mouse operation data points, they trained a classification model using 2,190 AI agent operation data points collected from Comet, Atlas, and other sources.
The figure below visualizes human and AI agent operation data in two dimensions using PCA (Principal Component Analysis) and t-SNE. Blue represents humans, and red represents bots or agents. The small neural network ultimately adopted achieved a ROC-AUC of 0.981, which indicates classification performance. ROC-AUC is an indicator where a value closer to 1 indicates higher classification performance, and 0.981 indicates that it was able to distinguish between humans and AI agents with high accuracy. However, it is stated that simply identifying 'access with few operations' as AI is insufficient, as humans who only perform one operation and then leave exhibit similar characteristics to AI agents.
It should be noted that the mechanism announced this time is not a new feature offered as a product, but rather a proof-of-concept research project envisioning future practical application. Akamai says that in future research, they will verify the detection performance at the request and session levels, whether the AI agent can distinguish unique features even on different websites, and will also explain the conditions necessary to make it a system that can be used in actual services.
Related Posts:
in AI, Posted by log1d_ts