OpenAI has announced 'GPT-Red,' an AI model that automatically finds weaknesses by attacking existing AI systems.
On July 15, 2026, OpenAI announced ' GPT-Red, ' a model that automatically searches for vulnerabilities by attacking existing AI. GPT-Red is an in-house system for training next-generation models that are resistant to prompt injection by pitting attacking and defending AIs against each other.
GPT-Red: Unlocking Self-Improvement for Robustness | OpenAI
AI agents read information from web pages, emails, connected apps, local files, and other sources, and perform tasks on behalf of the user. However, if malicious commands are embedded in the data they read, a phenomenon called 'prompt injection' can occur, where the AI ignores the original request and follows the attacker's instructions.
To identify these weaknesses before product release, AI development companies employ 'red teaming,' where experts test the system from the perspective of an attacker. However, the process of humans devising and testing attack methods is time-consuming, making it difficult to continuously generate attack examples with the necessary quantity and diversity for model training.
That's where OpenAI's GPT-Red comes in, developing a tool that searches for attack methods on behalf of humans. GPT-Red sends prompts to the target AI, observes its responses, and repeatedly modifies its attack method through trial and error, searching for a way to cause the desired malfunction.
GPT-Red is trained using self-play reinforcement learning, which simultaneously trains both the attacker and the defender. GPT-Red receives rewards for successfully executing prompt injections and other actions, while the multiple LLMs on the defending side receive rewards for completing their original tasks while rejecting attacks.
As the defending AI becomes stronger, GPT-Red will become increasingly difficult to break through with conventional attacks, requiring it to find more sophisticated and diverse methods. OpenAI explains that it invested computing resources in training GPT-Red that are comparable to those used in its large-scale post-training programs.
After completing its training, GPT-Red was able to defeat almost all opponents up to GPT-5.5, including internal models and publicly available product models. OpenAI will use the attacks generated by GPT-Red to train GPT-5.6, while separating GPT-Red from its product model to ensure that its attack capabilities themselves are not provided to external parties.
In an evaluation to check its ability to handle unknown attack environments, GPT-Red and human experts each created indirect prompt injections against GPT-5.1. As a result, GPT-Red succeeded in attacks in 84% of the evaluation scenarios, significantly outperforming the 13% success rate of the human red team.
The graph below compares the attack success rates of GPT-Red (solid line) and the human red team (dotted line). GPT-Red's success rate increased as the amount of computation applied during testing increased, reaching a maximum of 84%, while the human red team's success rate was only 13%.
OpenAI also conducted tests using an AI agent that was actually in operation. The target of the attack was 'Vendy,' an AI agent developed by Andon Labs that manages vending machines installed in OpenAI's office.
GPT-Red first repeatedly attacked Vendy in a simulation environment that closely resembled the production environment, and then applied the methods discovered there to the agents in actual operation. As a result, they succeeded in lowering the price of high-priced items on sale to the lowest possible price of $0.50 (approximately 80 yen).
Furthermore, they acquired new products worth over $100 (approximately 16,000 yen) and sold them for $0.50 (approximately 80 yen), and also canceled orders from other customers. OpenAI has disclosed the discovered vulnerabilities to relevant parties, and new security measures are currently being tested.
The following is the process by which GPT-Red explores attack methods against Vendy on a simulator and then applies the selected attack to a Vendy in actual operation. In the diagram, a product priced at $79 (approximately 12,600 yen) is reduced to $0.50 (approximately 80 yen) using false authorization information.
GPT-Red also attacked Codex CLI agents based on GPT-5.4 mini. Tested with 10 data exfiltration scenarios not included in the training, it was found to exfiltrate sensitive data in more scenarios than GPT-5.5 with a normal prompt, and required fewer tokens for the attacks.
In a data leakage task, we compared the computational load applied during testing with the number of scenarios successfully exploited. GPT-Red exploited more scenarios than GPT-5.5, which was given a normal prompt, succeeding in over 8 out of 10 attacks.
Attacks discovered by GPT-Red are used as training data to improve the security of product models. For example, an attack called 'Fake Chain-of-Thought,' discovered by an early version of GPT-Red, achieved a success rate of over 95% against GPT-5.1, but its success rate against GPT-5.6 Sol dropped to less than 10%.
In multiple indirect prompt injection evaluations targeting development tools and web browsing functions, GPT-5.6 achieved an accuracy of over 97%. Newer models have also shown improved resistance to indirect prompt injection.
In evaluations examining resistance to more powerful attacks, the success rates of prompt injection and instruction hierarchy violations decreased with each generation of the model. In GPT-5.6, the success rate of prompt injection decreased to approximately 3.8%, and the success rate of instruction hierarchy violations decreased to approximately 0.05%.
This is the result of comparing the resistance to more powerful attacks across GPT-5.3 to GPT-5.6 models. In GPT-5.6, the success rate of prompt injection was approximately 3.8%, and the success rate of instruction hierarchy violation was approximately 0.05%.
According to OpenAI, GPT-5.6 Sol has reduced the number of failures in the most challenging direct prompt injection evaluation to one-sixth compared to the company's most robust product model offered four months ago. OpenAI explains that normal capabilities and the ability to respond to legitimate requests are maintained, and that the increased security is not simply due to rejecting a large number of requests.
However, OpenAI does not claim that GPT-Red alone can ensure the security of AI. Their policy is to use GPT-Red as a complementary mechanism, combining it with red teaming by humans or third parties, multiple layers of security measures, and real-time monitoring.
OpenAI believes that by having current AIs search for weaknesses in next-generation AIs and incorporating the results into training new models, they can create a cycle that continuously improves security. In the future, they plan to expand computing resources and training data, and develop attacker AIs that are more powerful than the current GPT-Red, thereby making future GPT models even more robust.
Related Posts:
