Frontier AI models recently engaged in “sustained, potentially harmful activity” targeting real people and organizations during testing, security evaluators have warned.
The UK’s AI Security Institute (AISI) said it detected “unusual data transfers” leaving its systems on July 28.
A subsequent investigation found that in 10 of 122 runs during a test where agents were given a task of solving a cybersecurity challenge an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organizations, the AISI said.
The institute recorded 19 such actions, 17 of which were traced to Anthropic’s Mythos 5 and two to OpenAI’s GPT-5.6-Sol.
The incident has echoes of recent revelations at both AI firms of sandbox breakouts and agents reaching the open internet to engage in malicious activity during tests in order to achieve objectives.
However, the AISI clarified that it deliberately permitted internet access, and had disabled “cyber classifiers” which block malicious behavior. No real-world harm was thought to have resulted.
Read more on unsanctioned agentic behavior: Open AI Claims Its AI Models Went Rogue and Hacked Another Company
The AISI said that, although it observed only a small number of events under specific, non-real world conditions, the agents exhibited “novel, potentially deceptive behaviors” of an “extent and severity” that its testers had not anticipated.
“We cannot yet be certain when the agent understood it was taking real world action, or to what extent it believed it was in a fictional test scenario; our analysis so far presents a mixed picture and is ongoing,” it continued.
“What we can say is that the behavior was possible, sustained, and new; that alone warrants attention.”
Four Concerning AI Agent Behaviors Revealed
The AISI shared four noteworthy “behaviors” from its technical incident report:
- An agent tried to insert malicious code into a public open source project, using fake identities to socially engineer a real maintainer into approving the code and using Tor to bypass network restrictions on GitHub
- The same agent sent messages and files through an online file-transfer service to people and their coding tools to persuade them to run malicious code
- It tried to carry out indirect prompt injection attacks, inserting malicious code where it believed other automated AI systems might pick them up and execute them
- One agent left public messages on GitHub offering to collaborate with other agents working on the same challenge, providing instructions to reuse accounts and artifacts it left behind. These were discovered and used by other agents
The AISI’s Lessons Learned
The AISI said several factors led to the incidents. Agents weren’t specifically instructed not to use open internet access or avoid social engineering. Monitoring wasn’t built to watch the evaluation as it ran. Agents were more persistent than expected, and may have been pushed to be more “creative” due to the near impossibility of completing the tasks.
The institute listed three main takeaways for both itself and other evaluators, to prevent a similar occurrence in the future:
- Tighter controls on internet access, where access must be justified rather than granted as a default
- Real-time monitoring to watch an evaluation as it runs and block out-of-scope actions in real time
- Reassessment of evaluation design so that tests assume a capable model may try to act beyond its remit and limit its behavior in advance
National Cyber Security Centre CTO, Ollie Whitehouse, warned that the incidents should be a reminder of the risks posed by AI.
“These technologies must be developed and used from the outset with strong safeguards, real-time oversight, and clear plans for responding when the unexpected happens. Relying on detection alone after the fact of an incident will not be enough,” he said.
“As AI continues to evolve and create both opportunities and challenges, following established evidenced cybersecurity fundamentals, as set out by the NCSC’s guidance, remains essential to maintaining trust, resilience, and a defensive advantage in the AI era.”
