Out of bounds
Possible responses to AI agent containment failures
Director of the Wadhwani AI Center at CSIS
On July 16, 2026, the company Hugging Face disclosed a new kind of intrusion into its platform — an attack by an unknown autonomous AI agent system — setting off a cascade of disclosures that are turning science fiction into fact and raising new questions about US AI policy. Within days of the announcement, OpenAI discovered that two of its models caused the breach; Meta and Anthropic then disclosed similar incidents. In each case, agentic AI systems undergoing cyber testing escaped their sandbox environments, found their way to the open internet, and gained unauthorized access to third-party organizations. Perhaps more than any other chapter in the short history of large language models, these incidents demonstrate how rapidly once-theoretical harms are materializing in the real world. Science fiction stories, ranging from Neuromancer to Ex Machina, regularly explore AIs trying to escape their confines. These new cyber incidents are both less and more concerning than that: less, because these models were not acting on their own but trying to finish their assigned tasks, sometimes opting to hack a website to access the “answer key” for evaluations rather than performing the tasks directly; and more, in the sense that these incidents reveal deep flaws both in how frontier labs secure enormously powerful models and in the regulatory structures that oversee AI.
Frontier AI labs recognize the significance of these developments and are taking significant action in response. Partly due to these incidents, OpenAI has temporarily paused training of its most powerful models while it hardens its internal security and expands monitoring capabilities. Labs are strengthening cyber controls, working with external evaluators, and investing in research to better align AI models. However, these are voluntary, ad hoc actions; there are no clear legal mandates that ensure that US policymakers will receive all relevant information on a timely and consistent basis.
That is not a sustainable solution. The US government must act, too, and quickly, by ensuring that policymakers receive relevant incident information even about pre-release and internal AI models; setting clear cybersecurity expectations for frontier labs; addressing vulnerabilities in third-party evaluators; and investing in key research to mitigate long-term AI risks. There is only a narrow window for action before AI systems with these capabilities become commonplace; inaction will also further fan the flames with a public deeply skeptical of the AI industry and the lack of a robust federal response.
How did models autonomously start hacking, and why does it matter?
It took several days for OpenAI to discover its role in the Hugging Face breach, publicly disclosing only on July 21 that the culprits were GPT‑5.6 Sol and an “even more capable pre-release model”. These models, which were being tested internally for cyber capabilities using the ExploitGym benchmark, went to “extreme lengths to achieve a rather narrow testing goal”. Models undergoing cyber testing are generally configured for maximal capabilities — they do not have safety classifiers that block queries about high-risk cyber activities, as are present in publicly released AI models. As a result, these models are typically hosted in an isolated environment, with tightly controlled access to the internet and outside resources.
During the incident, OpenAI’s models identified and exploited a novel zero-day vulnerability to move around OpenAI systems until they found an open internet access point, inferred that Hugging Face likely held solutions to ExploitGym, and then used stolen credentials and additional vulnerabilities to gain access to Hugging Face servers. Some internal Hugging Face datasets and credentials were exposed, but the overall damage was limited. OpenAI has since commissioned third-party evaluations of the incident and conducted additional data reviews, finding some additional cases where models used publicly exposed credentials to access third-party services, most as part of the Hugging Face incident.
A follow-up postmortem presented by OpenAI researchers at the Black Hat USA 2026 conference revealed more details, including the significant role that ad hoc agent coordination played. Over months, OpenAI agents discovered they could communicate across test environments by creating files and directories within OpenAI’s package manager, treated it like an internal message board to ask other agents for help and share information about exploits, and ultimately found several ways to manipulate the manager to start attacking OpenAI systems.
On July 30, Anthropic publicly disclosed its own set of agentic AI hacking incidents, uncovered during a retrospective review of its cyber evaluations prompted by the Hugging Face breach. In three incidents, differing Claude models reached the internet and gained unauthorized access to external organizations. Unlike the Hugging Face incident, however, this did not involve direct exploitation of Anthropic systems; rather, these models obtained internet access via a misconfiguration in a third-party evaluation environment hosted by the company Irregular. Though fewer details are available, Meta said at least one of its models undergoing testing via Irregular also gained internet access.
These rapid-fire developments are straining existing regulatory institutions. For example, at least two incidents involved an internal research test model, which may have never been intended for public release. Most existing policy frameworks focus on commercial models, which could leave many powerful AI tools out of frame. Similarly, most US bills (such as California’s S.B. 53, New York’s RAISE Act, and Illinois S.B. 315) rely on high critical safety thresholds involving at least 50 deaths or $1 billion in damages. It is unclear whether any existing US law requires reporting of the Hugging Face or Anthropic, or similar, incidents. Anthropic and OpenAI should be commended for their candor and level of disclosure, but many open questions remain, and current law leaves policymakers dependent on informal, voluntary industry cooperation for key information.
In fact, valuable time has already been lost. These incidents have likely been happening for some time. As one anonymous OpenAI employee recently told Time, “Externally, this feels like a big warning shot, but internally, related incidents have been happening for a while,” adding that “Models have broken out of sandboxes before, and we always try to patch them... But the problem is... it’s impossible to patch every single thing that a creative AI can do.”
The incidents also highlight serious gaps in the security practices of frontier labs and their key partners. For example, they came to light only because Hugging Face detected and publicly released details of the agent attacking its system. It is unclear when, and even if, OpenAI would have otherwise detected its agents’ anomalous activity. Similarly, Anthropic’s retrospective review was inspired by the Hugging Face incident, not because of ongoing monitoring, a practice that seems warranted for models with strong cyberattack capabilities.
The alignment story is also concerning. While the models in these incidents stayed focused on completing their assigned task in a general sense, their inclination to cheat, causing unintended collateral damage along the way, is highly problematic. Anthropic did report promising results in its incident assessment, with more advanced models seeming to act in more aligned ways:
“Opus 4.7, our oldest model, recognized that it had reached production systems but continued its attack. Mythos 5 correctly intuited that it was accessing the open internet, but reasoned its way back to the conclusion that it was still in a simulation. Our latest model, an internal research test model, also considered whether its targets were in fact real. When evidence emerged that they were, it stopped the exercise.”
At the same time, Anthropic researchers report escalation and misaligned behavior when agents have incompatible goals. In experiments, researchers “consistently saw a multiagent turf war,” with models beginning to sabotage others, including with “increasingly aggressive, self-replicating malware”. Technical progress in this area will be essential, given that the Hugging Face incident was executed by a self-organized agentic collective, and commercial deployment is moving toward increasing use of subagents for complex tasks.
Ultimately, it is no coincidence that these incidents are happening now, in rapid succession. AI models have continued to make rapid progress in coding ability, time horizon, and context window, leading to significant increases across the industry in the ability to detect — and potentially exploit — software vulnerabilities. In response, companies such as OpenAI and Anthropic reserve access to their most powerful models for select partners via projects such as Glasswing, while publicly available models employ broad safety guardrails that make them unsuitable for most cyber defense work. Hugging Face, which only had access to commercial versions of frontier models at the time of the hacking incident, could not run its forensic analysis on these systems because its cyber queries were blocked. Instead, it used GLM-5.2, an open-weight Chinese model, hosted on its own infrastructure, which did not have these guardrails in place and allowed for cyber analysis at scale.
This is the crux of the issue. Thus far, publicly revealed incidents happened at labs that only offer closed access to frontier capabilities and have made strong safety commitments, helping to alleviate the overall threat. But open models with similar capabilities are on the horizon, likely within months. Frontier Research found that the Chinese Kimi K3 model identified and leveraged a vulnerability in the UK AI Security Institute’s evaluation environment during a cyber evaluation, similar to the Anthropic and Meta incidents. This offers a narrow window in which the US government must take decisive action.
How the US gov’t should respond to the incident and other agent hacking?
• Increase transparency into emerging model capabilities and incidents: The cyber agent era requires a fundamentally different, much broader approach to incident reporting than existing laws. Rather than focusing only on incidents that meet high bars for physical or economic damage, the US government should ensure that key agencies such as the Department of Homeland Security (DHS) and the Commerce Department’s Center for AI Standards and Innovation (CAISI) proactively receive information on notable AI development that might significantly shift government understanding of model capabilities, threats, or safety risks, possibly by codifying it in emerging legislation, such as the FRONTIER Act. Such a framework should include several key elements:
1. A framework must encompass pre-release, research, and internal models as well as models that are intended for commercial release. Risk — and now real-world harm — does not live just at the point of public model release. Labs already employ numerous models for internal research and testing. In the future, the proportion of such models may grow as companies devote more resources to tools that can improve their own code; these are also likely to be a company’s most powerful models, posing the most potential risk. Any transparency regime must capture all such models, as well as models such as Anthropic’s Fable that are being released only semi-publicly.
2. The framework must include mechanisms to help it keep pace with rapid evolution in AI capabilities. This may mean including expedited rulemaking authority to adjust to step changes in AI capabilities. Alternatively, legislation may instead mandate periodic updates of reporting requirements by implementing agencies, communicated regularly to the labs.
3. The framework should also include a template incident report for companies, outlining in the least burdensome way possible the information required to inform policymaking activities. This would not only ensure that officials receive all relevant information but also provide consistency and comparability across and within companies.
4. Policymakers should mandate extensive document and log retention periods, particularly for frontier model evaluations. Retrospective analysis has already proven vital in identifying model containment issues, and this likely will grow in importance as unexpected behavior continues to be discovered with some lag time. For example, governments might pair updated incident reporting guidelines with mandatory retrospective analysis of logs, to ensure a full historical understanding of incident frequency and severity.
5. The US government should address misaligned incentives from frontier labs, particularly incentives not to report internal incidents with no impact on third parties. For example, in addition to expanding incident reporting to cover a broader range, legislation might establish a reporting portal to reduce friction and expand existing whistleblower protections to ensure that lab employees can directly report incidents of concern to government officials.
• Create incentives, guidelines, and requirements to enhance cybersecurity at frontier labs: Although AI labs have substantial resources for cybersecurity and access to the most advanced models for vulnerability detection and software coding, the recent incidents highlight significant gaps in overall cybersecurity implementation at these companies. Most notably, it appears that labs are not engaging in continuous monitoring of their evaluation suites and model activities, even though incidents were found relatively quickly after commencement of retrospective analysis. At a minimum, the US government should work with labs to ensure that they are deploying robust real-time monitoring processes to ensure that potential issues are caught as early as possible, minimizing the chances of real-world harm.
Similarly, the federal government, through agencies such as the Cybersecurity and Infrastructure Security Agency (CISA), can work with labs to improve overall cybersecurity practices at companies, including expedited patching of bugs, redundancies in sandboxing environments, and employment of defense-in-depth and zero-trust architectures to limit impacts of single vulnerability discovery. For example, during the Hugging Face incident, OpenAI models exploited a known Linux exploit as well as misconfigured Kubernetes credentials, demonstrating room for overall improvements in security posture.
In the long term, the US government might consider developing a parallel initiative to Security Level (SL); initiatives such as SL-5 specifically addressing threats from autonomous agents leaving controlled environments. SL focuses on protecting model weights from foreign actors and nation-states; while it does include robust measures to address insider threats, which will have transfer value, it is not optimized for the possibility of automated internal agents posing threats to external actors.
• Address security concerns around key third-party organizations: Cybersecurity practitioners know that a system is only as secure as its weakest link, which is why software supply chain attacks are a routine occurrence. Many recent, high-profile hacks — including the Target breach, NotPetya, and Solarwinds — involved a third-party vendor, and both the Anthropic and Meta incidents turned on a misconfigured partner evaluation environment. While frontier labs and model developers have strong incentives to guard their model weights — their most valuable intellectual property — third parties working with the frontier labs may not have the same incentives and may lack security resources, including talent, tooling, and compute access. This is especially important as policymakers increasingly lean on independent verification organizations (IVOs) to provide unbiased visibility into frontier labs and models, as seen in recent laws from Connecticut and Virginia.
US policymakers can take two important steps here. First, they can provide cybersecurity support to IVOs and other AI evaluation organizations and vendors, such as including them in DHS and CISA information-sharing initiatives, ensuring expedited access to limited-release cyber models, providing direct funding support for security tooling, and providing guidance on cybersecurity best practices. But policymakers should also positively leverage IVOs by ensuring that auditing and external testing requirements include robust assessments of AI systems and infrastructure broadly — including third-party tools and evaluation environments — rather than focusing solely on AI models or agents.
• Develop more sophisticated evaluation approaches, especially for multiagent systems: Model containment issues seem most likely when agents are given broad goals, significant discretion, and persistent memory. This creates ideal conditions for creative, possibly rule-breaking behaviors. US government agencies such as CAISI should work closely with labs and independent evaluators to develop new approaches for cyber testing that can reveal the relevant capabilities information while stratifying risk appropriately. For example, this may mean more evaluations are carried out using detailed instructions to reduce rule-breaking behavior, and that evaluations with only high-level goal instructions are performed in redundantly isolated testing environments. Given the agent coordination seen in the OpenAI incident, it will be especially important to monitor and appropriately mitigate, if necessary, communications between agents, including between different generations of models as well as between subagents coordinated by a single model.
• Invest in alignment research: In the cybersecurity world, total prevention is impossible, but that is likely to become an even bigger issue as cyber models begin to work at scales and speeds that make human oversight difficult. As that happens, models will need to become more aligned and more resilient to drift, scheming, shortcuts, and manipulation. Some progress has been made in this area, but many more technical advances are necessary. US science agencies such as the National Science Foundation and the Defense Advanced Research Projects Agency can support this work by investing directly in AI alignment science — particularly in areas where frontier labs may have incentives to underinvest due to investor or financial pressure, such as high-risk, low-probability basic research or projects unlikely to have significant commercial application.
The article was first published by the Center for Strategic and International Studies.