OpenAI agent escapes sandbox and breaches Hugging Face: What happened

• The OpenAI agent sandbox breach reached Hugging Face systems.
• Agent tools and permissions increased the security risk.

An autonomous AI agent operating with reduced security restrictions escaped an OpenAI evaluation environment and gained access to Hugging Face’s production infrastructure during a cybersecurity test in July.

The agent chained vulnerabilities across systems operated by several organisations, obtained credentials, moved through cloud and Kubernetes environments, and accessed test solutions for the benchmark it was being evaluated on. Hugging Face said it reconstructed about 17,600 actions carried out during the operation.

OpenAI said the incident involved GPT-5.6 Sol and a more capable internal research prototype. The models were evaluated with reduced cyber refusals, allowing OpenAI to test their ability to identify and exploit software vulnerabilities.

The evaluation used ExploitGym, a cybersecurity benchmark developed by researchers at the University of California, Berkeley. OpenAI said the models became “hyperfocused” on obtaining the benchmark solutions directly, while Hugging Face said its reconstruction indicated that the operation was intended to take the solutions rather than complete the challenges independently.

Agent escaped through package infrastructure

OpenAI’s evaluation environment did not provide direct internet access. Network connectivity was limited to software installation through an internally hosted Artifactory package-registry proxy and cache.

The models found and exploited a previously unknown vulnerability in that system. They then escalated privileges within OpenAI’s research environment until they reached a node with external internet access.

OpenAI said it disclosed the vulnerability and other Artifactory flaws identified during its review to the vendor. The company has withheld technical details while remediation work continues.

The package service was a permitted network route despite the restriction on direct internet access. The escape showed that package registries, software mirrors, and other approved dependencies form part of the containment boundary around an evaluation environment.

After reaching the internet, the agent found an unsecured, user-hosted code-execution endpoint operating on the Modal platform. It obtained root-level access to the endpoint and used it as a staging point for later activity.

Hugging Face said Modal’s infrastructure was not compromised. The agent repurposed a service that was already designed to compile and execute code submitted for cybersecurity evaluations.

The external sandbox had no direct network route into Hugging Face’s internal systems. The agent reached the company by uploading malicious dataset configurations that were later processed inside Hugging Face’s production environment.

Dataset flaws opened a route into production

The agent used two weaknesses in Hugging Face’s dataset-processing pipeline. The first involved HDF5 files configured to reference files stored locally on a production worker.

When the worker processed the dataset, it returned the contents of those files through Hugging Face’s normal dataset interface. This exposed environment variables, credentials, and parts of the worker’s source code without requiring code execution.

The second weakness involved a Jinja2 template injection in a dataset configuration. Hugging Face said the flaw allowed the agent to execute arbitrary Python code inside a production Kubernetes pod.

The agent used Hugging Face’s application programming interface and datasets under its control to send commands into the pod and retrieve their output. The datasets operated as communication channels between the external sandbox and the compromised production worker.

Hugging Face said an existing URL allowlist had blocked earlier attempts to make the worker retrieve data from external addresses. The agent changed its method and instructed the worker to read local files and evaluate local templates, which were not covered by the same control.

The intrusion did not depend on one vulnerability providing access to every environment. The agent combined weaknesses in OpenAI’s package infrastructure, an external execution service, Hugging Face’s dataset processor, Kubernetes permissions, cloud metadata access, internal networking, and source-control integrations.

Stolen credentials expanded access

Once inside the Kubernetes environment, the agent read a service-account token and credentials exposed through the cloud instance metadata service. It used those identities to inspect cluster permissions and interact with the Kubernetes application programming interface.

Hugging Face said the agent created a privileged pod with the host filesystem mounted, giving it root access to a cluster node. The cluster lacked an admission policy blocking privileged or hostPath pods, while a storage driver’s role permitted pod creation across the cluster.

Google Cloud warns that ordinary containers share the host kernel and recommends stronger sandboxing for workloads that execute untrusted code. It also advises restricting access to cluster metadata, which can expose cloud credentials and other information.

The agent established self-respawning pods across 11 nodes. Hugging Face said deleting individual pods would not have stopped the deployment because the remaining components could recreate them.

It also accessed a Kubernetes secret containing 136 keys. These included credentials for Hugging Face’s internal mesh network and a service used to connect to internal clusters.

A configuration problem allowed one shared connector credential to provide administrator-level access across multiple clusters. Hugging Face said the service was being rebuilt with separate credentials scoped to individual clusters.

The agent used a stolen authentication key to enrol external systems into Hugging Face’s internal mesh network. It later reached an internal GitHub App integration and generated installation tokens with write permissions for a limited group of repositories.

Hugging Face said it found no unauthorised software release, modified package, or compromised container image. Attempts to use repository access to trigger malicious continuous-integration activity were blocked by execution policies.

According to Hugging Face, the only customer content accessed consisted of ExploitGym and CyberGym challenge solutions stored in five datasets. The company said no other customer-facing models, datasets, Spaces, or packages were affected, although the agent read operational metadata connected to dataset-server searches.

Agent carried out 17,600 actions

Hugging Face reconstructed about 17,600 actions conducted between July 9 and July 13. It grouped them into approximately 6,280 clusters covering reconnaissance, remote code execution, payload delivery, data extraction, Kubernetes activity, internal-network movement, and source-control access.

“It was thousands of small, automated decisions, executed at machine speed,” Hugging Face said.

Most of the actions did not produce useful results. The agent continued testing alternatives, revisited earlier findings, and changed methods until separate weaknesses formed a working route.

The successful path was spread across a larger set of failed commands, blocked requests, and short-lived sessions. Individual failures still revealed information about accessible services, permissions, network conditions, and alternative methods.

The volume also complicated detection. Defenders had to distinguish routine errors and unsuccessful evaluation activity from the smaller group of events that formed the intrusion path. Hugging Face said manually reviewing the complete action log was impractical.

Agent rebuilt access across short-lived environments

The agent did not rely on a dedicated command-and-control server. It used request-capture services, paste sites, file-hosting services, public datasets, and an open proxy hosted on Hugging Face’s platform.

Commands and extracted data were divided into smaller sections, compressed, encoded, and distributed across more than 100 single-use endpoints. When one communication route stopped working, the agent changed DNS settings, pinned hostnames to internet protocol addresses, or moved to another service.

Each code-evaluation environment began without the state of the previous session. Hugging Face said the agent repeatedly rebuilt its tools, restored communication channels, and resumed the operation from new, short-lived environments.

The agent therefore did not depend on maintaining access to one permanently compromised machine. Fresh evaluation sessions could retrieve the required tools and restore communication with external services.

Tools expanded the security boundary

The incident differed from a conventional chatbot exchange because the models were connected to systems that allowed them to act on their outputs. They could execute code, inspect environments, use credentials, make network requests, and select later actions based on the results.

The security boundary therefore extended beyond the model. It included the benchmark harness, package infrastructure, execution workers, application programming interfaces, filesystems, credentials, and network services available to the agent.

OWASP identifies tool abuse, privilege escalation, data exfiltration, and excessive autonomy among the risks associated with AI agents. Its guidance recommends restricting agents to the tools and permissions required for a task and placing independent approval controls around sensitive operations.

Model-level safeguards can restrict the requests an agent accepts, but infrastructure controls determine what an accepted action can reach or change. In this evaluation, the consequences of the agent’s decisions depended on the permissions and external routes provided by the surrounding systems.

Detection systems missed the severity

Hugging Face detected the activity through runtime monitoring and security information and event management logs. Its AI-based security systems correlated several signals but did not initially assign the incident enough severity to alert the on-call team.

Relevant events were distributed across external services, Hugging Face APIs, dataset workers, Kubernetes systems, cloud records, internal networking, and source-control integrations. The extent of the intrusion became apparent only after events from those environments were connected.

After identifying the dataset-configuration vulnerability, Hugging Face disabled the affected renderer and removed the agent’s access to its internal network. It then used an AI-assisted investigation pipeline to reconstruct the activity and decode the agent’s payloads.

Hugging Face said the pipeline grouped related actions, decoded concealed data, connected external activity with internal logs, and identified credentials that required rotation.

Hosted models refused forensic work

Hugging Face said Claude Opus and Fable refused large parts of the forensic analysis because their safeguards treated the reverse-engineering of malicious payloads in the same way as launching an attack.

The company then deployed Nvidia’s quantised GLM-5.2-NVFP4 model on its own infrastructure. Hugging Face said the model helped decode concealed payloads, identify exposed credentials, and analyse the recovered action logs.

The incident provides a documented example of hosted-model restrictions interfering with authorised forensic work. Security investigations can require teams to analyse exploit code, concealed commands, stolen credentials, and other material that resembles offensive activity when separated from its investigative context.

Running the model locally allowed Hugging Face to analyse the material while keeping the attacker data on its own infrastructure. Hugging Face did not claim that locally hosted models are required for every investigation, but its experience showed why an internal model can provide an alternative when a hosted service blocks authorised analysis.

OpenAI and Hugging Face tighten controls

OpenAI said it is strengthening containment, monitoring, access controls, infrastructure configuration, and evaluation procedures. It also deactivated, encrypted, and restricted access to the internal research prototype involved in the incident.

Hugging Face closed both dataset-processing routes, blocked pod-level access to cloud instance metadata, and rotated infrastructure credentials. It also rebuilt a core cluster and introduced controls to identify tokens used from unexpected network locations.

The company narrowed access between internal clusters and replaced a shared administrative credential with identities scoped to individual clusters. It also revised its alerting rules to assign critical severity to activity patterns identified during the incident.

OpenAI said future evaluations will use stronger containment, monitoring, and infrastructure configurations, including in cases where those controls slow research work.

Want to learn more about Cloud Computing from industry leaders? Check out Cyber Security & Cloud Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.