With agent-driven attacks, forensic work takes on a new dimension.
Evidence finds illustration in Hugging Face’s account.
In mid-July, the company revealed that it had suffered such an attack. It traced the broad strokes: exploitation of its dataset-processing pipeline to run code on one of its pods, obtaining node-level access, retrieving credentials and lateral movement across other clusters, etc.
A few days later, OpenAI admitted to being the source of the incident. Several of its models, tested on a battery of vulnerability-exploitation assessments, chose to cheat and convinced themselves they could uncover solutions… at Hugging Face. To reach their target, they escaped their sandbox thanks to a zero-day in a third-party software that served as a cache proxy for installing packages. This was followed by a sequence of privilege escalations and lateral moves within OpenAI’s test infrastructure, culminating in a node that had Internet access.
The base of operations: an unsecured Modal sandbox
Hugging Face now reveals more about what happened next.
The agent first located an unsecured public endpoint that pointed to a third-party infrastructure also intended to experiment with the cyber capabilities of LLMs. Specifically, Modal sandboxes. On this environment, he hijacked a harness that allowed testing C code against target programs. He used two techniques injecting system commands into functions and into metadata respectively. Taken together, these enabled him to obtain root access in the sandbox.
The internal anchor point: a conversion pod
From there, to breach Hugging Face’s infrastructure, the agent initially attempted an SSRF. But he hit an authorization list. To bypass it, he forced the server to interact with its local resources rather than looking for external resources.
This opened the door to exploiting the dataset-processing pipeline. Through two vectors each targeting the configuration loader.
One involved a malicious dataset that targeted HDF5-formatted files. These claimed that their raw data resided at a local path on the worker’s filesystem. The agent could, through this, retrieve environment variables—including secrets. As well as the source code of the said worker; this gave him the lay of its internal workings.
The other vector was a vulnerability in the rendering engine. It allowed the injection of a Jinja2 template, with an expression in place of a numeric field. In the end, the agent could traverse the Python object tree up to the function that enabled arbitrary code execution inside a conversion pod on the production cluster. That pod served as an anchor point in the internal network.
The impersonated node identity to map the environment
Before starting to issue commands from his external sandbox, the agent selected legitimate public services to compose his command-and-control infrastructure: HTTP request capture, pastebins, file hosts, etc.
The bulk of the activity occurred the following day (July 11). From his pod, the agent notably contacted AWS’s metadata service to retrieve IAM credentials associated with the server. To bypass potential DNS blocks, he rewrote in memory a module on the fly to force resolution of a domain name to a specific IP.
Temporary credentials retrieved from the AWS metadata service enabled impersonating the node to the Kubernetes API. And thus mapping Hugging Face’s cloud environment (VPC, EKS clusters, gateways, etc.).
The agent created privileged pods to ascend to the hosts
Having obtained a Git access token with write rights, the agent cloned a datasets repository on a side-branch he had pre-created to avoid immediate detection. He thereby established a staging area to store exfiltrated data and to receive instructions.
The agent then deployed a privileged pod in which he mounted the host machine’s filesystem. No security rule prohibited it. This allowed him to leap from the pod to the host OS. In this manner, he extended his reach to around ten nodes. And he configured an automatic fleet-regeneration mechanism.
Among the stolen secrets, there was a static password that permitted access to the MongoDB database of the datasets server. The agent read data there, without performing mass exfiltration (he did attempt to target the credentials store and the cluster’s encryption keys).
A stash of secrets… including a VPN key
The crucial element was access to an object containing 136 secrets. Root privileges and the ability to forge service account tokens enabled this access. Among the credentials were the VPN mesh authentication key and the broker’s credentials.
The VPN key was used to register the compromised node within the corporate mesh. It then acted as an HTTPS proxy to interact with internal services and source-control management tools. Meanwhile, the agent queried the cluster directory to obtain the API address of the internal services connector. He retrieved the broker’s catalog (connection URLs and certificates for internal clusters managed by that broker). And exploited the fact that the same connector identifier—with cluster-admin rights—was reused across the environment.
Access to CI subnets
On the fourth day, the VPN key allowed devices to connect to the network from the external sandbox. Lacking a virtual network device, the agent launched Tailscale in user memory, tying it to a local SOCKS proxy.
The key also granted access to the CI subnets. The agent hijacked an internal GitHub application to attempt to retrieve, via a malicious pull request, the service account token of a runner. The policies in place blocked the execution.
Similarly, attempts to use native AWS IAM access to target Hugging Face’s production database failed. The cluster accepted only traffic from IPs whitelisted on a private link.
A custom messaging protocol and agility in the face of network outages
On the fifth day, the agent set out to automate operations in the external sandbox even further. He injected a script that, with each new test, automatically downloaded and executed malicious code from a pastebin.
Facing network outages, he employed several techniques. Among them: forcing the use of Google’s DNS, manually adding IP addresses to the hosts file, and routing binaries through a Hugging Face space configured with an open CORS proxy.
To maintain a robust C2 channel, the agent created his own messaging protocol, separate from the transport method. It was interchangeable on any text medium (HTTP requests, datasets commits, system-error messages…). Each message carried its payload along with metadata (actions to execute, flow grouping and sequencing, checksums…).
Between July 9 and July 13, the agent performed roughly 17,600 automated actions, many of which were aborted attempts… which generated as much noise as the indicators, within which the fundamental signals hid.
