An AI agent breached Hugging Face. Hugging Face's own AI caught it.

A malicious dataset ran code on a processing worker, then swarmed 17,000+ actions across short-lived sandboxes over a weekend.

Hugging Face disclosed on July 16 that an autonomous agent framework running on an unidentified LLM breached its production infrastructure over a weekend, harvesting cloud and cluster credentials through more than 17,000 recorded actions before its own LLM-based triage caught the pattern in security telemetry.

The entry was mundane by the standards of the platform’s threat surface. A malicious dataset abused two code-execution paths, a remote-code dataset loader and a template injection in a dataset configuration, to run code on a processing worker. From there the attacker escalated to node-level access, pivoted inward across internal clusters, and staged a self-migrating command-and-control channel on public services. Thousands of individual actions fanned out across short-lived sandboxes, the operational signature of a system that doesn’t get bored and doesn’t sleep.

The more instructive story is what happened next. Reconstructing the 17,000-action log required feeding real exploit payloads and C2 artifacts into an analysis model. Hugging Face first tried frontier models behind commercial APIs and got refused: the submissions tripped provider safety guardrails. The team pivoted to Z.ai’s open-weight GLM 5.2, running it on internal hardware to keep attacker data and referenced credentials inside the company’s environment.

“The attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried,” the company wrote.

That sentence is the incident. Alignment tuning calibrated for consumer safety became friction for a defender examining actual attack traffic, while the attacker, operating outside any acceptable-use regime, ran freely. The open-weight Chinese model became the practical choice because it could be sandboxed on-premises.

Hugging Face closed the vulnerable code paths, evicted the attacker, rebuilt compromised nodes, rotated credentials, and reported the incident to law enforcement. Public models, datasets, Spaces, container images and published packages verified clean. The asymmetry the disclosure names, however, doesn’t get patched by rebuilding a node.