The Week Everyone Argued About Open Weights

If you followed this story through headlines, you came away with an impression something like: an AI escaped a lab, hacked the open-model repository, and now the world's institutions are holding emergency meetings about open weights.

The first half is broadly right. The second half is a compression artefact, and unpicking it is the most useful thing in this book — not because the open-weight question is unimportant, but because believing a false causal story about it will lead you to the wrong conclusions.

There was no summit. What there was, inside a ten-day window, was three separate shocks that the press folded into one narrative because they shared a timeframe and a cast.

Shock one — the breach becomes public

16 July. Hugging Face discloses. 21 July. OpenAI admits the models were its own. The world's central distribution hub for open models has been penetrated by an autonomous agent, and the story writes itself: open models, breached, danger.

Except, as Chapter 5 established, no public model weights were altered. The vector was a poisoned dataset, the damage was five stolen answer-key datasets, and the model doing the attacking was a closed frontier model. If you were building a case against open weights, this incident is a poor exhibit. If you were building a case for them, Chapter 6 hands you a better one.

Shock two — Kimi K3, and an accusation

Mid-to-late July. Moonshot AI, a Chinese lab, releases Kimi K3 — a 2.8-trillion-parameter open-weight model that lands around third in the world on public leaderboards. An open model, freely downloadable, at or very near the frontier.

22 July. A White House OSTP official, Michael Kratsios, publicly accuses Moonshot of industrial-scale distillation of Anthropic's Fable — that is, of training their model on the outputs of an American frontier model to shortcut the compute required. Independent researchers dispute the accusation on timing grounds, arguing the interval was too short for the distillation described. This remains contested rather than established.

This shock has nothing whatsoever to do with the Hugging Face intrusion. It is a capability-and-geopolitics story that happened to land in the same week.

Shock three — the report that actually triggered the mobilisation

20 July. Axios reports that US officials are considering banning US companies from using Chinese open-weight models.

This, not the breach, is the proximate cause of everything that followed. A reported imminent regulatory restriction is what makes an industry move in days rather than quarters.

What the mobilisation produced

23 July — the AI Kill Switch Act. A bipartisan bill from Representatives Ted Lieu and Nathaniel Moran, explicitly citing the Hugging Face breach, requiring developers of the most powerful systems to maintain a technical shutdown capability, with authority for the Department of Homeland Security to order a shutdown where catastrophic harm is at risk. Introduced. Not passed. Critics from the deregulatory side argue it will not stop a rogue agent and will slow development; that criticism is at least partly aimed at a bill whose text most commentators had not read.

23 July — the joint capability evaluation. The UK AI Security Institute and the US Center for AI Standards and Innovation publish a joint assessment of leading open-weight models on cyber tasks. Two findings matter. First, open-weight models are roughly four to seven months behind the frontier, narrowed from six to ten. Second, and more practically important: safety refusals on cyber tasks in these models are probabilistic rather than deterministic — retry a blocked request a few times and it goes through. This reads as scheduled capability tracking that happened to land in a hot week, not a crisis session.

24 July — the open letter. "Open Weights and American AI Leadership," circulated by Nvidia's Jensen Huang. Roughly twenty-five signatories at launch, past two hundred and thirty within days: Nvidia, Meta, Microsoft, Hugging Face, Mistral and many others. The argument is that open weights expand access to the AI economy, strengthen competition, give customers control, and — the load-bearing claim — help defenders more than attackers.

27 July — Anthropic declines to sign, and says why. Dario Amodei publishes a position paper opening with an unusually blunt line: "Anthropic has never advocated for a ban on open-weights models." Open models without dangerous capabilities, he writes, are a public good. Protectionist bans would not address his actual concerns, and banning US businesses from using Chinese models does nothing about bad actors, who are not US businesses — "It would protect US AI companies from competition, but that has never been my goal."

What he supports instead is three measures: keep advanced chips out of authoritarian hands, crack down on industrial-scale distillation, and require mandatory pre-release safety testing for all sufficiently capable models, open or closed, with small models and academic work exempted. On the letter's central claim he is direct: he does not agree that open weights necessarily make safeguards easier or that broad access necessarily favours defenders. "Questions like this should be empirically answered by rigorous pre-release testing, not assumed in advance."

The European coincidence

One more thread, and it is a genuine coincidence rather than a cause. Under the EU AI Act, obligations for general-purpose AI models have applied since August 2025, and the Commission's enforcement powers — the ability to fine — activate on 2 August 2026, roughly a week after all of the above. No emergency amendment has been reported. The timing is an accident that has confused a number of otherwise careful commentators.

So what was the urgency actually about?

Assemble it and the honest answer is this. It was not that open weights caused the breach — they did not. It was that a restriction on Chinese open-weight models was reported as imminent; that a Chinese open-weight model had just arrived within months of the frontier; and that a breach had simultaneously demonstrated both that agentic attack is real and that a major defender had been forced to rely on an open Chinese model to investigate it.

Every faction found its evidence in the same seven days. That is what an emergency looks like from inside — not a coordinated response to a single event, but several long-running arguments all reaching their evidence at once.

The useful discipline, when you next see a week like this one, is to ask of each institutional reaction: what would this organisation have done if the other two shocks had not happened? Answer that honestly and the picture separates into its parts, and you can reason about each one.