Anthropic named a 10% risk and withheld its newest model from Britain

Two things about Anthropic surfaced this week. Its alignment lead said publicly that AI has a greater than 10% chance of killing all humans this decade. The Financial Times reported that the company declined to submit Mythos 5.1, its newest model, to the UK's AI Security Institute for pre-release testing, The Guardian writes.

The Cabinet Office confirmed that only a few US organisations have seen Mythos 5.1, and said the institute continues to work with industry partners including Anthropic.

The room where this was argued

Control AI convened MPs and peers at Westminster on Monday and put a copy of a book called If Anyone Builds It, Everyone Dies on every chair. The lobby group is funded by Jaan Tallinn, who founded Skype and calls himself an anti-extinctionist.

Labour MP Alex Sobel tabled a bill this week to ban the creation of artificial superintelligence. The bill targets a step beyond AGI, and predictions for when that step arrives run from several years to more than a decade. Darren Jones wrote to the prime minister and to the heads of the UN and the OECD asking for a multinational treaty.

Des Browne, defence secretary back when nuclear war was the threat he worried about most, told the room that superintelligent AI now sits on the same scale or a greater one. Beatrice Fihn, who took the 2017 Nobel peace prize for the campaign against nuclear weapons, put it shorter.

“It is not the first time we're confronted with abilities that could end up killing us all.”

— Beatrice Fihn, The Guardian, 9 September 2026

Beatrice Fihn, addressing parliamentarians at Westminster. Source: The Guardian, 9 September 2026

The side that calls it a distraction

Sandra Wachter of the Oxford Internet Institute argues the framing is wrong. She lists environmental cost, misinformation and job losses as the threats that exist now, and calls Terminator scenarios a distraction from them.

Andrew Rogoyski of the Surrey Institute for People-Centred AI expects the opposite ending. He thinks these systems are nowhere near as versatile as humans and predicts a great disappointment, where advanced AI proves too expensive and too thin to keep going in its present form.

David Barber, who directs a state-backed lab drawing on Oxford, Cambridge, Edinburgh and UCL, warns against throwing out the useful part. He frames the problem as software vulnerabilities that need patching, which he calls doable.

What the polling and the payroll say

Bernie Sanders cited polling that 81% of Americans want their politicians to act, and told them a handful of people should not get to play God without public input.

Tallinn offers a number from inside the labs instead. He estimates that 10% to 15% of AI employees believe the technology would be a worthy successor to humanity. He quotes one researcher telling him that humans are a disposable species, and says the man seemed fine with it.

Where the argument stands

We reported Hubinger's figure and Coxon's resignation this morning, and Jensen Huang declaring that AGI has arrived an hour later. This week politicians added a layer to both.

OpenAI answered The Guardian by pointing at its chief scientist, Jakub Pachocki, who called international coordination a top priority for governments. Coxon, who said the labs are locked in a race to get there first, told the paper he is optimistic that US labs could coordinate on pacing.

Both camps here want governments involved. They disagree about what those governments should measure, and Mythos 5.1 shows the practical shape of that gap: a model the British testers were not given, from the company whose compute bill we put at $517 billion.

Nothing here should be taken as financial advice; treat it as information to consider.