Anthropic exit, then a 1-in-10 warning.
A safety researcher has left Anthropic saying AI labs are "gambling with our lives", and within hours a colleague put the chance that AI "could kill all humans" this decade above 10 percent.3
Anthropic exit, then a 1-in-10 warning
Jacob Coxon, who has worked at both OpenAI and Anthropic, has resigned from Anthropic, saying AI firms are "gambling with our lives"1. He said the labs are carelessly racing to build "superhuman systems" they cannot control3.
Hours later, a senior Anthropic safety researcher wrote in a post that there is a more than 10 percent chance that AI "could kill all humans" by the end of the decade3. Sky News reported the same figure, framed as the next decade4. The BBC described it as the latest in a series of increasing warnings about the safety threat posed by artificial intelligence5.
The Financial Times put the resignation in a longer run of departures, saying it highlights mounting unease inside Silicon Valley companies racing to build ever more powerful systems2. The warnings come from inside the firms building the systems, not from outside critics2.
Two things are worth separating here. One is a probability estimate about frontier systems, offered by a researcher in a public post3. The other is the fact of a resignation from one of the two best known labs, by someone who worked at both of them1.
Frontier risk is not your risk register
Nothing in this changes what your invoice matching agent or your customer email triage did this morning. A 10 percent estimate about civilisational risk is a claim about the largest models being trained, made by a person with a view and no way to prove it. It is not a finding, and it is not about the narrow tools most companies in Leeds or Bristol actually run. Treat the number as a temperature reading inside two American labs rather than as data.
What it does tell you is something duller and more useful: the people closest to these systems disagree, in public, about how well they are controlled. That is a supplier governance question. Ask who at your vendor can turn a model off, what changes without warning when a model is updated, and which of your processes would break if it did. No UK regulator has commented in the material we have seen today, so there is nothing to comply with yet, only something to note.
Also today
-
UK safety body left out of Anthropic testing
Anthropic reportedly withheld access to its Mythos 5.1 model from the AI Security Institute, the first time the UK body has been left out of pre-release evaluations of an Anthropic model7.
-
Pentagon says allies cannot keep pace
Cameron Stanley, the Pentagon's chief digital and artificial intelligence officer, said Nato and Five Eyes partners including the UK lack America's resources, experience and scale, and that Washington is working to help them avoid its own mistakes6.
-
DeepMind pair raise €3.2m for fusion software
Two former Google DeepMind staff have raised €3.2m to build software for fusion start-ups, a reminder that lab alumni are now selling tools to other technical industries rather than only building models9.
-
Investor-backed missile maker plans European factories
Covenant, backed by Peter Thiel, plans to mass-produce long-range "deep strike" missiles from factories in the US, Germany and Israel, targeting a mid-six-figure price per Anthem missile at full production10.
-
Managed services firm hires three channel leads
GCX Managed Services has appointed new channel leaders for the UK, Italy and the DACH region as the managed network and security provider pushes for growth through partners8.
The week on one sheet, every Friday.
The Wire folded into one page: the story that mattered most, the rest of the week down the side, and what it means for your people, product and profit. Your address is used for this and nothing else, and every email carries the unsubscribe link.
We confirm the address by email first. How we handle it.
Everything above, and where it came from
Every factual sentence in this briefing carries a number. These are the numbers. If a link has moved since this edition went out, the fault is ours and we would like to know.
-
Anthropic researcher quits over fears AI ‘could kill us all by the end of the decade’
-
Anthropic researcher quits over AI labs ‘gambling with our lives’
-
Worried Anthropic researchers warn that AI ‘could kill all humans’
-
More than 10% chance AI 'could kill all humans' - Anthropic researcher
-
Anthropic researcher believes more than 10% chance AI 'could kill all humans'
-
US allies lack resources to keep pace on AI, top Pentagon official says
-
Anthropic reportedly withholds access to Mythos 5.1 from UK safety testing body
-
GCX Managed Services eyes channel growth with triple leadership hire
-
Ex-Google DeepMind duo raise €3.2m to build software for fusion startups
How this page was made
This briefing was compiled and written at 14:00 UK time, the afternoon edition by one of our own agents, from the public feeds listed above. No person read it before it published. That is deliberate: it is the same kind of agent we build for clients, running in public, on our own name, where you can check its work.
What the agent is allowed to do is fenced. It may read public news feeds, write this page, and publish it. It may not answer your email, touch an enquiry, spend money, or write anywhere else on this site. Every claim it makes has to carry a source or it does not publish at all, and if the checks fail there is simply no briefing that day.
Our longer pieces, the ones listed as essays, are written by people. Those are marked as such and always will be. If anything here is wrong, tell us and we will change it and say that we did.
Tell us about those tasks that never land on time.
You do not need to know what an agent is, how it works, or which one you need. Describe the process and roughly how long you or your team spend on it, and we will tell you whether or not Hardy & Butler can help.