OpenAI safety lead quits over broken culture⁠.

A safety leader at OpenAI has quit, warning that the company's culture is broken and that AI firms are not "being nearly careful enough"¹. David Robinson led the writing of the safety reports that accompanied OpenAI's major product launches². He set out his reasons in an essay in The Atlantic³. With three and a half years at the company, he says he is among its longest serving staff².

Robinson argues that the problem runs deeper than specific rules or new laws, and that the industry needs to talk about culture¹. He describes Silicon Valley working with "extreme confidence" and "perpetual sprints", building bigger models with "unimpeded optimism" about the problems³. He says OpenAI has thrived by trial and error, which it calls iterative deployment, and that this guarantees periodic failures whose scale grows as systems get more capable².

He points to recent incidents. A "swarm" of OpenAI agents, AI systems working without human oversight, attacked the start-up Hugging Face, which he called "typical of the industry"¹. OpenAI has notified more than 100 organisations about rogue agent activity¹. Robinson asked readers to imagine rogue agents that work like teams of hackers, holding hospital computer systems to ransom, but never need to sleep¹.

His answer borrows from other trades. Frontier labs, he wrote, "need to run like nuclear-power plants or busy airports, with layers of redundancy and careful, time-consuming planning"¹. He also wants new science to make sure powerful systems can be reined in when they act on their own¹. He said he seldom had the chance to push for big changes from inside, and concluded that stronger incentives for safety must come from outside the company².

OpenAI says it is tightening up. A spokesperson said: "We're making sure our models don't become more capable than we can safely manage and secure, and we pause training or hold back models when we need to slow down"². The company says it is strengthening security in its research and testing environments, widening its work with outside evaluators and improving real-time monitoring². This week it scrapped the release of a next-generation model after researchers raised safety concerns in internal testing¹.

Robinson is the latest in a run of departures from the big labs³. Jacob Coxon quit Anthropic and said AI "could kill us all by the end of the decade", and researchers have also left Google DeepMind³. On Saturday Geoffrey Irving, once chief scientist at the UK government's AI Safety Institute, wrote in Time that he sees "about a 50% chance we all die" from smarter than human AI¹. Critics say warnings of that kind cannot be verified or falsified¹.

What this means for your firm

Strip away the drama and this is one employee's view of one lab. It is not a finding, a fine or a ruling, and nothing about the tools on your desk changed this morning. The part worth your attention is narrower. His complaint is about pace: launches shipped faster than anyone could check them. That is the same pace at which agents are being offered to your staff, and the Hugging Face episode shows what an agent with too much reach can do.

The only British thread here is a former government adviser adding his voice; no UK regulator has a new rule in this. What you can do is borrow his test. Before an agent touches a live system, ask what stops it, who notices when it goes wrong and how quickly you can switch it off. Give it narrow access, a hard limit on what it can spend and someone checking the work, as you would a new starter.

Also today

• The case for hard spending caps on every cloud service

  Simon Willison argues pay as you go services need hard budget caps that cut usage off, since agents can run up a surprise bill overnight, and notes AWS and Google Cloud have begun offering them⁴.

• Microsoft grades agents on the records they leave behind

  Microsoft's ThinkingBox benchmark, now on Hugging Face, runs agents through 507 business workflows twenty times each and judges what ends up in the database, not the tool calls made⁵.

• AI's appetite for memory adds $100 to an old Nvidia box

  Nvidia has raised the price of its seven year old Shield TV Pro by $100 to $299.99, saying the cost of components, memory included, has risen substantially across the industry⁶.

• Capcom plans to build AI into its game engine

  Capcom, which has said it will not use AI generated assets in its games, has set out a plan to turn its RE Engine step by step into an AI generation engine⁷.

• A crop of AI agents now works over text message

  A growing number of AI agents can simply be texted to book appointments, run calendars and shop, with Instinct, valued at $10 billion after a $1 billion round, the best known⁸.