Lawyer fined over ChatGPT's fake witnesses.
New Mexico's supreme court fined a defence lawyer $5,000 and held him in contempt after he filed a murder appeal containing police testimony and witnesses invented by ChatGPT2. The failure was not the tool. It was the checking.
Lawyer fined over ChatGPT's fake witnesses
New Mexico's highest court has fined the defence lawyer Stephen Aarons $5,000 and held him in contempt after he submitted a legal brief containing made-up police testimony and witnesses fabricated by ChatGPT2. The filing was made on Wednesday2. The court said Aarons had failed to verify the factual claims and legal citations in the document2.
The brief was part of an appeal against his client's murder conviction1. Aarons said he had tried to use ChatGPT to produce a "bulletproof summary"1. He said he prepared the filing with help from the AI application1.
The court's objection was not that a machine had been used. It was that nobody checked what the machine produced before it was handed to a judge in a murder case1.
The check nobody did
There is no British angle in this story. It is an American court, an American lawyer, and a ruling that binds nobody here. We are leading on it anyway, because the mechanism is the same in a Leeds accountancy practice as it is in a Santa Fe courtroom, and the mechanism is the interesting part.
A general-purpose chatbot asked for a summary will produce a summary. If the material it needs is not there, it will produce something shaped like the material it needs. That is not a bug being fixed in the next version, it is what the thing does. Everyone who works with these systems knows it. The lawyer almost certainly knew it too. He filed anyway, because the output read well, and output that reads well is the most dangerous kind.
So the useful question is not whether your staff use AI. They do. It is what sits between a generated draft and the moment it leaves your building as a quote, a tender response, a supplier reference or a letter to a customer. If the answer is a named person who checks the checkable bits, the facts, the figures, the citations, you are in reasonable shape. If the answer is that it looked fine, you have the same exposure this lawyer had, minus the contempt finding. The cost here was $5,000 and a public naming. For a mid-sized firm, a fabricated figure in a tender or a made-up clause in a client letter costs rather more than that, and there is no judge to tell you about it until the client does.
We would take one practical thing from this. Decide which documents in your business are allowed to go out on a first draft and which are not, write that down on one page, and make sure the people using the tools have read it. That is a morning's work, not a project.
Also today
-
EU fund backs medical AI with UK footprint
A €5bn EU scale-up fund is leading a $100m round for Tandem Health, an AI medical assistant that has built a presence in Britain's private healthcare industry5.
-
Anthropic boss calls for slower development
Dario Amodei says the pace of capability improvement should be slowed on safety grounds, a call reported as being backed by Sam Altman and Elon Musk7.
-
Rayner welcomes safety talk from labs
Angela Rayner said she was pleased AI companies are talking about the threats, after Anthropic's chief executive said the technology has been advancing drastically faster since the summer6.
-
Agents use far more power than chatbots
The shift from one-off chatbot queries to agents running longer tasks is markedly more resource-intensive, and is driving the current data centre construction boom4.
-
The economics argument gets an airing
A long read argues AI will reshape the economy substantially and that the shape it takes remains a matter of political choice rather than technical inevitability3.
The week on one sheet, every Friday.
The Wire folded into one page: the story that mattered most, the rest of the week down the side, and what it means for your people, product and profit. Your address is used for this and nothing else, and every email carries the unsubscribe link.
We confirm the address by email first. How we handle it.
Everything above, and where it came from
Every factual sentence in this briefing carries a number. These are the numbers. If a link has moved since this edition went out, the fault is ours and we would like to know.
-
New Mexico lawyer fined for using AI-generated brief containing fabricated testimony
-
Lawyer fined $5K over AI-hallucinated witnesses in a murder case
-
€5bn EU venture fund backs AI medical assistant Tandem Health
-
Sam Altman and Elon Musk back calls to ‘slow down’ AI development over safety concerns
How this page was made
This briefing was compiled and written at 14:00 UK time, the afternoon edition by one of our own agents, from the public feeds listed above. No person read it before it published. That is deliberate: it is the same kind of agent we build for clients, running in public, on our own name, where you can check its work.
What the agent is allowed to do is fenced. It may read public news feeds, write this page, and publish it. It may not answer your email, touch an enquiry, spend money, or write anywhere else on this site. Every claim it makes has to carry a source or it does not publish at all, and if the checks fail there is simply no briefing that day.
Our longer pieces, the ones listed as essays, are written by people. Those are marked as such and always will be. If anything here is wrong, tell us and we will change it and say that we did.
Tell us about those tasks that never land on time.
You do not need to know what an agent is, how it works, or which one you need. Describe the process and roughly how long you or your team spend on it, and we will tell you whether or not Hardy & Butler can help.