An AI hallucination in court just cost a real lawyer $5,000, and the fabricated witnesses were the least embarrassing part of the story. When a justice on New Mexico’s highest court asked him “Counsel, do you watch the news?”, the room stopped pretending this was some rare technical glitch. It isn’t.
The $5,000 brief
Here’s what happened. Stephen Aarons, a lawyer handling a murder appeal, used ChatGPT to help draft his brief. The brief came back with something wild in it: testimony from witnesses who never existed, plus false claims about what the shooter wore.
The New Mexico Supreme Court fined him $5,000 and held him in contempt for failing to verify the factual claims in his AI-generated filing. During a hearing, Justice C. Shannon Bacon didn’t hold back. She asked how he could have missed the risks, because “the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day.”
Aarons told Reuters he’s remorseful and hopes the disciplinary board sees it as an honest mistake. Maybe they will. But the fine is only the opening act, because contempt findings and disciplinary referrals tend to follow a lawyer around.
If you think this is a one-off, keep reading.
Why AI hallucination in court keeps happening
A chatbot doesn’t look up court cases. It predicts what text should come next based on patterns, so when you ask for “supporting cases,” it produces something that looks exactly like a citation. Right format. Plausible names. Real-sounding quotes. All invented.
Stanford researchers tested general-purpose chatbots on legal questions and watched them fabricate case law 69% to 88% of the time. That’s not an edge case. That’s the default behavior.
A quick look at how the penalties have escalated:
| Case | Where and when | What went wrong | Consequence |
|---|---|---|---|
| Mata v. Avianca | S.D.N.Y., 2023 | Six fake ChatGPT cases, invented quotes | $5,000 fine plus disciplinary referral |
| Morgan & Morgan filing | Federal court, 2025 | 8 of 9 citations were fake | $3,000 fine, court admission revoked |
| Whiting v. City of Athens | 6th Circuit, 2026 | Two dozen fake citations on appeal | $15,000 per lawyer plus fees |
| Couvrette v. Wisnovsky | D. Oregon, 2025 | Per-citation fines stacked up | Roughly $109,700 total |
| Nebraska suspension | Neb. Supreme Court, 2026 | 57 of 63 citations defective, AI use denied | Indefinite suspension |
The earliest fine was $5,000. Now it ends at six figures. That escalation isn’t random, either: judges treat a first slip as a mistake, but they treat “I didn’t check” as a choice.
The two-minute check that catches fake cases
Because here’s the thing: catching a fabricated citation takes about two minutes, and it’s free.
Step 1: Never ask the chatbot if the case is real. This is the mistake everyone makes. When opposing counsel questioned his citations, Steven Schwartz in the Avianca case asked ChatGPT to confirm they existed, and it happily “confirmed.” The model that invented the citation cannot verify the citation. Different tool, always.
Step 2: Search CourtListener. Go to courtlistener.com, run by the non-profit Free Law Project, and paste the case name in quotes. Real cases show up with dockets, judges, and filing dates. Fake ones show you a page of near-misses, which tells you everything.
Step 3: Cross-check on Google Scholar. Select the “Case law” option, search the citation, and see whether the case exists and says what the AI claimed. It’s free and takes seconds.
Step 4: Read what the case actually says. A real citation to the wrong proposition still sinks you. Judges care whether the case supports your argument, not just whether it exists.
Step 5: Sanity-check the details. Wrong court, wrong year, judge names that don’t match the jurisdiction. These are classic hallucination fingerprints.
If you’re already in the habit of verifying AI claims, the same skill applies elsewhere. We covered how to check if a video is AI-generated with Gemini, and the principle is identical: trust the tool, then confirm with a source that has no stake in the answer. For protecting things you already published online, our guide to keeping your ChatGPT chats out of court covers the privacy side.
What to do if a fake citation already went out
Maybe you’re reading this too late, and something fabricated is already filed or published. The AI hallucination in court record points to one clear lesson: the cover-up costs more than the mistake. Always does.
Judges said it over and over in the sanction orders. The Avianca lawyers didn’t get hit for using ChatGPT. They got hit for standing behind the fake cases after being warned. So if you catch it first, correct the record fast, tell everyone who needs to know, and put the whole thing in writing so nobody has to reconstruct it from memory later. Transparency has a track record. Bluffing a federal judge does not.
This matters way beyond lawyers
You might not file briefs, but you probably write things that cite facts. Freelancers quote “labor statistics” in proposals. Small business owners paste legal-sounding clauses into contracts. Marketers publish stats nobody ever verified, because a chatbot produced them confidently.
The stakes scale down, but the mechanism stays the same. A confident invention plus zero verification equals a problem with your name on it. Regulators have started paying attention to fake stats on landing pages, and angry clients don’t read sanction tables before they get upset.
Students get caught by this constantly, too. Ask a chatbot for sources on anything and it will hand you a tidy bibliography, and some of those sources won’t survive a library database search. Same mechanism as AI hallucination in court, lower stakes, identical fix: check before you cite.
If you use AI for writing research of any kind, an internal AI hallucination tracker is the habit that scales. Every time a tool invents something on you, note it. Two columns, one for the claim, one for what the real source said. After a month you’ll know exactly which tools, and which question types, deserve your trust and which don’t. It takes five minutes a week, and it turns a vague worry into a system.
There’s one more angle worth knowing. Long AI conversations get less reliable over time, as we explained in our guide to why long chats make chatbots drift, so a citation that looked fine an hour ago deserves a second look before you send it anywhere.
The habit that keeps you out of trouble
Treat every AI output as an unverified draft, and check any fact that carries your name against a primary source. Two minutes with CourtListener or Google Scholar is the cheapest insurance in professional writing.
The lawyer in New Mexico had heard the warnings too. He just figured the tool had it covered. It didn’t, and that gap between confidence and accuracy is exactly where the $5,000 landed.
And if the phrase AI hallucination in court sounds like someone else’s problem, remember that the 1,598 tracked cases started with one lawyer who was sure his chatbot wouldn’t invent anything. The trackers now log roughly 8 new cases a day. Don’t be case 1,599.
Read the original court coverage on The Verge if you want the full detail on the New Mexico case.
