AI is now embedded in legal research, drafting, and review. It's genuinely useful — and it will, with total confidence, hand you case law that doesn't exist and quotations that were never written. This guide covers what legal AI hallucinations are, why even "grounded" tools produce them, what the evidence actually says, and — the part most coverage skips — a concrete workflow to protect yourself.
What legal AI hallucinations are
A hallucination is a confident, plausible-sounding output that is false, with no internal signal that anything is wrong. In legal work they take a few recognizable forms:
- Fabricated cases: entirely invented decisions with realistic names and reporter citations that don't correspond to any real opinion
- Wrong or misstated statutes: real statutory sections cited for propositions they don't support, or provisions summarized inaccurately
- Fabricated quotes inside real citations: the most dangerous variety — a genuine, correctly-cited case with a quotation the court never actually wrote
That third category is the one to fear most. A fake case is caught the moment you look it up. A fabricated quote in a real case sails through a routine cite-check — the citation is valid — and the error hides in the language you didn't verify against the source.
Why they happen — even with "grounded" tools
A language model doesn't look facts up in a database; it generates the most plausible continuation of text based on patterns in its training. Sometimes the most plausible-sounding output is fiction.
"Grounded" or retrieval-augmented (RAG) legal tools try to fix this by retrieving real source documents and instructing the model to answer from them. This genuinely helps — it sharply reduces wholly-invented cases. But grounding is not a guarantee:
Where grounding still leaks
- • The model can misquote a document it correctly retrieved
- • It can attach a real quote to a proposition the source doesn't support
- • Retrieval can surface the wrong passage, and the model quotes it confidently anyway
- • It can blend language from multiple sources into a single fabricated "quotation"
In other words, grounding changes the distribution of errors — fewer fake cases, but the survivors skew toward the hard-to-catch quotes-in-real-citations kind.
The evidence
This isn't hypothetical. Two anchors and one tracker are worth knowing:
Mata v. Avianca (2023)
The landmark cautionary tale. Attorneys filed a federal brief citing multiple judicial decisions that did not exist — ChatGPT had fabricated them, complete with fake quotations. The court sanctioned the lawyers. It established, publicly and expensively, that unverified generative-AI output has no place in a filing.
Stanford RegLab / HAI (2024)
A 2024 study from the Stanford RegLab and the Institute for Human-Centered AI evaluated leading purpose-built, grounded legal-AI research tools and found they still hallucinated on a substantial share of queries — reported in roughly the 17–33% range depending on the tool and task.
Before publishing, re-confirm the exact percentages, the specific tools evaluated, and the precise attribution against the primary paper. Cite it honestly and attributably — never inflate or round it into a marketing number.
The Charlotin database (tracker)
Researcher Damien Charlotin maintains a widely-cited public database tracking court cases that involved AI-hallucinated citations. It's the best running record of how often these errors reach the courts and how far they travel before someone catches them. We cite it as a resource — it's the reference to consult, not something to reproduce.
How to protect yourself: a verification workflow
Most coverage stops at "be careful." Here's the underserved part — a concrete workflow you can actually run on every AI-assisted research task:
Independent cite-checks
Confirm every cited case and statute exists in an authoritative source (the reporter, the code) — separately from the tool that produced it. Don't let the AI verify its own citations.
Verbatim quote verification
Pull the primary source and read every quotation in context. Match the quoted words character-for-character against the opinion. A real cite with a fake quote is the error that hides.
Proposition check
Confirm the source actually supports the point it's cited for. A genuine quote can be attached to a claim it doesn't back.
Cross-model comparison
Ask the same question of multiple independent AI models. When one produces a case or quote the others can't reproduce — or that the others rate as unsupported — treat that disagreement as a flag to investigate.
Never file unverified output
No AI result enters a brief, memo, or filing until a human has verified the primary sources. Grounded ≠ verified.
Why cross-model comparison earns its place in the workflow
A single model can't reliably check itself — asked "are you sure?", it tends to re-assert the same error. Independent models fail differently: they have different training data and, in grounded setups, different retrievals. A fabricated case or misquote that one model produces usually isn't reproduced by the others, so it stands out as a low-consensus claim. That's your signal to go to the primary source. Consensus across models raises your confidence that something is real — it does not prove it, and it never replaces reading the opinion.
The professional-responsibility framing
This isn't only good practice — it maps onto core professional duties. The duty of competenceincreasingly includes understanding the tools you use and their failure modes; a lawyer who doesn't know that grounded AI still hallucinates quotes is missing something material about the tool. The duty of candor to the tribunal means you are responsible for the accuracy of what you file — full stop. An AI tool cannot hold that duty for you, and "the software generated it" was not a defense in the sanctioned cases.
- Treat AI output as a research lead, never as a citation of record
- The human signing the filing owns every case and quote in it
- Document your verification steps — it's both good practice and a record of diligence
Used well — with independent verification and cross-model comparison built into the workflow — AI makes legal research faster and helps surface the claims worth a second look. Used on trust, it will eventually put a fabricated quote in your brief. The difference is the workflow, and the workflow is yours to run.