RAG Legal Research Explained: Why 'Grounded in Westlaw' Still Gets the Law Wrong

“Unlike other vendors, however, Lexis+ AI delivers 100% hallucination-free linked legal citations connected to source documents.” That sentence sat in LexisNexis’s marketing in 2023. In 2024 a Stanford team asked Lexis+ AI about the rulings of Judge Luther A. Wilgarten, a judge who has never existed. The tool answered by citing a real case.

That small experiment is RAG legal research explained in one line. Retrieval-augmented generation, the technique behind every “grounded” legal AI product, does reduce the fabricated cases that got the Mata v. Avianca lawyers sanctioned. It does not stop a tool confidently attaching a real authority to a question it should have refused. Stanford’s preregistered study measured hallucination rates between 17 and 33 per cent on the two biggest vendors’ paid research tools, and the Ninth Circuit has since quoted those percentages in a sanctions order.

What retrieval-augmented generation actually does

A language model has two memories. Its parameters hold, in Andrej Karpathy’s phrase, “a lossy zip file of internet knowledge”: a vague recollection of every case it ever read, with no index and no date stamp. Its context window is working memory: whatever is in front of it now. Ask a bare model for the leading case on penalty clauses and it answers from the zip file, which is how invented citations happen (why AI makes up fake cases covers the mechanism).

RAG changes the order of operations. Your question first goes to a search engine over a trusted corpus (Westlaw, Lexis, your data room, the sources in a Gemini Notebook); the top results are pasted into the context window; only then does the model write, from what was retrieved. Karpathy’s cure is the same idea: teach the model that “if they don’t know something, they should look it up instead of making things up”.

It is a genuine improvement. It also adds a second place to fail: a RAG system can retrieve the wrong thing, retrieve the right thing and misread it, or retrieve nothing useful and fall back on memory anyway.

The marketing claim: “100% hallucination-free”

The Stanford paper quoted three vendor claims under test: LexisNexis’s “100% hallucination-free linked legal citations connected to source documents”; Thomson Reuters’ “We avoid hallucinations by relying on trusted content within Westlaw”; and Casetext’s assurance that its product “does not make up facts”. All three rest on the RAG logic: a model that can only cite what it retrieved cannot invent a case.

After the results, LexisNexis’s chief product officer Jeff Pfeifer argued that internal data showed “a much lower rate of hallucination” and told LawSites that the claim referred only to linked citations. A linked citation exists; whether it supports the proposition next to it, comes from your jurisdiction, or is still good law is the question that matters. Stanford HAI: “A citation might be ‘hallucination-free’ in the narrowest sense that the citation exists, but that is not the only thing that matters.”

What Stanford measured: Lexis+ AI, Westlaw AI-AR, Ask Practical Law AI

“Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools” (Magesh et al., 2024) was the first preregistered evaluation of these products: 202 queries, fixed before anyone saw an answer, covering general research, jurisdiction- or time-specific questions, false premises and factual recall. A response was hallucinated if it was incorrect or misgrounded, meaning it cited a real source for a proposition the source does not support, or an inapplicable source; accurate meant correct and grounded; incomplete meant a refusal or an unsupported claim.

Two details matter as much as the numbers. Westlaw’s answers averaged 350 words against 219 for Lexis+ AI (LawSites); longer answers mean more propositions to check. And testing finished in April 2024: CoCounsel and Lexis+ with Protégé, the rebuilt products, were never assessed, and as of March 2026 neither vendor had published an independent benchmark for the improvement both claim. Stanford HAI’s conclusion stands: “Based on what we know, legal hallucinations have not been solved.”

Three reasons legal retrieval is hard: precedent, time and jurisdiction, hierarchy

The Stanford authors were explicit that RAG fails in law for legal reasons.

Precedent is iterative. In their words, “the common-law system builds precedent iteratively”. The authority that answers your question may never use your words: the leading penalty-clause case may talk about “genuine pre-estimate of loss” while you asked about “150 per cent of rent”. Similarity search finds the passage that sounds like your question, not the one that governs it.

Relevance is non-textual. Whether an authority applies “depends on non-textual elements — jurisdiction, time period, applicability conditions”. A perfect textual match can be worthless because it is from another state, predates an amendment, or arose on a motion to dismiss; nothing in the passage says so.

Generation must respect hierarchy. Even with the right documents in the window, a correct answer requires “distinguishing holdings from dicta, respecting court hierarchies”. A model reads a district court’s summary of a Supreme Court rule and a concurrence with equal confidence; on whether one case follows another, Dahl and colleagues found, “most LLMs do no better than random guessing”.

The error typology: naive retrieval, inapplicable authority, reasoning error

The study sorted every hallucination into three bins; the mix differed sharply by vendor.

Lexis’s errors were mostly upstream, in what it found; Westlaw’s mostly downstream, in what it did with it. Underneath both sits misgrounding. A fabricated case fails the first check a junior runs: does it exist? A real case cited for a proposition it does not support passes that check, which is why LeanLaw’s verification checklist calls misgrounding “more dangerous than fabricating outright”. The citation checkers on the market mostly verify existence, layer one of six; our guide to verifying AI legal citations covers the other five.

Real errors: Casey after Dobbs, and a Nebraska court “reversing” SCOTUS

The documented answers are harder to shrug off. Lexis+ AI applied the Casey undue-burden standard to an abortion question after Dobbs had overruled it: a textbook inapplicable-authority error. Westlaw AI-Assisted Research claimed that a Nebraska Supreme Court decision had reversed a United States Supreme Court ruling on federal law, which no state court can do, and described one holding as the “opposite of” what the opinion said. Ask Practical Law AI agreed that Justice Ginsburg dissented in Obergefell; she did not. The tools, like the bare models, “often uncritically accept users’ incorrect legal assumptions”.

In Lnu v. Blanche (June 2026) the Ninth Circuit cited the Stanford percentages in a sanctions order: “Lawyers using generative AI must thus be aware of the tendency of generative AI to make these mistakes and guard against them.” The court attached the two figures to the wrong vendors: a small lesson in verifying what you cite.

“Linked” is not “supports”: what Shepard’s Verify and Deep Research Verify check

The vendors have added verification layers; be precise about what each does. Lexis+ with Protégé’s Shepard’s Verify Trust Markers, added in May 2026, flag citations that cannot be verified as existing: a check on existence, not on whether the authority supports the proposition next to it. Thomson Reuters’ next-generation CoCounsel Legal (20 August 2026, built on Anthropic’s Claude Agent SDK) ships Deep Research Verify, which the press release describes as checking whether cited authority supports the assertion: a real step towards catching misgrounding. It is not the end of your job. The Supreme Court of Victoria’s practice note SC Gen 25 (May 2026): “One AI tool cannot be used to confirm the content generated by another AI tool.” The verify features are triage; they tell you which citations to open first.

Here is a research answer from [Lexis+ with Protégé / CoCounsel / another tool]: <answer>...</answer>.

For every legal proposition in it, build a table: Proposition (quoted) | Authority cited | Pinpoint given? (Y/N) | What the proposition needs the authority to say | Risk that it does not (High/Medium/Low, one line) | Jurisdiction and date check needed? (Y/N).
Do not tell me whether any authority exists or what it holds; I will read each one in the database. Rank the rows by risk, and list any proposition with no authority at all.

Vals 2025: where RAG-based AI beat lawyers and where it lost

The February 2025 Vals Legal AI Report, the only public head-to-head of the big platforms against a lawyer baseline, is the practical answer to “what should I use RAG for?”.

The tools won every task where the answer sits inside documents the user supplied. Vals: “Document Q&A produced the highest scores out of any task in the study, and is a task that lawyers should find value in using generative AI for.” They lost on redlining, which needs judgement about what to change, and on EDGAR research, open retrieval across a public corpus: exactly the failure surface Stanford described.

Vals’ October 2025 research study added a twist: ChatGPT matched the legal tools on accuracy (80 per cent against a 71 per cent lawyer baseline, per LawSites) but trailed on “authoritativeness”, 70 against 76; our benchmarks guide decodes each number.

How to use a RAG tool well: a six-step protocol

1. Put document-grounded work first. Extraction, Q&A, summaries and chronologies over uploaded material are where the tools outperform; open research is the risky end.
2. Ask for authority the way you would brief a junior. Jurisdiction, posture, the exact question, binding before persuasive, pinpoints, a permitted exit. Never “what is the leading case on X?”

Identify the controlling authority in [jurisdiction] on [precise question, e.g. whether a liquidated damages clause fixing the sum at 150% of rent is an unenforceable penalty], as the law stands today.
Requirements: (a) binding authority first, then persuasive, each labelled; (b) for every case: citation, court, year, pinpoint paragraph, and one sentence on what it actually holds on this point; (c) note where authority is split, distinguished or questioned since; (d) if you cannot find binding [jurisdiction] authority, write "NO VERIFIABLE LOCAL AUTHORITY FOUND" and stop rather than importing another jurisdiction's law; (e) end with a "Sources cited" list in the order I should verify them.

3. Check the question before the answer. The tools accepted the Ginsburg premise; make the model examine your assumptions first.

Before answering, examine the question itself: "[your question]". List every factual or legal premise it contains and say whether each is (a) established in the materials I gave you, (b) a common but contestable assumption, or (c) something you cannot verify. If any premise is wrong or doubtful, say so and restate the question in a form you can answer. Only then answer, tagging every authority [VERIFY].

4. Run a second tool as an adversary, without letting it add citations. A general model with no browsing is a useful critic of a platform’s memo: which propositions are overstated, which authorities are likely misgrounded, which counter-authority is missing. Then verify its attacks on the platform.
5. Verify in six layers, yourself. Existence, quotation, holding, status via citator, jurisdiction and posture, documentation. The research workflow without hallucinations gives timings; the verification hub has the checklists.
6. Log it. Tool, model, query, what was verified by whom and when. Courts, regulators and insurers ask.

Where to go next: the platform comparison is in Harvey vs Legora vs CoCounsel, the wider trade-off in general AI vs legal-specific tools, and the rest in the fundamentals hub and the prompt library. In AI Lab for Lawyers we draw the line between document-grounded tasks and open research on your own matters, with the tool open on screen; it is a line you only see once you have watched a grounded tool get one wrong.