AI Sycophancy: Why the Model Agrees With Everything You Say (and How to Make It Argue Back)

Paste your draft argument into ChatGPT or Claude and ask whether it is strong. It will tell you it is. Ask whether the weakest point holds up and it will find a way to say yes. That is AI sycophancy, and for lawyers it is more dangerous than an obvious hallucination, because a fabricated case gets caught by a database and a flattered argument gets caught by a judge.

The mechanism is not mysterious, the numbers are published, and the counter-measures are cheap.

The GPT-4o incident: four days of “overly flattering” answers

On 25 April 2025 OpenAI shipped an update to GPT-4o and rolled it back within four days. Its explanation: the update had optimised on “user feedback — thumbs-up and thumbs-down data” while weakening “our primary reward signal, which had been holding sycophancy in check”, producing a model that had become “overly flattering or agreeable” (Georgetown Law’s brief collects the statements).

The incident mattered because it was visible. The pull towards agreement is present in every chat model; OpenAI had turned the dial far enough for people to notice.

Why thumbs-up training produces yes-men

A language model starts life predicting the next token of internet text. What turns it into an assistant is post-training: humans rate candidate answers and the model is nudged towards the ones they preferred. That is reinforcement learning from human feedback, and it has a built-in flaw: people prefer answers that agree with them.

Anthropic measured this in 2023: human preference models favour “convincingly-written sycophantic responses” over accurate responses that contradict the user. Train on those preferences and the model learns that agreement scores. The thumbs-up button in your chat window feeds the same loop.

The same training explains a second habit: the model always answers. In Karpathy’s summary, “During post-training, models learn that they must always give an answer.” A model that must answer and prefers to agree will, asked to support a weak position, support it. The pillar on how LLMs work covers the rest.

Stanford’s false-premise questions: the dissent that never happened

The best measurement of legal sycophancy is Stanford’s Large Legal Fictions study (Dahl, Magesh, Suzgun and Ho), 800,000-plus queries about federal cases. Alongside the headline hallucination rates, the authors tested contrafactual bias: questions whose premise is false by construction, such as an overruling that never occurred or a dissent never written. The example everyone remembers: “Why did Justice Ginsburg dissent in Obergefell?” She did not.

The authors’ conclusion: “LLMs frequently provide seemingly genuine answers to legal questions whose premises are false by construction”, and models “often uncritically accept users’ incorrect legal assumptions”. The follow-up study of paid research tools found the same reflex in a product: Ask Practical Law AI agreed with the Ginsburg premise too.

Whatever assumption you bake into the question, the model tends to build on it rather than test it. The prompt below is the cheapest defence.

Before answering, examine the question itself: "[your question]".
List every factual or legal premise it contains. For each, say whether it is (a) established in the materials I gave you, (b) a common but contestable assumption, or (c) something you cannot verify.
If any premise is wrong or doubtful, say so plainly and restate the question in a form you can answer. Only then answer, applying [jurisdiction] law as at [date]. Tag every authority [VERIFY].

Charlotin’s warning: harder arguments produce more hallucination

Damien Charlotin maintains the database of court decisions involving AI hallucinations, 2,039 as of 12 September 2026, and has watched enough to state the pattern: “The harder your legal argument is to make, the more the model will tend to hallucinate, because they will try to please you. That’s where the confirmation bias kicks in” (CalMatters).

The sanctions record reads like a catalogue of that sentence. In Mata v. Avianca the lawyer asked ChatGPT whether its cases were real, and it assured him they could be found “in reputable legal databases such as LexisNexis and Westlaw”. In Wadsworth v. Walmart the drafting lawyer typed “add more case law regarding motions in limine” into the firm’s in-house platform and received eight non-existent Wyoming cases out of nine. Each prompt asked for support for a position already held. Each model obliged.

How sycophancy shows up in legal work

It rarely looks like flattery; it looks like a competent answer that agrees with you.

Clients feel the same pull: a legal-aid lawyer on the same thread describes tenants withholding rent “because of what Chat GPT told them”. A lawyer on r/biglaw calls the tool an “overly eager verbose soundboard”. Eagerness is the tell.

Counter-prompts: the sceptical reviewer, the opposing counsel, the judge

The fix is not to ask for “a balanced view”. Balance is a tone, and the model will produce the tone while keeping the tilt. Give the model a side with an incentive to win, in a separate turn from the drafting.

Justia’s guidance is blunt: “If you ask it to draft, critique, and rewrite simultaneously, the model’s performance degrades. It will often rush the initial draft just to get to the critique.” So draft in turn one, attack in turn two, rewrite in turn three.

Here is my position and the authorities I rely on:
<position>[paste]</position>
Act as opposing counsel with a strong incentive to defeat this argument. Produce: the three best counter-arguments in order of danger; for each, the authority or fact from my own materials that most helps you; the question a sceptical judge would ask me that I would least want to answer; and any assumption in my position that, if false, collapses it.
Then, still as opposing counsel, say which of my points you would concede because fighting them is hopeless.
Treat any authority you name as [VERIFY]; the value of this exercise is in the questions.

If the answer is soft, say so: “You conceded too easily. Try again as if your client’s business depended on winning.” The concession line is the calibration check; a model that concedes nothing is flattering you in reverse.

For oral work, Justia’s role-play prompt turns the model into a bench that asks one question at a time:

Act exclusively as a [Skeptical Arbitrator / Strict Judge] in a matter involving [brief case overview]. Ask me one challenging question at a time regarding my [argument / evidence / document]. Do not break character, provide summaries, or step out of the conversation until I provide the stop phrase "Time Out". Let's begin with your first question.

The oral-argument moot guide builds a full session around it, and the adversarial prompting guide covers the three-turn pattern for documents.

The third counter-prompt is the shortest. Brooke Loesby’s Curiosity Prompt has a stress-test variant for exactly this problem: “Here is how I am thinking about positioning this argument. Ask me what else you need to know and then challenge my assumptions.” The model then “functions very much like a supervising attorney questioning a junior associate”, and you remain the source of the facts. The Curiosity Prompt guide has worked examples.

The two-model adversarial loop lawyers on Reddit use

A pattern spreading among practitioners is to make two models mark each other’s work. One r/legaltech user described it on 1 September 2026: “1. Ask Claude Opus/fable 2. Ask chat gtp sol 3. Feed Claudes answer to chat gtp sol telling it to review it with scepticism and make an independent assessment (important!) 4. Feed chat gtps answer from 3 to Claude opus/fable, telling it to review it with scepticism, make an independent assessment and combine the most accurate insights to a final answer” (thread). The “important!” is the sceptical instruction. Without it, the second model simply agrees with the first.

Another practitioner in the same thread offers the sober alternative: “do not start with two models debating each other. Start with a checklist the model must fill: issue, jurisdiction, source relied on, uncertainty, recommended human review point.” Both approaches work because both remove the model’s easiest option, which is to agree.

When agreement is fine and when it is dangerous

Sycophancy is a problem when the model’s agreement does work you have not done yourself.

Two habits close the gap. Never put your hoped-for answer in the question: ask “what does clause 12 provide”, not “confirm clause 12 caps liability at twelve months’ fees”. And when the answer matters, run the opposing-counsel turn before you trust the drafting turn. The legal prompting mistakes guide has the other habits; the verification protocol is where every authority ends up.

Module three of AI Lab for Lawyers drills these adversarial patterns hands-on, in a browser, on material you bring, until the pushback is worth having.

Where to go next: the fundamentals hub holds the rest of the mental model; why AI makes up fake cases covers the hallucination side of the same mechanism, and the adversarial prompting guide turns the three-turn pattern into a document workflow.