The AI Glossary for Lawyers: 40 Terms, Each With a Legal Example

A vendor tells you their tool is “grounded in authoritative sources” with “zero data retention” and an “agentic workflow layer”. Each phrase has a precise meaning and each is doing work in the pitch. Grounded in what? Retention of which data, on whose approval? Agentic meaning it plans its own steps, or runs a script? An AI glossary for lawyers is not about sounding technical; it is about knowing which question to ask next.

Forty terms follow, each ending with why you care.

Core mechanics: token, parameter, context window, temperature, inference

Token. The unit of text a model reads and writes: a word, part of a word or a punctuation mark; GPT-4’s vocabulary has roughly 100,277. Why you care: models cannot count letters reliably, and German needs more tokens per word than English.

Parameter. One of the billions of numbers set in training that encode what the model “knows”: Andrej Karpathy’s “lossy zip file of internet knowledge”. Why you care: unaided recall is weakest for the obscure state case you need.

Context window. Anthropic’s definition: “all the text a language model can reference when generating a response, including the response itself. This is different from the large corpus of data the language model was trained on, and instead represents a ‘working memory’ for the model.” Thinking tokens count too, and are billed. Why you care: what is in the window can be checked; what is not is recollection. See the context window guide.

Temperature. The randomness setting for choosing each token; Anthropic’s API reference warns that “even with temperature of 0.0, the results will not be fully deterministic”. Why you care: one answer is a sample, not a finding.

Inference. Running the trained model to produce an answer. Why you care: it is what you pay for, and “inference geo” decides where it happens.

Training vocabulary: pretraining, post-training, RLHF, fine-tuning, training data

Pretraining. The months-long phase in which the model reads internet text and learns to predict the next token, producing “just an expensive autocomplete”. Why you care: everything it knows unaided was fixed here.

Post-training. The shorter phase (“as little as a few hours”) that turns the base model into an assistant. Why you care: here it learned to always answer and to please the asker.

RLHF. Reinforcement learning from human feedback: people rate answers and the model is nudged towards the preferred ones. Why you care: Anthropic’s 2023 research found preference models favour “convincingly-written sycophantic responses” over accurate ones; OpenAI rolled back a GPT-4o update in April 2025 for being “overly flattering or agreeable”.

Fine-tuning. Further training of an existing model on a narrower dataset. Why you care: when a vendor says “proprietary legal model”, ask whether it is a fine-tuned general model, a prompted one or something trained from scratch, and whether your data is ever used.

Training data and cut-off. The corpus the model learned from and the date it stops. Why you care: Stanford’s Large Legal Fictions study found models most reliable on 1953-1969 Supreme Court cases and least on the oldest and newest; anything later must come through the context window.

Reliability vocabulary: hallucination, misgrounding, sycophancy, context rot, calibration

Hallucination. Confident, wrong output. Stanford’s legal research tool study counts a response as hallucinated if it is “incorrect or misgrounded” and accurate only if “correct and grounded”. Why you care: Damien Charlotin’s database listed 2,039 court decisions involving hallucinated material as of 12 September 2026, 811 of them involving lawyers. See why AI makes up fake cases.

Misgrounding. A real source cited for a proposition it does not support, or an inapplicable one (wrong jurisdiction, overruled). Why you care: more dangerous than a fabricated case; a checker who confirms existence stops there.

Sycophancy. The trained tendency to agree with the user. Why you care: in Stanford’s false-premise tests GPT-4 hallucinated on 53.1-69.1% of questions whose premise was wrong by construction.

Context rot. Anthropic’s docs: “As token count grows, accuracy and recall degrade, a phenomenon known as context rot.” Why you care: a 1M-token window is a capacity, not an instruction to fill it.

Calibration. How well expressed confidence matches accuracy. Why you care: Stanford found “a common thread across all models is a tendency towards overconfidence, irrespective of their actual accuracy”.

Retrieval vocabulary: RAG, grounding, citations, vector search

RAG (retrieval-augmented generation). Search first, then generate from what was found. Why you care: the major legal research platforms work this way and it does not solve hallucination; Lexis+ AI hallucinated on more than 17% of Stanford’s pre-registered queries, Westlaw AI-Assisted Research on more than 34%. See the RAG guide.

Grounding. Tying every statement to a passage in the supplied material. Why you care: quoting before concluding is the cheapest hallucination reducer for document work.

Below is [the counterparty's markup of our services agreement]. Task: identify every change that shifts risk to [PARTY_A].
Step 1: reproduce verbatim, in <quotes> tags with clause numbers, each passage changed or added.
Step 2: using only those quotes, list the risk-shifting changes: clause, operative words, why it shifts risk, severity (high/medium/low).
If no quote supports a point, do not make it. If an expected provision is absent, write "NOT IN DOCUMENT".
<document>
[paste]
</document>

Source-linked citation and verify markers. A citation rendered as a link to the source, sometimes flagged (Shepard’s Verify Trust Markers, Deep Research Verify). Why you care: the marker confirms the authority exists, not that it supports the proposition.

Vector search (embeddings). Retrieval by meaning rather than exact words, so “termination for convenience” and “cancel without cause” land together. Why you care: it finds paraphrases Boolean misses, and confused “moral turpitude” with “moral wrong doctrine” in Stanford’s test.

Product vocabulary: system prompt, custom instructions, Projects, custom GPTs, skills, plugins

System prompt. Standing instructions above every message that shape all answers. Why you care: every legal product is a general model plus a system prompt plus tools; Artificial Lawyer called Anthropic’s first legal plugin “basically just a very complex prompt”.

Custom instructions. The user-level version in ChatGPT or Claude: jurisdiction, spelling, house style. Why you care: “Lead your custom instructions with the safety rules, not your bio” (claudeforlawyers.com).

Projects. Folders in Claude or ChatGPT holding instructions and reference files for reuse across chats. Why you care: this is how a playbook becomes a repeatable review; never load privileged specifics into a shared Project.

Custom GPT. OpenAI’s shareable package of instructions, knowledge files and tools. Why you care: keep knowledge files small and anonymised.

Skill. A written procedure the model follows for a recurring task; Artificial Lawyer’s analogy: Claude is the brain, Cowork the hands, Skills the recipes, a Plugin the cookbook. Why you care: Zack Shapiro’s two-person firm runs on skills encoding one lawyer’s judgement.

Plugin. A bundle of skills, agents and connectors; Claude for Legal shipped 12 practice-area plugins on 12 May 2026. Why you care, from its README: “Skipping setup is the single most common reason a skill produces generic output.”

Agent vocabulary: agent, workflow, tool use, MCP, connector, computer use

Agent. Legora’s definition: “A chatbot receives a question and provides an answer… An agent receives a task, creates a plan of action, executes the plan, and returns a result. It’s execution technology. It does the work, end-to-end.” Why you care: “Agents do not sign documents. Lawyers do.” (Harvey). See the agents guide.

Workflow. A fixed sequence of steps; Legora calls chained workflows “automation… not agency”. Why you care: a workflow is auditable, an agent decides its own steps.

Tool use. The model calling something outside itself: a search, a calculator, a database. Why you care: it is Karpathy’s cure for hallucination (“look it up instead of making things up”), and each tool is another party holding data.

MCP (Model Context Protocol). An open standard for connecting models to tools and data, open-sourced by Anthropic in November 2024 and adopted by OpenAI, Google and Microsoft; iManage launched an MCP server in May 2026. Why you care: “MCP is now a procurement question.” See MCP for law firms.

Connector. A vendor-built link from a model to a named system (DocuSign, iManage, LexisNexis), often over MCP. Why you care: OpenAI notes MCP servers are third parties with their own retention.

Computer use. An agent operating a screen as a person does, “by seeing, clicking, and typing with its own cursor” (OpenAI on Codex). Why you care: this mode can file, send and pay, and California’s 2026 guidance says lawyers “must not permit AI systems to autonomously file documents”.

Confidentiality vocabulary: zero data retention, abuse monitoring, DPA, data residency, covered models

Consumer tier vs commercial tier. ChatGPT Free, Plus and Pro, Claude Free, Pro and Max, consumer Gemini and Copilot train on conversations by default; Business, Enterprise and API tiers do not. Why you care: a federal court held a defendant’s consumer-Claude exchanges to be neither privileged nor work product. See is ChatGPT confidential for lawyers.

Zero data retention (ZDR). The provider does not store prompts or outputs after serving the request. Why you care: OpenAI grants it “subject to prior approval” and only on eligible endpoints; Anthropic’s ZDR page limits it to eligible API and Claude Code for Enterprise customers and “still retains User Safety classifier results”. It does not cover claude.ai Team or Enterprise chat.

Abuse monitoring. Automated and human review of inputs for misuse. Why you care: “no training” is not “no human eyes”; OpenAI uses “specialized third-party contractors… solely to review for abuse and misuse”.

DPA (data processing agreement). The GDPR Article 28 contract making the vendor your processor. Why you care: necessary, not sufficient; in Germany it does not discharge § 203 StGB, and the BRAK says only “abstract” prompts belong in public tools.

Data residency and inference geo. Where data is stored and where the model runs. Why you care: OpenAI offers EU at-rest residency for Enterprise workspaces; Anthropic’s first-party API offers an inference_geo of “us” or “global” only, so EU residency for Claude comes via AWS Bedrock or Google Vertex; Microsoft’s EU Data Boundary excludes Anthropic models in Copilot.

Covered models. Anthropic’s term, effective 9 June 2026, for its Mythos-class models: “Prompts submitted to, and outputs generated by, covered models are retained for 30 days to support our safety work, on every platform where these models are offered.” Why you care: even ZDR customers accept 30-day retention on these.

Below are a vendor's security page, privacy policy and DPA. Answer these questions in a table: Question | Answer (quoted verbatim) | Document and clause | Gap.
Are inputs, outputs, files and embeddings excluded from training? Default retention? Is zero data retention offered, on what conditions, with what carve-outs (classifier scores, flagged content, legal holds, specific models)? Can abuse monitoring be disabled? Which subprocessors and regions? Is EU residency available for storage and inference?
Where a document is silent, write "SILENT". Quote, do not summarise.
<documents>
[paste]
</documents>

Regulation vocabulary: deployer, AI literacy, self-learning tool

Deployer. Under the EU AI Act, anyone using an AI system under their own authority professionally; the BRAK reads Article 3(4) as making law firms “Betreiber”. Why you care: the deployer carries the Article 4 duty.

AI literacy (Article 4). Since 2 February 2025 deployers must take measures to support staff AI literacy (softened from “ensure” by the July 2026 Digital Omnibus). The Commission’s Q&A says a firm whose staff use ChatGPT must comply. Why you care: no fine sits in Article 99, but sanctions are “more likely if there is proof of an incident due to lack of appropriate training”.

Self-learning tool. ABA Formal Opinion 512: a tool that learns from inputs; for such tools “a client’s informed consent is required prior to inputting information relating to the representation”, and boilerplate engagement-letter consent “is not sufficient”. Why you care: whether your tier trains by default decides whether you need consent. See the Opinion 512 guide.

How to use this glossary in a vendor meeting

Vocabulary pays off when it changes your next question.

The most useful line to add to any prompt is Brooke Loesby’s Curiosity Prompt.

I need [a first-draft response to this redline / a memo on X] for a matter in [jurisdiction]. Before you produce anything, ask me what else you need to know to give me the most accurate response, grouped by importance. Then wait for my answers.

Where to go next: the fundamentals hub holds the pillar this glossary supports, how LLMs work for lawyers, which turns the first fifteen terms into one picture; the Curiosity Prompt guide shows the interview pattern on real matters. Knowing the words is the entry ticket; in AI Lab for Lawyers you use them on your own files.