Legal AI Concepts: Part 1

What Legal AI Is, How It Works and Its Core Limitations

RN
Rohas Nagpal

Legal AI is the use of artificial intelligence to assist with legal work. It can help lawyers search authorities, analyse documents, review contracts, organise evidence, prepare chronologies, draft documents, monitor compliance and communicate complex material more clearly.

That definition is deliberately broader than generative AI. A chatbot that drafts a note is one form of Legal AI, but so is software that classifies discovery documents, extracts obligations from contracts or applies a rules-based eligibility test.

The following terms are related, but they are not interchangeable:

  • Artificial intelligence is the broad field of building computer systems that perform tasks associated with human intelligence, such as classification, prediction, language processing and planning.
  • Machine learning is an approach in which a system learns patterns from data instead of relying only on rules written by a programmer.
  • Generative AI produces new material, such as text, images, audio or code, in response to instructions and context.
  • A large language model (LLM) is a generative model trained to process and produce language. GPT-5.5 and other language models are examples.
  • Legal AI describes AI used for legal work. It may use an LLM, older machine-learning methods, rules, search technology or a combination of them.
Diagram showing Legal AI applications including research, drafting, contract review, compliance and dispute preparation
Legal AI is defined by the legal task it supports, not by one particular model or interface.

2. Forms of Legal AI

Legal technology used AI long before modern chatbots. Important forms include:

  • Technology-assisted review: ranks or classifies large document sets for discovery, investigations and disclosure.
  • Document classification and extraction: identifies document types, parties, dates, clauses, obligations and other fields.
  • Rules-based systems: applies defined legal or policy rules to supplied facts, such as a regulatory checklist or benefits test.
  • Predictive analytics: estimates outcomes or patterns using historical data. Any result depends heavily on data quality and may reproduce historical bias.
  • Search and recommendation systems: retrieve authorities, precedents and internal knowledge using keywords, semantic similarity or both.
  • Generative systems: summarise, explain, translate or draft new text using an LLM.
  • Workflow systems: combine several steps e.g. intake, document extraction, analysis, drafting, verification and lawyer approval.

A single legal product may combine all of these. A contract-review platform might first classify the document, extract clauses, retrieve the organisation's approved playbook and then use an LLM to explain deviations.

Example: One dispute, several forms of Legal AI

Consider a dispute arising from the termination of a software distribution agreement. The file contains the agreement, two amendments, 3,000 emails, unpaid invoices and notes of settlement discussions.

  • A rules-based system checks whether the contractual notice period was followed.
  • A document-extraction system identifies the parties, dates, invoice amounts and termination clauses.
  • A technology-assisted review system ranks the emails most likely to concern termination, non-payment or admissions.
  • A search system retrieves authorities on termination, waiver and acknowledgement of debt.
  • A generative system prepares a chronology and first draft of an internal case assessment.
  • A workflow system combines these steps and requires lawyer approval before any advice is sent.

The same matter may therefore involve several forms of Legal AI. Treating all of them as merely “chatbots” hides important differences in how they work and how they should be tested.

3. How large language models work

An LLM processes text as tokens and repeatedly predicts a suitable next token from the patterns it learned during training and the material supplied in the current request. At scale, this enables surprisingly capable drafting, summarisation, comparison and question answering.

But an LLM does not consult a mental statute book or reason as a lawyer does. Its output is generated from statistical patterns. It may reproduce a legal proposition accurately, combine two propositions incorrectly or invent a plausible-looking authority.

Training gives a model general capabilities. The material supplied at run time (e.g. the instructions, facts, documents, retrieved passages and tool results) helps adapt those capabilities to the particular matter. Neither training nor a large context window guarantees legal correctness.

Example: The invented authority

A lawyer asks an LLM whether a distributor can claim compensation after termination without the contractually required notice. The model gives a clear answer and cites a judgment with a realistic case name, court and citation.

The judgment does not exist.

This is possible because the model is generating a plausible sequence of words, not checking a verified table of authorities unless the system has been designed to do so. A professional-looking citation is therefore a claim to be verified, not evidence that verification has occurred.

Diagram explaining that a large language model predicts tokens and requires documents, sources, checks and lawyer review for legal work
An LLM generates language; a legal workflow must supply authority, controls and review.

4. Tokens and context windows

4.1 Tokens

A token is a unit into which a model's tokenizer divides text. A token may represent a short word, part of a longer word, punctuation, a number or text combined with surrounding whitespace. Tokenisation varies by model.

Tokens matter for three reasons: models charge and enforce limits by tokens; long documents use more of the available context; and the model's answer also consumes output tokens. A 200-page bundle is therefore not treated as one indivisible document.

4.2 Context windows

The context window is the maximum amount of tokenised material a model can process for one response. Depending on the model and API, it may include instructions, the user's prompt, chat history, retrieved passages, tool results and the generated answer.

If the request exceeds that limit, the system may reject it, truncate older material or select only part of the document collection. Even when everything technically fits, a model may overlook a small but decisive clause in a very long context. Lawyers should not treat “fits in the context window” as meaning “was analysed reliably.”

This is why serious document work often uses parsing, structured extraction and retrieval rather than placing an entire matter into one prompt. Part 4 explains that pipeline.

Example: The clause that technically fit but was missed

A lawyer uploads a 180-page contract bundle. Clause 14 allows termination on 30 days' notice. An amendment near the end of the bundle states that, during the first year, termination requires 90 days' notice.

The entire bundle fits within the model's context window. The model nevertheless summarises only Clause 14 and concludes that 30 days' notice was sufficient.

The failure is not caused by the context limit being exceeded. It is an attention and document-structure failure. This is why “the file was uploaded” and “the answer cited the contract” do not prove that every relevant provision was considered.

Diagram showing prompts, documents, conversation history and model output sharing a limited context window
The context window is a capacity limit, not a guarantee of attention or completeness.

5. Chatbots and Legal AI systems

A chatbot is an interface for interacting with an AI system. Some chatbots provide only a general conversation; others can search the web, analyse files, call tools or offer several models. The label “chatbot” therefore tells you little about the system's safeguards.

The important distinction for legal work is between an uncontrolled general-purpose interaction and a task-specific, governed workflow. A stronger Legal AI system may add:

  • approved source collections and jurisdiction filters;
  • matter-level access controls;
  • structured intake and required facts;
  • retrieval from contracts, authorities or evidence;
  • citations linked to the supporting passage;
  • deterministic checks for names, dates and citations;
  • audit logs and version records; and
  • risk-based lawyer review before external use.

That system may use one model or several. Multiple models can improve resilience or permit task-specific routing, but they also add cost, latency, privacy exposure and operational complexity. Model choice should follow testing, not marketing claims.

Example: Chatbot versus governed workflow

A client uploads a contract and asks, “Can the other side terminate immediately?”

A general chatbot may produce a fluent answer based on whichever clauses it notices. A governed Legal AI workflow may instead:

  1. confirm the governing law and the user's role in the transaction;
  2. identify the operative agreement and all amendments;
  3. extract every termination provision and cross-reference;
  4. retrieve the organisation's approved playbook and relevant authorities;
  5. show the supporting passages for each conclusion;
  6. flag missing facts and conflicting clauses; and
  7. require lawyer review before the answer is communicated externally.

The underlying model may be identical in both systems. The difference lies in the surrounding controls.

Legal AI architecture diagram showing models, legal documents, retrieval, verification, workflows and human review
Reliable Legal AI is a system of components and controls, not merely a model.

6. What lawyers can use Legal AI for

Useful applications include legal research, judgment analysis, drafting, contract review, due diligence, evidence and chronology analysis, litigation and arbitration preparation, mediation and negotiation preparation, regulatory monitoring, compliance analysis, knowledge management, translation, intake and client communication.

The appropriate design depends on the task. Extracting dates from a standard form is different from advising on an unsettled point of law. The second task needs more authority checking, jurisdictional expertise, explicit uncertainty and professional judgment.

Example: Same tool, different risk

Using AI to extract the dates and amounts from 500 standard invoices is a relatively bounded task. The output can be checked against the source documents and validated using deterministic rules.

Using the same AI to decide whether a limitation period was extended by an acknowledgement of debt is materially different. The answer may depend on the exact wording, date, authority of the sender, governing law and procedural context. The second task requires stronger source verification, legal judgment and escalation.

A useful starting question is not “Which AI is best?” but:

What precise legal task is being performed, what could go wrong, and what evidence would show that the result is acceptable?

7. Core limitations

Lawyers should understand at least six recurring limitations:

  1. Hallucination: the system may generate an unsupported fact, quotation, citation or legal proposition.
  2. Omission: it may miss a clause, exception, authority, fact or alternative interpretation without signalling that anything is absent.
  3. Staleness: training data, search indexes and internal precedents may not reflect current law.
  4. Context error: it may apply the wrong jurisdiction, confuse parties or blend material from different documents.
  5. Bias and inconsistency: results can reflect biased data and may change across repeated runs.
  6. Automation bias: polished language can cause a human reviewer to trust the answer more than the evidence warrants.

Case study: A polished answer that fails in six different ways

A Legal AI system is asked to assess a claim arising from the termination of a distribution agreement. It produces a clear, confident memorandum concluding that the claim is strong.

  • It cites a judgment that does not exist: hallucination.
  • It misses an amendment changing the notice period from 30 to 90 days: omission.
  • It relies on a provision that was amended after the system's source database was last updated: staleness.
  • It applies English law even though the agreement is governed by Indian law: context error.
  • When the question is asked again, it reaches a different conclusion without explaining why: inconsistency.
  • The reviewing lawyer accepts the memorandum because it is well structured and professionally written: automation bias.

No single safeguard would have prevented all six failures. Reliable use requires source control, matter context, verification, testing and meaningful human review.

These limitations do not make Legal AI unusable. They determine how it should be designed and supervised. Parts 2–6 explain the necessary instructions, document architecture, safeguards, testing and deployment controls.