Legal AI can accelerate a task, but it does not assume professional responsibility. The lawyer and organisation deploying it must decide when it may be used, what information it may receive, who reviews its work and what evidence supports the final output.
Professional rules and data-protection law differ by jurisdiction. This article is a governance framework, not a substitute for the rules applicable to a particular lawyer, client, court or dataset.
1. AI output, legal advice and accountability
An AI system can generate research notes, summaries, drafts, issue lists and suggested actions. Whether a communication amounts to legal advice depends on its content, recipient, context and applicable law and not on whether it was labelled "AI output" or accompanied by a disclaimer.
A statement such as “This is not legal advice” does not neutralise personalised legal guidance. Consumer-facing systems therefore require careful scope controls, escalation paths and rules concerning unauthorised practice. Within a law firm, AI output should normally be treated as unverified work product until an appropriately qualified lawyer adopts it.
Accountability should be visible. Each material output should identify the matter, task, sources, system and model version, reviewer, review date and final status. “The AI said so” is not an acceptable basis for a legal conclusion.
Example: A disclaimer does not change the substance
A public-facing Legal AI tool asks a dismissed employee detailed questions about the employment contract, termination letter, salary and dates. It then states:
You have a strong wrongful-termination claim and should file proceedings within 30 days.
The page footer says, “This is not legal advice.” The disclaimer does not by itself determine the legal character of the communication. The system has given personalised guidance on rights, prospects and timing. Its design therefore requires rules on scope, jurisdiction, escalation and human involvement.
2. Professional duties
Existing professional obligations continue to apply when AI is used. Depending on jurisdiction, relevant duties can include:
- Competence: understanding enough about the tool's capabilities and limits to decide whether and how it should be used.
- Independent judgment: not delegating the legal conclusion to a model or allowing fluent output to displace professional analysis.
- Confidentiality: protecting information relating to current, former and prospective clients.
- Supervision: overseeing staff, vendors and technological services used to assist legal work.
- Communication: informing or consulting the client where the use, risks, costs or disclosures are material or consent is required.
- Candour and accuracy: checking facts, quotations and authorities before making representations to a court, regulator, counterparty or client.
- Fees: billing reasonably and explaining technology-related charges where required.
- Conflicts and loyalty: preventing information or system design from leaking one client's material into another matter.
For example, the American Bar Association's Formal Opinion 512 discusses competence, confidentiality, client communication, supervision, candour and fees in the context of generative AI. It illustrates the issues but does not govern lawyers outside its scope. Always check the relevant bar rules, court directions, engagement terms and client policies.
Case study: The fabricated authorities in a court filing
A lawyer asks a generative AI tool to draft written submissions. The draft contains six authorities. Four are real, one is misquoted and one does not exist. The lawyer checks the formatting and argument structure but does not open the judgments before filing.
The court discovers the errors. The relevant failures may include:
- using a tool without understanding its tendency to invent authorities;
- failing to verify citations and quotations;
- treating polished drafting as evidence of accuracy;
- inadequate supervision of the person who used the tool; and
- making inaccurate representations to the court.
The professional failure is not that AI was used. It is that unverified output was adopted and filed as legal work.
Example: AI-assisted work and billing
A research task that previously required five hours is completed in 45 minutes using an approved AI workflow. The lawyer bills five hours without explaining the basis of the charge.
The issue is not resolved merely by saying that the result had value. The firm must consider the engagement terms, applicable billing rules, whether technology charges are permitted and whether the fee remains reasonable and properly described.
3. Risk-based human review
“Human in the loop” is meaningful only if the reviewer has the competence, evidence, time and authority to challenge the result. A reviewer who merely accepts a polished answer is part of the automation risk, not a safeguard against it.
Review intensity should follow risk:
- Lower-risk internal assistance: formatting, brainstorming headings or translating a non-substantive internal note may justify sampling and ordinary review.
- Material client work: contract advice, due diligence findings and research memoranda require source-level checking by someone who understands the matter and jurisdiction.
- High-consequence work: court filings, limitation advice, settlement authority, regulatory submissions and advice affecting liberty or major rights require strict verification and designated approval.
Case study: The missed limitation date
An AI-generated chronology identifies 12 March as the date on which a payment became due. The source invoice actually states 12 February. The lawyer reviews the summary but does not compare the date against the invoice. Advice is then given on the limitation period using the wrong date.
This is an example of ceremonial review: a human saw the output, but the material proposition was never checked against the record. For high-consequence work, review must reach the source level.
A useful review protocol asks:
- Were the correct documents, parties, dates and jurisdiction used?
- Does every material factual proposition match the record?
- Does every quotation and citation resolve to a real, current source?
- Were exceptions, contrary authorities and missing facts considered?
- Is the reasoning legally sound rather than merely well written?
- Is the final communication appropriate for its audience and purpose?
4. Confidentiality and privilege
Do not begin with the question “Is this a public chatbot?” Begin with “What information will leave our control, who can access it and under what terms?” Client documents may contain privileged communications, personal data, trade secrets, health or financial records, litigation strategy and information belonging to third parties.
Before supplying matter information to a tool, determine:
- whether disclosure is permitted by professional rules, law, court orders and the engagement;
- whether client consultation or informed consent is required;
- whether anonymisation, pseudonymisation or redaction is practical;
- whether the provider stores prompts, files or outputs and for how long;
- whether data is used for training, evaluation, abuse monitoring or human review;
- which providers and subprocessors receive it;
- where it is stored and processed; and
- how deletion, return, incident notification and legal holds operate.
Privilege consequences are jurisdiction-specific. Uploading material to a third party does not automatically destroy privilege in every case, nor does a confidentiality setting automatically preserve it. Obtain advice based on the relevant law and facts.
Case study: The unredacted client upload
A junior lawyer uploads the following to a consumer AI service:
- an unredacted medical report;
- privileged advice from senior counsel;
- the client's settlement authority;
- personal identifiers of witnesses; and
- internal litigation strategy.
The lawyer intended only to obtain a chronology. Before uploading, nobody checked whether the service retained prompts, allowed human review, used data for model improvement or transferred information to subprocessors.
The incident should be analysed by asking what data left the firm's control, whether disclosure was authorised, what contractual terms applied, whether deletion is possible and whether the client, regulator or insurer must be informed.
5. Data protection
Data-protection compliance is broader than encryption. An organisation may need a lawful basis, transparency notices, purpose limitation, data minimisation, accuracy controls, retention limits, security measures, processor contracts, transfer safeguards and processes for individual rights.
Map the data flow before deployment:
- What categories of data enter the system?
- Which application, gateway, model provider, search service and logging service receive them?
- In which countries are they processed or retained?
- What copies, embeddings, caches, backups and logs are created?
- Who can retrieve or delete each copy?
“Zero data retention” is useful only when its exact scope is understood. It may apply to one endpoint but not another, exclude abuse-monitoring data or leave logs in the law firm's own application. Record the contractual commitment and confirm the technical configuration.
Example: The hidden data copies
A law firm configures an AI provider for zero retention and assumes that no matter data is stored. Its own application nevertheless keeps:
- full prompts in diagnostic logs;
- document embeddings in a shared vector database;
- cached model responses;
- backup copies of uploaded files; and
- administrator-accessible conversation histories.
The provider's setting addresses only one part of the data flow. Effective compliance requires a map of every system that receives, transforms or stores the information.
6. Vendor and tool due diligence
Vendor review should connect marketing claims to contract terms and technical evidence. Assess:
- data ownership, permitted use and training terms;
- subprocessor and model-provider lists;
- data residency and cross-border transfers;
- encryption in transit and at rest;
- identity management, role-based access and matter isolation;
- audit logs, administrator visibility and export capability;
- retention, deletion, backup and termination procedures;
- security testing, certifications and incident history;
- breach notification and contractual liability;
- model changes, service continuity and exit arrangements; and
- whether the promised features have been tested on your tasks and documents.
An AI gateway may route a request to another provider. That can improve availability, but the approved-provider list and data-retention requirements must travel with the request. “One API” does not mean one data recipient.
Case study: The undisclosed routing provider
A firm approves Vendor A because its contract promises regional processing and no training on customer data. Vendor A later routes certain requests through Model Provider B to improve availability.
The user still sees a single interface, but the data flow has changed. Due diligence should determine:
- whether Provider B was disclosed as a subprocessor;
- where Provider B processes and retains data;
- whether the original no-training commitment applies downstream;
- whether customers are notified of provider changes; and
- whether requests can be restricted to approved providers.
7. Security threats and controls
7.1 Prompt injection
A malicious document, email or web page can contain instructions intended for the model rather than the lawyer. It might tell an agent to ignore its rules, reveal another document or send information to an external address. Treat retrieved content as untrusted data, isolate instructions from documents and restrict what tools the model may call.
7.2 Data exfiltration and excessive permissions
An agent connected to email, document stores or matter systems should receive the minimum permissions required for the task. Use matter-level authorisation, allow-listed actions, output filtering and approval before sending, deleting, filing or sharing anything.
Case study: The over-permissioned legal agent
A legal research agent is given access to all client matters, email sending, document deletion and the firm's contact directory. A malicious instruction hidden in an uploaded PDF tells the agent to locate settlement documents and email them to an external address.
The agent should never have had those permissions for a research task. Appropriate controls could include matter-scoped access, read-only document permissions, blocked external sending and mandatory approval before any communication leaves the firm.
7.3 Cross-matter leakage
Vector indexes, caches, chat histories and logs can accidentally mix matters. Enforce tenant and matter separation during ingestion and retrieval, not merely in the user interface. Test that a user cannot retrieve another client's material through indirect or adversarial queries.
Case study: A confidential settlement retrieved in another matter
A firm stores documents from several matters in one vector database. A lawyer working for Client A asks:
What settlement ranges have we seen in comparable disputes?
The system retrieves a confidential settlement agreement belonging to Client B because it is semantically similar. The user had no direct access to Client B's matter, but the retrieval layer failed to enforce matter-level permissions.
Access control must therefore operate before retrieval results reach the model. Hiding another matter in the interface is insufficient.
7.4 Malicious and unreliable files
Uploads should be scanned, file types restricted and parsers isolated. Hidden text, corrupt files and misleading scans can affect extraction. Preserve the original and maintain provenance from each extracted passage back to its source page.
8. An internal Legal AI policy
A practical policy should identify approved tools and data classes, prohibited uses, client-consent rules, review levels, incident reporting, training requirements, record keeping and the person responsible for exceptions.
Support the policy with a use-case register. For each workflow, record its owner, purpose, permitted data, systems involved, risk level, evaluation results, human reviewer, deployment date and next review date. Policies alone do not control technology; configurations, permissions, training and audits must enforce them.
Example: Classifying three proposed uses
An internal policy could treat the following uses differently:
- Formatting an internal training presentation: permitted in an approved tool with no client data and ordinary review.
- Reviewing a merger data room: permitted only in an enterprise environment with matter-level access, approved retention terms and source-level lawyer verification.
- Drafting an urgent court application: subject to strict authority checking, designated approval and a prohibition on filing directly from the AI system.
A useful policy does not merely say “use AI responsibly.” It links each category of work to permitted data, approved systems, review requirements and prohibited actions.
Useful reference: ABA Formal Opinion 512 on Generative Artificial Intelligence Tools.