Air-Gapped AI for Law Firms: Privilege Stays In-House
Air-gapped AI lets law firms run source-cited RAG over privileged case files and case law inside their own perimeter, with nothing sent to a cloud LLM.
Air-gapped AI for law firms is a large language model and retrieval system that runs entirely inside the firm’s own network, so privileged case files, client communications, and matter documents are never sent to a hosted LLM API. For litigation, corporate, and general-practice firms, this is the difference between an AI assistant a firm can defend to a client and one it cannot: the confidential material stays behind the firm’s own walls, no outside operator can reach it, and every answer traces back to a source. This page explains why sending privileged documents to a cloud LLM is a risk law firms should avoid, how on-premise RAG works over case files and case law, and what to check before you buy. It is general information, not legal advice.
Why shouldn’t law firms send privileged documents to a cloud LLM?
Law firms should not send privileged documents to a cloud LLM because doing so transmits confidential, privileged client material to a third party outside the firm’s control, which puts both confidentiality and privilege at risk. When a lawyer pastes a deposition transcript, a draft brief, or a client memo into a hosted chatbot, that content leaves the firm’s perimeter, may be logged or retained under terms the firm does not set, and can cross into a jurisdiction the client never agreed to.
For a law firm, three specific exposures make this hard to defend:
- Privilege risk. Attorney-client privilege and work-product protection rest on confidentiality being maintained. Disclosing privileged material to an outside AI operator can be argued to undermine the confidentiality the protection depends on.
- Confidentiality duty. A lawyer’s professional duty of confidentiality covers all information relating to a matter, not just what is filed. Routing that information through an uncontrolled endpoint is hard to square with that duty.
- Loss of custody. Once a document leaves the firm, the firm can no longer say with certainty where it was processed, who could read it, or whether it was retained.
The clean fix is architectural, not contractual: do not send the privileged material across the boundary at all. Teclops AI covers that reasoning in why your AI shouldn’t leave your walls.
What is air-gapped AI for legal work?
Air-gapped AI for legal work is an AI system that runs on an isolated network the firm controls, with no outbound connectivity, so the model, the case files, and every query stay inside one boundary. Nothing reaches the internet, no external API is called, and the assistant keeps working with the network cable unplugged.
In practice it has two layers. The model layer is an open-weight LLM whose weights sit on the firm’s own hardware. The retrieval layer is a RAG (retrieval-augmented generation) system that searches the firm’s own documents, briefs, contracts, and case law, then feeds the relevant passages to the model. When both run inside the isolated network, privileged content has no path out. For the general mechanics, see what air-gapped AI is and how air-gapped LLM deployment works.
How does on-prem RAG work over case files and case law?
On-premise RAG works over case files and case law by keeping retrieval and generation local: a question is answered from the firm’s own indexed documents, and the answer is grounded in the exact passages retrieved, never in data shipped to an outside model. The flow stays inside the firm:
- Local ingestion. Matter files, pleadings, contracts, precedent banks, and a case law library are ingested and embedded on the firm’s own hardware. The source files and the index never leave.
- Local retrieval. A lawyer asks a question in plain language, and retrieval finds the matching passages from local storage.
- Local generation. An on-site open-weight model composes an answer grounded in those passages, then cites them.
- Permission and audit. Access is enforced per user and per matter, and every query and answer is logged.
Because retrieval and inference both run locally, the assistant supports legal research and drafting without becoming a new route for client data to escape. Grounding answers in retrieved sources also directly attacks the fabricated-citation problem: the system answers from the firm’s real case law and flags when an answer is not there, rather than inventing a case that does not exist. It is a research and reference tool for licensed lawyers, not a substitute for legal judgment.
Hosted LLM API vs air-gapped on-prem RAG for law firms
The table below compares a hosted LLM API against air-gapped on-premise RAG on the axes a firm’s managing partner, general counsel, or IT lead actually evaluates.
| Property | Hosted LLM API | Air-gapped on-prem RAG |
|---|---|---|
| Where documents are processed | Provider’s servers, possibly cross-border | Inside the firm’s own network |
| Privileged content leaves perimeter | Yes, every request | No |
| Confidentiality control | Depends on provider terms | Firm keeps full custody |
| Used for model training | Depends on contract terms | No path to external training |
| Case citation accuracy | Can fabricate non-existent cases | Grounded in the firm’s real sources |
| Source traceability | Often none | Cited to the exact source passage |
| Access control | Coarse or none | Role-level and matter-level |
| Audit trail | Limited to what the provider exposes | Full, tamper-evident, internal |
The hosted API wins on convenience. The air-gapped deployment wins on every axis tied to privilege, confidentiality, and a defensible answer, which is why it is the sound posture for anything touching client matters. For the broader self-host decision, see on-prem vs cloud LLMs for sensitive data.
Which open-weight models can a law firm run on-site?
Law firms can run open-weight LLM families such as Llama, Qwen, Mistral, and Gemma entirely on-site, because their weights are downloadable and inference runs on local GPUs with no external API call. The firm owns the weights, serves them behind its own firewall, and updates them on its own schedule through controlled media.
For a legal deployment, screen every candidate on:
- License. Confirm the model permits commercial, self-hosted use with no conflicting clause. A model you cannot legally run inside the firm is disqualified regardless of accuracy.
- Long-document handling. Legal work runs on long briefs, contracts, and transcripts, so favor models that hold a long context and summarize accurately.
- Language coverage. Verify it handles the languages the firm works in, including Hindi and regional languages for Indian practice.
- Hardware fit. Filter to models whose weights fit the GPU memory you have at your expected concurrency.
The ordered method is in how to choose an open-weight LLM for on-premise use.
Why do lawyers need source-cited AI answers?
Lawyers need source-cited answers because a legal position cannot rest on an unverifiable or invented claim, and an AI that cannot show its source is not safe to rely on in practice. A source-cited system answers only from the firm’s own documents and case law and shows the exact passage behind each statement, so a lawyer can open the underlying case or file and confirm it before it reaches a client or a court.
Equally important is the refusal path. A trustworthy legal assistant says plainly when the answer is not in the firm’s sources rather than filling the gap with a plausible but fabricated citation. That is the difference between a reference tool and a professional liability. Teclops AI explains the mechanism in what source-cited answers actually mean, and the controls behind the deployment boundary in the Teclops AI security approach.
How does Samvad AI fit a law firm’s air-gapped stack?
Samvad AI is Teclops AI’s secure, source-cited RAG assistant that answers only from your own documents and deploys on-premise, air-gapped, or hybrid, switchable by configuration. For a law firm, that means lawyers can query matter files, briefs, contracts, and a case law library from inside the firm, with every answer citing the exact source passage and a clear statement when an answer is not in the sources. Samvad AI is permission-aware at the role and row level, so access can follow matter boundaries and ethical walls, is multilingual across English, Hindi, and more, and writes a tamper-evident audit log, so the firm can always answer where a document was processed and who could see it. Samvad AI is a research and drafting aid for licensed lawyers, not a source of legal advice. To see it on your own files, reach the Teclops AI team at teclops.ai@gmail.com.
Frequently asked questions
Does sending privileged documents to a public AI tool waive attorney-client privilege?
It can create a real risk. Attorney-client privilege depends on confidentiality being maintained, and disclosing a privileged document to a third-party AI provider outside the firm's control may be argued to undermine that confidentiality. Air-gapped AI sidesteps the question by never transmitting the document beyond the firm's perimeter. This is general information, not legal advice.
Can law firms use ChatGPT or hosted LLMs for case files and legal research?
For general, non-confidential research a hosted tool may be acceptable, but law firms generally should not paste client files, matter details, or privileged material into a hosted LLM, because that sends confidential data to a third-party endpoint the firm does not control. Firms that want AI over real matter content run it on-premise or air-gapped so nothing leaves their perimeter.
What open-weight LLMs can a law firm run on-premise?
Law firms can run open-weight LLM families such as Llama, Qwen, Mistral, and Gemma on their own GPUs, because the weights are downloadable and inference happens locally with no external API call. Confirm the license permits commercial self-hosted use, and weigh long-document handling and available hardware before deploying.
Can air-gapped AI do case law and precedent research?
Yes. An air-gapped RAG system can index a firm's own case law library, briefs, and precedent bank and answer research questions with citations to the exact passage, all inside the firm's network. Because it answers only from indexed sources and says when an answer is not there, it reduces the risk of a fabricated or non-existent citation.
How does on-prem RAG support a lawyer's duty of confidentiality?
On-premise RAG keeps client data inside the firm's own infrastructure, under its existing access controls, so no confidential material is transmitted to a third-party AI operator. This supports the duty of confidentiality and the duty of competence in using technology, though compliance always depends on the firm's full policy and supervision, not the software alone.