Air-Gapped AI for Hospitals: PHI Stays On-Site
Air-gapped AI lets hospitals query patient records and clinical guidelines locally while no PHI leaves the building. Here is how on-premise RAG makes it work.
Hospitals can run modern AI without patient data ever leaving the building by deploying an air-gapped, on-premise RAG system: clinicians query records and clinical guidelines locally, and no protected health information (PHI) reaches a public AI endpoint. This page explains why patient records cannot go to hosted AI tools, how on-premise retrieval keeps them inside the hospital, and what to check before you buy.
Why can’t hospitals send patient data to public AI endpoints?
Hospitals cannot send patient data to public AI endpoints because doing so transfers protected health information to a third party outside the hospital’s control, which breaks confidentiality obligations, consent limits, and data-residency rules. When a clinician pastes a discharge summary or lab report into a hosted chatbot, that PHI leaves the hospital perimeter, may be logged or retained by the provider, and can cross borders into jurisdictions the patient never consented to.
Health data is among the most sensitive categories of personal data. Under India’s Digital Personal Data Protection (DPDP) Act and comparable regimes elsewhere, personal data must be processed on a lawful basis, normally the patient’s consent or one of the Act’s narrow “legitimate uses,” and only by parties the hospital remains accountable for. A public AI endpoint is neither. The failure mode is not hypothetical: it is a routine copy-paste by a well-meaning clinician who just wants a faster answer.
What is air-gapped AI for healthcare?
Air-gapped AI for healthcare is an AI system that runs entirely on an isolated hospital network with no outbound connectivity, so the model, the patient records, and every query stay inside one controlled boundary. Nothing reaches the internet, no external API is called, and the system keeps working with the network cable unplugged.
In practice it has two layers. The model layer is an open-weight LLM whose weights sit on the hospital’s own GPUs. The retrieval layer is a RAG (retrieval-augmented generation) system that searches the hospital’s own documents, records, protocols, and guidelines, then feeds the relevant passages to the model. When both run inside the isolated network, PHI has no path out. For the general mechanics, see what air-gapped AI is and how it works.
How does on-premise RAG let clinicians query records safely?
On-premise RAG lets clinicians query records safely by keeping retrieval and generation local: a question is answered from the hospital’s own indexed documents, and the answer is grounded in the exact passages retrieved, never in data shipped to an outside model. The flow stays inside the hospital:
- Local ingestion. Records, care protocols, formulary data, and clinical guidelines are ingested and embedded on hospital hardware. The source files and the index never leave.
- Local retrieval. A clinician asks a question in plain language, and retrieval finds the matching passages from local storage.
- Local generation. An on-site open-weight model composes an answer grounded in those passages, then cites them.
- Permission and audit. Access is enforced per user, and every query and answer is logged.
Because retrieval and inference both run locally, the assistant supports clinical decisions without becoming a new route for data to escape. It is a retrieval and reference tool for licensed staff, not a substitute for clinical judgment.
How does a public AI endpoint compare to air-gapped on-prem RAG for hospitals?
Public AI endpoints and air-gapped on-premise RAG differ on exactly the axes a hospital’s security and compliance leads care about.
| Property | Public AI endpoint | Air-gapped on-premise RAG |
|---|---|---|
| Where PHI goes | Leaves the hospital to a third party | Stays inside the hospital network |
| Data residency | Provider’s regions, may cross borders | Hospital’s own data center |
| Consent alignment | Hard to bound to the patient’s consent | Kept within existing consent and policy |
| Answer grounding | General model knowledge, may hallucinate | Only the hospital’s records and guidelines |
| Source citation | Usually none | Exact source passage per answer |
| Access control | Coarse or none | Role-level and row-level |
| Audit trail | Outside your control | Tamper-evident log you own |
| Works offline | No | Yes |
For a fuller treatment of the self-host decision, see on-prem vs cloud LLMs for sensitive data.
Which open-weight models can hospitals run on-site?
Hospitals can run open-weight LLM families such as Llama, Mistral, Qwen, Gemma, and Phi entirely on-site, because their weights are downloadable and inference runs on local GPUs with no external API call. These models are the practical foundation for air-gapped clinical AI: the hospital owns the weights, serves them behind its own firewall, and updates them on its own schedule through controlled media.
Two constraints decide the shortlist. First, the license must permit commercial, self-hosted use with no conflicting clause, because a model you cannot legally run inside a regulated environment is disqualified regardless of accuracy. Second, the model must handle your clinicians’ and patients’ languages, which in Indian hospitals often means English and Hindi alongside regional languages. The ordered method is in how to choose an open-weight LLM for on-premise use.
Why do clinicians need source-cited answers?
Clinicians need source-cited answers because a clinical decision cannot rest on an unverifiable claim, and an AI that cannot show its source is not safe to trust near patient care. A source-cited system answers only from the hospital’s own records and guidelines and shows the exact passage behind each statement, so the clinician can verify the source before acting.
Equally important is the refusal path. A trustworthy clinical assistant says plainly when the answer is not in the hospital’s sources rather than filling the gap with plausible-sounding but ungrounded text. That is the difference between a reference tool and a liability. We cover why this matters in what source-cited answers actually mean.
What should you check before buying air-gapped clinical AI?
Use this checklist when evaluating any air-gapped AI for a hospital, diagnostics lab, or health system. Each row states what good looks like and the red flag.
| Criterion | What good looks like | The red flag |
|---|---|---|
| Data egress | No PHI leaves the network in any mode | Any telemetry or API call carrying record content |
| Deployment | On-premise, air-gapped, or hybrid by configuration | Cloud-only, or air-gap sold as a separate product |
| Grounding | Answers only from your records and guidelines | Falls back on the model’s general knowledge |
| Citation | Exact source passage shown per answer | Answers with no traceable source |
| Refusal | Says clearly when the answer is not in sources | Guesses to avoid an empty answer |
| Access control | Role-level and row-level permissions | One shared view of all records |
| Audit | Tamper-evident log of every query and answer | Logging that can be edited or is absent |
| Language | Handles your clinicians’ and patients’ languages | English-only |
Treat the checklist as a scorecard across vendors so the tradeoffs stay visible.
How does Samvad AI fit hospitals and health systems?
Samvad AI is Teclops AI’s secure, source-cited RAG assistant that answers only from your own documents and deploys on-premise, air-gapped, or hybrid, switchable by configuration. For a hospital, that means clinicians and staff can query records, protocols, and guidelines from inside the building, with every answer citing the exact source passage and a clear statement when an answer is not in the sources. Samvad AI is permission-aware at the role and row level, multilingual (English, Hindi, and more), and writes a tamper-evident audit log, which is why the Teclops AI security approach treats the deployment boundary as the first control. It does not make diagnostic or certification claims; it keeps sensitive health data where it belongs and makes it usable. To see it on your own records, reach the team at teclops.ai@gmail.com.
Frequently asked questions
Can hospitals paste patient records into public AI chatbots like ChatGPT?
As a rule, no. Pasting patient records into a public AI chatbot sends protected health information to a third-party endpoint outside the hospital's control, which in most jurisdictions conflicts with health-data confidentiality obligations and the consent basis the data was collected under. Hospitals that want AI on real records without that exposure run it on-premise or air-gapped, so no PHI leaves their perimeter.
Does on-premise AI help hospitals comply with the DPDP Act?
On-premise and air-gapped AI supports compliance with health-data laws such as India's DPDP Act by keeping personal data inside the hospital's own infrastructure, under its existing access controls and consent framework. It removes the cross-border transfer and third-party processing that public AI endpoints introduce, though compliance always depends on the hospital's full policy, not the software alone.
What open-weight LLMs can a hospital run on-premise?
Hospitals can run open-weight LLM families such as Llama, Mistral, Qwen, Gemma, and Phi entirely on-site, because their weights are downloadable and inference happens on local GPUs with no external API call. The right choice depends on the license, the hospital's languages, and the GPU memory available, so verify the license permits commercial self-hosted use before deploying.
Does air-gapped clinical AI keep working when the network is down?
Yes. An air-gapped system runs the model, the document index, and every query inside an isolated hospital network with no outbound connectivity, so it keeps answering with the internet cable unplugged. Updates arrive only through controlled physical media, which is also why PHI has no path to an outside service.
How do clinicians verify an AI answer drawn from patient records?
A source-cited RAG assistant shows the exact passage behind each statement, so a clinician can open the underlying record or guideline and confirm it before acting. It also says plainly when the answer is not in the hospital's sources, rather than guessing, which keeps it a reference tool rather than an unverifiable claim.