Enterprise AI vendor evaluation: a security and compliance checklist

Enterprise AI vendor evaluation is the due-diligence process a regulated buyer uses to confirm an AI or LLM vendor can handle sensitive data safely before signing. This checklist gives you seven groups of questions to ask any enterprise AI vendor, and for each question it shows what a good answer looks like and the red flag that should stop the deal. Use it as a procurement artifact: copy the questions into your RFP, send them to vendors, and compare the answers side by side.

This post is a procurement checklist for enterprise AI vendor selection. If you first need the underlying concepts, see AI data residency and sovereignty.

1. What is your data residency and deployment model?

Ask the vendor where your prompts and documents are physically processed, and which deployment models the product supports. The deployment model decides whether your data ever leaves your control.

• Ask: Can this run on-premise, air-gapped, or in our own VPC, or is it multi-tenant SaaS only?
• Good answer: On-premise, air-gapped, or single-tenant VPC deployment is available, switchable by configuration, with all processing inside infrastructure you control.
• Red flag: SaaS only, with prompts routed to a third-party model endpoint in another country and no self-hosted path.

Samvad AI deploys on-premise, air-gapped, or hybrid, switchable by configuration, so prompts and documents stay inside your walls. See how to deploy an LLM on-premise for the mechanics.

2. How is our data handled: training, retention, and deletion?

Ask whether the vendor uses your data to train models, how long they retain it, and how deletion works. Data handling is the single most common gap between consumer AI and enterprise-grade AI.

• Ask: Is our data, including prompts, embeddings, and outputs, ever used to train or fine-tune your models? What is the retention period, and how do we force deletion?
• Good answer: A written clause stating your data is never used to train shared models, with defined retention and verifiable deletion. In an air-gapped deployment, the no-training guarantee is architectural: your data physically cannot reach the vendor.
• Red flag: Training on customer data by default, opt-out buried in settings, vague retention (“as needed”), or no deletion guarantee.

3. How do you enforce access control?

Ask how permissions are enforced and whether the system integrates with your identity provider. Access control must live at the data layer, not in the user interface.

• Ask: Do you support role-level and row-level permissions, and SSO through our identity provider? Where are permissions enforced?
• Good answer: Permission-aware retrieval enforces role and row-level rules at query time, so a user only ever sees answers built from documents they are cleared to read, integrated with SSO.
• Red flag: A single shared index for all users, permissions handled only by hiding buttons in the UI, or no SSO.

Samvad AI is permission-aware at the role and row level, so the same question returns different answers to different users based on what each is cleared to see.

4. Does the system ground answers and cite sources?

Ask whether every answer is tied to a verifiable source and whether the system refuses when the answer is not in your data. An ungrounded AI invents confident, wrong answers.

• Ask: Does every answer cite the exact source passage? Will the system say “not found” rather than guess?
• Good answer: Each answer cites the specific source passage it was built from, and the system states plainly when an answer is not in your sources instead of fabricating one.
• Red flag: Free-text answers with no citations, or a model that always produces an answer even when your documents do not contain one.

For what a real citation should contain, see what source-cited answers actually mean.

5. What does your audit logging and explainability cover?

Ask what is logged and whether you can reconstruct any past answer. Auditors and regulators will ask you to explain why the system said what it said.

• Ask: Do you log prompts, retrieved sources, and responses per user? Is the log tamper-evident? Can we reconstruct a specific answer months later?
• Good answer: A tamper-evident audit log captures who asked what, which sources were used, and what was returned, so any answer can be reconstructed and explained.
• Red flag: Logs that capture only usage metrics, no record of which sources produced an answer, or logs the vendor can silently edit.

6. What is your security posture, and which standards do you align with?

Ask about the vendor’s security practices and certifications, and verify every claim with evidence. Treat unverifiable badges as marketing.

• Ask: What certifications do you hold, and can you provide the report? How do you handle encryption, secrets, and vulnerability management?
• Good answer: Named certifications backed by current reports you can review, encryption in transit and at rest, and a clear vulnerability disclosure process. A security-first vendor will also let your team review the deployment inside your own environment.
• Red flag: Claimed certifications with no report, or a refusal to let your security team inspect a self-hosted deployment.

Teclops AI is security-first and data-sovereign by design, with services delivered inside your own infrastructure. Confirm specific certifications in writing during procurement rather than assuming them.

7. What are our exit and portability rights?

Ask what happens to your data and configuration when the contract ends. Lock-in is a compliance risk, not just a commercial one.

• Ask: On termination, do we keep our data and embeddings in a portable format? Is the deployment ours to run independently?
• Good answer: Your documents, embeddings, and configuration export in open formats, and a self-hosted deployment continues to run in your infrastructure without the vendor.
• Red flag: Data trapped in a proprietary format, deletion the vendor controls, or a deployment that stops working the moment the contract lapses.

The enterprise AI evaluation checklist at a glance

Score each vendor on every group and require written answers, not sales claims.

#	What to verify	Pass looks like	Red flag
1	Deployment model	On-prem, air-gapped, or VPC	SaaS only, foreign endpoint
2	Data and training	No training, defined deletion	Trains by default, vague retention
3	Access control	Role and row-level at data layer	UI-only, shared index, no SSO
4	Grounding	Cites passage, refuses when unsure	Free-text, always answers
5	Audit logging	Tamper-evident, per-user, reconstructable	Usage metrics only, editable
6	Security posture	Named certs with reports	Unverifiable badges
7	Exit rights	Open-format export, runs without vendor	Proprietary lock-in

A vendor built for regulated workloads should pass all seven groups: on-premise or air-gapped deployment, no training on your data, permission-aware access, source-cited answers, tamper-evident audit logs, a verifiable security posture, and clean exit rights.

Samvad AI was built to these criteria. To run this evaluation against your own requirements, see our enterprise AI services or contact Teclops AI. This article is general guidance, not legal advice.