What Your Enterprise LLM Keeps: Privacy Risks, Opt-Outs, and Compliance
When employees paste contracts, customer records, or source code into AI tools, vendors may retain prompts, outputs, and metadata far longer than your team expects—and opt-outs from training rarely stop all retention. This post explains what GDPR and CCPA actually require, how to verify a vendor’s data-use controls, and the deployment steps that reduce exposure before your next AI rollout.
Enterprise LLMs Retain More Than Your Users Think
In March 2023, Samsung engineers pasted source code and internal meeting notes into ChatGPT, then discovered the hard way that “free” AI tools can become an unplanned data retention system. That incident was noisy because it involved source code, but the real problem is broader: most enterprise LLMs keep prompts, outputs, and metadata long enough to matter, and “don’t train on my data” is not the same thing as “delete my data.”
If your rollout assumes the vendor only sees the text on screen, you are already behind. OpenAI, Microsoft, Google, Anthropic, and AWS all draw different lines between training, logging, abuse monitoring, and customer-controlled retention. Those lines are usually buried in product-specific terms, not in the cheerful marketing page your procurement team forwarded.
The Retention Problem Is Usually in the Logs, Not the Model
The common mistake is treating model training as the only privacy risk. It is not. In practice, the prompt and response often land in application logs, abuse-detection queues, support tickets, telemetry pipelines, and sometimes human review systems before anyone asks whether the model itself was trained on them.
Microsoft Copilot for Microsoft 365, for example, is contractually separated from your tenant data for foundation-model training, but that does not mean your prompts vanish on contact. Microsoft still processes prompts and responses to deliver the service, and the surrounding Microsoft 365 compliance boundary depends on the workload, tenant settings, and whether you are using consumer Copilot, Copilot Studio, or a Microsoft 365 workload with its own retention rules. That distinction matters when legal asks why a deleted conversation is still discoverable in eDiscovery.
OpenAI’s business products have their own split: API and Enterprise customers are generally not used for training by default, while consumer ChatGPT has historically relied on user settings and account type. Anthropic and Google make similar promises in their enterprise offerings, but each vendor’s data-use terms, retention windows, and abuse-review practices differ enough that “we use an enterprise plan” is not a control. It is a billing tier.
GDPR and CCPA Care About Purpose, Minimization, and Deletion
GDPR does not care that your vendor has a shiny AI badge. Article 5 still requires data minimization, purpose limitation, storage limitation, and integrity/confidentiality. If an employee pastes customer PII into an LLM to draft an email, you now have a processing purpose to document, a retention period to justify, and a cross-border transfer story to explain if the vendor routes data outside the EEA.
Article 28 is where a lot of teams get sloppy. If the LLM vendor is a processor, you need a Data Processing Agreement that spells out subject matter, duration, nature, and purpose of processing, plus the categories of data and the obligations around deletion and subprocessors. If the vendor cannot tell you how long prompts and logs persist, your DPA is theater.
CCPA/CPRA is less ceremonial but just as annoying. California users have rights to know, delete, and opt out of certain sharing and selling, and service-provider / contractor language only helps if the vendor actually honors it. The trap is assuming “no training” equals “no retention.” It does not. A vendor can still retain data for security, debugging, fraud prevention, or service improvement, and that may be lawful if disclosed. Lawful, however, is not the same as acceptable for your risk model.
The Vendor Questions That Expose the Real Control Surface
Ask for the retention schedule in writing, not the sales deck. You want the default retention period for prompts, outputs, logs, embeddings, and admin audit records; whether deletion is immediate or asynchronous; whether backups are included; and whether support personnel can access content after deletion. “Up to 30 days” is not a retention policy if the vendor also keeps security logs for 180 days and support transcripts indefinitely.
Then ask a less fashionable question: can the vendor prove tenant isolation at the datastore level? Many AI features sit on top of shared services, and the meaningful control is not “we don’t train on your data” but whether one customer’s prompts can bleed into another customer’s logs, search indexes, or cached results. If the answer is hand-wavy, assume the architecture is too.
You also want the abuse-monitoring story. Vendors routinely reserve the right to inspect content for policy violations, malware, credential theft, or abuse. That is not exotic. It is how they keep their service from becoming a phishing factory. But if the vendor cannot tell you whether humans review content, under what triggers, and how long reviewed records persist, then your “opt-out” is mostly decorative.
The Deployment Controls That Actually Reduce Exposure
The best control is still not letting people paste crown-jewel data into a third-party prompt box. That sounds obvious until someone in Legal uploads a draft acquisition agreement to summarize redlines. Put the LLM behind an enterprise gateway or broker that can redact secrets, PII, and source code patterns before the request leaves your network. Netskope, Zscaler, and Microsoft Purview all have ways to inspect and classify content before it hits an external service, and that front-end control is worth more than a dozen policy memos.
Separate use cases by data class. A customer-support drafting assistant does not need access to raw CRM exports, and a code assistant does not need production database credentials. Use scoped workspaces, per-team tenants, and identity-based access controls so the model only sees the minimum corpus required for the task. If the same account can access HR complaints, source code, and M&A documents, you do not have an AI strategy; you have a data amalgamation problem.
Contrarian point: blocking all external LLMs is not automatically the safer move. Teams that ban the tools outright often drive usage into personal accounts, browser extensions, and copy-paste workflows that leave even less auditability. A controlled rollout with logging, redaction, and approved vendors is usually less risky than the shadow-AI swamp your policy created out of spite.
Verify the Claims Before the First Production Prompt
Do not accept a security whitepaper as evidence. Ask for the vendor’s SOC 2 report, DPA, subprocessors list, retention settings, and any independent attestations for data isolation. Then test the controls yourself with a canary prompt containing a fake but recognizable string, such as a synthetic customer ID or seeded secret, and verify where it appears in logs, dashboards, and support exports.
Also check whether your own environment is the weak link. If prompts are copied into ticketing systems, SIEMs, or browser histories, the vendor is not your only retention problem. A lot of “AI privacy incidents” are really internal logging mistakes with a chatbot attached.
The Bottom Line
Inventory every approved LLM path, then map what each one retains: prompts, outputs, embeddings, audit logs, and support records. If the vendor cannot give you a retention window and deletion method in writing, do not let it near customer data or source code.
Before the next rollout, enforce redaction at the gateway, segment by data class, and run a canary test to confirm what survives in logs and exports. If Legal cannot point to a DPA, retention schedule, and deletion workflow for the exact product in use, the answer is no until they can.