RAG Security in 2026: How to Stop Prompt Injection at Retrieval Time

Apache Struts, Okta, and the part of RAG people keep pretending is “just a data problem”

How do you stop prompt injection when the attack happens before the model ever sees a prompt? That’s the question worth answering, because once malicious text gets into retrieval, you are no longer defending a chatbot—you are defending a supply chain.

We already have the pattern. Apache Struts CVE-2017-5638 was “just” an input-handling bug until it became the entry point for the Equifax breach, with 147 million records exposed. Okta’s 2023 support-system incident showed the same old lesson in a newer costume: if an attacker gets into a trusted workflow, they don’t need to break crypto to do damage. RAG systems are now building the same kind of trust chain, except the payload is text, embeddings, and documents you let your own stack ingest.

Retrieval is the attack surface, not the model

Prompt injection at retrieval time works because most RAG pipelines treat documents as inert. They aren’t. A malicious PDF, wiki page, ticket, or synced Slack export can carry instructions that survive chunking, embedding, and retrieval. If your retriever surfaces that text into the context window, the model will often obey it more readily than your system prompt, because the model has no native concept of “trusted source” unless you build one.

This is not hypothetical theater. Anthropic’s 2024 responsible disclosure work demonstrated that models can be manipulated into assisting with dangerous tasks when safeguards fail. The important bit for you is not the headline; it’s the mechanism. The model didn’t “get hacked” in some cinematic sense. It was steered by content that changed its behavior. RAG just gives that steering wheel a database.

Poisoning the vector store is easier than breaking the model

You do not need a jailbreak if you can influence what gets indexed. That’s the ugly part. A compromised Confluence page, a poisoned SharePoint document, a malicious GitHub issue, or even a user-uploaded knowledge base article can become a retrieval-time payload. Once embedded, the text can be fetched by semantic similarity, not by exact match, which means your normal keyword filters miss the point entirely.

This gets worse in multi-tenant or loosely governed setups. If you index customer support transcripts into Pinecone, Weaviate, or Elasticsearch without strict provenance, a single injected document can contaminate answers across sessions. The vector store becomes a second-order trust boundary. And yes, this is the part where people say “we’ll just sanitize the input.” Great. Against what, exactly? A prompt injection payload is still just English with a bad attitude.

What actually works: provenance, segmentation, and retrieval-time policy

Start with provenance, not prompts. Every chunk should carry source metadata: document owner, creation channel, last modified time, ingestion path, and a trust score you compute, not one you copy from a dashboard. If a chunk came from an external upload, a support ticket, or a user-generated field, it should never have the same retrieval weight as an internal runbook or signed policy doc.

Then segment by trust domain. Do not mix customer-submitted content, internal operational docs, and vendor material in the same retrieval index unless you enjoy incident response as a lifestyle. Build separate indexes or at least separate namespaces with hard retrieval rules. A malicious document should not be able to outrank a curated policy just because it shares a few embeddings.

At retrieval time, enforce policy before generation. That means a gate that checks whether the retrieved text contains instruction-like content, references to secrets, tool calls, or attempts to override system behavior. You can use a classifier, but keep it boring: regex alone will miss obvious variants, and a pure LLM judge is expensive and flaky. The practical answer is layered controls—source trust, content heuristics, and a second-pass policy filter. Not glamorous. Effective anyway.

The contrarian bit: don’t rely on “prompt hardening” as your primary defense

A lot of advice says to make the system prompt stronger, more explicit, more defensive. That is not wrong; it is just incomplete enough to be dangerous. If your retrieval layer can surface arbitrary text, a stronger prompt is a speed bump on a highway. Useful, sure. Not where you put the brakes.

The better move is to reduce what the model is allowed to see. Summarize or transform untrusted content before retrieval, and strip imperative language from low-trust sources where possible. For example, a support transcript should be indexed as evidence, not as instruction-bearing prose. If you need verbatim text for auditability, keep it out of the primary retrieval path and fetch it only on demand with explicit user confirmation. That’s slower. So is cleaning up after a leak.

Tool use turns retrieval bugs into real-world actions

The risk jumps when RAG feeds agents that can send email, open tickets, modify configs, or query internal systems. A poisoned retrieval result can become a tool invocation if you let the model chain “answering” into “acting.” That is how a bad document turns into a bad change request, and then into an outage.

Put hard authorization checks outside the model. The model can propose actions; it should not be the authority that executes them. If an LLM suggests a Jira update, a PagerDuty page, or a GitHub PR, require deterministic policy checks on the exact action, target, and scope. This is the same lesson we learned from breach response years ago: trust the control plane, not the thing making the suggestion. Fancy language models do not get a free pass because they sound confident.

Detection needs to watch the retrieval layer, not just the chat log

You should log which chunks were retrieved, from where, with what similarity scores, and what downstream actions followed. If a model suddenly starts citing low-trust sources more often, or a single document repeatedly appears in failed or risky sessions, you have a signal. Most teams log prompts and responses and call it observability. That’s half a diary.

Look for changes in retrieval distribution, not just obvious malicious strings. Prompt injection often hides in benign-looking content that exploits instruction hierarchy, role confusion, or tool affordances. A document that causes the model to refuse policy, exfiltrate hidden context, or over-index on a single source is not “just relevant.” It is a canary with teeth.

The Bottom Line

Treat RAG as a trust pipeline, not a search feature. Separate sources by trust level, attach provenance to every chunk, and enforce retrieval-time policy before the model ever sees the text. If you let unvetted content into the context window, you have already lost the first round.

If the model can use tools, put authorization outside the model and log retrieval-to-action chains end to end. Otherwise you are one poisoned document away from turning your vector store into an attack path.

References

• Anthropic Responsible Disclosure: https://www.anthropic.com/news/anthropic-responsible-disclosure
• Apache Struts CVE-2017-5638: https://nvd.nist.gov/vuln/detail/CVE-2017-5638
• Okta Support System Incident Analysis: https://sec.okta.com/harfiles
• OWASP Top 10 for LLM Applications: https://owasp.org/www-project-top-10-for-large-language-model-applications/
• NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework