6.1 AI for Defence · Module 6 · Adversarial Risks

Prompt Injection

The number-one LLM application risk — and why ‘trusted’ documents can attack a model.

Adversarial~12 min

The big idea

OWASP ranks prompt injection as LLM01:2025 — the top risk for LLM applications. Direct injection is a user typing malicious instructions; indirect injection hides instructions inside documents, web pages, calendar invites or images that the model later reads. Demonstrated cases include manipulated AI search results and hidden prompts in academic papers. Mitigations: constrain model behaviour, validate inputs, filter outputs, restrict tool/plug-in scope, and require a human in the loop for sensitive actions.

Key insight: Indirect prompt injection means a model can be hijacked by content it merely reads — a document, a web page, an image — not just by what the user types. Never let an unsupervised agent take sensitive actions on untrusted content.

What you'll see

Direct vs indirect prompt injection, and the mitigations.

How it works and how to defend

The attack and the mitigations. Tap to expand.

Key points to remember

Prompt injection is OWASP's #1 LLM risk (LLM01:2025).
Indirect injection hides instructions in documents, pages, invites or images.
A model can be hijacked by content it reads, not just user input.
Keep a human in the loop for any sensitive or irreversible action.