Security fundamentals for AI-powered applications

If you build anything on top of an LLM — a chatbot, a copilot, an agent that reads tickets and calls tools — you are building a web application with a new class of input-handling bug at its core. The good news: the discipline is already written down. The OWASP Top 10 for LLM Applications (2025) is the canonical list, and OWASP followed it with a Top 10 for Agentic AI Applications in late 2025 (overview).

The one bug to understand first: prompt injection (LLM01)

Prompt injection has held the #1 spot for two consecutive editions, for a structural reason: LLMs process instructions and data in the same channel. There is no privilege boundary between "the system prompt" and "the document the user uploaded" — the model sees tokens. An attacker crafts input the model interprets as a new instruction, and the model often complies because it genuinely cannot tell the difference (Aembit's explainer).

The nastier variant is indirect injection: the malicious instruction arrives inside content your app ingests — a scraped web page, a PDF, a support ticket, a calendar invite — not from the chat box. Any feature that lets a model read third-party content and then act is an injection surface.

The rest of the LLM Top 10, in plain terms

• LLM02 Sensitive information disclosure — the model leaks training data, other users' data, or your secrets via context. What the active identity can't authorize must not surface.
• LLM03 Supply chain — compromised models, datasets, plugins, dependencies. Your app is only as trustworthy as everything flowing through it.
• LLM04 Data and model poisoning — tampered fine-tuning data becomes tampered behavior.
• LLM05 Improper output handling — the classic web bugs, re-entering through the model: if LLM output flows into HTML, SQL, shell commands, or templates unsanitized, you have XSS/SQLi/command injection with extra steps. Treat model output exactly like user input.
• LLM06 Excessive agency — the agent has more tools, broader permissions, or fewer approval gates than the task requires. This is how a chat feature becomes a data-exfiltration feature.
• LLM07 System prompt leakage — system prompts are not secrets. Never put credentials or security-critical logic in one.
• LLM08 Vector and embedding weaknesses — RAG pipelines need the same access control as the data they index; poisoned or cross-tenant embeddings reach the context window.
• LLM09 Misinformation — confident wrongness; mitigate with grounding, citations, and human review where stakes are high.
• LLM10 Unbounded consumption — inference is expensive; unthrottled endpoints are a denial-of-service and wallet-drain surface.

What agents add (OWASP Agentic Top 10)

When the model stops answering and starts doing, new failure modes appear: uncontrolled autonomy (AG01 — acting without approval gates), insecure tool integration (AG02), delegated identity abuse (AG03 — the agent impersonates users or escalates through tool chains), audit gaps (AG07), and cross-agent prompt injection (AG09 — malicious instructions propagating between agents). The theme is identical to decades of distributed-systems security: least privilege, explicit authorization, and logging (framework summary).

Seven defenses you can apply this week

1. Sanitize at the sink, not the model. Escape or validate LLM output wherever it meets HTML, SQL, a shell, or a template. You cannot prompt your way out of LLM05; classic input-handling discipline is the fix.
2. Least-privilege tools. Give the agent the minimum scopes it needs. Read-only by default; write access per-tool, per-task.
3. Human approval for irreversible actions. Deletes, payments, sends, deploys — gate them. This single control blunts most of AG01 and LLM06.
4. Separate identities. The agent's credentials are not the user's credentials. Delegated actions carry the user's authorization, checked server-side (AG03).
5. Rate-limit and cap. Per-user token budgets, request throttles, timeouts (LLM10).
6. Don't trust retrieved content. Label ingested documents as untrusted data in the prompt, apply access control to your vector store (LLM08), and consider architectural patterns like Simon Willison's dual-LLM approach for high-risk flows (references).
7. Log everything the agent does. Actions, tools, arguments, approvals. If you cannot reconstruct what happened, you cannot respond to an incident (AG07).

“The model is a new kind of input parser. Everything you already knew about untrusted input still applies — it just arrives wearing a conversational interface.”

Where to practice

None of this should stay theoretical. The PortSwigger Web Security Academy has free, legal, hands-on labs for the classic web vulnerabilities that LLM05 resurrects — and a growing set of LLM-attack labs. It is already wired into the LearnPath security roadmap as a primary resource. Learn the classics first: an XSS payload that flows through a model is still an XSS payload.

RRRTX Labs