LLM02: Insecure Output Handling (XSS, Command Execution, SQL)

Description

Insecure output handling arises when model outputs—HTML, Markdown, commands, SQL, code—are executed or rendered without validation. This enables cross-site scripting (XSS), command execution, data corruption, or privilege escalation in downstream systems. LLMs can produce plausible but unsafe content; treat outputs as untrusted input.

SEO keywords: LLM insecure output handling, LLM XSS, auto-execute commands, unsafe SQL generation, output validation.

Examples/Proof

  • HTML/Markdown rendering (Stored/Reflected XSS)

    • Output contains <img src=x onerror=alert(1)> or <svg onload=alert(1)>. If the chat UI renders unsanitized HTML, an attacker can execute scripts. See Web XSS: src/web-owasp-top-10/injection/stored-xss.md.

  • Auto-run code/commands (CI/agents)

    • The agent suggests rm -rf /tmp/* and the pipeline runs it automatically. Replace with echo POC > /tmp/poc.txt to safely test; if it executes, auto-run is enabled.

  • Dangerous SQL generation

    • Model proposes DROP TABLE or wide UPDATE without WHERE clause. If an executor tool runs it, confirm with a read-only environment; the behavior indicates missing guards.

Detection and Monitoring

  • HTML sanitization logs and CSP violations
    • Enable CSP (no inline scripts); monitor violations; block dangerous tags/attributes at the renderer.
  • Command/SQL allow-list mismatches
    • Log rejected commands/queries; alert on attempts outside policy; require explicit approvals.
  • Shadow execution
    • Dry-run code and SQL in a sandbox; compare planned vs executed actions and flag destructive operations.

Remediation (Layered)

  1. Never auto-execute
    • Insert human approval or policy gates; require diff/preview for any file/DB changes.
  2. Sanitize and sandbox aggressively
    • HTML-escape user-visible content; render Markdown in a safe mode; execute code/SQL in isolated, least-privileged sandboxes.
  3. Validate with schemas and policies
    • Enforce JSON Schema/Pydantic for structured outputs; parse commands/SQL and check against allow-lists and resource constraints.
  4. Use read-only by default
    • Default tools to read-only; require opt-in elevation with justification and auditing.

Prevention Checklist

  • HTML/Markdown sanitized; CSP enabled; iframes/safe renderers used
  • Commands and SQL parsed and validated against allow-lists
  • Structured outputs validated with JSON Schema/Pydantic
  • Shadow/dry-run execution path for code and DB changes