LLM02: Sensitive Information Disclosure (Leaked Context, Logs, Weights)

How LLM02: Sensitive Information Disclosure (Leaked Context, Logs, Weights) works

Sensitive information disclosure happens when an LLM integration hands regulated, privileged or proprietary data to a recipient never authorized to see it. The answer text is only one channel: retrieved chunks, tool-call arguments, reasoning traces, observability traces, gateway request logs, embedding exports and measurable properties such as latency, token length and log-probabilities are all disclosure surfaces. Two structural failures drive most findings - oversharing upstream, where a legacy ACL feeds the index with data the retriever then returns exactly as designed, and persistence, where data that has reached weights, adapters or embeddings stays extractable after the source document is deleted.

The 2026 edition keeps this entry at LLM02 and widens it rather than renaming it. Disclosure is framed across four lifecycle phases - training-time memorization in base models and LoRA adapters, inference-time context exposure, pipeline-time carryover into fine-tunes and telemetry, and observation-time inference from side channels - and severity turns on what the recipient can learn, not on whether the leak looked like natural language. Open-weights deployments cannot rely on rate limits, since extraction, membership inference and inversion run offline, and persistence makes the same finding an erasure problem under GDPR Article 17. The 2026 text assigns embedding-disclosure mechanisms to LLM09 and keeps only the regulatory consequence here; read LLM01 for the injection that triggers a leak and LLM08 for system prompt and reasoning-trace recovery.

Keywords: rag tenant isolation, prompt logging, training data extraction, pii leakage, trace redaction, retrieval authorization

Examples/Proof

• Cross-tenant retrieval
  • Widen the tenant filter on the intercepted retriever call, then ask for a summary. Foreign content in uncited prose proves authorization runs after retrieval. See the Cross-Tenant RAG Retrieval Leakage page.
• Unredacted trace storage
  • Send a request carrying the marker CANARY-1234 and a dummy bearer token, then read the observability project through its API. A verbatim match proves masking is off. See the Secrets In Prompt Trace Logs page.
• Weight-resident records
  • Run divergence and prefix-completion probes against a fine-tuned endpoint and diff against the base model. Verbatim records only in the tuned output prove adapter memorization. See the Training Data Memorization Extraction page.
• Redaction-layer bypass
  • Upload a PDF whose content is hidden only by a black rectangle over an unmodified text layer, then ask for a summary. Recovered text proves visual redaction is not redaction. Attacker-supplied text hidden the same way is injection, and belongs to LLM01.

Detection and Monitoring

• Retrieval attribution
  • Log the chunk ids entering each prompt with the caller identity; alert when a served chunk fails a replayed authorization check.
• Trace-store scanning
  • Run secret and PII detectors over the observability backend, not only the application. Alert on new public share links.
• Extraction patterns
  • Alert on repeated-token prompts, high-volume near-duplicate prefixes, and requests for log-probabilities or echoed prompts.

How to fix and prevent LLM02: Sensitive Information Disclosure (Leaked Context, Logs, Weights)

1. Authorize inside the index query
   • Build the retrieval filter server-side from a session-bound identity, not from a client-supplied claim list.
2. Mask before persistence
   • Apply masking in the tracing SDK and disable message logging at the gateway so prompts, attachments and tool arguments never reach the trace store intact.
3. Govern the training corpus
   • Classify, deduplicate and scrub PII before fine-tuning, seed canaries, and gate release on measured extraction rates.
4. Gate observable signals
   • Disable log-probabilities, echo and verbose error payloads in production, and budget queries per user and per session.

Prevention Checklist

• Per-tenant index isolation with retrieval-time authorization enforced server-side
• No secrets or regulated data in system prompts or tool arguments
• Trace and gateway logging masked, access-controlled, retention-capped, public sharing disabled
• Extraction and canary-recovery probes run as a release gate

Last updated 2026-08-19