Secrets In Prompt Trace Logs

Description

Most production LLM features have an observability tier behind them: a tracing project such as Langfuse, LangSmith or Helicone, a model gateway such as LiteLLM writing request logs to its own database, plus the usual APM and analytics sinks. These tools exist to capture the whole conversation, so by default they store the full prompt, retrieved chunks, uploaded attachment text, tool-call arguments, and whatever headers the SDK was handed. Masking is opt-in in the tracing SDK and message logging is on by default at the gateway, so a deployment that never decided retains everything.

That makes the trace store a second copy of every regulated record the assistant has touched, scoped for engineers rather than data subjects. The read paths are wide: one project-scoped key returns full history, public share links are unauthenticated URLs, and self-hosted instances are often reachable internally with open sign-up. Testing misses it because the application behaves correctly - the leak is one hop sideways, on a hostname that never appears in the product’s own docs.

Examples

Enumerate the observability tier

Work from the client bundle, the runtime environment and the egress. Tracing SDKs leave recognizable key prefixes and hostnames:

rg -n "langfuse|langsmith|helicone|pk-lf-|sk-lf-|lsv2_|api\.smith\.langchain" dist/
rg -n "LANGFUSE_|LANGCHAIN_|LANGSMITH_|HELICONE_|OTEL_EXPORTER" .env deploy/

Then run the feature through mitmproxy and watch for calls to cloud.langfuse.com, api.smith.langchain.com or api.helicone.ai. Any of those in the egress puts a trace tier in scope; a key in the bundle makes it reachable from outside.

Read a whole project with one key

Read keys are scoped to a project or workspace, never to a user, so a key from a bundle, a CI log or a verbose error page returns every prompt it covers. Langfuse uses Basic auth with the public key as username and the secret key as password:

curl -s -u "pk-lf-PLACEHOLDER:sk-lf-PLACEHOLDER" \
  "https://cloud.langfuse.com/api/public/traces?limit=5"

curl -s -X POST https://api.smith.langchain.com/api/v1/runs/query \
  -H "x-api-key: lsv2_PLACEHOLDER" -H 'Content-Type: application/json' \
  -d '{"limit":5,"is_root":true}'

curl -s -X POST https://api.helicone.ai/v1/request/query \
  -H "Authorization: Bearer PLACEHOLDER" -H 'Content-Type: application/json' \
  -d '{"filter":"all","limit":5,"offset":0,"sort":{"created_at":"desc"}}'

Send a request through the feature carrying the marker CANARY-1234, a dummy bearer token and a small attachment, then re-run the query. Marker, token and attachment text appearing verbatim in the stored input prove nothing is masked on the way in.

Harvest public trace share links

Langfuse and LangSmith both let a developer make one trace world-readable, and in Langfuse the flag is set from the SDK rather than the UI, so a share link can be created in code with nobody reviewing it:

rg -n "set_current_trace_as_public|set_trace_as_public|setTraceAsPublic|public=True|\"public\": *true" .

A shared Langfuse trace resolves at its ordinary path, cloud.langfuse.com/project//traces/, so the link is indistinguishable from an internal one; LangSmith shares sit under smith.langchain.com/public/ and are served by GET /api/v1/public//run. Open one from a clean browser profile: a rendered prompt with no login is the finding. Search the client’s ticket and chat history for the same paths, because that is where they get pasted.

Confirm the gateway retains unredacted prompts

LiteLLM sends full messages to its callbacks unless told otherwise. Check the three settings that matter:

litellm_settings:
  turn_off_message_logging: false      # default: prompts and responses go to callbacks
  redact_messages_in_exceptions: false # default: prompts reach Sentry on error
general_settings:
  store_prompts_in_spend_logs: true    # writes request content to the proxy database

Then read the spend log directly, remembering that request content also lands in proxy_server_request even when the messages column is empty:

SELECT request_id, messages, response, proxy_server_request
FROM "LiteLLM_SpendLogs" ORDER BY "startTime" DESC LIMIT 5;

A row containing CANARY-1234 confirms the gateway database is an unredacted prompt archive. Then force an upstream error and check the error tracker, not the HTTP response: with redact_messages_in_exceptions off the assembled messages array reaches Sentry as issue context, a third store on a third access-control model. What the client receives in that error belongs to the Reasoning Trace And Debug Leakage page.

Remediation

1. Mask before the data leaves the process
   • Use the tracing SDK’s masking hook so prompts, tool arguments and attachment text are redacted at trace creation, not by a downstream job.
   • Set turn_off_message_logging and redact_messages_in_exceptions at the gateway, and leave store_prompts_in_spend_logs off.
2. Never hand credentials to the trace tier
   • Strip Authorization headers and API keys from tool arguments and request metadata before recording, and treat any key seen in a trace as burned.
3. Lock down and inventory the trace stores
   • Inventory and revoke share links as a scripted sweep: LangSmith lists every shared entity at GET /api/v1/workspaces/current/shared, bulk-unshares with DELETE on the same path, and exposes per-run state at GET, PUT and DELETE /api/v1/runs/{run_id}/share. Rotate any key that has been in a client bundle or CI log.
   • On self-hosted deployments set AUTH_DISABLE_SIGNUP=true, keep the instance off internet-facing paths, and scope keys per environment.
4. Cap retention and scan continuously
   • Set short retention on trace and spend-log data, delete on the same schedule as the source records, and run secret and PII detectors against the trace backend itself.