Reasoning Trace And Debug Leakage

Description

An AI feature ships far more to the browser than the sentence the user reads. The transport is usually server-sent events or a websocket, and the frames carry whatever object the server-side SDK produced: reasoning or thinking blocks, tool-call names and their argument JSON, retrieved chunk text with document ids and scores, citation objects richer than the rendered footnote, gateway metadata, and token usage. Error paths add a layer, since a framework traceback or an echoed provider error frequently contains the assembled messages array. The shipped client contributes the rest: prompt fragments compiled into the JavaScript bundle, a source map, a reasoning-visibility flag, and the same strings inside mobile assets.

This is the cheapest hidden-context recovery available, because there is no refusal to defeat and no probe to tune - the material is already on the wire, addressed to you. Testers miss it because they assess the rendered interface, and the framework or SDK helper collapses the stream into one final string, so nothing in the UI hints at the fields that were discarded. Reasoning visibility is also configuration-dependent: providers gate it with a display or summary setting, so an empty thinking field on one route says nothing about another route, another model, or the same route after a config change.

Examples

Read the raw stream instead of the UI

Put the feature behind mitmproxy or Burp Suite to capture the app’s own request, then replay it with the stream flag set and keep the frames. Ask a question that forces both a retrieval and a tool call, so every field class appears in one capture.

curl -sN -X POST https://app.example.com/api/chat \
  -H "Authorization: Bearer $LAB_TOKEN" -H 'Content-Type: application/json' \
  -H 'Accept: text/event-stream' \
  -d '{"stream":true,"messages":[{"role":"user","content":"Which internal policy applies to a refund over the approval limit?"}]}' \
  | tee stream.log \
  | rg -n '"(reasoning|reasoning_content|thinking|summary|system|instructions|tool_calls|arguments|context|chunks|score|debug|trace_id)"'

Do not stop at a grep list. Enumerate every JSON path the endpoint emits, so an undocumented field still shows up:

rg -o '^data: (.*)$' -r '$1' stream.log | rg -v '^\[DONE\]$' | python3 -c '
import sys, json
paths = set()
for line in sys.stdin:
    try:
        obj = json.loads(line)
    except Exception:
        continue
    stack = [("", obj)]
    while stack:
        p, v = stack.pop()
        if isinstance(v, dict):
            for k, w in v.items():
                stack.append((p + "." + k, w))
        elif isinstance(v, list):
            for w in v:
                stack.append((p + "[]", w))
        else:
            paths.add(p)
print("\n".join(sorted(paths)))'

Any path the interface never renders is a candidate finding. Repeat for the websocket route and for the mobile client’s request, since clients get different verbosity. Then pull the structured fields out of the same capture:

rg -o '"(name|arguments)":"(\\.|[^"\\])*"' stream.log | head -20
rg -o '"(document_id|doc_id|chunk_id|source|namespace|score|distance|filter)":[^,}]*' stream.log | sort -u

Argument JSON arrives fragmented across frames, so concatenate the deltas before reading them. Reassembled, they expose the model’s view of the tool schema plus the values it chose - internal path templates, tenant column names, generated filter expressions. Retrieval debug objects expose chunk text, document identifiers and scores, including for chunks retrieved and never cited. The finding is that this plumbing reaches the client at all; whether the chunks crossed a tenant boundary belongs to the Cross-Tenant RAG Retrieval Leakage page.

Force verbose error payloads

Break the request shape in several ways and read what the integration returns.

for body in '{"messages":[{"role":"user","content":null}]}' \
            '{"messages":[],"temperature":9}' \
            '{"messages":[{"role":"user","content":"hi"}],"model":"does-not-exist"}'; do
  echo "--- $body"
  curl -s -X POST https://app.example.com/api/chat \
    -H "Authorization: Bearer $LAB_TOKEN" -H 'Content-Type: application/json' \
    -d "$body" | head -c 900; echo
done

Look for three observables: a traceback naming the framework and the prompt-template module, a validation error echoing the assembled messages array with the system entry intact, and an upstream provider error returned with the full forwarded request body. Drive an overlong input and a mid-stream abort too, since timeout and truncation handlers are the paths least likely to have been reviewed. The trace-store side of the same prompt retention is covered on the Secrets In Prompt Trace Logs page.

Pull prompts and debug flags out of the shipped client

Fetch the bundle and any source map, then search the assets the same way you would a mobile binary.

rg -n "You are |You must |systemPrompt|system_prompt|instructions:|<\|im_start\|>" dist/ -g '*.js' -g '*.map'
rg -n "debug|verbose|showReasoning|showThinking|showTrace|internalOnly|__DEV__" dist/ -g '*.js'

unzip -o app-release.apk -d apk-src >/dev/null
rg -n "You are |systemPrompt|reasoning|thinking" apk-src/assets apk-src/res

A full instruction block in the bundle is hidden context exposed with no request at all. Treat a client-side visibility flag as a control to test, not a note: flip it in the browser console or in the storage key it reads, re-issue the request, and compare the frames. If the reasoning field was on the wire with the flag off, the gate is cosmetic.

Remediation

1. Return a response contract, not the provider object
   • Map the provider response to an allowlist of fields at the edge, dropping reasoning, tool-call arguments, retrieval scores and gateway metadata by default.
   • Add a contract test that fails the build when a new field appears in the streamed or non-streamed response.
2. Suppress reasoning at the source
   • Where the provider offers a reasoning-visibility setting, set it explicitly to the non-exposing value on the server rather than relying on a client flag.
3. Make errors opaque
   • Return a generic message plus a correlation id, disable framework debug pages in production, and strip request bodies from any error the client can see.
4. Keep prompts and flags server-side
   • Assemble instructions server-side, remove prompt strings from client bundles and mobile assets, and do not publish production source maps.
   • Delete debug and verbosity switches from release builds instead of defaulting them off.
5. Monitor what leaves the edge
   • Alert on responses carrying reasoning, tool-argument or chunk-score keys, and on any completion matching a shingle of the assembled system prompt.