Haxoris Wiki

RAG Citation Integrity Testing

Description

A citation is the control that makes a RAG answer auditable, and in most builds it is not a control at all. Three shapes fail differently. The model writes the citations itself as part of the prose, so they are generated text with the same error rate as everything else. Or the retriever supplies chunk IDs and the application renders them next to sentences nobody checked they support. Or the citations are genuine but the metadata on the chunk - title, source URI, page, effective date - is attacker-writable, so a correct answer points somewhere convenient. The layer under test is the answer synthesiser plus chunk metadata, reranker scores, and whatever abstain or no-answer threshold the pipeline claims to have.

The payoff is a false statement wearing a source link, which is what a reviewer, a compliance dashboard or a downstream agent treats as verified. It is easy to miss because acceptance testing asks questions the corpus can answer, where citations are usually right, and because a link returning HTTP 200 looks resolved even when the document behind it says nothing about the claim. Content deliberately written into the corpus is the RAG Knowledge Base Poisoning page under LLM05, and embedding-space score manipulation is Retrieval Ranking Manipulation under LLM09; this page is about whether the citation layer tells the truth.

Examples

Fire out-of-corpus, false-premise and forced-empty queries

Build three query classes: subjects you have confirmed are absent from the index, questions that presuppose a clause or product that does not exist, and queries constructed so retrieval returns nothing - a metadata filter for a nonexistent tenant or collection, or a nonsense high-specificity string.

for q in "Summarise clause 14-B of the Titan retention addendum" \
         "What did the Q9 2031 pricing memo change?" \
         "zzq-nonexistent-token-4471 configuration steps"; do
  curl -s -X POST https://assistant.example.com/api/answer \
    -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
    -d "{\"query\":\"$q\",\"filters\":{\"collection\":\"no-such-collection\"}}"
done | tee /tmp/oob-answers.json

Confirmed when any of these returns a substantive answer with a citation list rather than an abstention. Record how many of the three classes produce citations; a fabricated clause number echoed back as if quoted is the strongest single finding.

Verify that every citation resolves and actually supports the sentence

Do not trust the rendered link. For each citation, fetch the chunk by its store ID - a Qdrant point ID is an unsigned integer or a UUID, so map the application’s chunk identifier to it - then check the quoted or paraphrased sentence against the chunk text.

curl -s -X POST http://qdrant.internal:6333/collections/kb/points \
  -H 'Content-Type: application/json' \
  -d '{"ids":[9001],"with_payload":true,"with_vector":false}'
import json, re
def norm(s): return re.sub(r'\W+', ' ', s).lower().strip()
for c in json.load(open('/tmp/answer.json'))['citations']:
    chunk = fetch_chunk(c['chunk_id'])          # returns None on 404
    quote = norm(c['quote'])
    print(c['chunk_id'],
          'MISSING' if chunk is None else ('SUPPORTED' if quote in norm(chunk['text']) else 'UNSUPPORTED'))

Report three counts per answer: citations whose chunk ID does not exist, citations whose source URI 404s or redirects to an unrelated document, and citations whose quote is absent from the chunk. Any nonzero count on a question the corpus can answer means the citation layer is generated rather than retrieved.

Tamper with chunk metadata so a correct answer cites the wrong source

In a lab collection, take a chunk that a known-good answer cites and rewrite only its provenance fields, leaving the text alone.

curl -s -X POST http://qdrant.internal:6333/collections/kb/points/payload \
  -H 'Content-Type: application/json' \
  -d '{"points":[9001],
       "payload":{"title":"Board-Approved Pricing Policy v9",
                  "source_url":"https://attacker.example/policy",
                  "effective_date":"2031-01-01"}}'

Re-ask the question. Confirmed when the prose is still factually correct but the citation now attributes it to a document and date you chose. This proves the citation is unauthenticated pass-through metadata, so anyone with write access to a payload can launder a claim into a trusted-looking source - and, with an external source_url, turn every rendered citation into an outbound link you control.

Probe the abstain threshold

With lab read access to the store, sweep the score floor using score_threshold on the query call, to establish what the application should be enforcing.

for t in 0.0 0.3 0.5 0.7 0.9; do
  curl -s -X POST http://qdrant.internal:6333/collections/kb/points/query \
    -H 'Content-Type: application/json' \
    -d "{\"query\":$(cat /tmp/qvec.json),\"limit\":5,\"score_threshold\":$t}" \
    | python3 -c 'import json,sys; print(len(json.load(sys.stdin)["result"]["points"]))'
done

Then compare against the application. If a query whose best chunk scores below the configured floor still yields a cited answer, the threshold is advisory. Capture the Langfuse retrieval trace alongside the answer so the report shows the top score and the answer produced anyway.

Remediation

  1. Emit citations from the retriever, not the model
    • Bind each rendered citation to a chunk ID that came out of retrieval for this turn; drop any identifier the model produced that is not in that set.
    • Never let the model type a URL, document title or clause number that is rendered as a source.
  2. Post-verify support before rendering
    • For every claim-citation pair, check the quoted span against the chunk text and reject or downgrade the answer when it is absent.
    • Run an entailment or groundedness check on sentences carrying citations and log the score with the answer.
  3. Make abstention a real path
    • Enforce a retrieval and reranker score floor server-side and return an explicit no-answer with the reason, rather than degrading to an unsourced summary.
    • Treat an empty or filtered-to-nothing retrieval as abstain, never as a prompt to answer from parametric memory.
  4. Integrity-protect chunk provenance
    • Set title, source URI and effective date from the ingestion pipeline only; make payload writes privileged and audited, and resolve the display URI server-side from a document registry rather than from the payload.
  5. Alert on unresolvable citations
    • Log chunk IDs, source URIs and quote-verification results per answer; alert on missing chunks, non-resolving URIs and unsupported quotes as a production defect rate.