Haxoris Wiki

Retrieval Ranking Manipulation

Description

A RAG answer is built from whatever survives the retrieval window, and that window is decided by mechanics an attacker can measure and compete against: cosine or dot-product distance in a known embedding space, a BM25 lexical leg, a fusion step merging the two, an optional cross-encoder reranker with its own top_n cut, chunk sizes and overlaps chosen at ingest, and often a recency or authority boost read off payload metadata. If you can write one chunk into the index, you can engineer it to occupy top-k for queries you choose, without the chunk containing a single instruction.

The payoff is control of grounding. The model answers from your text, cites it, and the authoritative chunk is simply absent from the context, so the failure presents as a confident, well-cited wrong answer rather than as an injection. Pushed further it becomes jamming: a blocker chunk engineered for a target query displaces the real source and the assistant claims the information is unavailable. It is easy to miss because evaluation measures answer quality on the questions the corpus was built for, and nobody diffs the retrieved id set before and after ingest. This page is about ranking mechanics only; durable corruption of the source corpus is the RAG Knowledge Base Poisoning page under LLM05, and instructions executing out of a retrieved chunk are the Indirect Injection Via Retrieved Content page under LLM01.

Examples

Fingerprint the encoder, the fusion and the reranker

You cannot optimise against a model you have not identified. Read the collection config and the schema.

# vector width narrows the encoder family: 384, 768, 1024, 1536, 3072
curl -s http://qdrant.internal:6333/collections/product-docs | python3 -c \
  'import json,sys; print(json.load(sys.stdin)["result"]["config"]["params"]["vectors"])'

# Weaviate names its vectoriser and reranker modules outright
curl -s http://weaviate.internal:8080/v1/schema | grep -Ei 'vectorizer|reranker|model'
curl -s http://weaviate.internal:8080/v1/meta | python3 -m json.tool | head -30

Confirmed when you can name the encoder checkpoint. Pull it from Hugging Face so candidate chunks can be scored offline before you upload anything.

Optimise a chunk for high similarity against a query bank

Write the queries you want to own, then hill-climb the chunk text against the local copy of the encoder, keeping the mean cosine over the whole bank rather than a single query.

from sentence_transformers import SentenceTransformer, util
m = SentenceTransformer("BAAI/bge-base-en-v1.5")
bank = m.encode(["what is our Q3 revenue projection",
                 "Q3 revenue forecast", "revenue guidance third quarter"],
                normalize_embeddings=True)
cand = "Q3 revenue projection. Revenue forecast third quarter. CANARY-1234."
print(util.cos_sim(m.encode(cand, normalize_embeddings=True), bank).mean())

Iterate: restate the query verbatim, append paraphrases and near-synonyms, and keep the chunk short so the pooled vector is not diluted. Confirmed offline when the mean cosine beats the best real chunk, then in the target when your chunk appears in top-k.

Win the lexical leg and the fusion step

Hybrid search blunts pure vector stuffing, so bait both legs. Weaviate reports how much each leg contributed, which makes tuning a loop:

{
  Get {
    Document(
      hybrid: {query: "Q3 revenue projection", alpha: 0.5,
               fusionType: relativeScoreFusion}
      limit: 10
    ) {
      title
      _additional { id score explainScore }
    }
  }
}

Qdrant’s equivalent is a Query API call with two prefetch branches - one dense, one sparse - merged by {“fusion”: “rrf”} or {“fusion”: “dbsf”}. Rank-based fusion rewards appearing in both lists at any position, so a chunk that is merely respectable on each leg beats one that is excellent on a single leg; score-based fusion rewards a dominant score instead. Read explainScore, adjust rare-term density and vector bait, resubmit. Confirmed when your id climbs while the authoritative id falls.

Flood near-duplicates and exploit boundaries, recency and the reranker

Upload several variants under distinct titles and source paths so deduplication and diversity filters do not collapse them, and put the bait at the head of each chunk so the overlap window carries it into the neighbour. If the pipeline boosts on a payload date field, set it forward. One variant, as a Qdrant upsert body against PUT /collections/product-docs/points:

{"points": [{"id": 9001,
  "vector": [0.011, -0.043, 0.377],
  "payload": {"title": "Q3 revenue projection (final)",
              "source": "poc-variant-a",
              "updated_at": "2026-12-01T00:00:00Z",
              "text": "Q3 revenue projection: CANARY-1234."}}]}

Flooding also defeats the reranker without beating it: a cross-encoder only reorders the candidate list handed to it, so if your near-duplicates fill the retriever’s candidate window the authoritative chunk is never scored at all.

Measure displacement, not presence

Capture the retrieved id set for each target query before and after ingest, from the retrieval trace - Langfuse spans or the citation list - and diff. The debug flag and the citation field names below stand in for whatever the target actually exposes.

for Q in "Q3 revenue projection" "revenue guidance third quarter"; do
  curl -s -X POST https://assistant.example.com/api/chat \
    -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
    -d "{\"message\":\"$Q (ref nonce-$RANDOM)\", \"debug\":true}" \
  | python3 -c 'import json,sys; print([c["chunk_id"] for c in json.load(sys.stdin)["citations"]])'
done | tee /tmp/poc-topk.log

Report the share of top-k held by your chunks, the rank the authoritative chunk fell to, and whether it left the window entirely. Four of five slots on an arbitrary query, with the real source at rank 11, is the result.

Remediation

  1. Cap single-source dominance in the window
    • Ceiling how many top-k slots one document, source URI or submitter may occupy, and fill the remainder from other sources.
    • Deduplicate at chunk level before embedding and use diversity-aware selection so near-duplicates cannot sweep the window.
  2. Keep untrusted content out of the same competition
    • Hold external and user-submitted material in a separate low-trust index, weighted down or excluded from grounded answers.
    • Normalise at ingest: strip zero-width characters, homoglyphs, invisible text and repeated query-term blocks.
  3. Screen ingest geometry
    • Reject vectors that sit unusually close to many unrelated frequent queries, and rate-limit documents per submitter.
  4. Harden the ranking configuration
    • Pin alpha, fusion algorithm, k and reranker top_n server-side, and size the reranker candidate window well above k so flooding cannot starve it.
    • Derive recency and authority from pipeline-observed values, never from attacker-controlled payload fields.
  5. Regression-test the retrieved set
    • Keep a golden query bank asserting named authoritative chunk ids stay in top-k, fail the build on displacement, and log the ids and scores behind every answer.