Haxoris Wiki

Model Registry Provenance Bypass

Description

Provenance bypass is changing which artifact serves traffic without touching the application. The reference in code is a name, not a digest: a Hugging Face repo id with no revision, an MLflow URI such as models:/support-classifier@champion, an S3 prefix, or a mirror URL. Whoever controls resolution of that name controls the weights, the tokenizer and any repo-side Python shipped with them.

The payoff is a model the deployment believes it verified. The paths are mundane: typo and namespace squats on public hubs, re-registering a deleted organisation name (model namespace reuse), a revision that follows main, a mirror that rewrites a blob, or a CI identity that can move a production alias unreviewed. It is easy to miss because the pipeline stays green and the model card is unchanged - only the loaded digest reveals the swap. Load-time code execution is covered by the Unsafe Model Artifact Deserialization page; this page is about which bytes arrive.

Examples

Unpinned revision follows a moving branch

Inventory every model reference and flag calls with no revision, which resolve to the default branch at pull time.

from transformers import AutoModelForCausalLM

# unpinned: resolves to main on every restart
AutoModelForCausalLM.from_pretrained("acme-labs/support-classifier")

# pinned: immutable commit
AutoModelForCausalLM.from_pretrained(
    "acme-labs/support-classifier",
    revision="9f8b7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f1a0b",
)

In a lab hub, push a new commit to main, restart the serving pod and diff the resolved commit. New weights in production with no PR and no approval is the finding. The same applies to hf download without —revision.

Namespace and mirror interception

Check which namespaces the pipeline depends on are claimable, then check what the mirror actually serves.

# does the namespace in the pipeline config still publish anything?
curl -s 'http://hub.lab.internal/api/models?author=acme-labs&limit=5'

# does the exact reference still resolve?
curl -s -o /dev/null -w '%{http_code}\n' http://hub.lab.internal/api/models/acme-labs/support-classifier

# force resolution through a mirror you control, with mitmproxy in path
HF_ENDPOINT=http://mirror.lab.internal:8080 \
  hf download acme-labs/support-classifier --revision main --local-dir ./pull
sha256sum ./pull/*.safetensors

An empty author listing plus a reference that no longer resolves means the name is free for someone else to take; treat the status code as hub-specific, since huggingface.co answers 401 rather than 404 for a repo it will not disclose. If a swapped blob loads without error, signature verification is absent - confirm with model_signing verify, passing the expected signer via —identity and —identity_provider, and record that it fails or was never run.

Alias rewrite by an under-privileged CI identity

MLflow stages have been deprecated in favour of aliases since 2.9, and alias moves are what promotion depends on. Use the CI token, not an admin token.

from mlflow import MlflowClient
c = MlflowClient(tracking_uri="http://mlflow.lab.internal:5000")
c.set_registered_model_alias("support-classifier", "champion", "7")  # 7 = unreviewed build
print(c.get_model_version_by_alias("support-classifier", "champion").version)

Then confirm what the running server loaded rather than what the registry claims.

# /v1/models returns served model ids only, never a digest
curl -s http://serving.lab.internal:8000/v1/models

# so take the digest from the mounted artifact itself
kubectl exec deploy/serving -- sha256sum /models/current/model.safetensors

A digest that does not match the approved release record proves the promotion path is unverified.

Remediation

  1. Resolve names to digests
    • Pin every model reference to a commit sha or content digest in the deployment manifest and fail closed on a missing pin.
    • Reject branch names, latest and floating aliases in CI policy checks.
  2. Verify signatures in the loader
    • Verify OpenSSF Model Signing or Sigstore attestations, and the signer identity, at load time; refuse unsigned artifacts.
  3. Own the resolution path
    • Mirror approved digests into a private registry, block egress to public hubs from build and serving networks, and pin HF_ENDPOINT to the mirror.
  4. Split build from promotion
    • Give CI an identity that can create versions but not move production aliases, and log every alias change with actor and target version.
  5. Assert the running artifact
    • Emit repo id, commit sha and file digests at process start and alert on any difference from the approved release record.