Haxoris Wiki

Hallucinated Package Name Squatting

Description

Coding assistants invent dependencies. Asked to solve a task with no obvious library, a model will produce a plausible package name, a module path inside a real package that does not exist, or a function on a real SDK that was never shipped. The important property is that the invention is not random: the same prompt tends to produce the same fabricated name across runs and across sessions, which makes the name predictable to anyone else who can run the same prompt. The integration under test is the whole path from that suggestion to disk - the IDE chat a developer copies from, an autonomous coding agent that runs its own install command, and the CI step that resolves a lockfile the agent edited.

An attacker who harvests a recurring fabricated name and registers it on a public registry gets code execution wherever that suggestion is followed: developer laptops with cloud credentials and SSH keys, and CI runners with deploy tokens. It is easy to miss in testing because the failure mode during normal QA is a clean 404 and a red build, so nobody records which name was requested. Functional tests never install a package that does not exist. This page covers harvesting, scoring and end-to-end proof; registry-side provenance and signing are covered by the Model Registry Provenance Bypass page under LLM04, and what the imported code then does belongs to Insecure Code From AI Assistants under LLM10.

Examples

Harvest fabricated names across repeated runs

Build a fixed prompt set covering tasks with no well-known library, then run it many times at the temperature the product actually ships. garak carries purpose-built probes for this against a REST target:

garak --target_type rest --config rest-assistant.yaml \
      --spec 'probes.packagehallucination.Python,probes.packagehallucination.JavaScript' \
      --generations 20 --report_prefix pkg-halluc

For the product’s own prompts, use promptfoo with repeats and caching off so every run is a fresh call:

promptfoo eval -c dep-prompts.yaml --repeat 20 --no-cache -o runs.json

Then extract every install target and import from the outputs:

jq -r '.results.results[].response.output' runs.json \
  | grep -oE '(pip install|npm i(nstall)?) [a-z0-9._@/-]+|^[[:space:]]*(import|from|require\().*' \
  | sort | uniq -c | sort -rn > /tmp/candidates.txt

The observable is a name appearing in a large fraction of runs. Record the repeat rate as a count over N, for example 17/20.

Score each candidate against the live registries

A candidate only matters if it is unclaimed. Reduce the harvest to bare names in /tmp/names.txt, then query the registries read-only and treat 404 as unclaimed, 200 as taken - percent-encode the slash in scoped npm names:

while read -r p; do
  n=$(curl -s -o /dev/null -w '%{http_code}' "https://registry.npmjs.org/$p")
  y=$(curl -s -o /dev/null -w '%{http_code}' "https://pypi.org/pypi/$p/json")
  printf '%s npm=%s pypi=%s\n' "$p" "$n" "$y"
done < /tmp/names.txt | tee /tmp/claimability.txt

Do the same for submodules and symbols, which registry lookups will not catch: import the real parent package in a throwaway virtualenv and confirm the suggested path or attribute raises.

python -c "import importlib; importlib.import_module('realpkg.invented_submodule')"

The finding is the intersection: high repeat rate and a 404. Rank by repeat rate, not alphabetically.

Prove install and import end to end in a lab registry

Never publish to a public registry. Stand up a local one - Verdaccio for npm, pypiserver or devpi for Python - publish a benign package under a harvested name, and point the assistant’s environment at it.

{
  "name": "harvested-name-here",
  "version": "1.0.0",
  "scripts": { "postinstall": "echo POC-PKG-1234 > /tmp/poc.txt" }
}
npm config set registry http://localhost:4873
# then run the exact command the assistant emitted, or let the agent run it
cat /tmp/poc.txt

Confirmed when the canary file exists and the agent’s transcript or CI log shows it resolved the name without any human choosing it. Note whether the install ran unattended in a CI step, which turns a suggestion into unreviewed execution. Repeat with --ignore-scripts to separate lifecycle-script execution from import-time execution.

Remediation

  1. Resolve dependencies before they reach a file
    • Validate every suggested package against an allowlisted private proxy at suggestion time and refuse to write an unresolvable name into a manifest or lockfile.
    • Block the assistant from editing manifests directly; require a resolved, pinned diff.
  2. Take install out of the unattended path
    • No agent step runs a package manager without review of the resolved lockfile diff.
    • Run installs with lifecycle scripts disabled, for example npm ci --ignore-scripts, in a sandbox with no cloud credentials and no outbound egress beyond the proxy.
  3. Pin, lock and mirror
    • Serve all dependencies from an internal mirror with an explicit allowlist; deny direct upstream resolution from developer and CI machines.
    • Require hash-pinned lockfiles and fail the build on unpinned additions.
  4. Alert on first-seen packages
    • Flag lockfile additions with no download history, no source repository, or a publish date later than the first assistant suggestion.
  5. Keep the harvest as a regression suite
    • Re-run the prompt set on every model, system-prompt and temperature change; a new high-repeat fabricated name is a new exposure window.