codebase-memory-mcp from source at our pin: 33 smoke checks passed, 3 failed

codebase-memory-mcp is a single C binary that indexes a repo into a local knowledge graph and answers structure questions over MCP, so a long agent session can query the call graph instead of walking the tree again. The project's GitHub description promises 99% fewer tokens. We pinned it, built it from source with the path upstream uses for its own smoke run, and ran its smoke invariants with no keys. At report time 33 invariants passed and 3 failed, and this lab measured no token savings at all.

The catch

The token multipliers you will see around this project (10x, 99%, 120x) are the vendor's figures. Our lab reports only what it ran, and it ran no token measurement.

Upstream's smoke.yml is opt-in and non-gating. It runs on manual dispatch or pushes to qa/smoke-** branches, so a green default branch says nothing about it. When our CI ran it at pin 494247225761, the binary built and three invariants failed: index-status, crasher-skipped-cli and hanger-skipped-cli. The lab now ships scripts/apply-smoke-invariants-pin-fix.py, which switches index_status to the flag form the daemon-side CLI expects and drops the crasher and hanger guards that upstream's gating pr-smoke job never runs. That is a patch to upstream's test script, and you should know it exists before you trust a green run.

The source build also needs Linux. On our Mac the native build stopped with a clang crash (reports/upstream-build.log), so the fixture-map test skipped locally with "binary unavailable". Upstream documents a Microsoft Defender false-positive class on some release artifacts, which is one reason we build from source at the pin and never use the curl installer.

What we ran (keyless)

export DISABLE_TELEMETRY=1 DO_NOT_TRACK=1
uv sync --locked --group dev
bash scripts/vendor-upstream.sh
uv run ruff check .
uv run pytest -q
python3 scripts/check-skills
python3 scripts/check-upstream-parity
bash scripts/upstream-cbm.sh          # Linux: source build + smoke invariants
uv run cbm-map tests/fixtures/tiny-repo --json

From reports/gate.json, run on October 14, 2026:

That CI result belongs to lab commit 056a8f6. The pin-fix script came after it, and later CI runs the same day, such as run 38519292875, finished green. The report file predates those runs, so regenerate it before quoting a pass.

How it is wired

The binary runs on your machine as an MCP server. It parses the repo with tree-sitter into a graph, and the agent calls MCP tools to search nodes and follow calls. Upstream says indexing and graph queries stay local, with a best-effort GitHub release update check after MCP initialize. The lab skill codebase-memory-map tells the agent to build the local map first and to hedge any vendor savings figure.

Minilab (open source)

Upstream is DeusData/codebase-memory-mcp: 46,590 stars, MIT, C, as of October 14, 2026. On a Linux machine, run bash scripts/upstream-cbm.sh and then uv run cbm-map tests/fixtures/tiny-repo --json to get the fixture node and hit counts this report could not record.

uv sync --locked --group dev && uv run pytest -q