agent-device passed 7 upstream CI gates at our pin, and doctor still says warn

agent-device gives coding agents a CLI, an MCP server and a typed Node API for driving iOS, Android, HarmonyOS, TV, web, macOS and Linux targets. We pinned agent-device@0.21.27, ran seven gates from upstream's own CI at commit 1658d637d64f, and collected doctor, devices and typed-client output as sha256-stamped evidence, all with no keys. Every CI job that ran passed, and doctor still returned warn because the machine had no phone tooling.

The catch

The keyless lane drives the device a CI runner has, which is a Linux desktop on ubuntu and simulators or the macOS desktop on a Mac. It does not touch a real phone. doctor --json reported success: true with status warn and the summary "No hard blockers found, but warnings need attention." The warnings are missing adb, hdc and Apple tools. To test Android or iOS hardware you install adb or Xcode and re-run doctor until the device checks pass.

The local evidence in our report came from a Mac. The full Linux evidence (doctor.json, devices.json, client.json and manifest.json with .sha256 sidecars) is an artifact of the CI lab job, and those files are gitignored. The optional web lane, which downloads a browser backend through web setup, runs only on manual dispatch, and it was skipped in this run. At the pin, upstream's src/ has no dedicated telemetry opt-out variable, so the lab sets DO_NOT_TRACK=1 and DISABLE_TELEMETRY=1 and relies on the web backend staying out of the default path.

What we ran (keyless)

bun install --frozen-lockfile
DISABLE_TELEMETRY=1 DO_NOT_TRACK=1 bun test --max-concurrency=1
DISABLE_TELEMETRY=1 DO_NOT_TRACK=1 bun run collect
bash scripts/upstream-ci.sh           # needs network, pnpm 11.17.0, Node 24.13

scripts/upstream-ci.sh runs pnpm install --frozen-lockfile and then these gates, copied from upstream's .github/workflows/ci.yml: typecheck, build, bundle-owner-files, unit-ci, tmpdir-leaks, integration-node and concurrency-torture.

From reports/gate.json, run on October 14, 2026, for lab commit a9c64f0 and CI run 38521651624:

On the ubuntu runner, the README records one device, with platform: "linux", id: "local", target: "desktop" and booted: true.

How it is wired

agent-device sits between the agent and a device you control. The agent runs doctor first, then devices, then drives the target through the CLI, the MCP server or createAgentDeviceClient, and closes the session in teardown. bun run collect writes each result with its sha256, byte size, tool version and timestamp into out/evidence/manifest.json. The lab skill device-evidence asks the agent to cite those evidence paths in its final report, because "a bare screenshot is not verification."

Minilab (open source)

Upstream is callstack/agent-device: 5,016 stars, MIT, TypeScript, as of October 14, 2026. Run bun run collect on your own machine and read out/evidence/manifest.json, then install adb or Xcode and re-run doctor to see which warnings clear.

bun install --frozen-lockfile && bun test --max-concurrency=1 && bun run collect