New node receives task before runner CLIs
Found by the matrix sweep (Task 11113) on dev EC2, 2026-10-29.
On a freshly installed self-hosted node (Ubuntu 22.04 arm64, `t11113-u2204-armb`), the first task dispatched to its vnode failed with:
`broken session: crashed — the ACP engine never started a session (spawn_failed): the engine 'claude-agent-acp' did not start: FileNotFoundError: [Errno 2] No such file or directory: 'claude-agent-acp'`
The vnode's runner-CLI install (`jaah-runner-cli-vn@<tag>.service`) logged `installed @agentclientprotocol/claude-agent-acp@0.93.0` at 11:44:37, after the task had already been placed. Restarting the task works. A user's first task on a new node, though, fails as a "broken session" with nothing telling them it was too early.
Ask: do not place a task whose engine the node does not have yet. Either placement checks the engine against the node's reported `runner_clis`, or the node reports its slots only once its runner CLI install has converged. If a launch does meet a missing engine, re-queue it with a readable wait reason instead of failing it.
Out of scope: the CLI pins themselves.
Done means:
- A test proves a task is not placed on (or is re-queued from) a node whose `runner_clis` lacks the task's engine.
- On a fresh dev node, a task started right after the node comes Online waits, then succeeds once the CLIs land.
Outcome
Placement now waits ("installing runner tools on <node>") rather than placing an ACP run on a Linux node that lacks the engine; it dispatches once the engine is attested. Quick gate and 419 tests green. PR #7336 queued to the merge lane. Live fresh-node check and deploy watch not done. Memory: none applied.
Artifacts
Activity
- Created
- Run
- Runner started
- Runner ended (done)