MicroVM session launch and identity

Task 5804 · Done · Coder · created · updated

Epic 5777 — MicroVM nodes. This child is the architecture's **C6: Session launch, accounts, identity**. Depends on the images/rail-executor child (provision flow, `microvm_session.prepare` stub, `vms.json`, guest §4.5 contract). Upstream: PRD task 5778 `prd`, Architecture task 5779 `architecture`. ## Original requirement (verbatim) Epic Goal 2: "The VM sees only what its task needs: its own worktree, its own config dir, and the token directory of the ONE account it runs under. Nothing else on the host, no sibling task, no other account, no NodeLink store, no EFS (port 2049), no instance metadata service." Goal 3: "NodeLink works: a NodeLink subscription OAuth account can run the task, the CLI's refreshed token is written back to the org's store, and the refreshed token is visible from other nodes. The current API-key-only limit on environments is removed." Goal 4: "Task identity comes from the VM itself (the per-VM vsock socket jaahd created), never from the guest's own claims, a uid, or process ancestry. Every control call (vocabulary tools, credential broker, jaah-forge, git askpass) works unchanged from inside the guest." Goal 8: "The gate commands run inside the VM; the task's commits reach `origin` and the push guards and hooks still apply. The leave-nothing-behind settle check still holds." Operator I9: "Every provider the fleet runs". I11: every session on a MicroVM node runs in its own VM. PRD R4: "The guest can read and write only its own worktree, its own config dir, and the token directory of the one account it runs under. It cannot see the rest of the host, a sibling VM, another account's token dir, the NodeLink mount as a whole, EFS (TCP 2049), or 169.254.169.254." R5: "Any account kind the fleet supports runs in a VM: subscription OAuth (local or NodeLink) and API key. For a NodeLink account, a refresh the CLI performs inside the guest lands in that account's file in the org's store and is readable from another node." R6: "The control plane learns a guest's task only from the per-VM vsock endpoint jaahd created for that VM. Nothing the guest sends, no uid and no process ancestry can change it. From inside the guest, the control MCP tools, `jaah-call`, the credential broker, `jaah-forge` and git's credential helper all work with the same commands and outcomes as on a static runner, and none of them can act for another task." R10: "The project's gate commands run inside the guest. Commits reach `origin`. The push guards, the forge-CLI deny, the Stop gate and the settle clean-tree check behave as they do on a static runner: a push to the integration branch is refused, and a settle with uncommitted or unpushed work is refused." ## Approach (settled) - **D9 worktree:** per-session host clone shared over virtio-fs (`cache=auto`). Node-local bare mirror per project `<state>/mirrors/<project>.git`, fetched by the daemon's system git login (as `ensure_clone` does). `git clone --local <mirror> <vmdir>/worktree`; reset `origin` to the forge URL; create the task branch; `install-access-guard.py` installs the pre-push hook. Every git dir inside the share (no `git worktree add`). `.env` seeding follows `create_worktree` minus the port band (no grid, R8). - **D10 config dir:** `<vmdir>/config` built from the account's layer — settings, plugins, hooks — converged exactly as `fleet_dispatch` does for a static launch. No token. Transcripts land there and are collected before destroy. - **D11 token dir** shared rw at `/jaah/token`: NodeLink account → `shared_storage.resolve_shared_auth_dir(slug)` (the only EFS path the VM sees); local account on a MicroVM node → `$JAAH_HOME/auth/.tokens/<slug>/` (same layout; the account-login flow on a MicroVM node writes there). Claude gets `CLAUDE_SECURESTORAGE_CONFIG_DIR=/jaah/token`; Codex/Gemini credential file in `/jaah/config` is a symlink into `/jaah/token` (as `shared_token_link`). API key, if any, from the broker over vsock. Every account kind = token dir + optional broker ask (no API-key-only restriction). **Risk K6:** EFS access point forces uid 3000:3000 (`shared_storage.py:12-13`) — `--translate-uid` must map guest 1000 → the host uid the NFS client presents; verify a guest write lands before claiming AC-2. - **D12 toolchain share (ro, identical paths):** `/usr/lib/jaahd`, `/usr/lib/skynetd`, `$HOME/.local/share/claude`, `$HOME/.local/lib/node_modules`, `<state>/toolchain/node`. Never `$HOME` itself, `$JAAH_HOME`/`~/.skynet`, `/etc/jaah`, or any secret. Implement `microvm_session.prepare(intent) -> Prepared{worktree, config, token, report, launch_dir, toolchain[]}` (replace C5c's stub) — the share list is the whole guest surface; test that the spec builder emits nothing else. - **Launch:** enact the `launch` intent (§4.2: `LAUNCH_KEYS` for runners/advisors, `{console_cmd}` for console sessions running `clauded_wt`, `{kind:"script", tool}`): write `launch/launch.sh` (+ `.settings.json`, `.sys`, `.task`) atomically (temp + rename) — writing it starts the session. Fill `runner_session_id` in `vms.json`. Works for Claude, Codex and Gemini. - **D13 identity:** `cred_broker` gains caller kind `VmCaller(vm_id)`, built from fds microvmd passes over `control_fds` (SCM_RIGHTS, tagged vm_id) for port 1024; resolve `vm_id → vms.json → task id / console session`. Request bodies, uids and `/proc` ancestry are never consulted for a VM caller. Guest `/run/jaah/creds/sock` (the path `jaah_cli.py` already dials) is forwarded by the base image's socat unit, so `jaah-call`, MCP control tools, `jaah-credential`, `jaah-forge` and `skynet-git-credential` work unchanged. - C4 emits the launch payload in parallel — test against the pinned §4.2 shape with fixtures. ## Acceptance criteria (verbatim) - AC-2 Token write-back. Oracle: during AC-1, force a token refresh inside the guest. On a second node mounting the same org store, read the account's `.credentials.json` modification time and expiry. Observe that they match the refresh. **Fails if** the second node still shows the pre-refresh token. - AC-3 Every provider. Oracle: run one real task each with Codex and with Gemini in a VM on the m8i node, through to `done`. **Fails if** any provider is refused or needs an API-key-only path that the others don't. - AC-5 Identity cannot be forged. Oracle: from guest A, call a control tool for its own task and see it succeed. Craft a request that claims task B (forged id field, fake uid, or a process tree that mimics B) and see it refused, and nothing written to B. **Fails if** anything in the guest's request changes the resolved task. - AC-6 Control calls unchanged. Oracle: inside the guest, `jaah-forge pr view`, a `git push` of the task branch (uses the credential helper), `jaah-call task.get` and a `task_artifact` upload all succeed, with no command changed versus a static runner. - AC-9 Guards hold. Oracle: from the guest, `git push origin HEAD:develop` is refused. A settle with an uncommitted file is refused. The Stop gate runs. (Here: build + unit/integration tests proving each mechanism; the live runs are C12's.) Credentials & reach: unit/integration tests — none beyond the worktree. Live AC-2/3/5/6/9 — dev console as `jaah/debug/main/operator-login`, the existing dev NodeLink account + second node (Q8), dev Codex/Gemini accounts (Q9) — **deferred** to C12 (dev reach refused until fleet egress is added, Q7). AC-16 — forge via `jaah-forge` — **verified**. Out of scope: no guest agent (N3); no tmux (N4); do not change static runner launch (`fleet_dispatch` static path), static worktrees, port grid, co-located nodes (N1). Do not keep or reintroduce `env_registry`/SO_PEERCRED env identity (C1 deletes it; N7). No console relay/answer delivery (C7), no egress relay (C8). Epic: "retiring static worktrees, co-located nodes (Multi-Node Hosts), the port grid or user slots; Mac/Windows hosts; GPU passthrough; any change to pricing beyond moving products to KVM-capable instance families." Done means: - `prepare` builds clone (origin = forge URL, task branch, pre-push hook installed), config dir (no token), token-dir path per account kind, toolchain list — tests; spec-builder test asserts no share outside the fixed tags/paths and none containing `$HOME`, `$JAAH_HOME`, `/etc/jaah`. - Pre-push hook in the clone refuses `HEAD:develop` — test. - `VmCaller`: a connection on VM A's listener resolves to A's task regardless of a forged task id/uid in the body — test; nothing written to B. - Launch writes `launch.sh` atomically for runner, console and script payloads; Claude/Codex/Gemini env + token links correct — tests. - K6 handling documented and tested (uid translation config). - `./dc check full` green on the PR; PR auto-merged and the Jaah Deploy for dev succeeds (AC-16).

Outcome

C6 merged (#5110): VM session prepare/launch, VmCaller identity, K6 owner map, guard protection. CI green; dev deploy green (f62a283). Full gate: 2 pre-existing docs stages red (untouched files). Live ACs deferred to C12.

Pull request

Artifacts

Activity

  • Draft
  • Ready
  • Runner started
  • Runner ended (done)