Run MicroVMs natively on macOS nodes

## Epic requirement (verbatim) "Implement microVM on Windows and MacOS" (Task 10021 PRD ask #1, `in_scope`). Operator decisions that bind every child (verbatim from the PRD/Architecture): - D1 **Full parity** with Linux. Each session gets its own Linux guest kernel. The CP egress allowlist is enforced on the node. Control calls are vsock-bound (or an equivalent per-VM channel). No NodeLink inside the VM. No root in the guest. The disk is retained on `system_failed`. Console, sizing and placement UX are the same. - D2 **All three:** fleet runners, advisor chats and human console sessions. D3 **The Linux guest's set** of providers. D4 macOS = Apple Silicon, Windows = x64. D5 cloud Windows too. D7 same plan feature (`billing_plans.microvm_allowed` + `orgs.microvm_allowed_override`). - AD1 Windows = WSL2 + the existing Cloud Hypervisor/KVM runtime (Windows 10 refused). AD2 Mac guests arm64 only, no Rosetta. AD3 Windows test PC = `win-t8703`. AD5 probe m8i.large exactly; metal (m7i.metal-24xl) is the fallback. AD6 Mac oracles deferred until the Mac is enrolled on dev. AD7 AC2(a) live is dropped; unit tests map every `microvm_prereq` code to its sentence. **Quality bar (verbatim).** Every refusal shown to an operator is one plain sentence naming what is missing and how to fix it (`docs/standards/ui-copy-standards.md`). No generic 400, no stack trace, no Linux-specific wording on a Mac or Windows node. Boot-to-runner latency on Mac and Win is reported in the build's evidence. It is not gated, but it must not be degraded by more than 2× against a Linux MicroVM node of similar size without the Architect recording why. ## This child: C3 — macOS runtime integration (Architecture §3.2 vsock transport, §3.4, §4.3 verification, §9.3 spec, §3.5) Requirements this child serves (PRD §4, verbatim): - R2. Sessions in VMs. On a marked Mac or Windows node, every runner, advisor and console session runs in its own VM with its own Linux guest kernel. Each session is a `microvms` row, never a `worktrees` row. - R3. Parity of contract. Every bullet of `docs/policies/microvm-nodes.md` holds on these OSes: sizing resolver and source display; placement admission by free RAM/disk and hypervisor usability, with readable wait reasons, and `failing_nodes`; the intent/fact rail; lifecycle: destroy at done, reset, delete or console end; retain on `system_failed` for `microvm_retention_hours`; liveness reclaim; the egress allowlist: default ∪ project additions − removals, plus the CP and forge hosts, with private, link-local and node-own addresses refused and each refusal logged; per-VM control-call binding: a guest can act only on its own task; no root and no `sudo` in the guest, with a rootless engine; no NodeLink mount in a VM; the API-key sign-in served without a key in the guest environment. - R4. Providers. Inside a VM, a Mac or Windows node runs every provider an Ubuntu MicroVM node runs. Placement and the account picker must not refuse such a provider because of the host OS when the session lands in a VM. The host-session rules stay as the ADR-0145 table states for unmarked nodes. - R6. Install and update. The macOS installer and the Windows service install and converge everything the VM runtime needs: binaries, images and helpers. The A2 prerequisite is the only manual step. Updating a node keeps its VMs' contract. - R7. Linux unchanged. Linux MicroVM behaviour, the static pool and the unmarked Mac/Windows behaviour do not regress. - R8. Docs describe the present. `docs/policies/microvm-nodes.md`, ADR-0129 (including the per-OS boundary statement), ADR-0145 / `node_platform.py` docstring, `apps/jaahd/README.md` (MicroVM section) and the macOS/Windows packaging docs describe the new state. Facts are single-homed, and `python3 tools/check_context.py` passes. The full settled Architecture (Task 10022) and PRD (Task 10021) are the epic's prerequisite artifacts; read them with `task_read`/`artifact_read` — §3 and §9.3/§9.4 in full — before building. Section numbers (§) refer to the Architecture. ### Approach (settled, realize it) Builds on the `jaah-vz` helper, the arm64 images/`disk_backend()`, the egress core seam and `guest_clis()` from the Foundation children. - **microvmd VMM backend switch** (`apps/jaahd/microvmd.py`): on macOS build the `jaah-vz run --spec` argv instead of CH's; spec schema (`microvmd.py:268-275`) gains `egress: "tap" | "vsock"` (absent = `tap`, Linux unchanged); on macOS `egress=vsock`, `tap`/`mac` null, ports 15001 and 15053 listed in `vsock_ports`. Wait for the ready line with the same boot timeout as CH's API socket; read the console pty from `<vmdir>/console.pty` (not `vm.info`); `stop {grace_s}` = SIGTERM, wait, SIGTERM, then SIGKILL. Winsize path (`microvmd.py:978-981`) unchanged. - **Process model**: LaunchDaemon `ai.jaah.microvmd` (`UserName=_jaahd`) runs `microvmd.py serve`; `jaahd-launch` (root) loads it when the marker exists and unloads it when the marker goes away (no root converge on macOS). `systemctl` calls in microvmd and `microvm_net` go behind `host_platform.unit_start/stop(name)`. - **vsock egress transport** (second caller of `relay_flow`/`answer_dns`): host side accepts vsock fds on 15001/15053 via microvmd's existing `control_fds` path (VM identified by the listener, not the body); header ASCII `"<dotted IPv4> <port 1-65535>\n"`, ≤ 64 bytes, read within 5 s, else close + refusal `bad_header`; non-IPv4 → refusal `ipv6`; DNS TCP-framed (RFC 1035 §4.2.2) on 15053. - **Guest side / seed** (`microvm_seed.py` gains `egress`, set from host OS: Linux and WSL `tap`, macOS `vsock`): with `vsock` the seed writes no network-config, a dummy `jaah0` (default route, `198.19.N.2`), guest nft rules (`tcp redirect to 127.0.0.1:15001` for the session uid; DNS to `127.0.0.53:53`), and the root-owned 0755 `jaah-egress-tunnel` unit (small Python, socat-free, in the base image) that reads `SO_ORIGINAL_DST` per flow and opens vsock 15001 with the header. - **Share uid**: `microvm_seed.guest_ids()` replaces `GUEST_UID = 1000` (`microvmd.py`) and `MAPPED_ID` (`microvm_liveness.py:39`): 1000 on Linux, `_jaahd`'s uid/gid on macOS; prepare makes the token dir `_jaahd`-owned (the K6 `token_owner` map, `microvm_session.py:39-43`, reduces to that). Risk R-b fallback: if VZ refuses mode-0600 token files to the mapped uid, copy the token dir into the VM dir with write-back on refresh — record which path holds. - **Lift the Linux-only gates for macOS** (otherwise no session ever runs on a Mac): `apps/jaahd/main.py:1173,1196` (facts + `runner_clis` from `guest_clis()` while marked — this is the path for the macOS `runner_clis` report) and `:1515` (rail), `:1896` (egress relay start); `reconcile/enact_signal.py:302` (`_bound_vm`); the "Linux-only machinery, off the macOS/Windows import graph" imports in `kill_gates.py:216`, `answer_delivery.py`, `console.py:528,1400,2150`; `microvm_host.py`'s tap reservation via `microvm_net` (skipped when `egress=vsock`); `microvm_facts.py:25`'s `host_platform.linux` import (go through `host_platform`). Windows stays gated here (its path is the bridge, the Windows-backend child's; file overlap at `main.py:1173,1196` — whichever merges second keeps both branches). Keep the import graph clean for an UNmarked node. - **Facts**: `free_ram_mib` from `host_platform.macos.read_mem` (free + inactive + purgeable), `free_disk_mib` from `statvfs`, `kvm_usable` from `jaah-vz probe` (keep the wire keys; generalise the docstring). - **`linux_arm64` verification**: a CI job on an arm64 Linux runner installs each provider's runner CLI as the arm64 base seed would; set each provider's `linux_arm64` flag (API + jaahd mirror) from that result (confirms or flips the Antigravity=False default). The arm64 layer build fails readably when a provider install fails. - Docs: the macOS packaging doc and the macOS ADR-0129 per-OS boundary paragraph (§3.5: each session has its own Linux guest kernel under Apple's hypervisor, through Virtualization.framework, with no guest network device; a VZ/`jaah-vz` escape reaches `_jaahd`, and through it every VM and the node credential; `jaah-vz` runs under the hardened runtime, not under seccomp/landlock — never more). Acceptance criteria this child serves (PRD §5, verbatim): - AC6 — Egress is enforced (R3). Verify: In a VM session: `curl -sI https://github.com` → succeeds; a host not on the project allowlist → refused; the node's own LAN IP and `169.254.169.254` → refused. Add the host to the project's egress additions (ProjectEditPage) → a new session reaches it. Each refusal is visible in the node's egress refusal log/report. - AC8 — The provider set is the Linux guest's (R4). Verify: On Mac (host supports only Claude Code), run a task with a Codex account (and one other non-macOS provider an Ubuntu MicroVM node runs) → it is placed in a VM and runs. On an UNmarked Mac, the same account is still refused with the ADR-0145 sentence. - AC15 — Linux and static behaviour unchanged (R7). Verify: `./dc check full` is green. On dev, a Linux MicroVM node still passes AC3. An unmarked Mac and Win still run a static task as before. - AC17 — Pins are licence-gated per architecture (A3). Verify: `python3 tools/microvm_pins.py` (the licence gate) passes with the arm64 guest images/binaries pinned beside the x86_64 ones. Ownership note (Planner): on-hardware Mac oracles (AC1–AC12, AC14 on Mac) belong to the Mac-verification child (AD6: no enrolled Mac yet). This child proves everything CI and integration tests can, so the Mac-verification child is verification, not the real build. Credentials & reach (Architect §8, verbatim rows): - AC1–AC12, AC14 on Mac | The operator's Apple Silicon Mac node on dev | **deferred** (AD6: "enroll it later"): the dev Nodes list with "All orgs" checked shows 9 nodes and none is macOS. C3 runs its Mac oracles once the Mac is enrolled. The build is not blocked, because C3's unit-level work and CI do not need it. - Build (C3 signing) | GitHub environment `macos-signing` secrets used by job `node-sign-macos` (`APPLE_DEVELOPER_ID_P12` and the others, named in `jaah-deploy.yml:18-22`) | **verified (by design)**: CI-held. No runner fetches them. If `APPLE_TEAM_ID` is unset, dev builds are ad-hoc signed with the entitlement (§3.1). - AC15 gate, AC16, AC17 | Worktree only (`./dc check full`, `tools/check_context.py`, `tools/microvm_pins.py`) | **verified**: nothing beyond the worktree. Out of scope (BA, verbatim — do NOT touch): - Do **not** change Linux MicroVM behaviour, its pins, its egress default list (`apps/api/app/microvm_egress.py`) or its sizing defaults, beyond what is needed to add the OS dimension. - Do **not** change the static worktree pool or dynamic worktrees on any OS. - Do **not** add a new plan term or billing price (D7). - Do **not** mount NodeLinks inside a VM on any OS. - Do **not** add Multi-Node Hosts / co-located nodes (`vnode_host`), GitHub Actions runners or SDK-agent support to macOS or Windows hosts. - Do **not** support Intel Macs, Windows on ARM, or cloud-launched macOS nodes. - Do **not** relax the host-session (unmarked) capability table of ADR-0145. - Do **not** describe the boundary as stronger than each hypervisor gives ("strong isolation", "defence in depth" — ADR-0129 anti-pattern). - No new console page. Existing surfaces only (A4). - Operate and verify Jaah only through the console. No CLI, SQL or cloud-console workaround for live state (CLAUDE.md hard rule). Architect boundaries (declined, do NOT build): a Windows native Hyper-V runtime; Rosetta in Mac guests; boot-latency optimisation (pre-booted snapshots); a guest NIC or `com.apple.vm.networking`; never rename `kvm_usable`/`kvm_error`. Also do NOT touch: the Windows bridge/distro; Linux tap/nft behaviour (a spec without `egress` must behave exactly as today). Done means: - Linux regression: all existing microvmd / microvm_seed / microvm_egress / microvm_liveness tests green unchanged; a spec with no `egress` key yields today's CH argv byte-for-byte (unit test). - macOS unit tests: argv builder emits `jaah-vz run --spec`; spec validation rejects `tap` non-null or missing 15001/15053 on macOS; ready-line parsing (ready / error line / timeout); stop sequence SIGTERM→SIGTERM→SIGKILL timing; vsock header parser (valid, >64 bytes, slow >5 s, IPv6, port 0/65536 → the right refusal reason logged to `egress.log`); DNS over 15053 framing round-trip via `answer_dns`; `guest_ids()` per OS; seed with `egress=vsock` contains no network-config, contains `jaah0`, the nft redirect and the tunnel unit (mode 0755, root). - `macos-15` CI import smoke: the MicroVM module graph (`microvmd`, `microvm_host`, `microvm_registry`, `microvm_client`, `microvm_console`, `microvm_egress`, `microvm_facts`, `microvm_images`, `microvm_session`, `answer_delivery`, `kill_gates`) imports on darwin — job log cited. - Darwin-patched integration test with a fake `jaah-vz`/microvmd: a rail launch intent → `microvm_host` writes a spec with `egress=vsock`, `tap`/`mac` null, ports 15001/15053 present → the fake ready line yields a registry row with no tap → a pane-write/answer intent for that task reaches the fake microvmd via `_bound_vm`/`deliver_vm_answer`, and `kill_vm` stops it; a marked darwin node's observation carries the `microvm` block and `runner_clis` = the base image's guest set. - AC8 (arm64 half) — the arm64 provider-install CI job log cited; each provider's `linux_arm64` flag matches its result (Claude, Codex and DeepSeek or Pi install; Antigravity confirmed or flipped). - AC17 still passes (`python3 tools/microvm_pins.py`). - The macOS bundle installs the LaunchDaemon plist (`UserName=_jaahd`) and `jaahd-launch` loads/unloads it on marker presence (unit test with launchctl patched). - AC15 — `./dc check full` green (cite GATE_RESULT).