Build jaah-vz Virtualization.framework helper

Task 10056 · Running · Coder · created · updated

## Epic requirement (verbatim) "Implement microVM on Windows and MacOS" (Task 10021 PRD ask #1, `in_scope`). Operator decisions that bind every child (verbatim from the PRD/Architecture): - D1 **Full parity** with Linux. Each session gets its own Linux guest kernel. The CP egress allowlist is enforced on the node. Control calls are vsock-bound (or an equivalent per-VM channel). No NodeLink inside the VM. No root in the guest. The disk is retained on `system_failed`. Console, sizing and placement UX are the same. - D2 **All three:** fleet runners, advisor chats and human console sessions. D3 **The Linux guest's set** of providers. D4 macOS = Apple Silicon, Windows = x64. D5 cloud Windows too. D7 same plan feature (`billing_plans.microvm_allowed` + `orgs.microvm_allowed_override`). - AD1 Windows = WSL2 + the existing Cloud Hypervisor/KVM runtime (Windows 10 refused). AD2 Mac guests arm64 only, no Rosetta. AD3 Windows test PC = `win-t8703`. AD5 probe m8i.large exactly; metal (m7i.metal-24xl) is the fallback. AD6 Mac oracles deferred until the Mac is enrolled on dev. AD7 AC2(a) live is dropped; unit tests map every `microvm_prereq` code to its sentence. **Quality bar (verbatim).** Every refusal shown to an operator is one plain sentence naming what is missing and how to fix it (`docs/standards/ui-copy-standards.md`). No generic 400, no stack trace, no Linux-specific wording on a Mac or Windows node. Boot-to-runner latency on Mac and Win is reported in the build's evidence. It is not gated, but it must not be degraded by more than 2× against a Linux MicroVM node of similar size without the Architect recording why. ## This child: `jaah-vz` Swift helper + signing + macOS probe/facts (Architecture §3.1, §9.3, §9.1 macOS column) Requirements this child serves (PRD §4, verbatim): - R1. Mark. An Apple Silicon macOS node and an x64 Windows node (self-hosted, or cloud-launched for Windows) can be marked MicroVM through the existing Node detail toggle. Conditions: the org's plan allows MicroVM, the node reports a usable hypervisor, and it has no held static worktrees. When the node does not qualify, the refusal is a readable sentence naming the missing prerequisite. The Linux-only `NO_KVM` sentence must not be shown to a Mac or Windows node for a reason that no longer applies. - R6. Install and update. The macOS installer and the Windows service install and converge everything the VM runtime needs: binaries, images and helpers. The A2 prerequisite is the only manual step. Updating a node keeps its VMs' contract. Pinned platform-report seam (Architecture §9.1, verbatim): | Field | Vocabulary | Linux | Windows | macOS | Null means | |---|---|---|---|---|---| | `kvm` (existing) | bool | `/dev/kvm` exists (unchanged) | the WSL KVM probe (§2.3) | `jaah-vz probe` `usable` | unknown, which is unusable | | `hypervisor` (new) | `kvm` \| `kvm-wsl2` \| `vz` | `kvm` | `kvm-wsl2` | `vz` | unknown (an old node). The console shows "Hypervisor unknown" | | `microvm_prereq` (new) | `windows_build` \| `wsl_missing` \| `wsl_no_kvm` \| `vz_unsupported` \| `vz_helper` | never sent (Linux keeps `NO_KVM`) | the first unmet one, in the order listed | `vz_unsupported` (probe `error: "unsupported"`) or `vz_helper` (helper missing, or probe `error: "entitlement"` or a non-zero exit) | nothing unmet | **Precedence.** `kvm is True` passes the hypervisor check, and a node sends no `microvm_prereq` with it. If one arrives anyway it is ignored. `kvm is not True` refuses with the `microvm_prereq` sentence, or with the "update this node" sentence when the code is null or unknown. An unknown code is dropped at ingest, like every other platform field. The full settled Architecture (Task 10022) and PRD (Task 10021) are the epic's prerequisite artifacts; read them with `task_read`/`artifact_read` when this body is shorter than you need. Section numbers (§) refer to the Architecture. ### Approach (settled, realize it) A Jaah-built Swift helper over Apple's Virtualization.framework (chosen over vfkit), source home `apps/jaahd/vz/` (Swift package), built on the `macos-15` CI job (`node-sign-macos`, `.github/workflows/jaah-deploy.yml:1282-1391`), shipped in the macOS bundle. It is a **CH-compatible subset** so microvmd only changes its argv builder (that switch is the macOS-runtime child's). - `jaah-vz run --spec <vmdir>/spec.json` reads `vcpus, ram_mib, disk_path, seed_path, shares[{tag,host_path,ro}], vsock_ports, pane_log`; ignores `tap`/`mac` (must be null on macOS); `egress` must be `vsock`; the `owner` on the `token` share is accepted and ignored. - Boot `VZEFIBootLoader` with per-VM variable store `<vmdir>/efi-vars`; disks `virtio-blk` (raw root, raw cidata seed); shares `VZVirtioFileSystemDeviceConfiguration`, one per tag. - **vsock**: one `VZVirtioSocketDevice`. For each port in `vsock_ports` (1024, 3000–3999, plus 15001/15053 for egress) a listener accepts the guest connection and connects to `<vmdir>/vsock.sock_<port>`, then splices (CH hybrid semantics). Host-initiated: listens on `<vmdir>/vsock.sock` and accepts `CONNECT <port>\n` → `OK <port>\n`. A guest port with no host listener: reset immediately, one line to `<vmdir>/vmm.log`, never retried/queued. - Console: `VZFileHandleSerialPortAttachment` on a pty; path written to `<vmdir>/console.pty` before the ready line. - **Ready**: exactly one stdout line `{"ready":true,"console":"<pty path>"}\n`; failure = one stderr line `{"error":"<code>","detail":"…"}` then exit. - **Exit codes**: 0 clean stop; 64 bad spec; 69 VZ unsupported; 70 VZ start/validation failure; 75 guest stopped with an error. - **Stop**: first SIGTERM → `requestStop` (ACPI), second SIGTERM → `stop`; no grace parameter. - **Probe**: `jaah-vz probe` exits 0 with `{"usable":bool,"error":null|"unsupported"|"entitlement"}`. - **Entitlement**: `com.apple.security.virtualization` only, never `com.apple.vm.networking`. `tools/macos_sign.py` keeps a binary's existing entitlements (`:27`); build with the entitlement and make `verify` assert it. Unsigned dev builds are ad-hoc signed with the same entitlements file. - **macOS facts**: `apps/jaahd/host_platform/macos.py` `kvm()` (today `None`, `:101`) returns the probe's `usable`; report `hypervisor: "vz"` and `microvm_prereq` per §9.1 (`vz_unsupported` on `error:"unsupported"`; `vz_helper` when the helper is missing, `error:"entitlement"`, or a non-zero exit). Acceptance criteria this child serves (PRD §5, verbatim): - AC1 — The mark turns on for a qualifying node (R1). Verify: Open Nodes → Mac (then Win) → MicroVM toggle → turn on. Observe: the toggle stays on, the MicroVM chip shows on the node, and the node's free static worktrees are gone from its worktree list. - AC2 — A non-qualifying node gets a readable, specific refusal (R1). Every state is covered: hypervisor prerequisite unmet, plan off, held worktrees, live VMs while turning off. Verify: (a) On Win with the A2 prerequisite disabled (and on a Mac where the Architect's prerequisite is unmet), turn on → observe a sentence naming that prerequisite; it must not say "needs a Linux node". (b) In an org whose plan and override deny MicroVM → observe "Your plan doesn't include MicroVM nodes." (c) With a user-held worktree → observe the held-worktrees sentence. (d) With a live VM, turn off → observe "Wait for N MicroVM… to end." - AC14 — Install is turnkey apart from A2 (R6). Verify: Enroll a fresh Mac and a fresh Windows machine with the per-OS one-liner from the console. Perform only the documented prerequisite. Turn MicroVM on → AC3 passes with no other manual step. Update the node package → a subsequent AC3 run still passes. Ownership note (Planner): this child delivers the helper and the macOS hypervisor fact; the Mac live runs of these oracles belong to the Mac-verification child (AD6). GitHub-hosted macOS runners cannot nest VMs, so CI proves the contract surfaces that need no guest boot. Credentials & reach (Architect §8, verbatim rows): - AC1–AC12, AC14 on Mac | The operator's Apple Silicon Mac node on dev | **deferred** (AD6: "enroll it later"): the dev Nodes list with "All orgs" checked shows 9 nodes and none is macOS. C3 runs its Mac oracles once the Mac is enrolled. The build is not blocked, because C3's unit-level work and CI do not need it. - Build (C3 signing) | GitHub environment `macos-signing` secrets used by job `node-sign-macos` (`APPLE_DEVELOPER_ID_P12` and the others, named in `jaah-deploy.yml:18-22`) | **verified (by design)**: CI-held. No runner fetches them. If `APPLE_TEAM_ID` is unset, dev builds are ad-hoc signed with the entitlement (§3.1). - AC15 gate, AC16, AC17 | Worktree only (`./dc check full`, `tools/check_context.py`, `tools/microvm_pins.py`) | **verified**: nothing beyond the worktree. Out of scope (BA, verbatim — do NOT touch): - Do **not** change Linux MicroVM behaviour, its pins, its egress default list (`apps/api/app/microvm_egress.py`) or its sizing defaults, beyond what is needed to add the OS dimension. - Do **not** change the static worktree pool or dynamic worktrees on any OS. - Do **not** add a new plan term or billing price (D7). - Do **not** mount NodeLinks inside a VM on any OS. - Do **not** add Multi-Node Hosts / co-located nodes (`vnode_host`), GitHub Actions runners or SDK-agent support to macOS or Windows hosts. - Do **not** support Intel Macs, Windows on ARM, or cloud-launched macOS nodes. - Do **not** relax the host-session (unmarked) capability table of ADR-0145. - Do **not** describe the boundary as stronger than each hypervisor gives ("strong isolation", "defence in depth" — ADR-0129 anti-pattern). - No new console page. Existing surfaces only (A4). - Operate and verify Jaah only through the console. No CLI, SQL or cloud-console workaround for live state (CLAUDE.md hard rule). Architect boundaries (declined, do NOT build): a Windows native Hyper-V runtime; Rosetta in Mac guests; boot-latency optimisation (pre-booted snapshots); never `com.apple.vm.networking`. Also do NOT touch: `microvmd.py` (argv switch is the macOS-runtime child's); Linux/Windows `host_platform`. Done means: - The `macos-15` CI job builds `jaah-vz`, signs it (Developer ID when `APPLE_TEAM_ID` is set, else ad-hoc) with `com.apple.security.virtualization`, and `tools/macos_sign.py verify` asserts the entitlement is present and `com.apple.vm.networking` absent — job log cited. - Contract tests on that job: `jaah-vz probe` prints valid JSON of the pinned shape and exits 0; `run` with a malformed spec exits 64 with one stderr JSON line; a spec with non-null `tap` exits 64; where VZ is unsupported on the runner, `run` exits 69. A pure-Swift unit test covers the `CONNECT <port>` / `OK <port>` host-initiated handshake parser and the vsock port→socket-path mapping. - jaahd unit tests: macOS `kvm()`/`hypervisor`/`microvm_prereq` mapping for probe `usable:true` (no prereq), `error:"unsupported"` → `vz_unsupported`, `error:"entitlement"` → `vz_helper`, helper missing → `vz_helper`, non-zero exit → `vz_helper`. - The macOS bundle (`packaging/macos/`) contains the signed helper (CI artifact listing cited). - `./dc check full` green (cite GATE_RESULT).

Outcome

Pull request

Activity

  • Draft
  • Ready
  • Runner started