MicroVM supervisor and tap network

Epic 5777 — MicroVM nodes. This child is the supervisor/network slice of the architecture's **C5: Node VM runtime** (split by the planner critic). Depends on the pins child (shipped `cloud-hypervisor`, `virtiofsd`, `CLOUDHV.fd` under `/usr/lib/jaahd/microvm/`). Upstream: PRD task 5778 `prd`, Architecture task 5779 `architecture`. ## Original requirement (verbatim) Epic Goal 1 (part): "Each dispatched task ... gets a fresh VM with its own kernel". Goal 2: "The VM sees only what its task needs ... Nothing else on the host, no sibling task, no other account, no NodeLink store, no EFS (port 2049), no instance metadata service." Goal 9 (part): "Nodes report whether they have KVM". PRD R1: "... No session on that node runs directly on the node. Every other node behaves exactly as it does today." R11 (part): "Placement never sends a session to a MicroVM node that has stopped reporting KVM." R16 (part): "A per-VM unprivileged user or seccomp/landlock stays only if the Architect keeps it as VMM-escape hardening, never for identity." Epic Replace-or-delete: "Any guest agent, custom guest kernel or image pipeline: do not build". "tmux: not in the guest." ## Approach (settled) - **D0:** every unit ships installed but inert. jaahd writes/removes `<state>/microvm.enabled` from the node's mark, read from `GET /api/agent/pool-tick` top-level key `microvm: bool` (pinned by C2; absent ⇒ false). Packaged `jaah-microvm.path` (`PathChanged=` on the marker) triggers `jaah-microvm-converge.service` (root oneshot, only input = marker exists): present → create tap pool + nft table, then start `jaah-microvmd`; absent → stop microvmd, remove table and taps. Unmarked node: no new process, no tap, no nft table. - **D1:** `jaah-microvmd.service`, `User=` the daemon user, `SupplementaryGroups=kvm`, same `NetworkNamespacePath` as its node's jaahd (empty on a plain node). Owns every `cloud-hypervisor`/`virtiofsd` process, console stream + ring buffer, per-VM vsock listener path. The postinst restarts it **only when no VM is live**, otherwise defers until VM-idle (risk K4 — own the postinst rule and its test). - **D2:** no root at VM time. CH needs only `/dev/kvm`. virtiofsd unprivileged with `--sandbox chroot`, `--translate-uid`/`--translate-gid` guest 1000 → daemon uid. CH seccomp (default) + `--landlock` with rules naming exactly that VM's dir and the ro toolchain paths. - **D3 / §4.6:** `jaah-microvm-converge` creates taps `jvm0…jvm63` owned by the daemon user; tap `jvmN`: host `198.19.(N).1/30`, guest `198.19.(N).2/30`, guest MAC `52:54:00:00:<N hi>:<N lo>`. Loads table `inet jaah_microvm`: prerouting `iifname "jvm*"`: `udp dport 53` and `tcp dport 53` → redirect `:5353`; `meta l4proto tcp` → redirect `:15001`; input `iifname "jvm*"`: accept `tcp dport {5353,15001}` and `udp dport 5353`, drop rest; input `iifname != "jvm*"`: drop `tcp dport {5353,15001}` and `udp dport 5353`; forward `iifname "jvm*"`: drop. Function single-homed in `apps/jaahd/microvm_net.py` (C9 calls it inside a vnode netns — keep it netns-agnostic). microvmd allocates tap index from free taps; full pool refuses admission with a reason. - **D4 (fact):** `platform.kvm` unchanged. microvmd reports `microvm.kvm_usable` after opening `/dev/kvm` rw as itself (`kvm_error` = errno text otherwise). Only on a marked node. - **§4.4 supervisor socket** `<run>/microvmd.sock` (0600, daemon uid), line JSON. Ops: `start {vm_id}`, `stop {vm_id, grace_s}`, `list`, `attach {vm_id, ro}` (upgrades to raw bytes, starting with ring replay), `resize {vm_id, rows, cols}` (sends `rows cols` to guest vsock port 1025), `write {vm_id, b64}`, `control_fds` (long-lived stream passing `(vm_id, port, fd)` via SCM_RIGHTS for every accepted vsock connection on ports 1024 and 3000-3999). VM dir `<state>/vms/<vm_id>/` written by jaahd before `start`: `spec.json` `{vm_id, vcpus, ram_mib, disk_path, seed_path, tap, mac, shares:[{tag, host_path, ro}], vsock_ports:[...], pane_log}`; share tags fixed: `worktree` (rw), `config` (rw), `token` (rw), `report` (rw), `launch` (ro), `toolchain-<n>` (ro) — anything else rejected by microvmd (test this: it is the R4 "nothing else" guard). microvmd creates `<vmdir>/vsock.sock_<port>` listeners before boot. - **D17 (supervisor half):** console via `--console socket=<vmdir>/console.sock` (fall back to `pty` if pinned CH v53 lacks socket mode — verify, risk K2); 256 KiB ring buffer; tee every byte to the session's pane-log path; fan out to `attach` subscribers. **D19b:** CH runs with `--platform serial_number=<vm_id>`. - **D21b:** `--memory size=<ram_mib>M,shared=on` and `--balloon size=0,free_page_reporting=on`; vCPUs never pinned. - **§4.3 facts** (sent as key `microvm` in `POST /api/agent/nodes/config-snapshot`, pinned by C2): `{kvm_usable, kvm_error|null, free_ram_mib (MemAvailable), free_disk_mib (statvfs(<state>).f_bavail), taps_free, base:{...}, layers:[...], vms:[{vm_id,state,tap,generation}], vm_dirs:[...], ch_pids, virtiofsd_pids (process scan by executable path, independent of microvmd's list), taps_busy (kernel tap state), egress_refusals:[...]}`. Report `base:null`/`layers:[]` until C5c fills them and `egress_refusals:[]` until C8 fills it — expose a hook for both. - **Risk K1:** prove CH opens a pre-created, pre-addressed tap `tap=jvmN` with no ip/mask as non-root. Fallback (decided, do not re-ask): `setcap cap_net_admin` on the shipped CH binary in postinst. - Add nfpm/postinst entries for `jaah-microvmd.service`, `jaah-microvm.path`, `jaah-microvm-converge.service`. - If this node has no `/dev/kvm`, cover with unit tests + a fake CH; real-boot proof is C12's (record in report what was proven where). ## Acceptance criteria (verbatim) - AC-1 (node half): "... VM gone: no cloud-hypervisor process, no VM disk, no tap/vsock socket for it on the node (`ps`, `ls` of the VM state dir). ... **Fails if** either run needs any manual step on the node, or leaves any VM artefact behind." — this child supplies `stop` cleanup of processes/sockets/tap and the facts that make it observable. Credentials & reach: none beyond the worktree (tests, fake CH). Real KVM boot: needs a KVM host — **deferred** to C12 (dev reach Q7; physical host asked of the operator in C12). AC-16 — forge via `jaah-forge` — **verified**. Out of scope: no tmux in the guest (N4); no guest agent, custom kernel, image pipeline (N3); no GPL binary (N5). Unmarked nodes byte-for-byte unchanged; do not touch static worktrees, port grid, `jaah-vnodehelper` (N1 — per-vnode instances are C9's). No image/overlay/seed/rail executor (C5c), no session prep (C6), no console relay/answer delivery (C7), no egress relay (C8). Epic: "retiring static worktrees, co-located nodes (Multi-Node Hosts), the port grid or user slots; Mac/Windows hosts; GPU passthrough; any change to pricing beyond moving products to KVM-capable instance families." Done means: - Marker on/off drives converge: tap pool + `inet jaah_microvm` created/removed; unmarked node runs no new process — tests on `microvm_net.py` (render nft ruleset matches §4.6 exactly) and the path/converge units. - microvmd ops `start/stop/list/attach/resize/write/control_fds` behave per §4.4 with a fake CH; `spec.json` share tags outside the fixed set are rejected — tests. - Ring buffer replays on attach and tees to pane log — test. - `kvm_usable`/`kvm_error` and the §4.3 block posted in config-snapshot — test. - postinst does not restart microvmd while a VM is live — test (K4). - K1/K2 findings recorded in the report (proven against the pinned binary where KVM is available, otherwise stated as pending C12). - `./dc check full` green on the PR; PR auto-merged and the Jaah Deploy for dev succeeds (AC-16).

MicroVM tap pool + nft table (root converge), jaah-microvmd supervisor, §4.3 facts, units. Full gate green; PR #5065 merged, dev Jaah Deploy succeeded. K2/D19b proven on pinned CH; K1 static, real boot pending C12. Follow-ups in report (mem-6603, mem-6604).