Matrix acceptance sweep on dev EC2

## Original requirement (verbatim) > Make sure Jaah nodes can install and work on Graviton-based AWS instances, earlier version of Ubuntu. It is ok if some features might not work, such as microVM. Should work with the same installation command with any arch specifics inside the bootstrap Operator scope additions (verbatim, PRD task 11088): "Install higher version of python on older versions of Ubuntu." · "Also add to scope Amazon Linux two latest versions." (resolved: AL2023 + the newest after it = AL2027 preview, architecture O2) · "Both graviton and x64 must work on supported versions of ubuntu or amazon linux" · "24.04 is supposed to have rootless Docker. Double check the sources" **Supported matrix S** (single home: `packaging/linux-support.toml`, created by the Foundation child "Report Linux distro/arch, gate capabilities"): x86_64 + aarch64 × Ubuntu 22.04, 24.04, 26.04, Amazon Linux 2023, Amazon Linux 2027 (preview) = 10 cells. Design inputs (read with `task_read`): PRD = task 11088, architecture = task 11089. §/D/R references below are the architecture's; C-n are the PRD's acceptance criteria. D18 applies to every build child: prove pre-merge with unit/integration tests plus container tests (`docker run --platform linux/{amd64,arm64}` under qemu binfmt, mem-26702 method); post-merge, after Jaah Deploy to dev succeeds, launch the child's own EC2 cells and fix forward. ## Scope (architecture Child F) Run C-1…C-10 on **all 10 cells of S** plus Ubuntu 20.04 on real EC2 in jaah-dev against dev.jaah.ai (instances tagged `jaah:test=task-<id>`, terminated after), with default settings. C-9 on existing dev Linux and Windows nodes; C-5 on every cell; C-3/C-7 on every new cell; GHA on every cell (reusing the GHA child's target repo). Record an evidence table (cell × criterion → pass/fail + the command output or console observation). Also run the cells moved here from sibling children: rootless Docker + vnode attach on AL2023/AL2027 × both arches, and a GHA runner job (incl. a browser job) on AL2023 arm64 and x64. **Closure rule:** this child is NOT done while any cell of S is red. Fix forward in this child (one or more PRs through the merge lane, each watched to a successful Jaah Deploy) until every cell is green. The only exception is a cell blocked by something only a human can clear (per the charter: one AskUserQuestion round, header Credential/Access/Human act/External); record it as an operator-visible exception with the observed failure. If any cell remains red or excepted at settle, the report states **"epic acceptance NOT met"** at the top and names each such cell. While testing the console, note anything broken or inconvenient per the project instructions. Credentials & reach: - Launch and terminate tagged EC2 test cells (post-merge) — `me/jaah/system/bootstrap-admin` → assume `OrganizationAccountAccessRole` in the `jaah-dev` org account, `us-east-1` — **verified** (broker fetch succeeded; `sts get-caller-identity` returned the assumed role in jaah-dev). O4 authorises mutations, limited to instances tagged `jaah:test=task-<id>`. - AMIs for every cell — SSM public parameters `/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-6.1-{x86_64,arm64}`, `…/al2027-preview-ami-kernel-7.1-{x86_64,arm64}`, `/aws/service/canonical/ubuntu/server/...` — **verified** (listed from h2). - Dev console UI and API, incl. C-7 API refusals — `org/console/dev-login` (operator) at `https://app.dev.jaah.ai` — **verified** (broker fetch succeeded; console returned 200 from h2). - Existing dev nodes for C-9 — dev console **Nodes** list, signed in as `org/console/dev-login` — **verified** (inventory at design time: 11 Ubuntu x64, 1 Ubuntu arm64, 1 Windows x64). - C-9 on an existing **macOS** dev node — **deferred**: dev has no macOS node; the macOS installer and bundle stay byte-identical, so regression is covered by `tools/tests/test_node_bootstrap_macos.py` in `./dc check full`. Re-check if a macOS dev node appears. - Test instances reaching the dev CP — dev tailnet, joined by the bootstrap's own key mint — **verified by design** (the first EC2 cell proves it). - GHA job oracle: a runner registers on the test node and runs one job — dev console → node → **GitHub Actions runners** tab; credential `me/users/roman@gmind.tech/github_admin` — **credential verified**; **target repo deferred**: dev has no GHA pool yet and creating or choosing a GitHub repo is outward-facing. The GHA child asks the operator for the repo in one round (header `External`) before its post-merge cell; the sweep child reuses it. If declined, GHA cells are recorded **unproven against O5**, never done. - Deploy to dev (C-5, C-9) — merge → `jaah-deploy.yml` develop path — needs nothing beyond the worktree. - Gates — `./dc check quick` / `full` — needs nothing beyond the worktree. Out of scope: Do NOT touch / do NOT do — - Do NOT re-design or widen scope (OS picker for EC2 create, MicroVM on arm64 — deferred follow-ups). - Do NOT leave test instances running. - Epic-wide negative constraints (PRD §5, verbatim): - MUST NOT make MicroVM work on arm64 Linux, or on any newly supported release. It is reported unavailable (C-7) unless it already works there. - MUST NOT change the cloud-launched EC2 image (`apps/api/app/aws/ec2.py` `UBUNTU_RELEASE` / `_UBUNTU_IMAGE_NAME`) or add an OS picker to the EC2 create flow. The stale comment at `ec2.py:1111` is the one allowed edit there (C-6). - MUST NOT change the macOS or Windows installers, bundles or their install commands (`node_bootstrap.py` macOS and Windows sections, `packaging/macos/`, `packaging/windows/`). - MUST NOT replace, relink or upgrade the distro's system `python3` (`/usr/bin/python3`). - MUST NOT add an OS or arch argument, flag or variant to the console's install command. - MUST NOT support Ubuntu 20.04, Debian, RHEL, Amazon Linux 2, or architectures other than x86_64 and arm64 as **supported**. They get only C-8's warn-and-try. - MUST NOT gate the support behind a feature flag, plan feature or setting. - MUST NOT bump unrelated pins in `packaging/platform-pins.toml`, `packaging/microvm-pins.toml` or `packaging/runner-cli-locks/` beyond what a supported cell needs to install. Done means: - Evidence table covers 10 cells × C-1…C-8, C-10, plus Ubuntu 20.04 (C-8), plus C-9 on existing Linux and Windows dev nodes, plus the AL vnode and AL GHA cells — every cell green with cited output; any non-green cell is only a human-blocker exception recorded per the closure rule, and the report then opens with "epic acceptance NOT met". - C-1 proven by comparing two console-copied commands (differ only by token/ID). - C-7 on every cell: each unavailable feature disabled/absent with a readable full sentence on node detail, MicroVM toggle, vnode attach, NodeLink attach, runner-provider choice; API attempt returns a refusal naming the reason. - macOS C-9 (deferred — no macOS dev node; covered by `tools/tests/test_node_bootstrap_macos.py` in `./dc check full`). - GHA cells: registered + one job each, or recorded unproven against O5 (deferred — no target repo) if the operator declined. - All test instances terminated (`aws ec2 describe-instances` with the tag shows none running). PRD acceptance criteria this child carries (verbatim): all of C-1…C-10 as stated in the PRD (task 11088 §3) — C-1 One command; C-2 Install succeeds; C-3 Online; C-4 Does real work; C-5 Upgrades; C-6 Rootless Docker / vnode on 24.04; C-7 Unavailable features are reported and hidden; C-8 Unsupported OS warns and tries; C-9 No regression on existing nodes; C-10 Available in dev, on by default. The full criterion and oracle text of each is embedded verbatim in the sibling children that own them (C-1/C-8 in the bootstrap child, C-2/C-4/C-5 in the interpreter child, C-3/C-7/C-9/C-10 in the capability child, C-6 in the rootless-Docker child) and in the PRD; run each oracle exactly as written there.