Bootstrap OS/arch detection and dnf branch

## Original requirement (verbatim) > Make sure Jaah nodes can install and work on Graviton-based AWS instances, earlier version of Ubuntu. It is ok if some features might not work, such as microVM. Should work with the same installation command with any arch specifics inside the bootstrap Operator scope additions (verbatim, PRD task 11088): "Install higher version of python on older versions of Ubuntu." · "Also add to scope Amazon Linux two latest versions." (resolved: AL2023 + the newest after it = AL2027 preview, architecture O2) · "Both graviton and x64 must work on supported versions of ubuntu or amazon linux" · "24.04 is supposed to have rootless Docker. Double check the sources" **Supported matrix S** (single home: `packaging/linux-support.toml`, created by the Foundation child "Report Linux distro/arch, gate capabilities"): x86_64 + aarch64 × Ubuntu 22.04, 24.04, 26.04, Amazon Linux 2023, Amazon Linux 2027 (preview) = 10 cells. Design inputs (read with `task_read`): PRD = task 11088, architecture = task 11089. §/D/R references below are the architecture's; C-n are the PRD's acceptance criteria. D18 applies to every build child: prove pre-merge with unit/integration tests plus container tests (`docker run --platform linux/{amd64,arm64}` under qemu binfmt, mem-26702 method); post-merge, after Jaah Deploy to dev succeeds, launch the child's own EC2 cells and fix forward. ## Scope (architecture Child D; §3.5 bootstrap, D16, D17, R4, R7) 0. **First step (R4)**: on a real AL2027-preview EC2 instance confirm `/etc/os-release` `VERSION_ID`, reachability of the Jaah rpm route and Tailscale's `install.sh`, and package names; correct `packaging/linux-support.toml` if `VERSION_ID` differs (fallback for Tailscale: warn-and-try). 1. **Detection prelude** in `_body` (`apps/api/app/node_bootstrap.py:716+`), the single home of install-time detection: `jaah_detect_platform` sets `JAAH_OS_ID JAAH_OS_VERSION JAAH_OS_LABEL JAAH_PKG(apt|dnf) JAAH_ARCH(x86_64|aarch64)`. A release outside S prints the **C-8 warning as the first `==> [jaah]` line**, naming the detected OS and the supported list rendered from `linux-support.toml` (says "preview" for AL2027) — never a hard-coded list. 2. **D16**: unknown `ID` falls through — apt present → Ubuntu branch, dnf present → AL branch, neither → fail at step "detect package manager"; an arch that is not x86_64/aarch64 fails at step "detect architecture". 3. **D17**: `jaah_step "<name>"` sets `JAAH_STEP`; the existing EXIT trap (`node_bootstrap.py:502-505`) prints `==> [jaah] FAILED at step: $JAAH_STEP` on non-zero exit. Every step is labelled. 4. **Branch**: `case $JAAH_PKG` — **apt**: the existing body, byte-identical for 26.04 except step labels; **dnf**: `dnf -y install ca-certificates curl git tmux`, Tailscale `install.sh`, `rpm --import` of the pubkey to `/etc/pki/rpm-gpg/RPM-GPG-KEY-jaah`, write `/etc/yum.repos.d/jaah.repo` mode 0600 (§3.4 shape; node token never logged), `dnf -y install jaah`. Root check, already-a-node branch, `/etc/jaah` writes and `jaah auth login` stay shared. 5. `render_cloud_init` keeps the dnf branch out (cloud nodes are Ubuntu 26.04 only; R7 16 KiB user-data cap); `render_fetch_command` (`:2916-2955`) is untouched (C-1). Credentials & reach: - Launch and terminate tagged EC2 test cells (post-merge) — `me/jaah/system/bootstrap-admin` → assume `OrganizationAccountAccessRole` in the `jaah-dev` org account, `us-east-1` — **verified** (broker fetch succeeded; `sts get-caller-identity` returned the assumed role in jaah-dev). O4 authorises mutations, limited to instances tagged `jaah:test=task-<id>`. - AMIs for every cell — SSM public parameters `/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-6.1-{x86_64,arm64}`, `…/al2027-preview-ami-kernel-7.1-{x86_64,arm64}`, `/aws/service/canonical/ubuntu/server/...` — **verified** (listed from h2). - Dev console UI and API, incl. C-7 API refusals — `org/console/dev-login` (operator) at `https://app.dev.jaah.ai` — **verified** (broker fetch succeeded; console returned 200 from h2). - Test instances reaching the dev CP — dev tailnet, joined by the bootstrap's own key mint — **verified by design** (the first EC2 cell proves it). - Deploy to dev (C-5, C-9) — merge → `jaah-deploy.yml` develop path — needs nothing beyond the worktree. - Container tests (D18) — docker + qemu binfmt on the runner's node (mem-26702) — needs nothing beyond the worktree host. - Gates — `./dc check quick` / `full` — needs nothing beyond the worktree. Out of scope: Do NOT touch / do NOT do — - Do NOT change the one-liner (`render_fetch_command`) or add any OS/arch flag/variant (C-1, PRD §5). - Do NOT add the dnf branch to `render_cloud_init` (R7). - Do NOT change the macOS/Windows bootstrap sections. - Do NOT restate the supported list anywhere but `linux-support.toml`. - Epic-wide negative constraints (PRD §5, verbatim): - MUST NOT make MicroVM work on arm64 Linux, or on any newly supported release. It is reported unavailable (C-7) unless it already works there. - MUST NOT change the cloud-launched EC2 image (`apps/api/app/aws/ec2.py` `UBUNTU_RELEASE` / `_UBUNTU_IMAGE_NAME`) or add an OS picker to the EC2 create flow. The stale comment at `ec2.py:1111` is the one allowed edit there (C-6). - MUST NOT change the macOS or Windows installers, bundles or their install commands (`node_bootstrap.py` macOS and Windows sections, `packaging/macos/`, `packaging/windows/`). - MUST NOT replace, relink or upgrade the distro's system `python3` (`/usr/bin/python3`). - MUST NOT add an OS or arch argument, flag or variant to the console's install command. - MUST NOT support Ubuntu 20.04, Debian, RHEL, Amazon Linux 2, or architectures other than x86_64 and arm64 as **supported**. They get only C-8's warn-and-try. - MUST NOT gate the support behind a feature flag, plan feature or setting. - MUST NOT bump unrelated pins in `packaging/platform-pins.toml`, `packaging/microvm-pins.toml` or `packaging/runner-cli-locks/` beyond what a supported cell needs to install. Done means: - Bootstrap-render tests: the warning list is rendered from `linux-support.toml`; cloud-init output size unchanged and contains no dnf branch; the 26.04 apt body is byte-identical apart from step labels; `render_fetch_command` output unchanged. - Container runs of the detection prelude on `ubuntu:22.04/24.04/26.04`, `amazonlinux:2023` (both arches) and `debian:12`: correct `JAAH_*` values; debian:12 prints the warning first naming "Debian 12" and the list; a forced failure prints `FAILED at step: <name>`; an image with neither apt nor dnf fails at step "detect package manager" and a faked unsupported `uname -m` fails at step "detect architecture" — cite output. - Post-merge on real dev EC2, AL2023 and AL2027 × x64 and arm64: C-1, C-2, C-3, C-4, C-5 (the console command byte-identical to the Ubuntu one; node Online within 5 min labelled e.g. "Amazon Linux 2023 · Arm"; a task settles done; unattended upgrade). - Post-merge Ubuntu 20.04 EC2: C-8 (warning is the first `==> [jaah]` line naming "Ubuntu 20.04" and the list; node Online or a named failing step). - `./dc check quick` green; merge lane `./dc check full` green; Jaah Deploy to dev succeeded. PRD acceptance criteria this child carries (verbatim; prove the cells named above, others are proven by their owning child or the sweep): | ID | Criterion | Verify (oracle) | |---|---|---| | C-1 One command | The install command the console shows for a self-hosted Linux node is byte-identical across every cell of S, with no per-OS or per-arch variant and no flags. All OS and arch detection happens inside the served bootstrap. | Open dev.jaah.ai → Nodes → create a self-hosted Linux node → copy the command. Do it twice and compare: the command differs only by the node's own token or ID, never by OS or arch. Run it unchanged with `sudo` on each cell's fresh instance. | | C-2 Install succeeds | On each cell, the command exits 0. The last output line reports success, and no step is skipped silently. A newer Python is provisioned where the distro's own is below the daemon's floor (22.04, and Amazon Linux if needed); the distro's `/usr/bin/python3` is never replaced or relinked. | On the instance: `echo $?` → `0`. `systemctl is-active jaahd` → `active`. `python3 --version` gives the same value as before the install. The daemon process runs an interpreter at or above the floor: `ps -o args= -C <daemon>` shows it, and `<that interpreter> --version` returns it. | | C-3 Online | Within 5 minutes of the install, the node shows **Online** in the console. Its detail page shows the correct OS release and architecture (for example "Ubuntu 22.04 · Arm" or "Amazon Linux 2023 · x86"), never a raw slug and never blank. | Open dev.jaah.ai → Nodes → the node. Observe the Online status and readable OS + arch labels in the list and on the detail page. | | C-4 Does real work | A fleet task dispatched to the node runs a Claude runner end to end and settles `done`. This proves tmux, git, the worktree pool, the Claude CLI and the node bus all work on that cell. | In the dev console, create a trivial task (e.g. "write hello.txt") on a project pinned to that node and start it. The task reaches done, and its session transcript appears in Audit > Sessions. | | C-5 Upgrades | When a newer `jaah` version is published, the node upgrades to it unattended and returns to Online with the new version, and its runners keep working. | Deploy a newer build to dev, wait for the update cycle (or trigger the console's update action). The node detail shows the new version and Online. Repeat C-4 once. | | C-8 Unsupported OS warns and tries | On an OS outside S (e.g. Ubuntu 20.04, Debian 12), the same command prints a clear warning first. The warning names the OS it detected and the supported list. Then it attempts the install anyway. If the attempt fails, the failure line names the step that failed; it is never a silent `set -e` exit. | Launch an Ubuntu 20.04 instance in dev and run the command. The warning is the first `==> [jaah]` line and names "Ubuntu 20.04" plus the supported list. Either the node comes Online, or the script ends with a named failing step. |

Merged #7307 (c914152, lane full gate green): bootstrap detects OS/arch, warns from linux-support.toml, names failed steps, adds dnf arm; cloud-init and one-liner unchanged. Container proofs pass; AL2027 VERSION_ID=2027 confirmed. Dev deploy fails on 11131's migration (draft #11175), so EC2 cells are deferred to the sweep. Memory: proposed 1 gotcha, 1 recipe.