Report Linux distro/arch, gate capabilities

Task 11107 · Done · Coder · created · updated

## Original requirement (verbatim) > Make sure Jaah nodes can install and work on Graviton-based AWS instances, earlier version of Ubuntu. It is ok if some features might not work, such as microVM. Should work with the same installation command with any arch specifics inside the bootstrap Operator scope additions (verbatim, PRD task 11088): "Install higher version of python on older versions of Ubuntu." · "Also add to scope Amazon Linux two latest versions." (resolved: AL2023 + the newest after it = AL2027 preview, architecture O2) · "Both graviton and x64 must work on supported versions of ubuntu or amazon linux" · "24.04 is supposed to have rootless Docker. Double check the sources" **Supported matrix S** (single home: `packaging/linux-support.toml`, created by the Foundation child "Report Linux distro/arch, gate capabilities"): x86_64 + aarch64 × Ubuntu 22.04, 24.04, 26.04, Amazon Linux 2023, Amazon Linux 2027 (preview) = 10 cells. Design inputs (read with `task_read`): PRD = task 11088, architecture = task 11089. §/D/R references below are the architecture's; C-n are the PRD's acceptance criteria. D18 applies to every build child: prove pre-merge with unit/integration tests plus container tests (`docker run --platform linux/{amd64,arm64}` under qemu binfmt, mem-26702 method); post-merge, after Jaah Deploy to dev succeeds, launch the child's own EC2 cells and fix forward. ## Scope (architecture Child A; §2, §3.1, §3.2, D4, D5, D6, D9) 1. **Single home of S.** Create `packaging/linux-support.toml` exactly as architecture §2 (schema 1; `arches = ["x86_64","aarch64"]`; five `[[release]]` rows with `id`, `version_id`, `label`, `pkg`, AL2027 `preview = true`). Rewrite `portfolio/orchestration/jaah-package-distribution.md` D1, `apps/jaahd/README.md:446`, `infra/jaah/README.md:39`, `infra/jaah/terraform/README.md:206`, `docs/architecture.md:217` to point at it, never restate the list. 2. **Report** (§3.1): `apps/jaahd/host_platform/linux.py` platform block gains `distro: {id, version_id}` from `/etc/os-release` `ID`/`VERSION_ID` and `features: {...}`. This child owns the `features` field and the **`mount_efs` probe**: true when `/sbin/mount.efs --version` exits 0 **and**, where `/usr/sbin/efs-proxy` exists, `efs-proxy --version` exits 0 (running the binaries catches a glibc mismatch). Once shipped, the probe always emits its key. The `rootless_docker` key is owned by the rootless-Docker child; leave it unemitted here. 3. **Ingest + storage**: allow-list at `apps/api/app/fleet_observe.py:759-787` — `distro.id` `[a-z0-9._-]{1,32}`, `version_id` printable ≤32 chars, `features` keys from a fixed allow-list (`rootless_docker`, `mount_efs`) with bool values. One additive migration: nullable `nodes.distro_id text`, `nodes.distro_version text`, `nodes.host_features jsonb`. 4. **Wire + label** (D4): `nodes.os` stays `ubuntu` (no OS migration); its operator label becomes **"Linux"** (`node_platform._LABEL`, `apps/web/src/lib/nodeOs.ts:10-14`). `routes/nodes/node_wire.py` adds `distro_label` for `os == "ubuntu"` rows only: the release `label` from `linux-support.toml`; unknown distro → `"<ID> <VERSION_ID>"` from the reported fields (e.g. "debian 12"), never blank (the payload carries no `NAME`; decided to avoid a third column); before first report → `"Linux"`. Web `nodeOsArchLabel` (`lib/nodeOs.ts:26-32`) renders `"<distro_label> · <Arm|x86>"` on the node list (`Nodes.tsx:762`) and detail (`NodeDetailPage.tsx:1439-1440`). 5. **Capability rules** (§3.2): `node_platform.node_reason(node, capability) -> Optional[str]` = declared-OS `reason(os, cap)` first, then host-fact rules (only when `node.os == "ubuntu"` and reported); `node_unsupported(node)` built from it. Rows: `microvm` unavailable when `distro_id` and `arch` present and not (x64 ∧ ubuntu ∧ 26.04) — arm64: "MicroVM isn't available on Arm Linux nodes yet." else "MicroVM needs an x86_64 Ubuntu 26.04 node."; `vnode_host` when `rootless_docker` key present and false — "Running several nodes on one machine needs rootless Docker, which this node doesn't have yet."; `shared_accounts` when `mount_efs` present and false — "NodeLink needs the EFS mount helper, which this node doesn't have yet." **D6: an absent fact = no restriction, decided per key.** 6. **Enforcement** switches from declared-OS checks to `node_reason`/`node_unsupported` at every site in §3.2: `nodelinks.py:219,654,1762,1951,2355`, `routes/nodelinks.py:524`, `routes/vnode_hosts.py:332`, `hosts.py:738`, `node_orders.py:552,625`, `routes/node_orders.py:206`, `routes/gha_runners.py:638`, `acp_runtime.py:147`, `agent_resolution.py:293` (via `session_reason`), `microvm_mode.blocker` (`:108-121`). Exception: `routes/nodes/cloud.py:197,358` keep declared-OS `supports`. A Multi-Node Host check reads the host's own `Node` row. API refusals name the reason (never 500/hang). 7. **Web surfaces** read the map: MicroVM toggle (`components/microvm/NodeMicrovmTab.tsx:501,531`) disabled with the reason; vnode attach (`pages/nodes/vnodeHosts/`) does not offer refused hosts and shows the reason; runner-provider picker (`AccountEditPage.tsx:510` `ProviderField`). Reasons render in full (not truncated, never a slug) at phone width too. Credentials & reach: - Launch and terminate tagged EC2 test cells (post-merge) — `me/jaah/system/bootstrap-admin` → assume `OrganizationAccountAccessRole` in the `jaah-dev` org account, `us-east-1` — **verified** (broker fetch succeeded; `sts get-caller-identity` returned the assumed role in jaah-dev). O4 authorises mutations, limited to instances tagged `jaah:test=task-<id>`. - AMIs for every cell — SSM public parameters `/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-6.1-{x86_64,arm64}`, `…/al2027-preview-ami-kernel-7.1-{x86_64,arm64}`, `/aws/service/canonical/ubuntu/server/...` — **verified** (listed from h2). - Dev console UI and API, incl. C-7 API refusals — `org/console/dev-login` (operator) at `https://app.dev.jaah.ai` — **verified** (broker fetch succeeded; console returned 200 from h2). - Existing dev nodes for C-9 — dev console **Nodes** list, signed in as `org/console/dev-login` — **verified** (inventory at design time: 11 Ubuntu x64, 1 Ubuntu arm64, 1 Windows x64). - Deploy to dev (C-5, C-9) — merge → `jaah-deploy.yml` develop path — needs nothing beyond the worktree. - Gates — `./dc check quick` / `full` — needs nothing beyond the worktree. Out of scope: Do NOT touch / do NOT do — - Do NOT add a new `nodes.os` value, an OS picker, or change the CHECK constraint (D4). - Do NOT change `routes/nodes/cloud.py` cloud-create capability checks (§3.2 exception). - Do NOT emit `rootless_docker` (rootless-Docker child) or touch packaging/bootstrap/interpreter work (other children). - Do NOT fail closed on an absent fact (D6/R8). - Epic-wide negative constraints (PRD §5, verbatim): - MUST NOT make MicroVM work on arm64 Linux, or on any newly supported release. It is reported unavailable (C-7) unless it already works there. - MUST NOT change the cloud-launched EC2 image (`apps/api/app/aws/ec2.py` `UBUNTU_RELEASE` / `_UBUNTU_IMAGE_NAME`) or add an OS picker to the EC2 create flow. The stale comment at `ec2.py:1111` is the one allowed edit there (C-6). - MUST NOT change the macOS or Windows installers, bundles or their install commands (`node_bootstrap.py` macOS and Windows sections, `packaging/macos/`, `packaging/windows/`). - MUST NOT replace, relink or upgrade the distro's system `python3` (`/usr/bin/python3`). - MUST NOT add an OS or arch argument, flag or variant to the console's install command. - MUST NOT support Ubuntu 20.04, Debian, RHEL, Amazon Linux 2, or architectures other than x86_64 and arm64 as **supported**. They get only C-8's warn-and-try. - MUST NOT gate the support behind a feature flag, plan feature or setting. - MUST NOT bump unrelated pins in `packaging/platform-pins.toml`, `packaging/microvm-pins.toml` or `packaging/runner-cli-locks/` beyond what a supported cell needs to install. Done means: - `packaging/linux-support.toml` exists as §2 and the five docs point at it with no restated list — `rg -n '22.04' <those docs>` shows only the pointer. - Rule-table unit tests cover every row incl. D6 never-reported and per-key absence (absent `rootless_docker` keeps vnode hosting; absent `mount_efs` keeps NodeLink) — tests green. - API refusal tests per enforcement site return a refusal naming the reason (4xx with the sentence), never 500 — tests green. - Web tests per surface (MicroVM toggle disabled + reason, vnode attach hides refused host + reason, provider picker) and the `distro · arch` label on list + detail — tests green. - Ingest rejects out-of-allow-list `distro.id`/feature keys — unit test. - Post-merge on dev (existing 26.04 x64 + arm64 nodes): list and detail show "Ubuntu 26.04 · x86" / "Ubuntu 26.04 · Arm" (readable, never a slug or blank); MicroVM is hidden/disabled with the reason on the arm64 node and offered on an x64 26.04 KVM node if one exists in dev (else proven by the API/unit test, and say so); the API refuses MicroVM on arm64 naming the reason; existing NodeLinks and vnode hosts stay available (R8). - `./dc check quick` green pre-PR; merge lane `./dc check full` green; Jaah Deploy to dev succeeded. PRD acceptance criteria this child carries (verbatim; prove the cells named above, others are proven by their owning child or the sweep): | ID | Criterion | Verify (oracle) | |---|---|---| | C-3 Online | Within 5 minutes of the install, the node shows **Online** in the console. Its detail page shows the correct OS release and architecture (for example "Ubuntu 22.04 · Arm" or "Amazon Linux 2023 · x86"), never a raw slug and never blank. | Open dev.jaah.ai → Nodes → the node. Observe the Online status and readable OS + arch labels in the list and on the detail page. | | C-7 Unavailable features are reported and hidden | Some features cannot work on a cell, e.g. MicroVM on any arm64 Linux, vnode hosting where C-6 disproves it, EFS NodeLink where no helper package exists for that OS. Each one is reported by the node as unavailable with a one-line human reason. The console does not offer that feature for that node and shows the reason where the feature would appear. The install still succeeds (C-2). This holds for every capability, on every console surface that offers one: node detail, MicroVM toggle, vnode attach, NodeLink attach, runner-provider choice. | For each cell: in dev.jaah.ai open the node's detail and every place that offers a node capability. Each unavailable feature is absent or disabled, with a readable reason (not truncated, not a slug). Attempting it via the API returns a refusal naming the reason, never a 500 or a hang. Each available feature is offered. | | C-9 No regression on existing nodes | Existing 26.04 x86_64 and arm64 nodes (cloud-launched and self-hosted) and existing macOS and Windows nodes upgrade to the new package and keep working. Their install command is unchanged. | After the deploy to dev, the existing dev nodes report the new version and Online. One task each completes on an existing Linux, macOS and Windows dev node. `./dc check full` is green. | | C-10 Available in dev, on by default | No flag, setting or plan feature gates support for S. It is live on the dev control plane for every operator once merged and deployed. | The flows C-1 to C-8 are run on dev.jaah.ai with default settings. No setting was changed to enable them. |

Outcome

Merged #7273 (full gate green). Linux nodes now report their distro and EFS helper; the API stores them and gates MicroVM, vnode hosting and NodeLink on them, with reasons. Console shows "Ubuntu 26.04 · Arm". Supported matrix is in packaging/linux-support.toml. Visual check passed. Not yet verified on dev: my deploy was superseded by a later one (still queued). Memory: none applied.

Pull request

Artifacts

Activity

  • Draft
  • Ready
  • Runner started
  • Runner ended (done)