Amazon Linux rpm channel and dnf

## Original requirement (verbatim) > Make sure Jaah nodes can install and work on Graviton-based AWS instances, earlier version of Ubuntu. It is ok if some features might not work, such as microVM. Should work with the same installation command with any arch specifics inside the bootstrap Operator scope additions (verbatim, PRD task 11088): "Install higher version of python on older versions of Ubuntu." · "Also add to scope Amazon Linux two latest versions." (resolved: AL2023 + the newest after it = AL2027 preview, architecture O2) · "Both graviton and x64 must work on supported versions of ubuntu or amazon linux" · "24.04 is supposed to have rootless Docker. Double check the sources" **Supported matrix S** (single home: `packaging/linux-support.toml`, created by the Foundation child "Report Linux distro/arch, gate capabilities"): x86_64 + aarch64 × Ubuntu 22.04, 24.04, 26.04, Amazon Linux 2023, Amazon Linux 2027 (preview) = 10 cells. Design inputs (read with `task_read`): PRD = task 11088, architecture = task 11089. §/D/R references below are the architecture's; C-n are the PRD's acceptance criteria. D18 applies to every build child: prove pre-merge with unit/integration tests plus container tests (`docker run --platform linux/{amd64,arm64}` under qemu binfmt, mem-26702 method); post-merge, after Jaah Deploy to dev succeeds, launch the child's own EC2 cells and fix forward. ## Scope (architecture Child C; §3.4, §3.5 jaah-pkg/pkgmgr, D7, D8, D10, D11-AL, D12, D15, R2) 1. **RPMs**: `build_jaahd_bundle.py --deb --rpm` (packager parameter in `_nfpm_package`, `:433-458`) builds `jaah` (noarch) and `jaah-python` rpm for x86_64 + aarch64 (reusing the interpreter child's pins). rpm `Requires: jaah-python >= 3.14.7`. **D8**: `packaging/postinst` stays the single maintainer-script home; rpm `%post` is a wrapper calling it with `configure` (`$1` ≥ 1). Account creation moves to `groupadd/useradd` (drop the `adduser` Depends); AppArmor steps run only when `apparmor_parser` exists. **D10**: MicroVM-only deps (`libseccomp2 libcap-ng0 dosfstools mtools nftables acl uidmap`) stay `Depends` in the deb, are weak deps in the rpm. 2. **Channel layout (D7)**: `s3://jaah-<env>-node-artifacts-<acct>/apt/rpm/<basearch>/` with `Packages/*.rpm` and signed `repodata/` (`createrepo_c`, `gpg --detach-sign --armor repodata/repomd.xml`, same GPG key); `jaah` noarch placed in both arch dirs. **No Terraform/IAM change** (existing grants are `apt/*`). 3. **Route**: `GET /api/agent/rpm/{object_path:path}` in new `routes/agent_rpm.py`; key allow-list `apt/rpm/(x86_64|aarch64)/(repodata/[^/]+|Packages/[^/]+\.rpm)`, anything else → the same 404; serve a 307 via `agent_apt.presigned_redirect` (`agent_apt.py:170`); auth via `routes/agent/auth.py:161` `_check_token` through agent_apt's Basic-auth shim (`agent_apt.py:81-105`); pubkey stays at `APT_PUBKEY_PATH`. 4. **CI**: `node-build` builds both formats; `node-publish` gets an rpm step in the same `jaahd-publish` environment signing with `JAAH_APT_GPG_PRIVATE_KEY`, concurrency group `rpm-publish-{env}`, downloading existing `repodata` first so earlier versions are kept; a verify step lists the exact version per arch. 5. **Node-side helpers (§3.5)**: shell library `/usr/lib/jaahd/libexec/jaah-pkg` (`pkg_kind`, `pkg_arch`, `pkg_install`, `pkg_installed_version jaah`, `pkg_refresh_jaah`, `pkg_install_jaah <v>` (`apt-get install --allow-downgrades jaah=<v>` | `dnf -y install --allowerasing jaah-<v>`), `pkg_remove_jaah`) used by `jaahd-update.sh` (`:313-445`), `jaah-uninstall` (`:100-154`), `jaah-gha-converge.sh`, `jaah-runner-clis-vn`, `postinst`. Python twin `infra/jaah/pkgmgr.py` (apt|dnf) used by `node_reprovision.py` (all `_apt_install` sites), `apps/jaahd/main.py:822`, `jaah_cli.py:618`, `upgrade_ops.py`, `environment.py`, `plugin_repair.py`. **Exhaustive gate test**: `rg -n "dpkg|apt-get|apt-cache|dpkg-query" apps/jaahd packaging infra/jaah infra/host --glob '!**/tests/**' --glob '!packaging/macos/**' --glob '!packaging/windows/**'` returns only the two helpers, `microvm_seed.py`, and lines with `# apt-only:`. 6. **dnf convergence**: nodejs22, tmux, git, bubblewrap from AL repos; ripgrep as a pinned upstream static binary per arch in `node_reprovision.py` `_RELEASE_ARCH` (`:249-268`) (D12); AL's own `amazon-efs-utils` (D11). 7. **Runner providers (D15)**: all five providers install on every cell; fix the `dpkg --print-architecture` gates that silently skip Antigravity off Debian (`postinst:806,855`, `jaah-runner-clis-vn:79,107`). If a provider proves impossible on a cell, add a `runner_cli:<p>` row to `node_platform.node_reason` keyed on distro/arch with a reason — never a silent skip. 8. Hand the rpm repo to the bootstrap child: the route serves real published rpms on dev after merge. Credentials & reach: - Publishing rpm to dev's bucket — the existing `jaahd-publish` environment and its `apt/*` grants (D7) — needs nothing new. - Deploy to dev (C-5, C-9) — merge → `jaah-deploy.yml` develop path — needs nothing beyond the worktree. - Container tests (D18) — docker + qemu binfmt on the runner's node (mem-26702) — needs nothing beyond the worktree host. - Gates — `./dc check quick` / `full` — needs nothing beyond the worktree. - Launch and terminate tagged EC2 test cells (post-merge) — `me/jaah/system/bootstrap-admin` → assume `OrganizationAccountAccessRole` in the `jaah-dev` org account, `us-east-1` — **verified** (broker fetch succeeded; `sts get-caller-identity` returned the assumed role in jaah-dev). O4 authorises mutations, limited to instances tagged `jaah:test=task-<id>`. - AMIs for every cell — SSM public parameters `/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-6.1-{x86_64,arm64}`, `…/al2027-preview-ami-kernel-7.1-{x86_64,arm64}`, `/aws/service/canonical/ubuntu/server/...` — **verified** (listed from h2). - Dev console UI and API, incl. C-7 API refusals — `org/console/dev-login` (operator) at `https://app.dev.jaah.ai` — **verified** (broker fetch succeeded; console returned 200 from h2). - Existing dev nodes for C-9 — dev console **Nodes** list, signed in as `org/console/dev-login` — **verified** (inventory at design time: 11 Ubuntu x64, 1 Ubuntu arm64, 1 Windows x64). Out of scope: Do NOT touch / do NOT do — - Do NOT create a new S3 prefix, IAM grant or Terraform change (D7). - Do NOT write the bootstrap dnf branch or detection prelude (bootstrap child) — this child's AL cells are proven end-to-end there. - Do NOT install rootless Docker on AL (rootless-Docker child) or change GHA runner install (GHA child). - Do NOT change the deb's MicroVM Depends (D10, C-9). - Epic-wide negative constraints (PRD §5, verbatim): - MUST NOT make MicroVM work on arm64 Linux, or on any newly supported release. It is reported unavailable (C-7) unless it already works there. - MUST NOT change the cloud-launched EC2 image (`apps/api/app/aws/ec2.py` `UBUNTU_RELEASE` / `_UBUNTU_IMAGE_NAME`) or add an OS picker to the EC2 create flow. The stale comment at `ec2.py:1111` is the one allowed edit there (C-6). - MUST NOT change the macOS or Windows installers, bundles or their install commands (`node_bootstrap.py` macOS and Windows sections, `packaging/macos/`, `packaging/windows/`). - MUST NOT replace, relink or upgrade the distro's system `python3` (`/usr/bin/python3`). - MUST NOT add an OS or arch argument, flag or variant to the console's install command. - MUST NOT support Ubuntu 20.04, Debian, RHEL, Amazon Linux 2, or architectures other than x86_64 and arm64 as **supported**. They get only C-8's warn-and-try. - MUST NOT gate the support behind a feature flag, plan feature or setting. - MUST NOT bump unrelated pins in `packaging/platform-pins.toml`, `packaging/microvm-pins.toml` or `packaging/runner-cli-locks/` beyond what a supported cell needs to install. Done means: - Container oracle: `amazonlinux:2023` × `linux/amd64` + `linux/arm64` installs `jaah` + `jaah-python` rpm from a local signed repo with `gpgcheck=1 repo_gpgcheck=1`; `%post` runs `postinst configure` to exit 0; a second install of a newer rpm runs the upgrade path without skipping steps (R2); `jaah-pkg` `pkg_kind` → dnf; the same image run of `ubuntu:24.04` → apt — cite output. - dnf convergence in both `amazonlinux:2023` arch containers: `rg --version`, `node --version` (v22), `tmux -V`, `bwrap --version`, `mount.efs --version` all succeed — cite output. - Route tests: allowed keys 307 to a presigned URL; any other key and a bad token return the same 404/401 as apt; token not logged. - Exhaustive package-manager gate test green; all five provider CLIs install in the AL containers (or a named `runner_cli` reason row exists, with a test). - Post-merge: Jaah Deploy to dev succeeded and its rpm verify step lists the exact version for x86_64 and aarch64; `dnf` against `<dev cp>/api/agent/rpm/$basearch` from an AL2023 EC2 instance (tagged `jaah:test=task-<id>`) resolves `jaah` and `jaah-python` with signature checks on. - No regression on deb nodes: existing dev 26.04 nodes upgrade through `jaah-pkg` and stay Online (C-9). - `./dc check quick` green; merge lane `./dc check full` green. PRD acceptance criteria this child carries (verbatim; prove the cells named above, others are proven by their owning child or the sweep): | ID | Criterion | Verify (oracle) | |---|---|---| | C-9 No regression on existing nodes | Existing 26.04 x86_64 and arm64 nodes (cloud-launched and self-hosted) and existing macOS and Windows nodes upgrade to the new package and keep working. Their install command is unchanged. | After the deploy to dev, the existing dev nodes report the new version and Online. One task each completes on an existing Linux, macOS and Windows dev node. `./dc check full` is green. |

Merged #7294 (c066f8a06, lane full gate green): jaah/jaah-python rpms, signed dnf channel under apt/rpm + /api/agent/rpm, jaah-pkg/pkgmgr, dnf convergence; AL2023 containers pass on both arches. Dev rpm verify lists 1.3.71/1.3.73. Open: EC2 dnf check and C-9 wait on dev rollout, blocked by 11131's migration (draft #11175). Memory: none applied.