Instance-type allow-list table core

Task 5647 · Done · Coder · created · updated

## Original requirement (verbatim, epic) Move all product related configuration from Settings to Products section in the Operator. Operator's ask, verbatim: "Create a task to merge these two concepts. User should be able to manage the list of supported instances - create UI with support of family, lookup from actual aws, showing params (cpu, ram, other) and price on-demand and current spot. user adds which we want to allow. on the hosting products section - when adding products only allow selecting from available, prefill the name from instance type, also allow easy navigation, by family, etc. show exclamation mark if the product has an instance type that is no longer allowed." Itemized requirement 1 (verbatim): "**Allowed instance types become a managed list, not a text knob.** It is one operator-owned set, the single source both the cloud-node launch path and the hosting products read. It replaces `CLOUD_NODE_INSTANCE_TYPES` and carries the current values over in a migration." Why (verbatim): products sold `m6i.2xlarge`/`m6i.4xlarge` while the allow-list held `m8i.*`; a tenant was charged $620 for a Dedicated host that `cloud_node_launch.require_allowed_instance_type` then refused. Full PRD: Task 5632; full design: Task 5633 (read with `task_read` if you need more than this body). ## This child (C1): the allow-list core — table, owner module, seam swap, knob retirement, list/disallow API You own the ONE home of the allow-list. Siblings build on your contracts below; keep them exact. ### Approach (settled by the Architect — realize, do not reopen) - **AD1 seam swap.** Keep `cloud_node_launch.allowed_instance_types()` and `require_allowed_instance_type()` (`apps/api/app/cloud_node_launch.py:80-128`) with UNCHANGED signatures; their bodies delegate to a new module `apps/api/app/instance_types.py`. Every existing reader then follows with no call-site change: launch `cloud_node_launch.py:778`, `node_group_reconcile.py:345`, `routes/node_groups.py:266`, `routes/nodes.py:2280`, `hosting_provisioning.py:319`, `routes/cloud_nodes.py:471`, and the `Knob.instance_type` select/validation in `operator_settings.py` (~`:2642`, `:2792`, empty-list message `:2795`, doc comment `:94`). - **Pinned contract §4.1:** ```python # apps/api/app/instance_types.py (new; the ONE home of the allow-list) def allowed(session: Session | None = None) -> list[str]: ... # ordered by position; [] on DB error (fail-closed, logged at warning) def allowed_set(session: Session | None = None) -> frozenset[str]: ... # the stale predicate's input def rows(session: Session) -> list[AllowedInstanceType]: ... # with spec snapshot, ordered by position NOT_ALLOWED_HINT: Final = "Products › Instance types" # cloud_node_launch.allowed_instance_types(session=None) -> instance_types.allowed(session) # require_allowed_instance_type messages: # empty: "no EC2 instance types are allowed — allow one in Products › Instance types before creating a cloud node" # refused: "instance type '{t}' is not allowed — Products › Instance types permits: {a, b}" ``` AD2: no cache — one indexed SELECT; uses caller's session or opens `SessionLocal()` like `operator_settings._read_overrides`. - **Table §4.2:** migration `0418_allowed_instance_types` (next after `0417_automations`; renumber at rebase if taken). Table `allowed_instance_types` with the `EntityMixin` entity shape (no slug; `name` = the instance type) plus `instance_type TEXT NOT NULL UNIQUE` (regex `hosting_prices.INSTANCE_TYPE_PATTERN`), `position INTEGER NOT NULL`, `family TEXT NOT NULL`, `vcpu INTEGER NULL`, `memory_mib INTEGER NULL`, `architecture TEXT NULL` (`x86_64`|`arm64`), `network_performance TEXT NULL`, `current_generation BOOLEAN NULL`, `gpu_count INTEGER NULL`, `specs_region TEXT NULL`, `specs_at TIMESTAMPTZ NULL`. Operator-global (not OrgScoped). Register it where `rental_prices` is registered: `tests/test_entity_shape.py`, `test_org_delete_order.py`. - **AD4 migration seed:** effective value = `system_state[STATE_KEY].value["cloud_node_instance_types"]` override, else `os.environ["CLOUD_NODE_INSTANCE_TYPES"]`, else the frozen literal `t4g.micro,t4g.small,t4g.medium,t4g.large,t4g.xlarge`. Parse as today (split `,`, strip, dedupe, keep order); empty → no rows; specs NULL; family = text before first `.`; position = order. Do NOT import `operator_settings` in the migration. AD5: leave the orphaned override key in `system_state` (additive migrations). - **Retire the knob:** delete `cloud_node_instance_types` from `CATALOG` (`operator_settings.py:908-918`); nothing reads `CLOUD_NODE_INSTANCE_TYPES` afterwards. **AD19:** re-anchor the headscale panel in `apps/web/src/pages/Settings.tsx:99` from `cloud_node_instance_types` to `cloud_node_tailscale_credential_id`, or the panel disappears. - **HTTP §4.3 (your half):** ``` GET /api/instance-types require_operator + system:view 200 {"types":[{"instance_type":"m8i.2xlarge","family":"m8i","vcpu":8,"memory_gib":32.0,"architecture":"x86_64", "network_performance":"Up to 15 Gigabit","current_generation":true,"gpu_count":0, "specs_known":true,"position":3, "usage":{"products":[{"id":"…","name":"…","active":true}],"node_groups":[{"id":"…","name":"…"}],"live_hostings":1}}]} DELETE /api/instance-types/{instance_type}[?confirm=true] require_operator + system:edit 204 | 404 unknown | 409 {"detail":{"message":"m8i.2xlarge is in use — disallowing stops new orders and scale-out; running machines are untouched.","usage":{…}}} ``` AD14 usage = active `rental_prices` kind=instance rows with that key, node-group lines on that type, live hostings (`node_rentals.instance_type`, status not terminated/failed). `memory_gib` = `memory_mib/1024`. `specs_known` = vcpu and memory present. Add `SENTENCE_TEMPLATES` entries for both handlers (`tests/test_activity_kinds.py` enforces it); audit rows come from the middleware. - **Tests that set the knob today** (~13 files: `tests/test_cloud_node_launch.py`, `test_cloud_node_routes.py`, `test_node_groups_routes.py`, `test_node_group_reconcile.py`, `test_nodes_route.py`, `test_hosting_provisioning.py`, `test_plan_bundles.py`, `test_virtual_nodes.py`, `test_operator_settings.py` tier census, web `Settings.test.tsx`, `Settings.roles.test.tsx`): add ONE shared test helper/fixture that seeds `allowed_instance_types` rows, and switch them to it — no per-file copies. Update `apps/web/src/types.ts:841` comment and `infra/jaah/terraform/hosting/README.md` step 3 wording to name Products › Instance types. ### Acceptance criteria (verbatim from PRD; your share) - **AC2.1** The allow-list is a set of DB rows managed only from Products › Instance types; `CLOUD_NODE_INSTANCE_TYPES` / `cloud_node_instance_types` is no longer read by any code path and no longer shown. - **AC2.2** Migration seeds the set from the effective current value, preserving order; empty effective value → empty set. - **AC2.3** Every reader uses the set: cloud-node launch, node-group create/reconcile, `routes/nodes.py` check, Dedicated provisioning, hosting-product create (R6), tenant catalogue (R7) and order start (R8). — YOUR share: launch, node-group create/reconcile, nodes.py, provisioning (via the seam). Catalogue/order/product-create are sibling C4. - **AC2.4** Empty set: launch, node-group and Dedicated order all refuse with a message naming Products › Instance types; … — YOUR share: launch and node-group refusals (the Dedicated-order and empty-state UI parts belong to C4/C6). - **AC3.1** … from the allowed list they can disallow (D8 confirmation when in use). Each change is audited like other operator setting changes (who/when/what). — YOUR share: DELETE + usage + 409 + audit. - **Quality bar:** a removed type must be refused IMMEDIATELY on every ECS task (no cache); a DB error must fail CLOSED (refuse), never allow everything; the refusal message must tell the operator exactly where to fix it (Products › Instance types), never "Settings". Credentials & reach: All API/component tests — None. DB-backed tests need Postgres — **verified**: the PR's CI runs them against Postgres (`.github/workflows/test.yml`), as does the merge lane's `./dc check full`; a slot without Postgres relies on that CI run as its DB oracle. Out of scope: Do NOT touch the AWS browse/allow endpoints (C2), the catalogue/order/rental-price guards (C4), any Products page UI (C5/C6/C7). Do NOT delete the orphaned `system_state` override key (AD5 follow-up). Do NOT change knob keys/validation/tiers of any other knob. Do NOT stop, terminate or modify running nodes/hostings. Do NOT add boto3/botocore. Do NOT change the spot/on-demand launch market logic or `arch_for_instance_type`. Epic: Shared tiers and storage sizes (unchanged), refunding the failed order, GCP/Azure. Done means: - AC2.1 — `grep -rn "cloud_node_instance_types\|CLOUD_NODE_INSTANCE_TYPES" apps/` returns only the migration (and its test); operator-settings registry test no longer lists the knob. - AC2.2 — migration test with (a) DB override `m8i.large,m8i.xlarge`, (b) env only, (c) neither → default list, (d) empty string → empty set; order preserved. - AC2.3 (your share) — API test: remove a type from the set → `require_allowed_instance_type` / the launch route refuses it with a message that points to Products › Instance types (not Settings). - AC2.4 (your share) — API test per entry point (cloud-node launch, node-group create) with an empty set → refusal naming Products › Instance types. - AC3.1 (your share) — API test: DELETE → row gone; in-use disallow without `confirm=true` returns 409 with the usage list; with `confirm=true` deletes; activity/audit row written. - Fail-closed — unit test: DB error in `instance_types.allowed()` → `[]` and a warning log. - Headscale panel still renders in Settings (existing Settings tests green after re-anchor). - `./dc check quick` green; PR CI (Postgres) green.

Outcome

Core had merged in #4941, but it forked develop into two 0419 Alembic heads and the dev deploy failed. #4954 renumbers the allow-list migration to 0420 on top of 0419. Quick gate, CI gate and the develop Jaah Deploy are all green. Full tier left to CI (no local Postgres).

Pull request

Artifacts

Activity

  • Draft
  • Ready
  • Runner started
  • Ready
  • Runner started
  • Runner ended (done)