Move AWS onto provider layer

Task 8773 · Done · Coder · created · updated

## Requirement Original requirement (epic, verbatim): "Add support for GCP and Azure with the same functionality as AWS: - Your Cloud (former Your AWS) - NodeLink may work on GCP and Azure - What else we have AWS specific Out of Scope: Jaah Hosted (still remains AWS only)" Operator constraint (PRD R9, verbatim): "Make sure as much code as possible is shared between all three providers with minimum specific." Acceptance this child owns (PRD, verbatim): - AC9.1 The provider-specific part of each capability (launch, lifecycle, network resolve, image resolve, pricing, storage, break-glass, labels) sits behind one provider interface; orchestration, durable workflow, bootstrap, routes, UI flow and tests are shared, with AWS refactored onto the same interface. → verify (Architect/code-review): the routes and `launch_cloud_node`-level orchestration contain no `if provider == ...` branching; a provider is added by implementing the interface and registering it. - AC3.4 AWS behaviour is unchanged: an AWS launch, stop/start, resize, delete still pass. → verify: repeat AC3.1–3.2 on an AWS connection on dev CP; existing AWS test suites green in `./dc check full`. - AC3.3 A launch whose first zone has no capacity falls back to the connection's other zones (parity with AWS extra subnets); a permanent failure leaves no orphan DB row and shows the provider's error text. → verify: in a test, inject a capacity error on zone 1 and observe success on zone 2; on dev, launch with an invalid machine type and observe the readable refusal and no stray node row. (This child: the SHARED loop, tested with FakeProvider.) - AC2.4 (API half) Only providers whose plan switch is ON for the org appear in the provider picker; a provider OFF renders nothing (no upsell), and the API refuses it with 403 "<Label> is not on your plan. Upgrade in Billing." → verify: act as a tenant with GCP off: picker lacks GCP; a direct POST of a GCP connection returns 403. - AC2.3 (API half) Blank network fields auto-resolve at save to a default the row then shows (parity with AWS "Auto — default VPC"). - AC5.2 (API half) Add node's Instance type list for a connection shows only that provider's allowed types. - AC8.3 Every server door (connection create/update, launch, lifecycle actions, storage create) enforces the provider switch. → verify: API test per door returns 403 with the switch off. (This child: connection create/update/lookup, launch, browse/allow.) ## Scope (Architecture §1, §2.2, D2, D3, D9, D10, D12) 1. AWS adapter in `apps/api/app/clouds/` implementing the protocol from the Foundation child by wrapping the existing `app/aws/*` client — **no AWS request bytes change** (risk 8). Register it in `REGISTRY`. 2. `cloud_node_launch` public functions (`launch_cloud_node`, `terminate_cloud_node`, …) stay facades with unchanged signatures, routing through `REGISTRY[infra.provider]` (D2) — 12 modules import them incl. Hosting and node groups, which must keep working untouched. 3. Shared fallback loop in `launch_cloud_node` over `placements()` + `is_capacity_error()` (D3), with `find()` recovery of a lost launch answer; durable `provision_cloud_node` (`durable/cloud_node.py:635`) persists the `InstanceRef` in `Node.infra` (`provider`, `zone`, `scope`). 4. HTTP API §2.2: `GET /api/cloud-connections/providers` → `[{key, label, account_noun, credential_type, fields}]`, only switched-ON providers; `GET /api/cloud-connections/lookup/{field}?provider=&credential_id=&region=&<parent>=` (old `regions`/`subnets`/`security-groups` paths stay as aliases); create/update body `{provider, name, description, credential_id, region, root_volume_gb, image, network}` (AWS also accepts legacy flat fields), response always carries `provider` and the RESOLVED `network`. Remove the `_AWS_PROVIDER` refusal (`routes/cloud_nodes.py:82-84, 255-260`) in favour of `require_provider`. 5. Node read gains `cloud: {provider, provider_label, account_noun, instance_id, region, zone, instance_type, disk_gb, state, actions}` for a Your Cloud node, `null` otherwise; `actions` computed server-side — this child ships `["delete"]` only; the lifecycle child adds the rest. 6. Instance types: allow-list read gains `provider` filter; `GET /api/instance-types/browse?provider=&region=` and `POST /api/instance-types/allow {provider, instance_type}` via `catalogue()`/`prices()` (fills `windows_capable`); AWS browse behaviour unchanged. 7. Node groups refuse a non-AWS connection with 422 "Node groups run on AWS only for now." (N4). 8. `after_enrolled` call site on the existing first-connect path, best-effort (a failure logs and retries next heartbeat) (D12) — adapters own the body. 9. D9: when a node launched into a customer-supplied subnet never connects, the launch row shows "Node did not connect — the subnet needs outbound internet (NAT or public IP)" after the existing connect timeout. Recon: `apps/api/app/cloud_node_launch.py:865` (launch), `:1106` (_run_instances), `:1154-1330` (power/resize), `:705-716` (subnet fallback); `apps/api/app/cloud_connection_network.py`; `apps/api/app/routes/cloud_nodes.py`; `apps/api/app/instance_types.py`, `apps/api/app/instance_type_browse.py`; tests monkeypatch module functions / `httpx.MockTransport` (`tests/test_cloud_node_launch.py:697`). Credentials & reach: Unit/integration tests in children | none beyond the worktree (`./dc check quick`/`full`, httpx MockTransport fixtures) | **verified**: nothing external. AWS regression on dev CP (AC3.4) | roster `gmind/system/gmind-aws`, copied to dev credential `jaah/test/aws` by the epic's "Dev credential stubs" child | **verified** for the Architect stage; the live run happens in the epic's "Dev oracle run" child. Out of scope: Do NOT touch Jaah Cloud / Jaah Hosted (`apps/api/app/hosting_*.py`, `HostingPicker.tsx`, `infra/jaah/terraform/hosting/`, rental billing) — remain AWS only (N1); its callers of the facades must work with zero edits. Do NOT touch Jaah's platform AWS usage: `apps/api/app/bedrock/`, CloudWatch node journal, SQS alarm relay, WAF, ECS, KMS, Athena, Cost Explorer under `apps/api/app/aws/` (N2). No GCP/Azure equivalent of the hosting account, headscale, or control-plane infra (N3). Dynamic node groups and Spot market stay AWS-only (N4). No keyless federation (N5). No change to how AWS behaves for the user (N7). No new migration (the Foundation child owns the only one). Do NOT build lifecycle routes (stop/start/type/disk — the lifecycle child), console UI, GCP/Azure adapters, or NodeLink. Do NOT fold AWS network columns into `network` JSONB (F3). Do NOT remove the old lookup aliases (named follow-up after one release). Done means: - AC9.1 — verify: `grep -nE 'provider *(==|!=) *["\x27]|== *"(aws|gcp|azure)"|provider in \(' apps/api/app/routes/cloud_nodes.py apps/api/app/durable/cloud_node.py` and the orchestration functions of `apps/api/app/cloud_node_launch.py` return nothing; a test registers `FakeProvider` under a new key and launches/terminates through `launch_cloud_node` with no other change. - AC3.4 — verify: `test_cloud_node_launch.py`, `test_cloud_node_workflow.py`, `test_aws_ec2*.py`, `test_hosting_*.py` pass with no edits beyond import paths; a request-capture test shows byte-identical AWS RunInstances/Terminate params before and after; live dev-CP AWS repeat (deferred — runs in the "Dev oracle run" child once `jaah/test/aws` exists). - AC3.3 shared loop — verify: a FakeProvider test raises a capacity error on placement 1 and succeeds on placement 2; a permanent error leaves no node row and surfaces the provider text verbatim (no secret). - AC2.4/AC8.3 API — verify: with GCP OFF, `/providers` omits GCP and POST/PUT/lookup of a GCP connection, launch on it and browse/allow with `provider=gcp` each return 403 with the exact text (one test per door). - AC2.3 API — verify: an AWS connection saved with blank network returns concrete resolved values; reread returns the same. - AC5.2 API — verify: allow-list read with `provider=aws` excludes rows of other providers. - Node group with a non-AWS connection → 422 "Node groups run on AWS only for now." — test. - `after_enrolled` invoked once on first connect, failure retried on next heartbeat — test; D9 message shown on a never-connecting customer-subnet launch — test. - `./dc check quick` green locally; merge lane `./dc check full` green; Jaah Deploy of the PR succeeds.

Outcome

Lane fix: rebased; GCP adapter onto the extended protocol, registry in ProviderKey order, seam test on INFRA_PROVIDER_KEY, cross-org census names `field`. Quick gate green, 411 affected tests pass; PR #6555 re-queued. Live AWS repeat deferred to Dev oracle run. Memory: used mem-23019, mem-18424.

Pull request

Artifacts

Activity

  • Draft
  • Ready
  • Runner started
  • Runner ended (done)
  • Run
  • Runner started
  • Runner ended (done)