GCP compute adapter and credential
## Requirement
Original requirement (epic, verbatim): "Add support for GCP and Azure with the same functionality as AWS:
- Your Cloud (former Your AWS)
- NodeLink may work on GCP and Azure
- What else we have AWS specific
Out of Scope: Jaah Hosted (still remains AWS only)"
Acceptance this child owns for GCP (PRD, verbatim):
- R1 Two new credential types: **GCP** (service-account JSON key; project id derivable from it) and **Azure** (tenant id, client id, client secret, subscription id). Secrets sealed and tested like the `aws` type. (This child: GCP.)
- AC1.1 Settings › Credentials › Add offers "GCP" and "Azure"; saving a valid key and pressing Test shows success; an invalid/revoked key shows the provider's own error text, never the secret. → verify: on dev CP add each type with the operator's real key and with a deliberately wrong secret; observe pass and the readable refusal.
- AC1.2 No secret value appears in any API response, log line, or the credential list. → verify: open the saved credential's edit form (secret fields empty/masked), and `GET` the credential via the console's network tab — no secret field value.
- AC3.1 From Add node › Your Cloud, choosing a GCP (then an Azure) connection and an allowed machine type launches an Ubuntu VM (same Ubuntu release AWS uses) in the customer's account; the node appears in Machines, reaches online (daemon connected over the tailnet) and can run a task. → verify: on dev CP launch one node per provider, dispatch a trivial task to it, observe it complete.
- AC3.2 Node detail on a GCP/Azure node offers the same actions an AWS node offers today — stop, start, change machine type, grow disk, delete — and each takes effect in the provider (observed in the provider console) and in Machines. Delete removes the VM and its disk. (This child: the GCP adapter calls.)
- AC3.3 A launch whose first zone has no capacity falls back to the connection's other zones; a permanent failure leaves no orphan DB row and shows the provider's error text. (This child: GCP `is_capacity_error` + `placements`.)
- AC4.1 With `windows_nodes_allowed` ON, the OS control offers Windows on GCP/Azure connections for machine types that run it (as for AWS, Task 8744); the Windows node launches, its Jaah service connects and it runs a task. With the switch OFF, Windows is not offered on any provider. (This child: GCP Windows image + bootstrap transport + `windows_capable` in catalogue.)
- AC5.1 (data) Products › Instance types … lists that provider's machine types with specs and current on-demand and spot $/hour from the provider's pricing source … A price that cannot be read shows a one-line reason, never blocks the list.
- AC7.1 Every GCP/Azure node is launched reachable by the provider's own shell path when its daemon/tailnet is dead (GCP: OS Login / IAP SSH; Azure: Run Command or serial console), set up best-effort like the SSM profile (a credential that lacks the permission still launches, with a warning naming what to grant). → verify: on a dev node stop the Jaah daemon, then open a shell through the provider's console.
- AC7.2 GCP/Azure launches are MicroVM-capable: the node reports `kvm_usable = true` on allowed types that support nested virtualisation, and a MicroVM session runs on it when `microvm_allowed` is on. → verify: mark a GCP and an Azure node MicroVM, dispatch a task, observe it run in a VM.
- AC7.3 Every resource Jaah creates in GCP/Azure carries labels/tags equivalent to AWS's `ManagedBy=jaah` plus its node/org, and the "What Jaah creates" table lists them per provider. → verify: inspect the VM, disk, firewall/NSG and storage in each provider console. (This child: GCP compute resources.)
## Scope (Architecture §3 GCP column, verbatim decisions)
- New package `apps/api/app/gcp/` on stdlib + httpx, NO cloud SDK (D1); PyJWT + cryptography (already pinned — update their requirements comments that name "the one importer").
- Pinned auth seams the NodeLink children import: `app.gcp.auth.token(cred: Credential, scopes: Sequence[str]) -> str` (cached in-process per credential id + updated_at, never logged; raises `CloudNodeError` with Google's readable text, never a secret) and `app.gcp.rest.call(cred, method, url, json=None) -> dict` incl. long-running operation polling.
- Credential type `gcp` in `apps/api/app/credentials/registry.py`: `service_account_json` (secret), `project_id` (optional, derived from the key); Test = mint token + `GET compute/v1/projects/{p}`.
- Network auto-resolve: NOT the auto-mode `default` network: blank network ⇒ create custom-mode VPC `jaah-vpc` + regional subnetwork `jaah-nodes` 10.88.0.0/20, only ingress rule `jaah-node-iap` (IAP `35.235.240.0/20`, tcp:22 and tcp:3389, target tag `jaah-node`); a customer-named network is used as-is and resolve warns when it finds an allow rule from 0.0.0.0/0 targeting the node. Egress: customer subnet ⇒ no public IP; auto-created `jaah-vpc` ⇒ ephemeral external IP per VM.
- Connection fields: `project_id` (defaults from the key), `zones`, `network`, `subnetwork`, `firewall`.
- Zone fallback via connection `zones`; capacity = `ZONE_RESOURCE_POOL_EXHAUSTED`/`…_WITH_DETAILS`.
- Idempotent launch: `instances.insert?requestId=<uuid5(node_id, zone)>` + instance name `jaah-node-<node>`; `find` by name.
- Image: `ubuntu-os-cloud` family for the same Ubuntu release AWS resolves; Windows `windows-cloud` family `windows-2022`.
- Bootstrap: Linux metadata key `user-data` (cloud-init GCE datasource); Windows metadata `sysprep-specialize-script-ps1` (runs once; must tolerate a second run). VM launched with NO service account; `after_enrolled` removes `user-data` / specialize metadata via `instances.setMetadata`.
- Nested virt: `advancedMachineFeatures.enableNestedVirtualization=true` when catalogue says capable (not E2, not ARM, not AMD except N4D).
- Break-glass: Linux metadata `enable-oslogin=TRUE` + IAP SSH (tcp:22); Windows IAP RDP (tcp:3389) + `serial-port-enable=TRUE`; warning names missing `compute.firewalls.create` and operator role `roles/iap.tunnelResourceAccessor`.
- Labels `managed-by=jaah`, `jaah-node`, `jaah-org-id`, `jaah-purpose` (lowercase, label charset).
- Stop/start `instances.stop`/`start`; change type `instances.setMachineType` (stopped); grow disk `disks.resize`; terminate `instances.delete` (boot disk `autoDelete=true`).
- Catalogue `machineTypes.list` in a zone of the region; prices from Cloud Billing Catalog `services/6F81-5844-456A/skus` with the GCP credential's OAuth token (`cloud-platform` scope, NO API-key field): vCPU×core SKU + GiB×RAM SKU per series; spot = "Spot Preemptible" SKUs; shared-core/GPU types and a disabled API ⇒ one-line reason ("Enable the Cloud Billing API in <project>").
- Register the adapter in `REGISTRY`. Add GCP rows to the "What Jaah creates" table in the `cloud_node_launch.py` docstring and the credential reader-doc wording (`apps/docs/content/using-jaah/credentials/add-a-credential.md`).
Credentials & reach: Unit/integration tests | none beyond the worktree (`httpx.MockTransport` recorded fixtures) | **verified**: nothing external. GCP live oracles (AC1–AC7 GCP) | `jaah/test/gcp` (type GCP): service-account key in one dev project. Roles: `roles/compute.instanceAdmin.v1`, `roles/compute.networkAdmin`, `roles/compute.securityAdmin`, `roles/file.editor`, `roles/serviceusage.serviceUsageConsumer`. APIs enabled: `compute.googleapis.com`, `file.googleapis.com`, `cloudbilling.googleapis.com`. Org policy `compute.disableNestedVirtualization` must be off for the project (AC7.2). No `iam.serviceAccountUser`. The human doing break-glass needs `roles/iap.tunnelResourceAccessor` + `roles/compute.osAdminLogin`. | **deferred**: not on the roster; operator BA D6 "You'll add creds later — create stubs for creds and tell me to fill in". Live runs happen in the epic's "Dev oracle run" child.
Out of scope: Do NOT touch Jaah Cloud / Jaah Hosted (N1) or Jaah's platform AWS usage under `apps/api/app/aws/` and `apps/api/app/bedrock/` (N2). No GCP equivalent of the hosting account, headscale, or control-plane infra in `infra/jaah/terraform/` (N3). No node groups / Spot market on GCP (N4). No Workload Identity Federation (N5). No macOS (N8). No `google-cloud-*` SDK (D1). No migration. Do NOT edit shared orchestration, routes or console UI (other children); do NOT build Filestore/NodeLink (NodeLink children) beyond the pinned auth/rest seams. No `if provider ==` branching outside `app/gcp/` and the adapter.
Done means:
- AC1.1/AC1.2 (GCP) — verify: tests: a valid fixture key Tests green; a revoked-key fixture surfaces Google's error text; no response/log/exception string contains any part of the private key (assert on captured logs and the credential GET body); live dev-CP run (deferred — `jaah/test/gcp` not yet filled; runs in "Dev oracle run").
- AC3.1/AC3.2/AC3.3/AC4.1 (GCP adapter) — verify: MockTransport tests per method assert the exact request (requestId, name, metadata keys, no service account, `autoDelete`, nested flag only on capable types, Windows uses `sysprep-specialize-script-ps1`), capacity codes map to `is_capacity_error`, `find` recovers by name, `terminate` deletes the boot disk; FakeProvider-shared orchestration test passes with the GCP adapter registered; user-data within `user_data_limit`. Live (deferred — as above).
- AC5.1 data — verify: price test pins n2-standard-4 and e2-standard-4 against the catalog fixture; e2-micro returns a one-line reason, never raises; disabled Billing API ⇒ list still returned with the reason.
- AC7.1–AC7.3 (GCP) — verify: tests assert OS Login/serial metadata, the `jaah-node-iap` rule, labels on VM/disk/firewall/VPC; a credential lacking `compute.firewalls.create` still launches with a warning naming it; the docstring table has GCP rows. Live (deferred).
- AC9.1 — verify: no provider-name branching added outside `app/gcp/` + its adapter (grep).
- `./dc check quick` green; merge lane `./dc check full` green; Jaah Deploy of the PR succeeds.
Outcome
GCP adapter (app/gcp: auth token, rest+LRO, network, catalog/prices, GcpProvider registered) and gcp credential type + Test; quick gate + 1397 tests green; review lane pass; PR #6552 merged (d36ff2ff), Jaah Deploy succeeded. Windows uses startup-script, not specialize (documented). Live GCP runs deferred to oracle child. Memory: none applied.
Artifacts
Activity
- Draft
- Ready
- Runner started
- Runner ended (done)