Azure compute adapter and credential

Task 8775 · Done · Coder · created · updated

## Requirement Original requirement (epic, verbatim): "Add support for GCP and Azure with the same functionality as AWS: - Your Cloud (former Your AWS) - NodeLink may work on GCP and Azure - What else we have AWS specific Out of Scope: Jaah Hosted (still remains AWS only)" Acceptance this child owns for Azure (PRD, verbatim): - R1 Two new credential types: **GCP** (service-account JSON key; project id derivable from it) and **Azure** (tenant id, client id, client secret, subscription id). Secrets sealed and tested like the `aws` type. (This child: Azure.) - AC1.1 Settings › Credentials › Add offers "GCP" and "Azure"; saving a valid key and pressing Test shows success; an invalid/revoked key shows the provider's own error text, never the secret. → verify: on dev CP add each type with the operator's real key and with a deliberately wrong secret; observe pass and the readable refusal. - AC1.2 No secret value appears in any API response, log line, or the credential list. → verify: open the saved credential's edit form (secret fields empty/masked), and `GET` the credential via the console's network tab — no secret field value. - AC3.1 From Add node › Your Cloud, choosing a GCP (then an Azure) connection and an allowed machine type launches an Ubuntu VM (same Ubuntu release AWS uses) in the customer's account; the node appears in Machines, reaches online (daemon connected over the tailnet) and can run a task. → verify: on dev CP launch one node per provider, dispatch a trivial task to it, observe it complete. - AC3.2 Node detail on a GCP/Azure node offers the same actions an AWS node offers today — stop, start, change machine type, grow disk, delete — and each takes effect in the provider (observed in the provider console) and in Machines. Delete removes the VM and its disk. (This child: the Azure adapter calls.) - AC3.3 A launch whose first zone has no capacity falls back to the connection's other zones; a permanent failure leaves no orphan DB row and shows the provider's error text. (This child: Azure `is_capacity_error` + `placements`.) - AC4.1 With `windows_nodes_allowed` ON, the OS control offers Windows on GCP/Azure connections for machine types that run it (as for AWS, Task 8744); the Windows node launches, its Jaah service connects and it runs a task. With the switch OFF, Windows is not offered on any provider. (This child: Azure Windows image + Custom Script Extension + `windows_capable`.) - AC5.1 (data) Products › Instance types … lists that provider's machine types with specs and current on-demand and spot $/hour from the provider's pricing source … A price that cannot be read shows a one-line reason, never blocks the list. - AC7.1 Every GCP/Azure node is launched reachable by the provider's own shell path when its daemon/tailnet is dead (GCP: OS Login / IAP SSH; Azure: Run Command or serial console), set up best-effort like the SSM profile (a credential that lacks the permission still launches, with a warning naming what to grant). → verify: on a dev node stop the Jaah daemon, then open a shell through the provider's console. - AC7.2 GCP/Azure launches are MicroVM-capable: the node reports `kvm_usable = true` on allowed types that support nested virtualisation, and a MicroVM session runs on it when `microvm_allowed` is on. → verify: mark a GCP and an Azure node MicroVM, dispatch a task, observe it run in a VM. - AC7.3 Every resource Jaah creates in GCP/Azure carries labels/tags equivalent to AWS's `ManagedBy=jaah` plus its node/org, and the "What Jaah creates" table lists them per provider. → verify: inspect the VM, disk, firewall/NSG and storage in each provider console. (This child: Azure compute/network resources.) ## Scope (Architecture §3 Azure column, verbatim decisions) - New package `apps/api/app/azure/` on stdlib + httpx, NO `azure-mgmt-*` SDK (D1). - Pinned auth seams the NodeLink children import: `app.azure.auth.token(cred: Credential, scope: str = "https://management.azure.com/.default") -> str` (client-credentials grant at `login.microsoftonline.com/{tenant}/oauth2/v2.0/token`; cached per credential id + updated_at; raises `CloudNodeError` with Azure's readable text, never a secret) and `app.azure.rest.call(...)` incl. long-running operation polling. - Credential type `azure`: `tenant_id`, `client_id`, `client_secret` (secret), `subscription_id`; Test = token + `GET /subscriptions/{id}`. - Network auto-resolve: RG `jaah-<conn>` (create if blank), VNet `jaah-vnet` 10.88.0.0/16 + subnet `jaah-nodes`, NSG `jaah-node` (deny inbound). Egress: customer subnet ⇒ NIC only; auto-created ⇒ Standard public IP per VM (inbound denied by NSG). Connection fields: `resource_group`, `vnet`, `subnet`, `nsg`, `zones`. - Zone fallback via `zones`; capacity = `SkuNotAvailable`/`AllocationFailed`/`ZonalAllocationFailed`. - Idempotent launch: ARM PUT of named resources (`jaah-node-<node>` VM, `-nic`, `-ip`), `If-None-Match: *`; `find` by name. - Image: `Canonical` Ubuntu offer for the same release AWS resolves, Gen2; Windows `MicrosoftWindowsServer:WindowsServer:2022-datacenter-azure-edition`. - Bootstrap: Linux `osProfile.customData` (base64; cloud-init Azure datasource) — NOT `userData` (IMDS-readable); Windows via Custom Script Extension with the script in `protectedSettings` (must tolerate a second run). - Nested virt: no flag; capability = series allow-list in the adapter (Dv3+/Dsv3+/Ev3+/Esv3+/Fsv2 and later); truth stays the node's own `kvm_usable` report. - Break-glass: boot diagnostics ON (managed) ⇒ serial console; Run Command needs nothing (VM agent); warning if boot diag refused. - Tags `ManagedBy=jaah`, `jaah:node`, `jaah:org-id`, `jaah:purpose`. - Stop = `deallocate` (stops billing) / `start`; change type PATCH `hardwareProfile.vmSize` (deallocated); grow disk PATCH managed disk `diskSizeGB` (deallocated, then start); terminate DELETE VM with `deleteOption=Delete` on OS disk, NIC, public IP. - Catalogue `resourceSkus` (filter location, resourceType virtualMachines); prices from the Retail Prices API (unauthenticated) `armRegionName`+`armSkuName`, `priceType Consumption`, spot via meterName "Spot"; miss ⇒ one-line reason. - Register in `REGISTRY`; add Azure rows to the "What Jaah creates" table in the `cloud_node_launch.py` docstring and the credential reader-doc wording. Credentials & reach: Unit/integration tests | none beyond the worktree (`httpx.MockTransport` recorded fixtures) | **verified**: nothing external. Azure live oracles (AC1–AC7 Azure) | `jaah/test/azure` (type Azure): service principal with `Contributor` on the dev subscription (it creates the `jaah-<conn>` resource group); resource providers registered: `Microsoft.Compute`, `Microsoft.Network`, `Microsoft.FileShares` | **deferred**: same operator answer D6 ("You'll add creds later — create stubs for creds and tell me to fill in"); live runs happen in the epic's "Dev oracle run" child. Out of scope: Do NOT touch Jaah Cloud / Jaah Hosted (N1) or Jaah's platform AWS usage under `apps/api/app/aws/` and `apps/api/app/bedrock/` (N2). No Azure equivalent of the hosting account, headscale, or control-plane infra in `infra/jaah/terraform/` (N3). No node groups / Spot on Azure (N4). No Azure federated credentials (N5). No macOS (N8). No `azure-mgmt-*` SDK (D1). No migration. Do NOT edit shared orchestration, routes or console UI; do NOT build Azure Files/NodeLink beyond the pinned auth/rest seams. No `if provider ==` branching outside `app/azure/` and the adapter. Done means: - AC1.1/AC1.2 (Azure) — verify: tests: valid fixture Tests green; a wrong-secret fixture surfaces Azure's AADSTS text; no response/log/exception contains the client secret (assert captured logs + credential GET body); live (deferred — `jaah/test/azure` not yet filled; runs in "Dev oracle run"). - AC3.1/AC3.2/AC3.3/AC4.1 (Azure adapter) — verify: MockTransport tests assert the exact ARM bodies (`customData` not `userData`, `If-None-Match: *`, `deleteOption=Delete` on disk/NIC/IP, CSE `protectedSettings` for Windows, deallocate for stop), capacity codes map, `find` by name; shared FakeProvider orchestration test passes with Azure registered. Live (deferred). - AC5.1 data — verify: Retail Prices fixture test returns on-demand + spot for two sizes; a missing meter returns a one-line reason, list still returned. - AC7.1–AC7.3 (Azure) — verify: tests assert boot diagnostics enabled, tags on VM/disk/NIC/IP/NSG/VNet/RG; a refused boot-diag call still launches with a warning; docstring table has Azure rows. Live (deferred). - AC9.1 — verify: no provider-name branching added outside `app/azure/` + its adapter (grep). - `./dc check quick` green; merge lane `./dc check full` green; Jaah Deploy of the PR succeeds.

Outcome

Shipped app/azure (Entra auth, ARM REST+LRO, Retail Prices, AzureProvider registered) and the azure credential type/Test; quick gate + 1123 tests green, review lanes pass; PR #6550 merged (c3b67a0), Jaah Deploy succeeded. Live Azure oracles deferred until jaah/test/azure is filled. Memory: none applied.

Pull request

Artifacts

Activity

  • Draft
  • Ready
  • Runner started
  • Runner ended (done)