GitHub deploy ingestion into deployments

Task 12269 · Queued · Coder · created · updated

## Requirement (verbatim) 1. "Deploy ingestion: consume forge deployment_status / workflow_run events on the existing `github` trigger source (and each supported forge's equivalent) into a `deployments` table (project, environment, status, commit range, started/finished) mapped to the merged PRs it contains. Writes the ledger's merged → deployed edge and starts its survival clock. First consumer: Jaah Deploy on the Jaah project, replacing the runner charter's manual deploy watch." Settled scope: GitHub only (D3; other forges deferred). The deploy-watch instruction stays (D10). Ledger edges are recorded as pending `production_qa_edges` rows until epic 12243 lands (D1). D5: "Each project names its production environment (Jaah: `main`). Every deploy is recorded with its environment. Only production deploys drive the deployed edge, the survival clock, suspect linkage and regression checks. A failed deploy to any environment counts as a deploy failure." Design: architecture artifact of Task 12249, §1.2 A-3/A-4/A-5/A-15/A-16, §5.1 steps 1–8, §4 `deployments.reconcile` and its keyed effects, §13 risks 2–5, 15. ## Scope (architecture C3) - Recorder handlers on `record_only` `github` triggers: `push` → `branch_pushes` (stored payload trimmed); `workflow_run` → upsert run attempt (only `deploy_workflows`); `workflow_job` completed failure → earliest failing job + first failing step; `deployment_status` → `deployment_environments` (correlate by run id in `target_url`/`log_url`, fallback `(head_sha, workflow)`; unmatched kept and re-correlated by cron for 30 min). Pin `deployment_status` correlation fields with recorded fixtures first. - A-4 classification (`shipped`, `status` with conclusion winning, `skipped` for success-without-shipped, anchor = shipped to production AND `run_event == 'push'`; dispatch runs never anchor; environment fallback). - `reconcile` (idempotent, `reconciled_hash`-keyed): range walk over `branch_pushes` (first/resolved/partial/pending/rollback, 500-hop cap), `deployment_commits`, `deployment_prs` from `merge_lane_rows.merge_sha` ∪ `task_pr_changes.merge_sha`; `edges.on_deployment` → `deployed` edge per PR task on anchors only; `incidents.record_deploy_failure` (type `deploy_failed`, severity `error`, fingerprint `deploy:{env}:{workflow_path}:{failing_job or 'run'}`, facts run_url/failing_job/failing_step/range/prs/run_event) once the job is known or 2 min after finish; `incidents.resolve_deploy_failures` on the next successful anchor of the same workflow/environment; calls the linkage/regression/probes stubs. - Cron job `deploy_reconcile` (pending ranges 10 min then partial, partial recompute 24 h, 2-min failure grace, 30-min status re-correlation). `branch_pushes` retention 180 days. - **Connect deploys** button in the Production QA settings slot: asks for deploy workflow path(s), creates the two `record_only` github triggers (*Deploys*: push/workflow_run/deployment_status; *Deploy jobs*: workflow_job), shows ingress URL + secret the existing way. Not gated (measurement). - `fire_intake()` wake keeps latency under 60 s. Relaxed AC (operator R2, flagged for Finish): AC-18 "log tail in body" → "run + job + failing step in body; tail fetched by the runner". This child records those facts; the filing is the incident→task child's. Credentials & reach: - UI checks on dev — `org/console/dev-login` (org Admin), `https://dev.jaah.ai` — credential **verified**; environment **deferred** for this node (private 10.60.x; operator R4: run from a tailnet node) - Scratch repo `pqa-test` + webhooks (AC-1..5, 18) — `me/users/roman@gmind.tech/github_admin` — **verified** (`credential_get` returned a token; `api.github.com` answered 200 from this node) - AC-5/19/27 on the Jaah project — `org/console/dev-login`; Jaah repo webhook config needs repo admin — **deferred** — tailnet constraint; the repo webhook is configuration the operator adds in GitHub (PRD §6 allows) - AC-29 gates — worktree only — **verified** Out of scope: Do NOT add deploy ingestion for GitLab, Bitbucket or Azure DevOps (D3). Do NOT remove or edit the Jaah project's "watch Jaah Deploy" instruction (D10). Do NOT change `.github/workflows/jaah-deploy.yml`'s deploy behaviour. Do NOT add a CP-side forge read (no compare/log API calls; A-5, R2). Do NOT raise the 1 MiB ingress body cap (A-16; deferred follow-up). Do NOT create `task_outcomes` (12243). Do NOT file tasks (incident→task child). Epic out of scope (verbatim): "building an APM or log store; replacing the customer's monitor; auto-merging a revert without the project's policy; any scoring or ranking logic (the outcome ledger epic reads the states this epic writes)." Done means: - AC-1: "within 60 s of GitHub emitting the `workflow_run`/`deployment_status` event, a row shows the project, environment `production`, status, commit range (from-sha..to-sha) and the merged PR(s) it contains, linked to their tasks." — on dev from a tailnet node with `pqa-test`, compare GitHub Recent deliveries timestamp with the row's received time; plus fixture-driven intake test asserting the row. - AC-2: "two PRs merged before one deploy appear on the same deployment row. A PR merged after that deploy's head sha appears only on the next one." — fixture sequence test (push, push, run) + live check. - AC-3: "a cancelled run shows status `cancelled`, files no task, writes no incident and starts no survival clock." Superseded Jaah runs (success without a successful deployment_status) record `skipped` — fixture tests from recorded Jaah Deploy payloads. - AC-4: "a successful deploy to a non-production environment shows on Deployments with its environment. Its PRs get no deployed edge and no survival clock." — test: non-production success → zero `production_qa_edges`; production push anchor → one pending `deployed` edge per PR task. - AC-5: Jaah Deploy runs on `develop` (env `dev`) and `master` (env `main`) appear within 60 s once the operator adds the webhook (deferred to the operator's webhook configuration; fixture test with recorded Jaah payloads now). - Deploy failure: `failure` writes one `deploy_failed` incident per failing job; a second failure of the same job updates it; a different job is a separate incident; the next successful anchor resolves it — tests. `workflow_dispatch` runs never anchor; rollback detection; out-of-order events converge (reconcile re-entrancy test). - AC-26: all of the above with `production_qa` OFF still writes rows — test. - AC-29: `./dc check quick` green; changed-area tests green; merge lane full gate green.

Activity

  • Draft
  • Ready