Native post-deploy health probes
## Requirement (verbatim)
6. "Native probes (later phase): project-declared health URLs and smoke commands run from the project's own node right after a deploy, writing the same incidents table."
7 (verbatim): "Every behaviour change — filing a task, filing a revert, running probes — is behind the plan feature `production_qa` with org override and per-project opt-in."
Settled: A7 "Native probes run on a node that serves the project, after its production deploy succeeds. A smoke command runs with the same sandbox and permissions as a runner session on that node. Nothing new is granted." Architect A-8: a scripted, LLM-free persona in a CP-leased slot (the merge-lane pattern) over a new node-bus vocabulary verb; smoke commands authored by `projects:edit`. **Security surface** (commands executing on a node): route the PR through the autonomous security review lane.
Design: architecture artifact of Task 12249, §10, §1.2 A-8, §13 risk 12, fingerprints `probe:{env}:{probe_id}`.
## Scope (architecture C9)
- Settings editors in the Production QA slot: health URLs (https only, ≤10, `expect_status`), smoke commands (≤5, `timeout_s ≤ 300`), `projects:edit` only; validation server-side.
- `probes.schedule` (production push anchor, gate ON, ≥1 probe): hidden system task, persona `deploy_probe` (scripted, registered like `merge_lane.LANE_PERSONA`), `dedup_key=probe:{deployment_id}`, `started_by_id = production_qa_enabled_by`. Gate OFF → nothing scheduled.
- `tools/deploy_probe.py` in the slot: HTTP GETs (10 s, no cross-host redirects), smoke commands through the lane's `sandbox_argv`; reports `production_qa.probe_result {deployment_id, results:[{probe_id, kind, ok, status|exit_code, duration_s, output_tail ≤4 KiB redacted}]}` on a new vocabulary verb; settles itself.
- Result intake: each failure → `health_breach` incident (`probe:{env}:{probe_id}`, `deployment_id`); a later passing probe resolves it; results feed the Deployments "Probes" tab.
Credentials & reach:
- AC-24 probes — a connected node serving `pqa-test` on dev, able to take a slot — **deferred** — the dev control plane is unreachable from the architect's node; confirm a node on dev serves `pqa-test` from a tailnet node before the oracle
- UI checks on dev — `org/console/dev-login` (org Admin), `https://dev.jaah.ai` — credential **verified**; environment **deferred** for this node (operator R4: run from a tailnet node)
- AC-29 gates — worktree only — **verified**
Out of scope: Do NOT grant probes any permission beyond a runner session's sandbox (A7). Do NOT run probes with the feature OFF or the project not opted in. Do NOT add a migration. Epic out of scope (verbatim): "building an APM or log store; replacing the customer's monitor; auto-merging a revert without the project's policy; any scoring or ranking logic (the outcome ledger epic reads the states this epic writes)."
Done means:
- AC-24: "a project admin declares health URLs (expected status) and smoke commands in project settings. After a successful production deploy, with the feature ON and the project opted in, they run from a node serving the project (A7). A failure writes a `health_breach` incident (fingerprint = project, env, probe) and files a task per AC-15. With the feature OFF, probes do not run. Results show on the Deployments row detail." — declare one URL returning 500 and one smoke command `exit 1`, deploy, observe two incidents and two tasks (live on dev from a tailnet node once a node serves `pqa-test`); plus tests for scheduling, gate OFF (no probe task), result intake, resolve on pass.
- Security: non-https URL, >10 URLs, timeout >300, cross-host redirect, and an edit by a non-`projects:edit` user are refused — tests; output tail redacted and capped at 4 KiB; autonomous security review lane run and its findings closed in the PR.
- AC-26 (OFF check): with the feature OFF, no probe task is created and deployment measurement is unaffected — test.
- AC-29: `./dc check quick` green; changed-area tests green; merge lane full gate green.
Activity
- Draft
- Ready