Build outcome ledger core and writer

Task 12279 · Running · Coder · created · updated

Epic 12243 (plan task 12246). Full settled inputs: the PRD (Task 12244) and the Architecture (Task 12245) of this epic — read them with `task_read`/`artifact_read` on those tasks; section numbers below (R…/AC…/§…) cite them. Where PRD and the original proposal (Task 12195) differ, the PRD governs. ## Original requirement (verbatim) > "1. Ledger: one `task_outcomes` row per build attempt that settles or fails. Attribution: agent (built-in provider or SDK agent + commit), model family and concrete model, account auth_kind, persona, review tier, routing decision id (nullable, future), project, org, node class. Timeline: ready → first active → PR opened → enqueued → merged → deployed → survival window end. Gate facts: lane verdicts per push, dispatch_attempts, pushes after enqueue, non-runner commits before merge, security findings from review lanes / CI scans. Cost: priced tokens over all attempts (failed included), node minutes, lane minutes." > "2. Outcome state, CP-written and never runner-claimable (like `outcome_no_pr_reason`): merged → deployed → survived | reverted | fixed_forward | incident_linked; terminal failed | abandoned. Survival window default 14 days, a plan term with org override. `deployed` and `incident_linked` edges arrive from the production QA epic; until then survival is judged on the window without a deploy edge." > Done means (epic, verbatim): "Every build task that settles after release has a `task_outcomes` row with attribution, timeline, gate facts and cost; a settle without one is a gate failure in tests." PRD requirements carried (verbatim from PRD §4): R1.1 "Each dispatch of a pipeline persona (D1) writes exactly one `task_outcomes` row (D2). The row exists from dispatch. Its outcome state is filled in when the dispatch settles, fails or is abandoned." R1.2 attribution snapshot at dispatch, never rewritten (agent: built-in provider or SDK agent id + bundle commit; model family and concrete model; the dispatched account's `auth_kind`; persona; `review_tier` (D7, written at settle); `routing_decision_id` nullable, always null; project and org; node class D9 = hosting `jaah_cloud`|`self_hosted` × isolation `static`|`microvm`). R1.3 "Each is null when the event did not happen. A missing event is never zero or epoch." R1.4 gate facts incl. lane verdicts **per push** kept as history, `dispatch_attempts`, pushes after enqueue, non-runner commits before merge. R1.5 cost per dispatch (priced tokens, node minutes, lane minutes); "all attempts" summed by the metric layer. R1.6 "The row is CP-written only. No runner tool can create or modify it, except `review_tier` (R1.2) and the findings (R3.x) passed on the settle call." R2.1 states `pending`, `merged`, `deployed`, `survived`, `reverted`, `fixed_forward`, `incident_linked`, `failed`, `abandoned` (+ `settled` and inherited terminal for analysis rows). R2.2 survival window plan term `outcome_survival_days`, default 14, org override. R2.4 analysis rows inherit the epic result (D5). D6: a dispatch that ends without settling is `abandoned` and counts; only operator cancel/delete/reset/move is excluded. ## Approach (settled — realize, do not reopen) Architecture §3.1, §3.2, §3.3, §3.6 (survival term only), §4, §5, §6.1, §6.3, §7 (a)(b)(c), Pinned seams 1 and the `outcome.head_pushed` verb of seam 2. - New package `apps/api/app/outcome_ledger/` with `states.py::TRANSITIONS` (one declarative table; a test holds every writer to it), `writer.py` exposing exactly the pinned API: `open_row`, `attach_session`, `end_row(..., review_tier=None, findings=None)`, `on_merged`, `add_head`, `set_commit_facts`, `set_head_verdict`, `exclude`, `apply_link` (real: recomputes the origin's state from its links per §4 LIVE-link rule). Callers never write the tables directly. `dimensions.py::model_family` (single home) and the complexity bucket (§6.3 edges, clarity bump, language extension map). - One additive migration: `outcome_links` (§3.3, incl. UNIQUE(origin_outcome_id, kind, evidence_ref)), `task_outcomes` (§3.1, incl. `task_id` ON DELETE SET NULL, `dispatch_seq` never reset, UNIQUE(task_id, dispatch_seq), the three indexes) and `task_outcome_heads` (§3.2). - Writer seams (§5 table, every row): `lease_drain._stamp_dispatched_model` AND `microvm_placement.py` stamp → `open_row` (pipeline personas + shadow coders only; system personas get no row); `note_start` → `attach_session`; `runner_settle.end_attempt` → failed/abandoned by source; every DONE writer (`task_terminal.settle_task`, `runner_settle._settle_done`, `settle_coder_on_merge`, `settle_coder_no_pr`) → `end_row('settled')`; `task_prs.record_attested` and `merge_lane.apply_event` merged → `on_merged`; lane failed/merged/superseded → `set_head_verdict(source='lane')`; self-merge `pr_provenance.commits` → `set_commit_facts`; reset/delete (before the cascade, every task `_rows_the_cascade_will_remove` lists)/project move → `exclude` with §5 scope rules; cost priced from `Task.metrics["sessions"][agent_session_id]` only (never task totals). - Daemon: `apps/jaahd/guards/guard-pre-push.py` step 3 posts `outcome.head_pushed` (POST `/api/agent/tasks/{task_id}/outcome-heads` `{head_sha, ref, pushed_shas ≤ 250}`), best effort, never blocks a push; verb added to `apps/jaahd/agent_vocabulary.py`. - `review_tier` optional on `task_complete`/`task_fail` (SettleIn + fail body; ≤16 chars, normalised to minor/standard/major/security/unknown, never 422); tool schemas in `agent_vocabulary.py`; prompt sweep: `dispatch_compose/role_prose.py` and `portfolio/fleet/runner-charter.md` name it (ai-code-policy, prompt text as executable). - `outcome_survival_days` numeric plan term (beside `static_slots`, `_LIMIT_OVERRIDES`, `INHERITABLE`), in the operator plan form (`pages/products/plan/planForm.ts`) and the org overrides form (`pages/orgs/overridesForm.ts`); `survival_ends_at` fixed at merge. - `outcome_ledger_tick` cron (15 min, `cron_system_activities._system`): (a) merged → survived past window with no LIVE link, (b) analysis inheritance (D5; no merged build child → failed; no build children → stays settled), (c) catch-all reconcile → abandoned `reason='reconciled'` with a warning. Idempotent. - Org-transfer rules for `task_outcomes`, `task_outcome_heads` and `outcome_links` in `tenancy/transfer*.py` (move with task/project) so `test_transfer_rules.py` passes. Settled interpretations (Architecture §18, kept by the Planner): language is a filter, not a stratum key; the success denominator counts final rows only (pending/merged unjudged rows are out of numerator and denominator); only LIVE links (confirmed, or proposed with confidence ≥ 0.5) move or block an outcome; the pages inherit the Command center's edition gate; reset excludes only unfinished rows. Credentials & reach: - pytest ACs: none beyond the worktree (`./dc` stack) — verified (worktree). - Dev UI flows as Jaah-org Owner and as operator: `me/jaah/debug/main/operator-login` (operator, switches into tenants); environment `https://app.dev.jaah.ai` (public dev console; the internal `dev.jaah.ai` ALB did not answer from h2 within 15 s) — verified: credential_get ok; `app.dev.jaah.ai` → HTTP 200 from h2. - Live dev DB reads: `tools/cp_psql.sh dev` → broker refused `org/aws/cp-psql-reader` — not needed by any oracle (every AC is a UI flow or a test). Out of scope: the forge scan, detectors and link APIs (child "Add outcome scan and defect detection"); backfill (child "Backfill outcome ledger history"); metrics/read APIs; every UI except the two plan-term forms; security findings storage (`findings` is accepted by `end_row` as a no-op parameter until the security child); benchmark. Out of scope (epic, verbatim): "any routing behaviour (the routing epic reads this ledger); new production signal sources (the production QA epic writes deployed/incident_linked); a public index or tenant-to-tenant sharing; re-scoring tasks older than a one-off backfill of merge, lane and cost facts." Do NOT touch (PRD A8): no change to `apps/api/app/agent_resolution.py` resolution order, `apps/api/app/jaah_scheduler.py` account placement, or `apps/api/app/jaah_routing/` (`routing_decision_id` stays null); no deploy ingestion, monitor adapters, incidents table or new trigger source kinds (`apps/api/app/triggers.py` untouched; `deployed_at`/`incident_linked` are columns/states nothing writes); no public page, no unauthenticated endpoint, no tenant-to-tenant view, no tenant name on any operator-plane aggregate; no retired or rewritten weekly outcome-review job (`apps/api/app/operator_dashboard/review.py`, `apps/api/app/operator_dashboard/quality.py`, D4); backfill writes merge, lane and cost facts only, never re-runs LLM scoring, never re-opens old tasks; benchmark mode never merges a shadow PR and never changes the origin task's dispatch. No CP forge call (ADR-0156): forge facts come only through the `outcome_scan` runner. Migrations additive only (new tables + nullable/defaulted columns). Done means: - AC1.1 "Every dispatch of every pipeline persona after release has exactly one row. **Verify:** a new API test drives dispatch → settle, dispatch → fail, and dispatch → session-lost → re-dispatch → settle, and asserts 1, 1 and 2 rows. Run `pytest apps/api -k task_outcomes`. Observe it pass. A settle path that writes no row must make this test fail (the body's \"a settle without one is a gate failure in tests\")." — Planner tightening: the test drives **each** DONE writer in §5 (`settle_task`, `_settle_done`, `settle_coder_on_merge`, `settle_coder_no_pr`), the lane reopen, and the microvm stamp path, and asserts the catch-all reconcile never had to close a row on those paths; an `outcome_scan`/system-persona dispatch yields zero rows; a fixture `before_flush` assertion fails if any writer of `dispatched_model` skips `open_row`. - AC1.2 "Attribution is complete. **Verify:** in the same test, every R1.2 field except `routing_decision_id` is non-null for a built-in-provider dispatch and for an SDK-agent dispatch (which carries agent id + commit). `review_tier` is `unknown` when the settle omits it." - AC1.3 "Runners cannot claim a row. **Verify:** a test calls the agent settle route with any outcome field other than `review_tier` and findings, and observes a 422 or the field ignored. The row is unchanged." - AC2.1 "**Verify:** with a test clock, a `merged` row with no links becomes `survived` exactly when the window ends, not a day earlier. With an org override of 3 days it becomes `survived` at day 3. Observe both in the `pytest` test." - AC2.2 "Dev flow. **Verify:** on `dev.jaah.ai`, as the operator, open the Jaah org's plan terms and observe \"Survival window: 14 days (plan)\". Set the override to 7 and observe \"7 days (org override)\"." (then set it back to inherit, through the UI) - AC2.3 "**Verify:** a test settles an analysis child of an epic whose two coder children survive. The analysis row becomes `survived`. When one coder child is `reverted`, the analysis row becomes `reverted`." - Cost is per dispatch: a test with two sessions on one task asserts each row's `cost_usd` equals that session's priced fold, and the sum equals the task total (no multiplication). - Per-push history: a test posts two `outcome.head_pushed` calls and a lane `failed` then `merged` event and asserts two heads with verdicts fail/pass, `pushes_after_enqueue` counted, and `nonrunner_commits` = PR commits not in ∪ pushed_shas. - `apply_link`: a test inserts a confirmed link on a survived row → defect state; dismissing it → back to survived (or merged inside the window); a proposed link < 0.5 neither moves nor blocks. - `pytest apps/api -k "task_outcomes or outcome_ledger"` and the daemon guard test (`apps/jaahd`) green; `test_transfer_rules.py` green. - Gate (epic Done means, verbatim: "Required checks: ./dc check quick green; changed-area tests green; merge lane full gate green.") — `./dc check quick` green; changed-area pytest/vitest green; merge lane `./dc check full` green; Jaah Deploy of the PR succeeds.

Activity

  • Draft
  • Ready
  • Runner started