Agent performance profiles and outcome ledger system

Jaah records cost, tokens and time per task (task_run_metrics.py, Task.metrics), a human 1–5 rating and a weekly LLM-scored sample by persona, but attributes no outcome to the agent, model or configuration that produced it. Build the outcome ledger and the profiles over it. PRD: the attached proposal report (Task 12195), decisions table and "Epic 1". 1. Ledger: one `task_outcomes` row per build attempt that settles or fails. Attribution: agent (built-in provider or SDK agent + commit), model family and concrete model, account auth_kind, persona, review tier, routing decision id (nullable, future), project, org, node class. Timeline: ready → first active → PR opened → enqueued → merged → deployed → survival window end. Gate facts: lane verdicts per push, dispatch_attempts, pushes after enqueue, non-runner commits before merge, security findings from review lanes / CI scans. Cost: priced tokens over all attempts (failed included), node minutes, lane minutes. 2. Outcome state, CP-written and never runner-claimable (like `outcome_no_pr_reason`): merged → deployed → survived | reverted | fixed_forward | incident_linked; terminal failed | abandoned. Survival window default 14 days, a plan term with org override. `deployed` and `incident_linked` edges arrive from the production QA epic; until then survival is judged on the window without a deploy edge. 3. Defect attribution: a revert commit, a later PR/task referencing the PR (forge fixes/reverts link; same files in-window with a defect-shaped title at lower confidence), or an operator's explicit "caused by" link. Store confidence; operator confirm/dismiss on the task page; confirmed links are labels. 4. Metrics per configuration (agent × model family × persona × review tier) with roll-ups per agent, model, project, org: success rate (survived ÷ settled+failed, operator-cancelled excluded); first-pass acceptance (merged with zero lane failures, zero re-dispatch, zero human fix commits ÷ merged); defects introduced per 100 merges; rework (lane failures + re-dispatches + post-enqueue pushes + human fix commits per merged task, plus rework hours); completion time ready→merged median/p90 and agentic time; cost per successful task (Σ cost of all attempts ÷ survived); security issues per 100 tasks, credited to the detecting lane and debited to the authoring configuration. Wilson 95% intervals; a configuration ranks only at ≥ 20 outcomes, else "insufficient data"; stratify by persona and a complexity bucket (diff size, files touched, repo language, clarity score). 5. Surfaces (console UI style guide): Agents › Performance profile per agent/model/configuration; Compare 2–4 configurations on identical strata; org Leaderboard (filters persona, project, period, min n); task Outcome panel shows the ledger row and attribution links; Command center widgets read the ledger in place of the weekly sample; CSV export per org. Operator plane: Fleet › Agent index with cross-org aggregates per configuration, each cell ≥ 5 orgs and ≥ 100 outcomes, never a tenant name. 6. Benchmark mode (last phase): plan feature + per-project opt-in, budget-capped, default sample 5% of coder tasks: clone a sampled task to a shadow task on an alternative configuration in its own worktree; the shadow PR is never merged (closed with a body naming the duel); gate pass, blind-review verdict and diff distance to the merged PR give a pairwise result; a Bradley-Terry rating per configuration is shown beside the observational metrics. 7. Measurement is always collected (ledger, metrics, pages); only benchmark mode changes behaviour and only behind its plan feature and project opt-in. Dogfood on the Jaah project first. Out of scope: any routing behaviour (the routing epic reads this ledger); new production signal sources (the production QA epic writes deployed/incident_linked); a public index or tenant-to-tenant sharing; re-scoring tasks older than a one-off backfill of merge, lane and cost facts. Done means: - Every build task that settles after release has a `task_outcomes` row with attribution, timeline, gate facts and cost; a settle without one is a gate failure in tests. - A revert of a merged PR flips its outcome to reverted within one cron pass; a fix-forward PR with a forge link flips the origin to fixed_forward with confidence shown; an operator dismissal reverts it. - Agents › Performance renders a profile for each built-in provider with intervals and grey "insufficient data" below 20 outcomes; Compare shows two configurations on identical strata. - Fleet › Agent index refuses a cell under 5 orgs or 100 outcomes. - With benchmark mode ON for a project, a sampled task produces a shadow task, a closed unmerged shadow PR, one pairwise record and an updated rating; with it OFF or the plan term absent, no shadow task exists. - Required checks: ./dc check quick green; changed-area tests green; merge lane full gate green.