Add outcome scan and defect detection
Epic 12243 (plan task 12246). Full settled inputs: the PRD (Task 12244) and the Architecture (Task 12245) of this epic — read them with `task_read`/`artifact_read` on those tasks; section numbers below (R…/AC…/§…) cite them. Where PRD and the original proposal (Task 12195) differ, the PRD governs. Depends on the ledger core child (writer API, `outcome_links` table).
## Original requirement (verbatim)
> "3. Defect attribution: a revert commit, a later PR/task referencing the PR (forge fixes/reverts link; same files in-window with a defect-shaped title at lower confidence), or an operator's explicit \"caused by\" link. Store confidence; operator confirm/dismiss on the task page; confirmed links are labels."
> Done means (epic, verbatim): "A revert of a merged PR flips its outcome to reverted within one cron pass; a fix-forward PR with a forge link flips the origin to fixed_forward with confidence shown; an operator dismissal reverts it."
PRD requirements carried (verbatim from PRD §4 A3): R3a.1 "The detectors each produce an attribution link (origin PR → defect evidence, kind, confidence 0–1, state): a revert commit on the integration branch naming the merged PR or its merge sha; a later PR or task with a forge \"fixes\"/\"reverts\" reference; a later PR inside the window touching the same files, with a defect-shaped title (lower confidence); an operator's explicit \"caused by\" link. The link state is `proposed`, `confirmed` or `dismissed`." R3a.2 "Detection runs as a CP cron over every supported forge (D13). Each pass completes within one cron interval." R3a.3 "Owner/Admin can confirm or dismiss a link, and add a \"caused by\" link, on the task page. Confirmed links are exported as labels (CSV, A5). A dismissal returns the outcome to its prior state." D13: GitHub, GitLab, Bitbucket and Azure DevOps through the existing forge layer.
## Approach (settled — realize, do not reopen)
Architecture §3.6 (scan state + `outcome_scan_interval_minutes`), §7 (d) + due-project rule, §8 entirely, §9.2 link routes + Scan now, §10, Pinned seam 2 (`outcome_scan.plan`/`outcome_scan.report` + wire fixture `apps/api/tests/fixtures/outcome_scan_wire.json` read by a CP test and a daemon test).
- `outcome_scan` hidden system persona (`persona_registry.PERSONA_RECORDS`, non-pipeline, done criterion `outcome`, model tier `low`, one open per project); brief in `dispatch_compose.roles._ROLES["outcome_scan"]` (the model does no judging); git person via `git_access.project_forge_person(project)` — move `merge_lane._bot_pr_person` there, both callers read it (single home); no person → not filed + a per-project `cron_alarms` alarm.
- Toolbox `tools/outcome_scan.py` packaged to `/usr/lib/jaahd/tools/` like `tools/merge_lane.py` (steps 1–5 of §8, paged caps).
- `jaah-forge` read verbs on all four adapters (`prs(state=merged, since=…)` with title/body/files/merged_at; `checks_for_sha`; new CLI verb `pr provenance <n> --json`, adding Bitbucket and Azure `provenance()`); fixture test per adapter; no guard change (read-only).
- CP judging `outcome_ledger/detectors.py`, pure functions; confidence table §8 (revert 0.95; fixes_ref 0.80/0.60; same_files 0.30 +0.1/extra file capped 0.5; caused_by 1.0 confirmed); LIVE-link 0.5 gate; UNIQUE(origin_outcome_id, kind, evidence_ref) so re-runs are idempotent and a dismissed link is never re-proposed. Per-head forge CI for non-lane heads → `writer.set_head_verdict(source='forge_ci')`; lane-project `pr_commits` → `writer.set_commit_facts`.
- One additive migration (the `outcome_links` table already exists from the ledger core child): `projects.outcome_scan_last_filed_at/cursor_sha/cursor_at/outcome_scan_interval_minutes` + `orgs.outcome_scan_interval_minutes`. Global setting in `app/operator_settings/catalog_tasks.py` (default 60, floor 15); resolution `outcome_ledger.settings.scan_interval(project)` project → org → global; org-settings and project-settings (governance tab, beside `merge_lane_batch_max`) fields.
- APIs: `POST /api/outcome-links/{id}/confirm`, `/dismiss`, `POST /api/tasks/{id}/outcome-links` (caused_by), `POST /api/projects/{id}/outcome-scan` (Scan now) — all `require_org_admin`; responses return the link and the origin row's new state.
- Stale-scan alarm (§8) and the "no git person" state exposed on the read model for the Performance tab.
Credentials & reach:
- pytest ACs: none beyond the worktree (`./dc` stack) — verified (worktree).
- Dev UI flows as Jaah-org Owner and as operator: `me/jaah/debug/main/operator-login` (operator, switches into tenants); environment `https://app.dev.jaah.ai` (public dev console; the internal `dev.jaah.ai` ALB did not answer from h2 within 15 s) — verified: credential_get ok; `app.dev.jaah.ai` → HTTP 200 from h2.
- Live dev DB reads: `tools/cp_psql.sh dev` → broker refused `org/aws/cp-psql-reader` — not needed by any oracle (every AC is a UI flow or a test).
Out of scope: the task Outcome panel UI and the live GitHub sandbox oracles (child "Build task Outcome panel" owns AC3.1–AC3.3 in the browser and creates the sandbox repo); metrics; security-scan findings (the security child adds `security_check_patterns` use); benchmark.
Out of scope (epic, verbatim): "any routing behaviour (the routing epic reads this ledger); new production signal sources (the production QA epic writes deployed/incident_linked); a public index or tenant-to-tenant sharing; re-scoring tasks older than a one-off backfill of merge, lane and cost facts."
Do NOT touch (PRD A8): no change to `apps/api/app/agent_resolution.py` resolution order, `apps/api/app/jaah_scheduler.py` account placement, or `apps/api/app/jaah_routing/` (`routing_decision_id` stays null); no deploy ingestion, monitor adapters, incidents table or new trigger source kinds (`apps/api/app/triggers.py` untouched; `deployed_at`/`incident_linked` are columns/states nothing writes); no public page, no unauthenticated endpoint, no tenant-to-tenant view, no tenant name on any operator-plane aggregate; no retired or rewritten weekly outcome-review job (`apps/api/app/operator_dashboard/review.py`, `apps/api/app/operator_dashboard/quality.py`, D4); backfill writes merge, lane and cost facts only, never re-runs LLM scoring, never re-opens old tasks; benchmark mode never merges a shadow PR and never changes the origin task's dispatch. No CP forge call (ADR-0156): forge facts come only through the `outcome_scan` runner. Migrations additive only (new tables + nullable/defaulted columns).
Done means:
- AC3.4 "**Verify:** unit tests cover each detector on GitLab, Bitbucket and Azure DevOps fixtures. Run `pytest apps/api -k defect_attribution` and observe them pass." (GitHub fixtures too.)
- AC3.1/AC3.2/AC3.3 API half (test-level twin of the dev flows): a test feeds an `outcome_scan.report` with a `git revert <merge-sha>` commit and asserts the origin row is `reverted` with a `revert` link at confidence 0.95 after one tick; a report with a merged PR body `Fixes #<origin PR>` flips the origin to `fixed_forward` with confidence 0.80; `POST …/dismiss` as Owner/Admin returns the origin to its prior state (`merged` or `survived` per window) and records `decided_by`; as a Member the confirm/dismiss/caused-by routes return 403; a same_files-only match (≤0.5) neither moves nor blocks survival; re-running the same report creates no duplicate link and never re-proposes a dismissed one.
- Scan filing: a test asserts an idle project files nothing, a due project files exactly one hidden `outcome_scan` task, a second is not filed while one is open, and the stale-scan alarm fires once per project; `outcome_scan` dispatch writes zero `task_outcomes` rows.
- Wire contract: the CP test and the daemon test both read `apps/api/tests/fixtures/outcome_scan_wire.json` and pass; each forge adapter fixture test passes (`pytest apps/jaahd -k forge`).
- Toolbox: a test runs `tools/outcome_scan.py` against a temp git repo holding a revert commit and more than the per-call caps of commits, with `jaah-call`/`jaah-forge` stubbed; it asserts the revert is reported, every page is sent with `last` on the final page, and the cursor advances only on the last accepted page.
- Dev end-to-end: on `https://app.dev.jaah.ai` as Jaah-org Owner, click **Scan now** on the Jaah project; observe one hidden `outcome_scan` task that settles `outcome`, and the project's last-scan time/cursor advance in the UI.
- The scan interval resolves project → org → global in a unit test, and the three settings fields render and save in the console (org settings, project settings governance tab, operator settings).
- Gate (epic Done means, verbatim: "Required checks: ./dc check quick green; changed-area tests green; merge lane full gate green.") — `./dc check quick` green; changed-area pytest/vitest green; merge lane `./dc check full` green; Jaah Deploy of the PR succeeds.
Activity
- Draft
- Ready