Live-verify the outcome ledger's revert, fix-forward and benchmark
(a) Gap: three of the epic's "Done means" items were never observed in the real environment. Each child reported them as unverified because dev had no coder run after release:
- AC1.4: a post-release coder task's Outcome panel shows attribution, timeline, gate facts and cost.
- AC3.1/3.2/3.3: a revert flips the outcome to Reverted within one cron pass, a `Fixes #N` PR flips the origin to Fixed forward with its confidence shown, and Dismiss restores the prior state.
- AC6.1: with benchmark ON, a sampled task produces a shadow task, a closed unmerged shadow PR, one pairwise record and an updated rating.
Today only pytest/vitest and the lane's full gate cover them (Tasks 12284 and 12288 reports).
(b) Where: Task 12284's body names the oracle setup. Run it on `https://app.dev.jaah.ai`, entirely through the console UI. In the Jaah org's dev project "Outcome ledger sandbox" (`jaah-ai/outcome-ledger-sandbox`; create it through the console's new-repo flow if it is missing), create a small coder task, let it merge, push `git revert <merge-sha>` to the sandbox's integration branch, then merge a second PR whose body says `Fixes #<origin PR>`, using Scan now. Never revert anything on `jaah-ai/jaah`. Benchmark: in the Jaah project (override and opt-in are already ON per Task 12288), set the sample rate to 100% in the UI, let one coder task merge, observe the duel, then restore 5%. Record both changes with their times.
(c) Fix: none expected. If an oracle fails, fix the defect in its home module (scan detectors, writer, benchmark) as part of this task and re-run the oracle. If all pass, settle `no_pr` / `already_shipped` with the observed evidence.
(d) Acceptance, each with a screenshot in the report:
- AC1.4: the Outcome panel of a post-release coder task shows agent, model, auth kind, persona, review tier, node class, the timeline, gate facts with units, and cost.
- AC3.1: the state reads "Reverted", with a high-confidence link naming the revert commit, after one scan pass.
- AC3.2: the origin reads "Fixed forward", with its confidence shown.
- AC3.3: after Dismiss, the origin is back to its prior state and the link reads "Dismissed by <name>".
- AC6.1: the origin links a shadow task; the shadow PR is Closed (not merged) and its body names the duel; Performance shows one more duel and an updated rating for both configurations.
(e) Out of scope: the CE import fix and the BT interval fix (their own follow-ups); any routing change; turning benchmark on for any org other than Jaah.
Done means:
- Every oracle above has been observed on dev with a screenshot, or its defect has been fixed, merged through the lane, and the oracle re-observed.
- If code changed: `./dc check quick` green, changed-area tests green, merge lane full gate green, and the PR's Jaah Deploy succeeded.
Outcome
Pull request Pull request Pull request
Activity
- Ready
- Runner started