Recommend and Auto routing modes

Task 12297 · Queued · Coder · created · updated

Epic 12251, child C4. Needs C3b (policy), C3c (panel + routes module), C2 (`projects.experiment_budget_share`). ## Original requirement (verbatim) Ask 3: "Policy: Thompson sampling over configurations in the allowed set; objective expected survival probability per dollar with a latency cap; exploration capped at a per-project budget share, default 10%. …" Ask 4 (part): "Recommend shows the choice and reason on the task form and the dispatch plan decision, one click to accept; Auto dispatches on it inside org policy, every decision visible and overridable on the task." Epic Done means (verbatim): "In Recommend, the task form and dispatch plan decision show the choice with its reason; the accepted/overridden rate is recorded." "In Auto on a project with two allowed providers, both receive dispatches over time with the exploration share bounded by the project knob". ## Settled decisions A1: exploration share = dollars over rolling 30 days, exploratory priced cost ÷ project's total routed coder spend ≤ knob (default 10%). A10: surfaces are the task create/edit form and the task Routing panel; "Accept" writes the pick into the task's agent/model fields as a human pin. R7: Auto — "Change" lets a human override before dispatch or for the next re-dispatch, recorded. ## Approach (Architect §4.4 C4 part, §4.5, AD4, AD14) - Budget step in `policy.py`: numerator = Σ expected_cost_usd of the project's coder rows exploratory ∧ used='router', status ∈ {pending,locked,final}, last 30 days (pending/locked count as reservations); denominator = Σ of all used='router' coder rows same window/statuses; refresh_picks processes a project's tasks sequentially in one transaction; if exploratory and (num+cost)/(den+cost) > knob → pick greedy, `exploration_capped=true`; knob 0 → never exploratory. - `POST /api/routing/preview` (draft in, indicative pick out, no writes; seeded from a hash of the draft — indicative only). `POST /api/tasks/{id}/routing/accept` (AD14): sets task fields with `pin_source='human'`, stamps pending row `recommend_response='accepted'`, `frozen=true`; finalize records used='router' when actual == frozen router_pick. Any other human change to agent/provider/model while a Recommend pick is pending records `overridden` + `human_pick` and freezes the row; finalize precedence per AD14 (frozen accepted evaluated before pin fallback). Create form posts `routing_accepted_config` when "Use this" was clicked. Unanswered → `no_response`. - UI: `RecommendationBlock` on task create/edit form (pick: agent, tier, review configuration; full reason; "Use this"); panel actions "Use this" (Recommend) and "Change" (Auto, an ordinary edit). Credentials & reach: - Dev UI flows (AC5, AC6, AC8 pin in Auto project): `me/jaah/debug/main/operator-login` and `org/console/dev-login` — verified. - AC7 sim, AC8 API: the worktree only — verified. - AC7 dev part: dev Jaah org accounts on two built-in providers — deferred (operator to confirm; observable only after C9). Out of scope: changing account quota semantics or spillover; training a model from scratch; routing analysis personas before coder tasks have data; public marketplace billing; any ledger metric definition (owned by the outcome ledger epic). MUST NOT apply Recommend/Auto to analysis personas. MUST NOT touch placement/`gate_and_tier`. Done means: - AC5 Recommend surfaces (coder task in the Jaah project on dev). → Verify: create a coder task and observe on the task form the pick (agent, tier, review configuration), the full reason sentence (not truncated, wrapping at 390px phone width) and a "Use this" button. Open the task page's Routing panel and observe the same. Click "Use this" and observe the task's agent/model fields set. Dispatch and observe the decision row with response `accepted`. - AC6 Accepted / overridden rate recorded. → Verify: on dev, accept one recommendation, override one (pick another agent), and ignore one. Open the scoreboard and observe accepted / overridden / no-response counts of 1/1/1 for that period and project. - AC7 Auto spreads across two allowed providers (simulation part). → Verify: an API simulation test seeds the ledger-stats interface with two providers of close expected value, runs N=500 dispatches, and observes both providers chosen at least once and exploratory spend share ≤ knob (+1 dispatch tolerance). Repeat with knob 0% and observe only the greedy pick. Tightened: knob 0 → greedy ≥ 99%; a 20-task same-pass fan-out keeps reservations within the share (R4b). - AC8 Fallbacks. → Verify: … On dev, pin a task's agent in an Auto project, dispatch, and observe the pinned agent used and "human-pinned" in the reason. - Gates (AC16): `./dc check quick` green; changed-area suites green; merge lane `./dc check full` green.

Activity

  • Draft
  • Ready