Routing decision record and Shadow mode

Epic 12251, child C3a. Needs C1 (`agent_router/modes.py` effective_mode/mode_basis, `policy_settings.py`). ## Original requirement (verbatim) Ask 1: "Routing decision record: `routing_decisions`, one per dispatch — candidates with features and scores, the chosen configuration, mode, the human's final pick when it differed, a plain-language reason. The ledger row links to it so every decision is graded by its outcome." Ask 3 (part): "Falls back to today's precedence chain whenever the allowed set is one, data is below the minimum, or the mode is off; with the feature OFF resolve_agent and placement stay byte-identical." Epic Done means (verbatim): "With the ledger present and routing OFF, every dispatch still has a routing_decisions row in Shadow and the scoreboard shows router vs default survival and cost over a period; resolve_agent and placement are byte-identical to today (regression test)." ## Settled decisions - R1: every dispatch of every persona writes one row, every mode incl. feature OFF; a re-dispatch is a new row; stable id for the ledger. A2: written from release on, even before the ledger exists. - R3: writing the Shadow record cannot delay, fail or reorder a dispatch; a write failure is logged and dispatch proceeds. - A4: human pins win; playbook never overridden. ## Approach (Architect AD2, AD3, AD5, AD6, AD7, §3.1, §4.1 — pinned) Migration: ``` routing_decisions id uuid pk; org_id uuid (OrgScoped); project_id int fk projects; task_id bigint fk tasks on delete cascade attempt int not null -- = tasks.dispatch_attempts after the claim bump status text check in ('pending','locked','final','superseded'); inputs_hash text not null; frozen bool default false dispatch_id text null; persona text; mode text check in ('shadow','recommend','auto'); mode_basis jsonb features jsonb; candidates jsonb; router_pick jsonb; default_pick jsonb; actual jsonb null used text check in ('router','default') fallback_reason text null check in ('allowed_set_one','data_below_minimum','mode_off','human_pinned','system_pinned','playbook','vendor_bound','persona_not_routed','candidate_unavailable','no_pick') exploratory bool default false; exploration_capped bool default false; expected_cost_usd numeric(12,4) null recommend_response text null check in ('accepted','overridden','no_response'); human_pick jsonb null reason text not null; stats_version text not null ('none' until C9) created_at, locked_at, finalized_at timestamptz unique (task_id, attempt) where status <> 'superseded'; index (org_id, finalized_at), (project_id, finalized_at), (status) where pending/locked Config = {kind:'builtin'|'agent', provider, agent_id, tier, model, review:{lanes:[{lane, provider, agent_id, tier}], blind:[...]}} tasks.pin_source text null check in ('human','system') -- backfill 'human' where agent_id or model set ``` - AD3: the pick lives ONLY in the side table, never on `tasks` (any tasks write bumps `updated_at`, `models/tasks.py:478`, and moves list windows). Lifecycle pending → locked (at lease, with dispatch_id) → final (at claim) or superseded. Computed in a new fleet-wide `agent_router.refresh_picks(session)` once per lease pass beside `_fail_unrunnable_agents` (`lease_drain.py:2275`), per-task SAVEPOINT, never raises; recomputed only while unlocked, unfrozen and `inputs_hash` changed. Never computed in `build_dispatch_plan`. - AD2: wrapper `agent_router.effective_choice(session, task, project)`; `resolve_agent` (`agent_resolution.py:208-233`) is NOT edited. Switch all six callers: `routes/fleet.py:895`, `:1213`, `lease_drain.py:1497`, `:2283`, `microvm_placement.py:356`, `placement_grants.py:366`. Grep test fails on any new direct caller outside `agent_router`. `_unrunnable` renamed public `unrunnable_reason` (rename only). - `model_override(session, task)` at both compose sites (`lease_drain.py:2058`, `microvm_placement.py:381`); `set_actual_model` at `_stamp_dispatched_model` (`lease_drain.py:2134`) and `microvm_placement.py:393`. - AD5: `finalize_on_claim` in `routes/tasks/dispatch.py:407-438` after the claim UPDATE wins, in a SAVEPOINT, exceptions logged and swallowed; no locked row → a default-only final row with `fallback_reason='no_pick'`. Declined lease unlocks back to pending; stale locks released by refresh_picks. - AD6: `pin_source='human'` on human writes of agent_id/bound_provider/model (create/update routes), NULL when a person clears all three; `'system'` on runner `task_create` with model (`apps/jaahd/agent_vocabulary.py:581,604,633`), Runner cron model (`lease_drain.py:2049-2050`), planner/epic children with a model. AD7: route only while `bound_provider IS NULL` and no slot held, else `vendor_bound`. - `agent_router/personas.py`: `REVIEW_PERSONAS = ("reviewer","security_reviewer","tester")`; `ROUTED_PERSONAS = ("coder",) + REVIEW_PERSONAS`. - `policy.pick(session, task, project, *, attempt) -> PickDraft` as a DEFAULT-ONLY STUB here (router_pick = default_pick, used='default', fallback per mode/pin/persona); C3b replaces it. Reason sentence must still be a readable full sentence. - Public API per Architect §6: effective_choice, model_override, refresh_picks, lock_on_lease, unlock, finalize_on_claim. Credentials & reach: - AC1 SQL check: `org/aws/cp-psql-reader` via `tools/cp_psql.sh dev` — deferred (broker refused it; operator will grant). Fallback oracle (C5): operator scoreboard `dispatches` equals sessions started on Audit › Sessions for the same window. - AC2 and API tests: the worktree only (`./dc` stack + pytest) — verified. Out of scope: changing account quota semantics or spillover; training a model from scratch; routing analysis personas before coder tasks have data; public marketplace billing; any ledger metric definition (owned by the outcome ledger epic). MUST NOT alter `resolve_agent` behaviour, `model_catalog` precedence, `_best_account`/`jaah_scheduler.gate_and_tier` or placement output when the effective mode is off. MUST NOT create `task_outcomes`. No UI here (panel is C3c). Done means: - AC1 Shadow row on every dispatch (all modes, ledger absent or present, all personas). → Verify: … run `tools/cp_psql.sh dev` with the pinned query `select count(*) from sessions s where s.task_id is not null and s.started_at > :release and not exists (select 1 from routing_decisions d where d.task_id = s.task_id and d.status = 'final' and d.finalized_at <= s.started_at + interval '15 minutes')` and observe 0 (deferred — cp-psql-reader not yet granted; fallback via C5's scoreboard). Also run an API test asserting that a record-write failure still dispatches. - AC2 Byte-identical when OFF (feature off; feature on with project mode inherit). → Verify: a regression test runs a fixture matrix of tasks covering playbook / agent_id / bound_provider / project default agent / project default provider / built-in, each persona, and model set or unset, through `resolve_agent`, model resolution and `_best_account` on develop-equivalent logic and on the new code, and asserts equal outputs. Run `./dc check quick` and the named test; observe pass. Tightened: the test also asserts `tasks` rows (incl. `updated_at`) are unchanged across a Shadow refresh_picks pass, and that `build_dispatch_plan` makes no `policy.pick` call. - Grep test: no direct `resolve_agent` caller outside `agent_router`. - Lifecycle tests: pending→locked→final per attempt; re-dispatch gets a new row; declined lease unlocks; MicroVM path finalizes; pin_source backfill and system writers. - Gates (AC16): `./dc check quick` green; changed-area suites green; merge lane `./dc check full` green.