Roll out intelligent routing on dev and run

Task 12440 · Running · Coder · created · updated

Epic 12251 (outcome-driven routing and agent marketplace) is fully merged on develop: PRs #7602, #7605, #7622, #7629, #7632, #7633, #7634, #7642, #7643, #7644, #7645 and #7662. The Finish phase could not confirm the epic works on the dev control plane, because every child deferred its dev oracles. ## What was observed (Finish, 2026-11-02) - **Dev rollout never done.** Task 12290 deferred its D7 rollout. No later task did it. Without the rollout, Recommend and Auto can't be observed anywhere. - **Deploy state.** Jaah Deploy run 41364718422 (780ac61) reached `apply (dev)` = success, with only `platform bundles` failing (tracked as 12433). That build contains every PR above except #7662. Run 41368846759 (91d9404, carrying #7662) was still pending. - **Dev unreachable from the Finish node.** `curl https://dev.jaah.ai/api/health` timed out with HTTP 000. `tools/cp_psql.sh dev` was refused: `org/aws/cp-psql-reader`. ## Do 1. **Confirm the deploy.** Check that the Jaah Deploy run containing 91d9404 (or later) applied to dev. If it failed, fix it and merge again, per the project rules. 2. **Perform the D7 rollout through the console UI only (no SQL, no API calls):** - Turn ON the `intelligent_routing` override for the dev Jaah org. - Set the org's maximum mode to Auto. - Set the Jaah project to Recommend. - Create or choose a second test project and set it to Auto, with two built-in providers' accounts present in Credentials › Runners. - Turn Blind review ON for the Jaah project, with "Prefer a different vendor" ON. - If the dev org lacks a second provider account, ask with header `Credential`. 3. **Run the dev oracles from the children's Done-means, in a real browser on app.dev.jaah.ai.** Use `me/jaah/debug/main/operator-login` and `org/console/dev-login`. - AC1 (12295): a task's Routing tab returns 200 and shows its Shadow decision. - AC3 (12296/12301): the operator Routing scoreboard at 30d shows router vs default survival and cost. - AC4 and AC9 (12290): demoting the org maximum lowers the projects, with one audit entry each. - AC5 and AC6 (12297): the Recommend block appears on the task form and the Routing panel. Accept one task, override one and ignore one, then check that the scoreboard counts 1/1/1. - AC7 (12301): the Auto project's dispatches spread over both providers, with exploration ≤ the project knob. - AC10–AC12 (12298): a Major coder task spawns a blind reviewer and a tester on another provider, its findings are listed, and a security finding appears on the ledger row. - AC13 (12299): the review lanes are named on the Routing tab. - AC14 (12300/12301): the marketplace shows ledger figures or "Insufficient data" or "No ledger data yet", and no self-reported figure. - Capture a screenshot for each. 4. **Fix every oracle that fails,** in the smallest change, through the merge lane. 5. **Small polish while there:** - (a) `apps/api/app/routes/agent_marketplace.py` lines 9-10: the docstring still says ledger figures are `no_ledger` until wired. Correct it. - (b) At 390px, the task form's "Use this" button sits partly under the sticky Save/Go footer. Make it fully visible without scrolling into the footer. Out of scope: any change to the router policy, the ledger metrics, quota/spillover semantics, or placement. No new routing features. Do NOT turn `intelligent_routing` on for any plan or on any production (app.jaah.ai) org. Done means: - Dev runs a deploy containing 91d9404 or later, and that Jaah Deploy run is green (or superseded by a green one). - The dev Jaah org and its projects are configured as in step 2, through the console. - Every oracle in step 3 is observed on dev, with a screenshot in the report. Each failure is fixed and merged, or is named with its blocker. - Items 5(a) and 5(b) are fixed. - Required checks: ./dc check quick green; changed-area tests green; merge lane ./dc check full green.

Activity

  • Ready
  • Runner started