Split routes/agent.py into package
## Requirement (verbatim, epic 7555)
> Break large modules into smaller and more structured ones. E.g. "Very large modules: routes/[tasks.py](http://tasks.py) has 12,867 lines and [models.py](http://models.py) has 8,770."
Settled upstream (PRD = task 7553, Architecture = task 7554): split the 26 files over 3,000 lines frozen on 2026-10-18 — one domain per module, one PR per module (paired tests ride along), every caller rewritten with no shim, external contracts and DB schema unchanged, no size gate. This child is ONE of 21 independent PRs; no child depends on another (overlap is textual, resolved at rebase by the codemod).
## This child: `apps/api/app/routes/agent.py` (7,708 lines) → package `apps/api/app/routes/agent/`, plus its paired test `apps/api/tests/test_tasks_agent_routes.py` (~3,900 lines; tests `/api/agent/tasks/*`) (Architecture §5.3)
Modules, each with its OWN `APIRouter` (AD8: same `prefix`/`tags`; `main.py` includes them explicitly in a fixed order):
- `auth.py`: `_check_token`, `_authenticate_*`, `_read_scope_for`, `caller_node_id`, `_wire_project`, `require_caller_task`, `_check_token_for_*`, `_guard_task_creating_caller`, `_CALLER_TASK_ID_RE`, `MAX_WIRE_TASK_ID`, `HOST_PRODUCER_CALLER`.
- `accounts_worktrees.py`; `pool_tick.py` (roster, known slots); `slot_intents.py` (ack, holder, launch claim); `fleet_observe.py` (observe, alerts, telemetry).
- `node_config.py`: config, egress, logs, catalogs, liveness, identity, `NODES_LIVENESS_SQL`.
- `releases.py`: release, autoupdate, flags, port grid, nodelink.
- `gha_runners.py`.
- `task_settle.py` (name it `task_terminal.py` if `app/task_settle.py` makes it confusing): settle/complete, fail, PR, questions, runner session, append, follow-up, `_MANIFEST_REFUSAL_LIMIT`, `_settle_outcome_criterion_task`.
- `task_reads.py`: by-id, attachments with `_content_disposition`, bundle, claim, system tasks, needs-attention.
- `runner_liveness.py`: breaks, heartbeat, console exit/resume.
- `agent_filing.py`; `task_secrets.py` (git tokens, task credentials, API key, Bedrock, usage, memory).
C4 route order: `/tasks/by-id`, `/tasks/needs-attention` and `/tasks/system/open` are included before any router that could declare `/tasks/{task_id}`; a comment at the include site states the rule.
Patch seams (C2): `test_task_done_at.py:95–163` (`dispatch_and_audit_sync`, `_settle_compliance_group`, `_settle_outcome_criterion_task`, `settle_analysis_child_on_artifact`, `settle_coder_on_merge`, `settle_coder_no_pr`); `test_fix_jobs_finish.py:25`.
Repoint (C5): `apps/api/test_g3_task_binding.py:61–465` (QUICK tier, `tools/check.sh` ~1558); `tests/test_task_state_write_sites.py:134–139`; `test_tenancy_scope_sites.py:320`; `test_tenancy_plane_sites.py:512`; `test_org_filter_sites.py:48–184`; `test_tenant_create_surfaces.py:194`; `tests/test_data_access_events.py:31`; `test_authz_policy.py` `_INFRA_MODULES` (:65) gains the new `app.routes.agent.*` modules carrying `@infra` routes.
Test split: `test_tasks_agent_routes.py` splits by its endpoint banners into `test_tasks_agent_routes_<facet>.py`; shared helpers to a sibling helper module.
Fan-in: ~22 sibling routers (mostly via `_check_token`), lazy service imports, ~60 tests. Living docs: 5 doc files point at `routes/agent.py`.
## Shared rules (Architecture §3, binding)
- C1 Package shape: `X/__init__.py` is a docstring module map only — no names, no `__getattr__`, no `*`.
- C2 Patch seams: a moved name that tests patch (`setattr(<alias>, "<name>"` or `"app.<path>.<name>"`) is reached by other new modules as `<module>.<name>` via `from app.X import <module>`; tests patch it on the defining module; drop `raising=False` once the new home is confirmed; names the monolith imports and tests patch on its namespace are reached through their defining module, or the test patches every consuming submodule.
- C3 Codemod import forms: parenthesised `from app.X import a, b`; `from app import X` + `X.attr`; `import app.X as m`; string targets `"app.X.name"`/`"app.X:name"`; `sys.modules["app.X"]`; function-local imports stay function-local (agent.py has 7; reason unchanged).
- AD6 Codemod: a transient rewrite script (stdlib `ast` or `sg`; no libcst) kept in `$JAAH_REPORT_DIR`, never committed; re-run after every `git rebase origin/develop`. Scope: `apps/ tools/ infra/ packages/`.
- C5 Source-reading and path-keyed tests/registries are repointed, never deleted or skipped. Discovery: `git grep -nF '<old path>' -- tools/ apps/ infra/ packages/ '*.toml' '*.json' '*.yml' '*.sh'` for the old production AND old test path; re-key every hit.
- C6 `logging.getLogger(__name__)`; update logger-name assertions; no infra metric filter keyed on the old dotted path.
- C7 Every living-doc pointer to the moved path points at the package dir or owning submodule; a deleted path never appears in backticks.
- C9 Pydantic schemas live with the routes that use them; shared ones in a module named for the wire shape; no catch-all `schemas.py`; rename no schema class.
- AD7 Test siblings `test_<subject>_<facet>.py`; helpers to a sibling helper module, not `conftest.py`.
- Every new module opens with a one-line docstring naming the one domain it owns; no `utils`/`common`/`helpers` grab-bag.
- AD9 Delivery: merge lane is ON — do NOT arm auto-merge. Rebase on fresh `develop` and re-run the codemod immediately before each `pull_request_enqueue`; enqueue after every push. Watch Jaah Deploy: on failure fix and enqueue again; cancelled/superseded → leave it.
- Run `./dc check full` only on the committed, rebased tree, never in parallel with `./dc test` on the same slot.
- Small internal cleanups allowed only where no caller outside the module can observe them.
Credentials & reach:
- AC5, AC6, AC7, O4, O-patch, O-catalog, O-mappers, O-cycle, O7 build | None beyond the worktree; the slot's local `./dc` stack (project p1-6) | verified on node h2: `./dc up` exit 0; `./dc run api alembic check` → "No new upgrade operations detected."; the in-process `app.openapi()` returned 726 paths and 1,037 schemas
- AC9 / AD9: PR view, enqueue, Jaah Deploy watch | `jaah-forge`, pre-authenticated for the task, plus the `pull_request_enqueue` control tool; GitHub `jaah-ai/jaah` | verified: `jaah-forge api get repos/jaah-ai/jaah/actions/workflows` from node h2 listed the "Jaah Deploy" workflow
- AC10 dev UI check, O7 node-healthy check | `jaah/debug/main/operator-login` (roster; covers app.jaah.ai and app.dev.jaah.ai); env `https://app.dev.jaah.ai` | verified: `credential_get` succeeded (value not recorded); HTTPS GET of app.dev.jaah.ai from node h2 returned 200. `dev.jaah.ai` (internal ALB) timed out from h2, so it is not used. `gmind/skynet/dev` is on the roster but its base_url is an old host, so it is not used.
- AC10 "a fleet runner claims a ready Task"; O7 "dev node healthy" | Same operator login; the console Nodes and Tasks pages on app.dev.jaah.ai | verified: signed in to app.dev.jaah.ai as the operator login in a browser from h2; the Nodes page listed 3 nodes, all `online` (including ts2-node6), which are the runners a deploy updates
Out of scope: Do NOT touch `packages/agent-sdk/examples/python/**` (N1). Do NOT split any other file — the other frozen files each have their own child (N7), and files ≤3,000 lines today are follow-ups (N2). Do NOT add a line-count ceiling, lint rule or gate (N3). Do NOT keep `app.routes.agent` names importable through a shim, facade, `__getattr__` redirect, star re-export or alias module (N4). Do NOT change HTTP paths/methods/schemas, DB schema, node↔CP wire format, MCP/`jaah-call` verb names, env-var names, or user-visible UI (N5). No new features, dependency bumps or behavioural fixes riding along; a bug found while moving code is named in the PR body as a deferred follow-up (N6). Do NOT rewrite git history or touch files under `apps/api/migrations/versions/` (N8).
Done means:
- AC1 No §4 file remains over 3,000 lines, and no module created by the split is over 3,000 lines. — Run `git ls-files | grep -E '\.(py|ts|tsx|js|sh)$' | grep -v -E 'node_modules|alembic/versions|migrations/|packages/agent-sdk/examples/' | xargs wc -l | awk '$1>3000 && $2!="total"' | sort -rn` on `develop` after the PR; observe `routes/agent.py` and `test_tasks_agent_routes.py` absent and no new path under the split directory or new test siblings listed.
- AC2 Each new module owns one domain and declares it (R2). Fails if a module is a grab-bag (`utils2`, `misc`, `part2`, `helpers` mixing domains) or its docstring names two unrelated domains. — `for f in <new files>; do head -3 "$f"; done` → every file opens with a one-line ownership docstring/comment; reviewer (code-review lane) confirms each module's contents match its stated domain and its name. Record the module→domain table in the PR body.
- AC3 Old path removed, no shim (R3). — Architecture's restated AST oracle (stdlib-`ast` script in `$JAAH_REPORT_DIR`) over `apps/ tools/ infra/ packages/` for P=`app.routes.agent` fails on: (a) `from P import N` where N is not a submodule file; (b) `from app.routes import agent [as m]` / `import app.routes.agent [as m]` with any `m.<attr>` naming a non-submodule; (c) `from P.<sub> import N` where N is not defined in that submodule; (d) a string target `"P.<Name>"`/`"P:<name>"` whose first component after P is not a submodule; (e) the package `__init__` AST check `python -c "import ast,sys;t=ast.parse(open(sys.argv[1]).read());print([n for n in t.body if not isinstance(n,(ast.Expr,ast.Import,ast.ImportFrom)) or isinstance(n,ast.ImportFrom) and n.names[0].name=='*'])" <init>` printing anything but `[]`. Cite the zero-finding output in the PR body.
- AC4 Routes unchanged (A-2). — O4: on the base sha and after, `./dc run api python -c "import json; from app.main import app; print(json.dumps(app.openapi(), sort_keys=True))" | tail -1 > openapi.json`; `diff <(jq -S .paths before.json) <(jq -S .paths after.json)` and the same for `.components.schemas` are both empty. Route precedence: existing route tests for `/tasks/by-id`, `/tasks/needs-attention`, `/tasks/system/open` pass.
- AC5 DB schema unchanged (A-2). — `alembic check` in the `./dc` API container prints "No new upgrade operations detected"; the PR adds or changes no file under `apps/api/migrations/versions/`.
- O-patch: for every `setattr`, string-patch or `raising=False` target touched, `python -c "import importlib; m=importlib.import_module('<mod>'); getattr(m,'<name>')"` succeeds; list them in the PR body.
- AC6 Full gate green. — `./dc check full --gate-result --timeout 1500` on the committed, rebased tree → `GATE_RESULT status=pass` (includes `tools/check_context.py`).
- AC7 Coverage not reduced (R6). — Before/after per touched suite: `pytest --collect-only -q | tail -1` → after ≥ before; cite both numbers in the PR body.
- AC8 Docs pointers current (R7). — `git grep -nF 'app/routes/agent.py' -- '*.md' '*.toml' '*.json' '*.yml'` and the same for `tests/test_tasks_agent_routes.py` → 0 hits (except git-history-only prose); `python3 tools/check_context.py` → exit 0.
- AC9 Delivered one PR per module (R4). — `jaah-forge pr view <n>` → diff touches only this module (+ its paired test, callers, docs); merged into `develop` through the merge lane after `pull_request_enqueue` (not auto-merge), shown MERGED.
- AC10 Availability on dev — the product still works for the operator after each PR deploys (no flag involved; refactor ships ON by deploy). — Watch **Jaah Deploy** for the merge commit succeed on GitHub; then open `https://app.dev.jaah.ai` in a browser: Tasks list loads; open an epic and a child Task detail — header, description, children, comments, sessions render with no console errors; create a Task, edit and save it, Start it; Nodes page lists nodes; Credentials page lists credentials; a fleet runner claims a ready Task (proves the `/api/agent/*` paths). Observe each succeeds as it did before the PR. Tasks created for the check are titled `[split-check P3]` and deleted afterwards.
- AC12 Quality: the result is more navigable, not just shorter. Fails if a split produces circular-import workarounds (function-local imports added only to dodge cycles), duplicated code across new modules, or modules so thin (one trivial function) that a reader must open many files to follow one flow. — Code-review lane on the PR: `git grep -n "^\s\+from app\." <new files>` new function-local imports each justified in a comment; reviewer verdict recorded in the PR body.
Outcome
Split routes/agent.py into 13 one-domain modules (app/routes/agent/) and its test into 9 facets; no shim, OpenAPI identical; lane full gate green, merged d8e264ed, deploy ok. Follow-ups #7670, #7701 (drafts).
Artifacts
Activity
- Draft
- Ready
- Runner started
- Runner ended (done)