Drop codemap from prompts and search
Epic context (Task 6294, verbatim): Implement the five improvements from the Project Memory audit (Task 6283). The audit scanned 251 memory-ON runner sessions (2026-10-11..14): only 21 distinct non-codemap units ever reached a prompt, codemap chunks were 42% of injected bytes with zero reads, 24 of 74 search/read tool calls failed on argument names, 16 accepted runner proposals never became injectable, and memory_feedback was called once. The five changes below are one coherent flywheel fix; each is independently shippable.
Full settled inputs: the BA PRD (R-*/AC-*) and the Architecture doc of epic 6294 (sections cited below). This body carries everything this child needs.
## Requirement (verbatim from the epic)
2. Take codemap out of prompts and search; retire old generations.
Codemap units never fill a prompt slot (select.py ~151-152: exclude, do not just sort last) and never rank in memory_search results unless kind=codemap is requested explicitly. Keep them for the hook tier on a Read/Glob of their exact directory (tools/project_memory_tier.py). When tools/project_memory_node.py re-maps a directory, the capture path (apps/api/app/project_memory/capture.py _candidates, insert-only today) retires that directory's previous chunks in the same write.
BA requirements (verbatim):
- R-B1: `select_for_prompt` never returns a `codemap` unit. Codemap is excluded from the candidate set, not sorted last. A prompt with fewer than 8 qualifying non-codemap units carries fewer units. It is not padded with codemap.
- R-B2: `memory_search` returns no `codemap` unit unless the caller passes `kind=codemap`. This holds with a query, with no query, with a `paths` filter and with a `persona` filter. It covers both the runner-facing `memory_search` tool and the console unit search (Project › Memory search box). The console's **kind** filter set to `codemap` must still list them. [Architect: the console uses `console.list_units`, not `search.py`, so the console inventory and search are unaffected — only the runner-facing search hides codemap.] Decided: the console list with no kind filter does still show codemap (an inventory); only the runner-facing search hides them.
- R-B3: The hook tier keeps surfacing a codemap unit on a Read/Glob of its exact directory. That is unchanged and covered by a test. Codemap never surfaces on a Bash command.
- R-B4: When a codemap ingest for directory D lands, every previously `accepted`/`stale` codemap unit whose directory is D and which is not in the new generation is set to `retired`, with `valid_to=now` and `injectable=false`, in the same DB transaction as the insert. After the write, exactly one generation is live for D. Chunks of other directories are untouched. A chunk whose text did not change (same content hash) is kept, not retired and re-inserted. Pre-existing duplicate generations are cleaned up the first time each directory is re-mapped.
## Approach (settled by the Architect)
- B1: add `ProjectMemory.kind != "codemap"` to the `select_for_prompt` WHERE (select.py:133); delete the codemap sort key (:151-152) and fix the docstring (:114-116).
- B2: in `search.py` `search_units` (:190-196), when `kind` is unset add `ProjectMemory.kind != "codemap"`. The console uses `console.list_units`, so the inventory is unaffected. Update `_TOOL_MEMORY_SEARCH`'s summary in `apps/jaahd/agent_vocabulary.py` (drop "code-map notes", add "codemap only with kind=codemap").
- B3: no code; extend `tools/tests/test_context_router_memory.py::test_codemap_units_fire_only_on_a_read_or_glob_of_their_directory` (:507) with subdirectory, parent and Bash negatives.
- B4 (pre-mortem: today `batches()` in tools/project_memory_node.py:766 splits by count/bytes, so a directory can span requests):
1. Node: codemap batching packs whole directories — a directory never spans two requests (≤60 chunks × 1200 B fits the 100-candidate/200 KB caps, project_memory_node.py:67,71).
2. CP, `capture._candidates`, same transaction as the inserts: group the batch's STORED (inserted or merged) codemap candidates by directory (`paths[0]` `<dir>/*` with `/*` stripped; title prefix `Code map: <dir>` only when `paths` is empty). For each directory whose stored set is complete (one unnumbered chunk, or `(1/N)…(N/N)` for a single N), set `status='retired', injectable=false, slice=false, valid_to=now, status_reason=CODEMAP_SUPERSEDED` on every codemap unit of the project with status in (`accepted`,`stale`,`proposed`), `paths @> ARRAY['<D>/*']`, and id not among this batch's stored ids. Any rejected chunk of D or an incomplete set → retire nothing. Leave `flagged` units alone. Match on `paths`, not on the (truncatable) title.
3. `vocabulary.py`: `CODEMAP_SUPERSEDED = "superseded by a newer code map"`.
Credentials & reach:
| Oracle | Credential / identity / environment | Status |
|---|---|---|
| All unit and integration tests (AC-A*, B*, C*, D*, E1-E3, X1, X2) | None beyond the worktree. Postgres comes from the slot's `./dc` stack under `./dc check full` | verified: the worktree and `./dc` exist in every coder slot. The BA noted Postgres is absent in analysis slots, so tests run in the coder's gate |
| Dev flows AC-B2, AC-D3, AC-E1 (a memory-ON **dev** runner's `memory_search`, `[memory]` line, settle votes) | `jaah/debug/main/operator-login` on `https://app.dev.jaah.ai`. The coder signs in to the dev console as operator, dispatches a short probe task in a memory-ON project on dev, and reads that task's transcript in Audit › Sessions. This follows the hub rule "operate Jaah only through the UI". The coder's own session is NOT relied on: it may be dispatched by the prod CP and so run prod code | verified: credential on the roster and `task_credential_get` succeeded (value not recorded); dev console `GET /` → 200 and `/api/health` → 200 from node `h2`. The probe task itself is created at build time, not here |
| Deploy watch (operator directive: Jaah Deploy of each PR) | `jaah-forge` (pre-authenticated per task) | verified by construction: provided to every build runner; no credential to fetch |
Out of scope: Do NOT change codemap chunking (`CODEMAP_MAX_PARTS`, body cap), `CODEMAP_BUNDLE_SHARE`, or the hook tier's directory-match rule; no `generation` column; no one-time sweep of old generations (deferred F4); no profiling of the node codemap job.
Epic out of scope (verbatim): "the two immediate data fixes (purging mem-7834's login triple and retiring mem-2/mem-3) are operator actions in Project > Memory, not code in this epic; profiling the node-side codemap job and curate/nightly cron cost; changing the OFF-arm behaviour (memory OFF must remain byte-identical, see apps/api/tests/memory_off_fixture.py); any new memory kind; org-scoped memory."
Do NOT edit `apps/api/tests/memory_off_*.golden.json`, the `memory_off_fixture.py` pins, or `portfolio/fleet/runner-charter.md`. No new kind, feed, flag, setting or env var. Schema changes additive only (coding-standards migration-overlap rule).
Done means:
- AC-B1 — With 3 non-codemap and 10 codemap units all matching, `select_for_prompt` returns only the 3 non-codemap units. The composed PROJECT MEMORY section contains no `Code map:` title. With 0 non-codemap matches, the section is absent, as today's no-match behaviour. — `pytest apps/api/tests/test_project_memory_dispatch.py -k codemap`; observe pass
- AC-B2 — `memory_search` with a query, with no query, with `paths`, and with `persona` returns zero codemap units when `kind` is unset or non-codemap. With `kind=codemap` it returns them. The console list with Kind=codemap still lists them. — `pytest apps/api/tests/test_agent_memory.py -k search` + console test (`test_project_memory_console.py -k codemap`: Kind=codemap lists them, the unfiltered list still includes them); observe pass. Dev flow: in a dev runner, `memory_search q="how is project memory used"`. No `Code map:` hit is returned.
- AC-B3 — The hook tier surfaces a codemap unit on Read of a file in exactly `<dir>` and on Glob of `<dir>`. It does not surface it for a subdirectory, a parent, or a Bash command. — `pytest tools/tests/test_context_router_memory.py -k codemap`; observe pass
- AC-B4 — Ingesting generation 2 of directory D, where 1 chunk is unchanged and 2 have changed text, leaves exactly gen-2's chunks live for D. Gen-1's changed chunks are `retired` with `valid_to` set, in the same transaction. Chunks of directory E are untouched. Re-ingesting an identical generation retires nothing. — `pytest apps/api/tests/test_project_memory_capture_routes.py -k codemap`; observe pass. Also: an incomplete set, or a set with one rejected chunk, retires nothing; `tools/tests/test_project_memory_node.py` proves no directory spans two batches.
- AC-X1 — Memory-OFF parity: `memory_off_prompt.golden.json` and `memory_off_lease_payload.golden.json` are unchanged (no diff in the PR), and the OFF golden test passes. — `cd apps/api && pytest tests/test_project_memory_dispatch.py -k memory_off_prompt_golden`; `git diff --stat origin/develop -- apps/api/tests/memory_off_*.golden.json` is empty
- AC-X2 — Whole gate green. — `./dc check full` exits 0
- AC-X3 — Availability: every capability above is ON for any project whose memory arm is ON, with no new flag, setting or env var gating it, and it is live on the dev control plane after deploy. — Dev flow AC-B2 observed on `dev.jaah.ai`
- The post-merge Jaah Deploy run on `develop` succeeds in GitHub (operator directive; a canceled/superseded run is left alone); cite the run.
Outcome
Codemap excluded from prompts and runner memory_search (kind=codemap still works), superseded codemap generations retired per directory, node packs whole directories. Full gate green; PR #5437 merged; deploy 38604561138 succeeded. Dev runner probe unverified: dev operator account isn't a Gmind member.
Artifacts
Activity
- Draft
- Ready
- Runner started
- Runner ended (done)