Demote unused units; console usage sort

Task 6299 · Done · Coder · created · updated

Epic context (Task 6294, verbatim): Implement the five improvements from the Project Memory audit (Task 6283). The audit scanned 251 memory-ON runner sessions (2026-10-11..14): only 21 distinct non-codemap units ever reached a prompt, codemap chunks were 42% of injected bytes with zero reads, 24 of 74 search/read tool calls failed on argument names, 16 accepted runner proposals never became injectable, and memory_feedback was called once. The five changes below are one coherent flywheel fix; each is independently shippable. Full settled inputs: the BA PRD (R-*/AC-*) and the Architecture doc of epic 6294 (sections cited below). This body carries everything this child needs. ## Requirement (verbatim from the epic) 3(a) Record an injection count per unit (events.py ~65-68 stores only last_injected_at today) and drop `injectable` after 20 injections with zero read/followed/non-provisional cited events; expose the count in Project > Memory. 5 (part). Update comparison.py read_rate (~183-237) so "used" reflects the new semantics. In Project > Memory (apps/web/src/components/projects/ProjectMemoryTab.tsx ~186-208) show injected/used/followed counts with a sort, so an operator can retire dead units by eye. BA requirements (verbatim): - R-C1 (injection count): A unit's injection count is the number of distinct tasks with an `injected` event for it, on any channel (prompt or hook). Historical events count. A unit's use count is the number of distinct tasks with a use event as defined once in R-E3. - R-C1 (demotion): When a non-codemap unit reaches an injection count ≥ 20 with a use count = 0, it loses `injectable`. It stays `accepted` and searchable. It records a distinguishable reason the console can show as "unused". It is NOT re-made injectable by `corroborate` on evidence that existed before the demotion; a new use event or confirm vote after demotion may restore it. The threshold 20 lives in `vocabulary.py` as one named constant. Codemap is exempt. Runs at least in the nightly job. - R-E3: A unit is used by a task when that task has a `read` event, a `followed` event, a non-provisional `cited` event, or a `confirm` vote from that task (tool or settle). The one definition lives in one place and is shared by `comparison.read_rate`, the C1 demotion rule, and the console's "Used" count. Historical `followed` events still count. - R-E4: Every unit row in Project › Memory shows Injected, Used and Followed counts, each in tasks; Read and Cited stay visible. The list can be sorted by Newest (default), Injected, Used and Followed, each descending — server-side over all units matching the current search/kind/status filter, not client-side over the newest `LIST_LIMIT` rows. A unit demoted under C1 shows a visible "unused" marker instead of the plain `accepted` chip. The sort control follows the tab's existing filter controls in style and labels, per `docs/standards/ui-copy-standards.md`. ## Approach (settled by the Architect) - Single home in `apps/api/app/project_memory/events.py` (contract): `USE_EVENT_KINDS = ("read","followed","cited")` replacing `events._USE_KINDS` (:20), `nightly._used` (:145-147) and `comparison._USED` (:37, which lacks the provisional filter); `use_event_clause(event)` (event-only, non-provisional — graduation, slice ranking `nightly._using_tasks` :354-361/:372/:534 and the `last_used_at` decay stamp keep exactly this); `used_by_task_clause(unit_id_col, task_id_col)` (event clause OR a `confirm` vote of any source by that task); `usage_counts_subquery(project_id)` → unit_id, injected, read, cited, followed, used (DISTINCT tasks). - Scope guard: the vote-inclusive "used" feeds ONLY read_rate, demotion and the console Used count. Do not change graduation, slice ranking or decay. - `vocabulary.py`: `UNUSED_AFTER_INJECTIONS = 20`, `DEMOTED_UNUSED = f"unused after {UNUSED_AFTER_INJECTIONS} injections"`. - `nightly.py`: new pass `demote_unused` after `retrust`, before `decay`, committing like its siblings. Demote: `status='accepted' AND injectable AND kind<>'codemap'`, ≥ threshold distinct injected tasks, zero used tasks → `injectable=false, status_reason=DEMOTED_UNUSED`, add project to `report.changed`. Restore: `status_reason=DEMOTED_UNUSED`, ≥1 used task, `trust >= TRUST_FLOOR` → `injectable=true, status_reason=NULL`. `corroborate` already requires `status_reason IS NULL` (curate.py:680), so no change there. No counter column — counts derive from `memory_events` (one row per task/unit/kind). - `comparison.read_rate` (:207): use the shared predicate correlated on (task, unit). - API: `GET /api/projects/{id}/memory/units` gains `sort` ∈ `newest|injected|used|followed` (default newest; unknown → 422). `console.list_units` LEFT JOINs the usage subquery onto the filtered units and orders `<count> DESC, number DESC` before `LIMIT LIST_LIMIT`. Rows gain `used: int` and `demoted_unused: bool` (computed from the constant server-side; the client never string-matches). - Web (`ProjectMemoryTab.tsx` + `projectMemory.ts`): Sort `TextField select size="small"` after Status, same pattern as Kind/Status, `data-testid="memory-sort"`, options Newest/Injected/Used/Followed; `sort` in the query key and `listUnits` filters; facts row gains Used and Followed beside Injected/Read/Cited via the same `Fact` component; `StatusChip` shows **unused** when `demoted_unused`. One-word labels. Optional: a `near_duplicate` regression test on the mem-6130/mem-11178 texts (not required). Credentials & reach: | Oracle | Credential / identity / environment | Status | |---|---|---| | All unit and integration tests (AC-A*, B*, C*, D*, E1-E3, X1, X2) | None beyond the worktree. Postgres comes from the slot's `./dc` stack under `./dc check full` | verified: the worktree and `./dc` exist in every coder slot. The BA noted Postgres is absent in analysis slots, so tests run in the coder's gate | | Dev flows AC-A4, AC-E1, AC-E4 (Project › Memory on dev) | `jaah/debug/main/operator-login` (generic_login, covers dev and main, operator and tenants) on `https://app.dev.jaah.ai` | verified: on the roster, `task_credential_get` succeeded (value not recorded); read-only probe `GET /api/health` → 200 from node `h2` | | Deploy watch (operator directive: Jaah Deploy of each PR) | `jaah-forge` (pre-authenticated per task) | verified by construction: provided to every build runner; no credential to fetch | Out of scope: Do NOT change graduation thresholds, slice ranking, decay windows, the trust formula or `corroborate`'s rule; no denormalised counter column; no change to `followed` minting (sibling hook/settle children own it); do NOT retire or merge mem-2/mem-3/mem-6130/mem-11178 by code (operator action; E5 is no-code per the BA). Epic out of scope (verbatim): "the two immediate data fixes (purging mem-7834's login triple and retiring mem-2/mem-3) are operator actions in Project > Memory, not code in this epic; profiling the node-side codemap job and curate/nightly cron cost; changing the OFF-arm behaviour (memory OFF must remain byte-identical, see apps/api/tests/memory_off_fixture.py); any new memory kind; org-scoped memory." Do NOT edit `apps/api/tests/memory_off_*.golden.json`, the `memory_off_fixture.py` pins, or `portfolio/fleet/runner-charter.md`. No new kind, feed, flag, setting or env var. Schema changes additive only (coding-standards migration-overlap rule). Done means: - AC-C1 — A non-codemap unit with 20 distinct injected tasks and no used event (R-E3) is `injectable=false` after the nightly run, carries the visible "unused" marker, and is still returned by `memory_search`. With 19 injections, or with 1 use event, it stays injectable. A codemap unit with 20 hook injections stays injectable. After demotion, the `corroborate` pass with no new evidence does not restore it. A new confirm vote does. — `pytest apps/api/tests/test_project_memory_nightly.py -k unused`; observe pass - AC-E3 — `comparison.read_rate` counts a unit as used when a task has read, followed (new), non-provisional cited, or a confirm vote. A provisional PR cite alone is not used. The C1 rule and the console "Used" count return the same number for the same fixture, from one shared definition. — `pytest apps/api/tests/test_project_memory_console.py -k read_rate`; observe pass - AC-E4 — Project › Memory shows Injected, Used, Followed, Read and Cited per unit. The sort control offers Newest, Injected, Used and Followed. Choosing Injected orders the rows by descending injected count across all filtered units, including a unit older than the newest `LIST_LIMIT`. A demoted unit shows "unused". Labels are short and not truncated at 360/768/1280 px. — `npx vitest run src/components/projects/ProjectMemoryTab.test.tsx` (apps/web) + API sort test; observe pass. Dev flow: open `dev.jaah.ai` → Projects → jaah → Memory, choose Sort: Injected. The top rows are the most-injected units, with counts visible. - Quality floor (tightened): the counts sit beside the existing facts row without wrapping the unit title at 360/768/1280 px — attach screenshots at those widths to the report; a layout that wraps or truncates the title FAILS. - AC-X1 — Memory-OFF parity: `memory_off_prompt.golden.json` and `memory_off_lease_payload.golden.json` are unchanged (no diff in the PR), and the OFF golden test passes. — `cd apps/api && pytest tests/test_project_memory_dispatch.py -k memory_off_prompt_golden`; `git diff --stat origin/develop -- apps/api/tests/memory_off_*.golden.json` is empty - AC-X2 — Whole gate green. — `./dc check full` exits 0 - AC-X3 — Availability: every capability above is ON for any project whose memory arm is ON, with no new flag, setting or env var gating it, and it is live on the dev control plane after deploy. — Dev flow AC-E4 observed on `dev.jaah.ai` - The post-merge Jaah Deploy run on `develop` succeeds in GitHub (operator directive; a canceled/superseded run is left alone); cite the run.

Outcome

Nightly demotes never-used units (20 injections, 0 use); one shared USE definition for read_rate, demotion and the console; Memory tab gains Used/Followed and a server-side sort. Full gate green, PR #5448 merged, Jaah Deploy succeeded, verified on dev at 360/768/1280. UX follow-ups are in the report (mem-2, mem-10065).

Pull request

Artifacts

Activity

  • Draft
  • Ready
  • Runner started
  • Runner ended (done)