Project memory: open the pipe, honest trust

Task 6659 · Done · Epic · created · updated

Operator ask (Task 6581 console, verbatim): "File all 5 as an epic. It is important that all memory related mechanism stays behind the feature flag on the billing plan." Context: the attached report (Project Memory Audit II, Task 6581) measured 53 memory-ON runner sessions on Oct 14–16 after the five Oct 14 fixes (Tasks 6296–6302, 6315, 6316). Findings in one line each: every runner proposal is accepted then parked as "canon check inactive" (no memory model credential; the rules-only fallback holds any unit touching a context-router route, which is every unit), so 0 proposals have ever reached a prompt; settle votes let habit outvote verification (mem-2, fixed Oct 9, holds trust 0.96 after 18 reflexive confirms); 23 of 47 propose calls are refused and 12 facts lost; 211 hook surfacings produced 0 follow-up reads. ## Constraint that binds every subtask: a billing-plan feature switch Today project memory is gated only by operator knobs (`runner_project_memory` kill switch, `project_memory_default` org-tier default, `project_memory_holdout_percent`, `memory_llm_credential_id` in `apps/api/app/operator_settings.py`; resolution in `apps/api/app/project_memory/switch.py::effective_memory`). There is NO plan switch: `billing_plans` carries `compliance_enabled`, `nodevault_allowed`, `nodelink_allowed`, `merge_lane_enabled`, but nothing for memory, and `app/plan_terms.py` has no memory term. Subtask 0 (first, the others depend on it): add `project_memory_enabled` as a billing-plan switch with the standard per-org override, following the existing plan-switch recipe (mirror commit 544a3f27d / `merge_lane_enabled`: plan column + `orgs.<name>_override`, additive migration, `plan_terms` SWITCH_DEFAULT_KNOBS / OPERATOR_PLANE_SWITCHES / PlanTerms / effective, operator_settings default knob, `routes/billing_plans.py`, `apps/web/src/lib/planSwitches.tsx`, plan editor UI). `effective_memory` must return `(False, "plan")` when the org's plan term is off, ahead of task/epic/holdout/default; `memory_active` must read the same term so a running task loses memory within the settings-cache window when the plan flips. Every memory mechanism — prompt section and hook bundle, `memory_search`/`memory_read`/`memory_propose`/`memory_feedback`, settle votes, the curate/nightly crons' work for that org's projects, the node helper's background jobs, the Project › Memory tab — must be inert for an org whose plan term is off, answering the existing `MEMORY_OFF_DETAIL` refusal where a runner tool is involved. Default: off on every plan except where the operator turns it on. Each later subtask ships its change inside this gate and adds a test proving the mechanism is inert with the term off. ## The five improvements (one subtask each, in this order after subtask 0) 1. Open the pipe: never hold a `propose`-feed unit with no path to release. In `apps/api/app/project_memory/curate.py` (canon_verdict ~455–456, promote_unit ~559–562, recheck_held ~571–656): without a memory model, a `propose`-feed unit that passes the regex secret gate is accepted injectable; keep the rules-only hold for settle-mined, stop-delta and distilled units; make `recheck_held` (or a rules-only equivalent) release the already-held `propose` backlog on the next tick. Show the held count in the Project › Memory tab banner (`apps/web/src/components/projects/ProjectMemoryTab.tsx`) so a stuck pipe is visible. Done means: a new proposal reaches a later dispatch prompt in the local stack with no model credential set; the held backlog count drops to the settle/distill units only. 2. Weighted trust: a verified flag beats a reflexive confirm. `votes.py` 24–38, `nightly.py::trust_of`, `routes/agent_memory.py::record_settle_votes`: a settle confirm for a unit that was only `injected` in that task weighs 0.25; a flag carrying a note, a `stale` verdict, or a merged-PR citation weighs 1.0; one flag with a note drops `injectable` until a later evidence-backed confirm. Backfill the gotcha/qa path auto-fill (`bundle.py` 48–63, Task 6298) over existing units so the stale verifier can judge units like mem-2 (paths empty, test file named in prose). Done means: with the formula, mem-2's current 63 confirms + 2 flags no longer keeps it injectable once one flag carries a repro note; the backfill gives mem-2 a path and the next verify pass marks it stale. 3. `memory_used` becomes an evidence field. `record_settle_votes` mints a confirm only when the task has a `read`/`followed`/non-provisional `cited` event for the unit, or its `mem-N` appears in the summary or PR body; other listed units are recorded as "seen" (no vote; still counts against unused demotion). Reword the prompt line (`dispatch_compose.py` ~3430) to "list a unit only if you read it, ran its command, or cite it". Done means: a settle listing only prompt ids mints zero confirm votes; one listing a searched-and-read unit mints one. 4. Halve proposal churn: accept and label instead of refusing. `gates.py` 31–37 / 93–120, `vocabulary.py` 152–187, `apps/jaahd/agent_vocabulary.py` alias table: truncate an over-long title/body at the limit and answer with a warning instead of 422; accept a gotcha without a repro as search-only with reason "needs repro"; `task_state`/`time_bound` answers name the offending phrase, and "after Task N" alone no longer counts as task state when the body states a rule; add a `repro` argument appended to the body as a `Repro:` line. Done means: the 12 abandoned proposals in the report's list, replayed against the local stack, each land on the first call. 5. Hook tier earns its lines. `tools/project_memory_tier.py` 304–361, `tools/context_router.py` 791–815 / 1197–1266, `apps/jaahd/fleet_account_settings.json` (UserPromptSubmit wiring): drop the per-turn prompt match; fire on Bash command-prefix and Edit/Read anchored-path matches only; for gotcha/recipe carry up to 300 characters of body instead of the lead; count a unit surfaced in 20 sessions and never read as unused, the same demotion the prompt channel uses. Done means: a session replay shows no `[memory]` line on UserPromptSubmit, a Bash match shows the body, and the unused counter increments from hook surfacings. Out of scope: configuring the production memory model credential (an operator action in Settings › Runners › Project memory, not code); retiring mem-2/mem-3 and purging mem-7834 (operator does it in the console); any change to the OMC notepad/project-memory tier; new embedding providers. Done means (epic): every subtask merged to develop with `./dc check full` green and Jaah Deploy succeeded; a single plan switch gates every memory mechanism and is off by default; with the switch on and no model credential, a runner proposal reaches a later prompt; the report's before/after table can be re-measured by re-running the Task 6581 scan after two days of runs. Required checks: ./dc check full

Subtasks

  • [business_analysis] Project memory: open the pipe, honest trust Done
    PRD settled: S0 plan switch project_memory_enabled (default off, gates every memory mechanism) then S1–S5 with oracles; mem-2 stale needs creation-time anchoring; S4 replay corpus defined (report's 12 not recoverable). mem-14220, mem-17928.
  • [architecture] Project memory: open the pipe, honest trust Done
    C0 (plan switch, memory_active(session) gate, scoped curate/nightly, 403/410) first; C1–C5 in parallel after it. C2 adds a node-resolved creation-time anchor, and S5-4 moves to C3's use_pairs fix. Seams are pinned in §2. Used mem-14220 and mem-17928.
  • [plan] Project memory: open the pipe, honest trust Done
    6 coder children, one per subtask, split by the Architect's boundaries: C0 plan switch (phase 1); C1/C2/C3/C5 (phase 2); C4 after C2 (phase 3, restore dep). S4-2: followed PRD over Arch §2.2 Repro precondition. No Required checks line (mem-11178). Traps tightened into Done means. Used mem-14220, mem-17928.
  • Add project memory billing-plan switch Done
    Project memory now behind a billing-plan switch (off by default); every memory mechanism inert when off. Full gate green, PR #5601 merged, deploy OK, dev override ON for Gmind. Dev runner legs unverified (tests cover); follow-up draft #6688. Used mem-2.
  • Release held proposals, show held count Done
    With no model, proposals are served and the held propose backlog is released per tick with a secret re-scan. The Memory tab shows a Held count and held rows. Full gate green; PR #5628 merged and Jaah Deploy OK. The dev-console leg was not run (tests and the local stack cover it). UX draft #6713 filed.
  • Weight settle votes, backfill anchored paths Done
    Weighted settle votes (0.25 reflexive), noted-flag hold + restore, nightly gotcha/qa path backfill anchored at creation. Full gate green; PR #5627 merged (c14a05eb8), Jaah Deploy OK. Dev mem-2 check deferred to node package; follow-up draft #6757.
  • Mint memory confirms only on evidence Done
    Settle confirms memory only on use evidence (read/followed/cited or mem-N in summary); others seen. use_pairs ignores settle confirms, fixing Used and hook demotion. PR #5612 merged, deploy OK. Full gate green bar 1 unrelated flake. Dev runner-prompt/Used legs unverified (tests cover). Draft #6711.
  • Fire hook memory on signals only Done
    Hook [memory] now fires only on Bash prefix or path matches, never on prompts; gotcha/recipe show a 300-char body excerpt. Fixed bundle byte-cap overshoot. Full gate green, PR #5625 merged; deploy superseded by the next run. No UI affected; dev runner checks wait for the package.
  • Accept and label memory proposals Done
    memory_propose now cuts with warnings, takes repro and paths strings, keeps repro-less gotchas as "needs repro", and names the refused phrase. The corpus lands on its first call. PR #5647 merged; deploy superseded. UI check OK. Drafts #6765, #6766; dev leg awaits the package.
  • [finish] Project memory: open the pipe, honest trust Done
    All 6 coders (6670-6675) merged (#5601,#5628,#5627,#5612,#5625,#5647); code verified on develop, plan switch gates all runtime memory. Filed 6796 (Audit memory list off-plan). #5647 deploy pending; dev.jaah.ai unreachable, live leg unverified.
  • Audit › Project memory: hide off-plan projects Done
    Audit memory comparison list now hides off-plan projects; API and plan-off compose tests added. Full gate green via the merge lane, #5711 merged (77fa57c6a). Its deploy was superseded by the next develop deploy. Visual check OK. Filed draft #6917 (empty picker).

Artifacts

Attachments

Activity

  • Created
  • Updated
  • Runner started
  • Ready
  • Ready
  • Ready
  • Runner started
  • Runner ended (done)
  • Runner started
  • Runner ended (done)
  • Runner started
  • Draft
  • Draft
  • Draft
  • Draft
  • Draft
  • Draft
  • Draft
  • Ready
  • Ready
  • Ready
  • Ready
  • Ready
  • Ready
  • Ready
  • Runner ended (done)
  • Runner started
  • Runner ended (done)
  • Runner started
  • Runner started
  • Runner started
  • Runner started
  • Runner ended (done)
  • Runner ended (done)
  • Runner ended (done)
  • Runner ended (done)
  • Runner started
  • Runner ended (done)
  • Runner started
  • Ready
  • Runner ended (done)
  • Runner started
  • Ready
  • Runner started
  • Ready
  • Runner started
  • Ready
  • Runner started
  • Ready
  • Runner started
  • Ready
  • Runner started
  • Ready
  • Runner started
  • Ready
  • Runner started
  • Runner ended (system_failed)
  • Runner ended (system_failed)
  • Runner started
  • Run
  • Runner started
  • Runner ended (done)
  • Runner ended (done)