Accept and label memory proposals
Epic 6659: Project memory, open the pipe, honest trust. This is child C4 (improvement 4, PRD S4). It builds on C0's plan gate and on C2's generic restore branch in `votes.cast_vote`, which is why it runs after C2.
Sources: PRD (Task 6660) §3.4, and Architecture (Task 6661) §1 C4 and §2.2.
## Original requirement (verbatim)
"It is important that all memory related mechanism stays behind the feature flag on the billing plan." … "Each later subtask ships its change inside this gate and adds a test proving the mechanism is inert with the term off."
"4. Halve proposal churn: accept and label instead of refusing. `gates.py` 31–37 / 93–120, `vocabulary.py` 152–187, `apps/jaahd/agent_vocabulary.py` alias table: truncate an over-long title/body at the limit and answer with a warning instead of 422; accept a gotcha without a repro as search-only with reason \"needs repro\"; `task_state`/`time_bound` answers name the offending phrase, and \"after Task N\" alone no longer counts as task state when the body states a rule; add a `repro` argument appended to the body as a `Repro:` line. Done means: the 12 abandoned proposals in the report's list, replayed against the local stack, each land on the first call."
## Acceptance criteria (verbatim, PRD §3.4)
- **S4-1 Truncate.** A title over 80 characters or a body over 1200 characters is truncated at the limit, on a word boundary where possible. The unit is accepted, and the answer carries a `warnings` entry naming the field and the original length. An over-limit list (more than 20 items) is cut to 20 with a warning. A `paths` string is accepted as a one-item list (or split on commas) with a warning, not a 422. Any other schema error (unknown kind, empty title or body) stays 422.
- **S4-2 Gotcha without repro.** It is accepted as `injectable=false` (search-only) with `status_reason="needs repro"`. A later `memory_feedback` confirm or a re-proposal with a repro makes it eligible as usual.
- **S4-3 Refusals name the phrase.** A `task_state`/`time_bound` refusal answers `{rejected, phrase}`, where `phrase` is the exact matched text, so the retry is one edit.
- **S4-4 Narrower matching.** "after/since Task N" alone is not task state when the body also states a rule, meaning it contains a relation term (`vocabulary.py:50-53`). Like `time_bound`, `task_state` ignores the `Repro:` line and fenced blocks, because a repro citing a PR or run id is evidence, not task state.
- **S4-5 `repro` argument.** `memory_propose` takes an optional `repro` string, declared in `apps/jaahd/agent_vocabulary.py` and in the route model, and the CP appends it to the body as a `Repro: <text>` line. A gotcha with `repro` passes the repro gate. The body limit applies after appending, which truncates with a warning per S4-1.
- **S4-6 Replay corpus lands on the first call.** Each entry gets an `id` (accepted, possibly search-only with a reason) on its first call. No `rejected` and no 422. The corpus is:
- the 4 real proposals in artifact `refused-proposals-oct14-16.json`;
- one synthetic case per reported refusal class: an 81+ character title; a 1201+ character body; `paths` as a string; a `repro` argument; a gotcha without a repro; "after Task N" in a body that states a rule; a body with "still" or "next" used as a rule ("the next free slot").
The coder commits it as a test fixture.
- **S4-7** Genuine session summaries (`session_summary` gate), canon duplicates, secrets and locator-only qa are still refused. The relaxation does not open these.
- **S4-8 Inert** (S0-7): with the term off, propose answers 409 `MEMORY_OFF_DETAIL`, with or without `repro`.
- **Q2** Every runner-facing refusal and warning names its cause in plain words (field, phrase or limit). **Q3** Limits are named constants in `vocabulary.py`; the tool description matches them, with a test.
## Approach (Architect §1 C4, §2.2; settled except where recorded below)
- **Clamp, single-homed.** `vocabulary.clamp_proposal(title, body, paths, personas, repro) -> Clamped(title, body, paths, personas, warnings)`. It works in this order:
1. Append `\nRepro: <repro>` FIRST.
2. Cut to `TITLE_MAX`/`BODY_MAX` on a word boundary with `…`.
3. Split a `str` `paths` on commas.
4. Cap lists at `LIST_MAX_ITEMS`/`LIST_ITEM_MAX`.
Each cut adds a warning such as `"body cut to 1200 characters (was 1430)"`.
- **Propose route** (`routes/agent_memory.py:339-367`) calls the clamp before building `Candidate`, and `Candidate.__post_init__` stays strict for other feeds. `MemoryProposeIn` gains `repro: Optional[str]`, and `paths` accepts `str | list[str]`. `MemoryProposeOut` gains `warnings: list[str] = []`, `phrase: Optional[str]` and `status_reason: Optional[str]`.
- **Gates** (`gates.py:95-120`):
- Add `exclusion(...) -> Exclusion(reason, phrase) | None` carrying `m.group(0)`. `exclusion_reason` becomes a thin wrapper.
- `task_state` scans the stripped text, with Repro lines and fenced blocks removed.
- The "after/since Task N" family does not fire when the stripped body has a `RELATION_TERMS` hit.
- `time_bound`'s `next`/`still` require the temporal sense: not followed by a noun-phrase head such as `free`/`available`. Pin this with the corpus.
- **Gotcha without repro** (feed `propose` only) is stored with `injectable=False` and `status_reason = NEEDS_REPRO` (new constant `"needs repro"`). Add `NEEDS_REPRO` to `vocabulary.RESTORABLE_REASONS` (C0), so C2's generic `cast_vote` restore branch handles the confirm path. Other feeds keep rejecting.
- **Re-proposal with a repro.** Change `promote_unit` only. When `near_duplicate` (`curate.py:542-547`) matches a survivor with `status_reason == NEEDS_REPRO` and the incoming body has a `Repro:` line or fenced block (`gates._REPRO_LINE`), then after `merge_units`:
- (a) copy the incoming body onto the survivor;
- (b) clear `NEEDS_REPRO`;
- (c) set `injectable = votes.restorable_injectable(unit)`.
With no near-duplicate, the new unit is eligible and the old one stays `NEEDS_REPRO`.
- **Planner decisions, made under Autonomous ON. They resolve a conflict between PRD S4-2 and Architect §2.2.**
- Decided that a full-weight confirm restores a `NEEDS_REPRO` unit with NO `Repro:`-line precondition. PRD S4-2 says "a later `memory_feedback` confirm … makes it eligible as usual", and Arch §2.2's extra precondition would make that oracle fail. The `Repro:`-line condition applies only to the re-proposal path.
- Decided, on purpose, that any confirm C2 weighs at 1.0 restores a `NEEDS_REPRO` unit. That includes a settle confirm backed by a read event, not only `memory_feedback`. This is "as usual".
- A noted flag on a `NEEDS_REPRO` unit keeps `NEEDS_REPRO` (Arch §2.2), so a later full-weight confirm can still restore it. Pin this with a test.
- **`apps/jaahd/agent_vocabulary.py:740-754`:**
- add a `repro` `ToolArg`;
- `paths` accepts a string;
- rewrite the summary to describe warnings and phrase;
- keep `strict_args`.
Extend `tools/tests/test_terminal_types_parity.py` so that 80, 1200 and 20 in the summary equal `TITLE_MAX`, `BODY_MAX` and `LIST_MAX_ITEMS`. This node half ships via the jaah package.
- **Corpus fixture** at `apps/api/tests/fixtures/memory_propose_corpus.json`. It holds the 4 real refused calls from BA Task 6660's `file` artifact `refused-proposals-oct14-16.json`, read via `task_read_artifact`, plus the 7 synthetic classes. Before committing, check the real calls for secrets and for paths outside the repo.
Shared edit points (rebase expected): `curate.promote_unit` (C1 edits the canon hold there), `vocabulary.RESTORABLE_REASONS` (C2 adds `FLAGGED_WITH_NOTE`), `routes/agent_memory.py`, and `tools/tests/test_terminal_types_parity.py` (C5 also extends it).
Credentials & reach (verbatim rows from Architecture §4):
- All unit/API/web tests, `./dc check full`, and the "local stack" oracles (S4-6 replay) | Nothing beyond the worktree and its `./dc` stack's Postgres | verified: worktree present, gate runs in-slot. **Decided:** the "local stack" oracles run as API integration tests. They drive the real agent routes through the FastAPI test client against the slot DB, with a test-minted node caller. `jaah-call` targets the node's control plane (dev/prod), never the `./dc` stack, so the live leg is the dev runner check.
- Corpus input (C4) | BA Task 6660 `file` artifact `refused-proposals-oct14-16.json`, read through `task_read_artifact` | verified in the BA output (PRD §2). Read access uses the control tools, with no credential.
- Dev console checks (S4-2/5), operator sign-in on `https://app.dev.jaah.ai` | `jaah/debug/main/operator-login` | verified: on the roster, `task_credential_get` succeeded, and the dev health probe returned 200.
- PR merge and Jaah Deploy watch (E1) | `jaah-forge` (authenticated per task) | verified.
- Dev oracles needing the new node code (S4-5 dev) | Dev nodes running a jaah package built from the C4 merge | deferred: the package does not exist until this child merges and Jaah Deploy publishes it. Confirm the node's package version on the console Nodes page first. The API/tool tests stand as the oracle meanwhile.
Out of scope:
- From the epic, verbatim: "configuring the production memory model credential (an operator action in Settings › Runners › Project memory, not code); retiring mem-2/mem-3 and purging mem-7834 (operator does it in the console); any change to the OMC notepad/project-memory tier; new embedding providers."
- Do NOT relax the `session_summary`, canon-duplicate, secret or locator-only-qa gates.
- Do NOT change `merge_units` or nightly consolidation.
- Do NOT relax `Candidate` strictness for the settle/stop_delta/distill/codemap feeds.
- Do NOT rewrite C2's restore branch; only add the reason.
- Make no migration.
Done means:
- S4-1 — an API test per case (long title, long body, >20 items, `paths` as string) asserts HTTP 200, a non-null `id` and the warning naming field and original length. The cut is on a word boundary. Unknown kind or empty title/body still gives 422.
- S4-2 — an API test shows a gotcha with no repro is stored `injectable=false` with `status_reason="needs repro"` and appears in `memory_search`. A later `memory_feedback` confirm makes it injectable. A noted flag on it keeps `needs repro`. Re-proposal tests: (i) a near-duplicate with `repro` leaves one injectable unit whose body carries the `Repro:` line; (ii) a non-matching one leaves the new unit injectable and the old one `needs repro`. In the Memory tab the unit shows the reason.
- S4-3 — an API test asserts `phrase` for each `task_state`/`time_bound` pattern family.
- S4-5 — an API test shows `repro` is appended as a `Repro:` line BEFORE the body cut, so a gotcha whose repro would be lost to truncation still passes the repro gate. Truncating before appending FAILS. On dev, after the package lands, a runner's `memory_propose` with `repro` returns an id and no `malformed_call`.
- S4-4/S4-6 — the committed corpus fixture holds 4 real and 7 synthetic entries. An API test iterates it through the real propose route against the slot DB and asserts an `id` on the first call for every entry, with no `rejected` and no 422.
- S4-7 — the existing gate tests stay green, plus a secret-hit case is still refused.
- S4-8 — an inert test using `plan_off_project` asserts 409 `MEMORY_OFF_DETAIL` with and without `repro`. Name it in the PR body.
- Q2/Q3 — every warning and refusal names its field, phrase or limit. `test_terminal_types_parity.py` pins the tool summary's 80/1200/20 to the `vocabulary.py` constants.
- E1 — the PR is merged to develop through auto-merge with `./dc check full` green, and the Jaah Deploy run succeeded (or was superseded).
Outcome
memory_propose now cuts with warnings, takes repro and paths strings, keeps repro-less gotchas as "needs repro", and names the refused phrase. The corpus lands on its first call. PR #5647 merged; deploy superseded. UI check OK. Drafts #6765, #6766; dev leg awaits the package.
Artifacts
Activity
- Draft
- Ready
- Runner started
- Runner ended (done)