Make ACP usage cost cumulative

Task 7481 · Done · Coder · created · updated

Child C1 of epic 7425 (ACP runtime: every provider lane on ACP). Planned by Task 7428. ## Requirement (verbatim) Epic goal: "Goal: every provider and auth lane can run on jaah-acp-host. On it, usage is counted exactly once, a usage limit triggers spillover, a question waits for the operator, and the provider seam has one home. Each fix removes its catalogue gap or legacy-only row in `apps/jaahd/agent_providers.py`, so Settings › Providers tells the truth. While a legacy path still exists, it keeps working." Outcome 1 (A4): "Usage is exact. An ACP `usage_update.cost` is a running total: keep the latest, never sum, and never add it on top of the per-turn costs (cumulative 0.10 then 0.25 ⇒ 0.25)." Current behaviour (PRD §3): `acp/events.py:336-339` turns `usage_update` into a usage item. `_usage` (`:241-250`) tags it `agent_reported` only when cost is its only field. `session_events.py::usage_of` (`:355-387`) **sums** every `agent_reported` cost (`:376-379`), including the prompt result's `_meta["jaah.ai"].usage` cost (`events.py:359`). So a cost-only running total is summed (over-count). A `usage_update` that carries `used`/`size` alongside cost becomes `source=tokens`, and its cost is **dropped** (under-count). Nothing distinguishes a running total from a per-turn cost. Decided (PRD): the latest value wins even when it is lower, because the agent's running total is authoritative. Billing rules are out of scope, so R1 changes only aggregation, not pricing. The cumulative cost rule applies to `usage_update.cost` only; per-turn costs still sum when a session has no `usage_update` cost. A cost riding with context fields is counted. ## Approach (architecture §1, verbatim) **Problem found in recon.** There are two layers of summing, not one: 1. `session_events.usage_of` sums every `agent_reported` cost (`session_events.py:376-378`). 2. Ingest folds each batch onto a running total read back from `Task.metrics["sessions"][id]` (`routes/agent_sessions.py:738-743`, `_stream_totals` :778-787, `totals_from_entry` `session_events.py:319-338`). Only `session_end` recomputes from every stored event (`iter_all`). A "keep the latest" fix inside `usage_of` alone would still be summed across batches (R1.3). The running-total state must therefore survive the metrics round-trip. **Chose:** an explicit cost-kind marker on the wire, plus split running totals. **Over** inferring "cumulative" from `source`. **Over** having the API recompute from all events on every batch (an S3 read of the full stream per batch). Pinned wire contract (node → API, this child owns both sides): - `acp/events.py`: the `usage_update` branch (:336-339) writes the new key `cost_kind: "cumulative"` into the usage item whenever `reported_cost_usd` is not None. `USAGE_KEYS` gains `cost_kind`. Every other item writes `cost_kind: null`. `_usage` excludes `cost_kind` from the "only cost ⇒ agent_reported" test. - Chose a new key over keying off `source`, because a cost-only `usage_update` is rewritten to `source="agent_reported"` (`_usage` :246-248, pinned by `test_acp_events.py:179-180`), which makes it indistinguishable from a per-turn `agent_meta` cost. **Recorded scope of R1.6:** the two `usage_update` wire-shape assertions in `test_acp_events.py::test_usage_update_and_plan` (:176 and :179-180) gain `cost_kind`. Every token assertion stays unmodified. - A `usage_update` with `used`/`size` keeps `source="usage_update"` (`test_acp_events.py:176`); `usage_of` then drops its cost. The cumulative branch below runs first, whatever the source, so R1.4 holds. - `UsageData` (`session_events.py:103`) gains `cost_kind: Optional[Literal["cumulative"]] = None`. It already has `extra="ignore"`, so old and new nodes are both accepted. Readers and writers ship in one PR. Aggregation (`usage_of`): - An item with `cost_kind == "cumulative"`: set `totals.cumulative_cost_usd = cost`. Replace, never sum; a lower value still wins (R1.5). The item adds no tokens, no `items`, and no model bucket. It must not fall through to the `source=tokens` path, which today invents an `unknown` bucket. - An `agent_reported` per-turn item: `totals.turn_cost_usd += cost` (today's behaviour, renamed). - `UsageTotals.reported_cost_usd` becomes the derived property `cumulative_cost_usd if not None else turn_cost_usd`, so every existing reader (`_fold_stream_usage`, billing) keeps one name (R1.2). - Tokens are untouched (R1.6). Persistence (`_fold_stream_usage`, `totals_from_entry`): - The metrics entry gains two additive keys, `reported_cumulative_cost_usd` and `reported_turn_cost_usd`, and keeps writing `reported_cost_usd` (the effective value every consumer reads). - `totals_from_entry` restores both split fields. A legacy entry with only `reported_cost_usd` is read as `turn_cost_usd` (in-flight sessions keep today's semantics; `session_end` recomputes from all events). Additive, no schema change: `Task.metrics` is JSONB. Seam (architecture §9): `acp/events.py` is shared with C2 (`_turn_usage`); disjoint functions. If C2 lands first, its prompt-result items must write `cost_kind: null` after rebase. Credentials & reach: - Replay, unit, API and golden tests; `check_provider_branches.py`; `./dc check full` — Nothing beyond the worktree — verified (tests and gate are local to any slot) Out of scope: Do NOT touch billing rules — "No price, tier, markup, debit or plan rule changes: no edits to `apps/api/app/billing/**`, `apps/api/app/token_billing/**` pricing tables, `line_pricing_tier` or `priced_by_time`. R1 and R2 change only how reported values are read and aggregated." Do NOT touch `_turn_usage` token reading (C2), the chat ↔ terminal hand-off (Task 7370), registry and SDK v2 agents (Task 7429), Settings › Providers layout. "No secret in any persisted surface". "No new provider-name branches (`tools/provider_branches.toml` allowlist growth)." Done means: - R1.1 A session's `reported_cost_usd` from ACP `usage_update.cost` equals the **latest** cumulative value received (by event sequence), never a sum of `usage_update` costs. — verify: a replay or unit test feeds `usage_update` costs 0.10 then 0.25 (USD). Observe `usage_of(...).reported_cost_usd == Decimal("0.25")`. - R1.2 Per-turn agent-reported costs (`_meta["jaah.ai"].usage` on a prompt result) are never added on top of a session that has a `usage_update` cost. — verify: a test feeds `usage_update` 0.10, a prompt-result cost of 0.10, then `usage_update` 0.25. Observe 0.25. - R1.3 The running total survives batching: costs that arrive in separate ingest batches of one session (`routes/agent_sessions.py` stream folding) still give the latest value, not a sum of per-batch latests. — verify: an API test posts 0.10 and 0.25 in two batches for one session. Observe the stored session total 0.25. - R1.4 A `usage_update` whose cost rides with context-window fields (`used`/`size`) still counts as the running total. Its cost is no longer dropped. — verify: a test with `used`+`size`+`cost` 0.25 gives `reported_cost_usd == 0.25`; plus an `acp/events.py` unit test that such a `usage_update` carries `cost_kind: "cumulative"`. - R1.5 (states) A session with no `usage_update` cost keeps today's behaviour: per-turn agent-reported costs sum. A non-USD cost is still ignored. A session with no usage at all reports None. A decreasing cumulative value (agent restart or resume) is taken as the latest. — verify: one test case per state, asserting each; plus a case where a legacy metrics entry holding only `reported_cost_usd` is restored as the turn sum. - R1.6 Token counts are unchanged: they sum per turn as today. — verify: the existing `test_acp_events.py` and `session_events` tests pass, with only the two `usage_update` cost wire-shape assertions (:176, :179-180) gaining `cost_kind`; no token assertion modified. - T.5 `python3 tools/check_provider_branches.py` passes, and no provider-name branch is added to its allowlist. — verify: run it and observe "OK"; `git diff tools/provider_branches.toml` adds no `[[allow]]`. Required checks: ./dc check full

Outcome

The ACP usage_update cost is now a running total: the latest value is kept, never summed, and per-turn costs are not added on top. It survives batching. Quick gate and API/jaahd tests green; PR #6046 merged as 195a6bc. Its deploy was cancelled when newer deploy 1dd1fba replaced it. Follow-up: draft #7511 (UI label).

Pull request

Artifacts

Activity

  • Draft
  • Ready
  • Runner started
  • Runner ended (done)