Add trigger health notices and digest
Epic 5336 — Event-driven Task triggers. Upstream: PRD = Task 5337 `prd.md`; Architecture = Task 5338 `architecture.md` (artifact 3566). Read §3.1 `trigger_notices`, §3.9, §3.10 (the stubs you fill, already merged) and §6 IN FULL.
This child owns the ONLY schema migration of its wave: `trigger_notices` (+ unique index on `(user_id, kind, (summary->>'day'))`), the `email_templates` type CHECK gaining `trigger_failed` and `trigger_digest`, and the seeded-enabled `trigger-digest` system activity (daily 08:00 UTC).
## Original requirement (epic body, verbatim)
> limits: concurrency cap, queue depth, per-run budget, rate limit; health: Active / Degraded / Failed / Paused with reason, auto-pause after five consecutive failures or ten missed days, a deliveries log …, a daily digest
## Requirements (PRD, verbatim)
**R11 budget** Per-run budget (USD, optional; **advisory only**, D7): when a filed Task's `cost_usd` exceeds it, the breach is flagged on the Task, in the deliveries log and in the digest, and the trigger is marked **Degraded** with reason "budget exceeded on Task #n". **The run is never stopped.**
**R12** … A trigger's failures are: preflight failure, template or filing error, and a filed Task ending `system_failed`.
**R13** - **N days without a successful run** (a successful run is a delivery that dispatched, or was cleanly filtered or deduped): **off by default**, N set per trigger (D6). Either threshold sends the trigger to **Failed** with the reason, e.g. "5 consecutive preflight failures" or "no successful run for 7 days". The owner gets an in-app notification and an email at that moment, not only in the digest.
**R15** Once a day, each trigger owner who had at least one delivery or one non-Active trigger in the last 24 h gets **one email** and **one in-app (bell) notification** (D8). It shows per-trigger counts by outcome, current health with reason, budget breaches and links. No activity means no digest (A14).
## Approach (architecture §6, settled)
- Fill `trigger_health.on_failed` (a `trigger_failed` notice + email via `email_notify.deliver_to_user`, new template type `trigger_failed`), `fold_filed_outcomes` (filed Task reaching `system_failed` → +1 once via `outcome_counted`; `done` → budget-breach clearing per §3.9), `fold_budget` (`task_run_metrics.compute_run_metrics` for non-terminal or terminal-within-24h filed Tasks; over `budget_usd` → delivery `budget_exceeded_usd` + trigger breach fields; NOTHING stops the run). Extend the pure `trigger_health` with the budget-Degraded rule ("budget exceeded on Task #n ($x of $y)").
- Bell: derive `trigger_failed` and `trigger_digest` kinds from `trigger_notices` exactly as `billing` derives from `billing_notices` (`routes/inbox.py`); no general notifications table.
- `trigger-digest` activity: idle auto-pause (`auto_pause_idle_days` set, enabled, `coalesce(last_success_at, created_at) < scheduled_at - N days` → Failed "no successful run for N days" + `on_failed`), using `scheduled_at` as its clock; then per owner with activity in the 24 h before `scheduled_at`: one `trigger_digest` notice + one email (counts by outcome, health+reason, budget breaches, links); idempotent per (owner, day); nothing on a quiet day.
- Web: Task detail shows the budget flag from the delivery (`trigger_budget_exceeded_usd`), e.g. "Over trigger budget: $0.42 of $0.01"; the bell renders both kinds readably.
## Acceptance (PRD, verbatim)
- **AC-11.3** A budget of $0.01 is exceeded by any real run. The Task finishes normally, and the trigger shows Degraded with the budget reason. → verify: on dev.
- **AC-12.1** Each of the four states can be produced on dev and shows its badge and reason. (Here: budget-Degraded.)
- **AC-13.3** The days rule fires after N days without success. → verify: an automated test with an injected clock. Days cannot be waited out in a dev check.
- **AC-15.1** On dev, the digest is sent to the owner's verified email and appears in the inbox bell, with correct counts for the previous 24 h. → verify: run the digest job on dev (or wait for it) and observe both.
Out of scope: Do NOT stop, kill or fail any runner for budget. Do NOT add a general notifications table. Do NOT change the processor pipeline order or other children's sources. PRD §5 (verbatim, binds): No deferred source in this epic. No skill attachment. No budget enforcement. No GitHub reads that decide filing. No public exposure beyond the two fenced paths. Cron behaviour is unchanged (do not touch cron health/alarms). No new licence for any persona other than advisor. Slack: no Events API. Secrets: no webhook secret, bearer token or credential in any digest, notice, email or log.
Done means:
- AC-11.3 — on dev, a trigger with budget $0.01 files a Task that runs to `done` normally; the trigger list/detail shows "Degraded" with "budget exceeded on Task #<n> ($x of $0.01)" as visible text; the Task detail shows the budget flag; a later Task at/under budget clears it.
- AC-13.3 — a unit test with an injected `scheduled_at` showing the idle rule fires at N days and not at N-1, and is off when unset.
- R13 notify — when a dev trigger goes Failed (e.g. 5 failing deliveries), the owner gets a bell notice and an email at that moment. Cite both.
- AC-15.1 — run the digest activity on dev (fire it from the console's activity run-now): the owner receives one email and one bell notice whose per-trigger counts match the deliveries log for the prior 24 h; a second run the same day sends nothing new; a quiet owner gets nothing.
- Filed Task ending `system_failed` → `consecutive_failures +1` exactly once (test).
- `./dc check full` green.
Outcome
Trigger health notices, advisory budget, idle auto-pause and daily digest merged (PR #4754, migration 0398) and deployed to dev. Full gate green apart from develop's Task 5399 price tests. Still to do: the dev checks for AC-11.3, R13 and AC-15.1, because dev can't be reached from this node.
Artifacts
Activity
- Draft
- Ready
- Runner started
- Ready
- Runner started
- Ready
- Runner started
- Awaiting response
- Awaiting response
- Runner resumed
- Runner ended (done)