Cron alarms file into project "cto", which exists

Task 6270 · Done · Coder · created · updated

Found by the Finish phase of epic 6213 (SOC 2 Type II in-house evidence, R2: "a check that fails or drifts raises the existing alarm/paging path — never silent"). ## Defect Every cron alarm — the 13 compliance-evidence collectors (`compliance-evidence:<id>`), wedged crons, the reconciler, node liveness — files a hidden `adhoc` Task into the project NAMED `cto`: `apps/api/app/node_liveness.py:61` `ALARM_TASK_PROJECT = "cto"`, used by `apps/api/app/cron_alarms.py:80,110` and `node_liveness.py:642,694,736`. The name resolves through `project_identity.legacy_project_id` (`apps/api/app/project_identity.py:38-47`) in the operator org (`routes/agent.py::_wire_project`, line 455). The project was renamed: the operator Directory on BOTH control planes lists a `jaah` project and no `cto` project. ## Reproduction (observed 2026-10-14) - Prod `app.jaah.ai/projects` (operator, All orgs): projects df, finance, hcd, hj, Invoicing, jaah, Mobile app, n8n, oam, oam-demo, Reports, sales, timesaver, timesaver2, TimeSaver app — no `cto`. - `task_search persona=adhoc` in the jaah project: 0 rows ever. - Dev (Task 6225 report): the drift/error evidence runs called `raise_alarm`, no Task was filed; locally the same call logs `404: project 'cto' not found`. Dev's backups/securityhub/paging/cloudwatch collectors drift daily and nobody is told. - So the drift leg is SILENT on both environments: `raise_alarm` swallows the 404 into one log warning (`cron_alarms.py:96-98`) and `resolve_alarm` treats 404 as the quiet case (`cron_alarms.py:122-123`). ## Fix Stop resolving the alarm project by a hard-coded name. Add one operator setting "Alarm project" (a project id picker) in `app/operator_settings.py`, modelled on `compliance_github_credential_id`; `cron_alarms.raise_alarm`/`resolve_alarm` and `node_liveness`'s alarm calls pass `project_id` from it. While it is unset or names no row, (a) the setting shows that state in Settings, and (b) `raise_alarm` logs at ERROR naming the setting (not a quiet warning). Remove the `"cto"` constant. Add a readiness-checklist owner action to set it on prod (pointer beside I-15), since the alarm path is SOC 2 CC7.2/CC7.3 evidence. Out of scope: do NOT change alarm persona/state/dedup semantics, the collectors, their schedules, or paging (SNS). Do NOT create or rename projects on dev/prod by any route except the console. Do NOT touch `seed_dev_profile.py`'s local `cto` fixture beyond what tests need. Done means: - With the setting pointed at a project on dev, `Run now` of a drifting collector (e.g. `backups` or `securityhub`) opens exactly one hidden alarm Task in that project, and a second run refreshes it rather than adding one → verify on dev console: Task visible, screenshot. - With the setting unset, the run's ERROR log names "Alarm project" and Settings shows it unset → verify: unit test + dev screenshot. - `grep -rn '"cto"' apps/api/app/node_liveness.py apps/api/app/cron_alarms.py` → 0 hits. - Existing cron_alarms / node_liveness tests pass (updated only for the id path); `./dc check full` green; Jaah Deploy succeeds on dev.

Outcome

Fixed the Alarm project save, which 422'd on dev for any project outside org zero. PR #5465 merged; full gate green; dev deploy OK. On dev a drifting backups run filed one hidden alarm, #5009 in jaah, and a re-run refreshed it. Deferred: owner sets it on prod (I-16); pickers truncate.

Pull request

Artifacts

Activity

  • Ready
  • Runner started
  • Ready
  • Run
  • Runner started
  • Ready
  • Runner started
  • Ready
  • Runner started
  • Runner ended (done)