Fix cron follow-up pass: DBOS.trigger schedule
A cron fire that is dropped while a pass is running is supposed to earn one immediate follow-up pass (`apps/api/app/cron_runtime.py::_follow_up_after_release`, line ~303). That follow-up never runs: `DBOS.trigger_schedule` raises "cannot be called from within a workflow", because `_release_then_follow_up` is called from inside the cron tick workflow (`cron_runtime.py:376` in a done-callback, `:408` in a finally).
Evidence from CloudWatch over the last 24h:
- dev: about 800 warnings a day.
- main: about 1,700 warnings a day.
- Mostly from the `lease-drain` and `task-pipeline` crons.
- The log line is: `WARNING cockpit-cron: cron <id>: the follow-up for a dropped fire did not enqueue (DBOS Error: DBOS.trigger_schedule cannot be called from within a workflow)`.
Effect: when a task changes state while a pass is running, the early wake is lost and the task waits for the next interval (up to 10s for lease-drain, 15s for task-pipeline). The comment in `routes/fleet.py::build_dispatch_plan` added by Task 5579 already assumes the follow-up works.
1. Make the follow-up enqueue from outside the workflow context, so `trigger_schedule` is accepted. Examples: hand it to a plain thread or the serving loop after the workflow returns, or use the DBOS-sanctioned in-workflow equivalent if one exists in dbos 3.2.0. Keep "at most one follow-up per pass" and "never raises".
2. Add a test that drives a dropped fire through the real wrapper and asserts that a follow-up tick is enqueued. It must fail on today's code.
Out of scope: changing cron cadences, the coalesce window, or the lease-drain/dispatch logic.
Done means:
- The new test fails before the fix and passes after it.
- `./dc check full` is green.
- After deploy to dev, CloudWatch `/ecs/jaah-dev-cp` shows zero "follow-up for a dropped fire did not enqueue" warnings over 30 minutes. Run `filter @message like /follow-up for a dropped fire did not enqueue/ | stats count(*)`.
- A follow-up actually runs: a debug or info line, or a cron tick within seconds of a dropped fire, is visible in the same log group.
Outcome
Follow-up trigger now runs in an empty contextvars.Context so DBOS accepts it; new test red→green; full gate green; PR #4893 merged + Jaah Deploy OK. Dev: 0 warnings over 17:26–17:59 UTC, follow-up ticks seen after dropped fires. Main pending promotion.
Artifacts
Activity
- Created
- Run
- Runner started
- Runner ended (done)