Fix migration 202610290030 control plane rollout
Every Jaah Deploy to dev since Task 11131 merged fails at `apply (dev)`: "the control-plane rollout failed and ECS rolled back to jaah-dev-cp:1406". Runs 40799965317 (337c84445) and 40800827524 (41e1bb737), 2026-10-29.
The new api container never listens on :8000; its log ends at
`Running upgrade 202610282358 -> 202610290030, Same-org keys on every project and credential reference` (log group /ecs/jaah-dev-cp, stream api/api/b87a517808134dab99fd15b1732efe0f). The web sidecar answers /api/health with 502, ECS kills the task after its ELB health checks fail, and the rollout gives up. The likely cause is that adding 26 foreign keys (`apps/api/migrations/versions/202610290030_project_credential_same_org_keys.py`) takes locks the still-serving previous release holds, or validates too slowly to finish inside the health-check window. Confirm from the logs and DB before fixing.
Make the migration finish within the rollout while the previous release serves: for example `ADD CONSTRAINT … NOT VALID` then `VALIDATE CONSTRAINT` (or a deferred validation step), a `lock_timeout` with retry, or a split into smaller revisions. Then confirm a develop deploy goes green.
Out of scope: changing which references get same-org keys (Task 11131's design); the deploy workflow's health-check timings.
Done means:
- A Jaah Deploy run on develop completes `apply (dev)` with success, and its api log shows revision 202610290030 applied.
- The migration's tests still pass; `./dc check full` green.
Outcome
No change needed: b9e4f1cb0 (Task 11184) had already fixed the 202610290030 rollout. Develop deploy 40808129669 and every run after it are green. Dev DB is at 202610290030 with all keys valid, and the migration's 4 tests pass. Memory: used mem-30558.
Artifacts
Activity
- Created
- Run
- Runner started
- Runner ended (done)