Reduce control plane polling from idle nodes

Task 13921 · Running · Coder · created · updated

From the lag analysis in Task 13902 (report attached, item 2). In 24 h on prod, nodes made about 1.9 M requests against about 0.1 M from the console: - pool-tick: 611 k (every 5 s per node; `DEFAULT_IDLE_INTERVAL` in apps/jaahd/main.py) - fleet/observe: 559 k - memory/bundle: 230 k, about 2 GB of responses - accounts: 109 k - worktrees: 56 k - sessions/work: 54 k - gha-runners desired/report: 104 k - guard/bundle: 45 k All of it competes with console reads on one CPU-bound process. Requirements: 1. Back off idle nodes. Use a longer interval when nothing changed, and snap back to fast polling on activity or a bus nudge. 2. Answer unchanged bundles and lists (memory/bundle, guard/bundle, accounts, worktrees) with 304 or an empty delta keyed by ETag or `since`. 3. Where the node bus already exists, push changes instead of polling. Out of scope: autoscaling, relay and client changes. Done means: - Agent request volume on dev drops by at least 50% for an idle fleet. Show it with a before/after Logs Insights count. - Dispatch latency is unchanged. A queued task still starts within the current time on dev.

Attachments

Activity

  • Created
  • Updated
  • Run
  • Runner started
  • Awaiting response
  • Awaiting response
  • Runner resumed