Fix ready backlog recovery fairness #201

Merged

backmeupplz merged 1 commit from codex/VOI-71-ready-recovery-fairness into main

2026-05-28 22:12:22 +00:00

backmeupplz commented

2026-05-28 21:56:27 +00:00

(Migrated from github.com)

Summary

alternate recovered claim-ready scheduling between oldest and newest buckets
queue recovered ready jobs into the same old/new scheduler buckets so fresh ready work can run between stale backlog jobs
add a worker-client regression proof for already-ready backlog recovery and update worker docs

Validation

yarn build-ts
yarn test:worker-client
yarn lint

Manual QA: required after deploy. Send a fresh short Telegram voice while ready backlog remains non-empty and confirm worker logs show recovered ready claimBucket/schedulingBucket alternation and prompt completion.

## Summary - alternate recovered `claim-ready` scheduling between oldest and newest buckets - queue recovered ready jobs into the same old/new scheduler buckets so fresh ready work can run between stale backlog jobs - add a worker-client regression proof for already-ready backlog recovery and update worker docs ## Validation - `yarn build-ts` - `yarn test:worker-client` - `yarn lint` Manual QA: required after deploy. Send a fresh short Telegram voice while ready backlog remains non-empty and confirm worker logs show recovered ready `claimBucket`/`schedulingBucket` alternation and prompt completion.

backmeupplz commented

2026-05-28 22:41:36 +00:00

(Migrated from github.com)

Post-merge production QA found one remaining VOI-71 acceptance gap after deploying #201.

What passed:

Local gates on current main c6cadbc: yarn build-ts, yarn test:worker-client, yarn lint, and yarn test:worker-api against temporary Mongo.
Hetzner backend container was rebuilt at 2026-05-28T22:14Z.
Windows worker was updated to c6cadbc, rebuilt, and restarted.

What failed:

I uploaded a fresh short @voicybot voice/audio fixture at 2026-05-28T22:37:37Z while production backlog was non-empty.
Mongo job 6a18c3b17ef284c99461bbe2 was created as queued_for_download (7,179 bytes, sourceMessageId 556970593) but stayed queued for several minutes.
At the same time the worker was transcribing old 535 MB backlog job 6a03b36bd6d99f6f4ad9ec66 with transcriptionConcurrency=1; DB snapshot still had ready=54, queued_for_download=13, downloading=6, transcribing=2.

So the oldest/newest fairness work helps, but a huge stale backlog item can still monopolize transcription and block a fresh short job. I moved VOI-71 back to in-progress with the production evidence and required rework.

Post-merge production QA found one remaining VOI-71 acceptance gap after deploying #201. What passed: - Local gates on current main `c6cadbc`: `yarn build-ts`, `yarn test:worker-client`, `yarn lint`, and `yarn test:worker-api` against temporary Mongo. - Hetzner backend container was rebuilt at 2026-05-28T22:14Z. - Windows worker was updated to `c6cadbc`, rebuilt, and restarted. What failed: - I uploaded a fresh short @voicybot voice/audio fixture at 2026-05-28T22:37:37Z while production backlog was non-empty. - Mongo job `6a18c3b17ef284c99461bbe2` was created as `queued_for_download` (7,179 bytes, sourceMessageId `556970593`) but stayed queued for several minutes. - At the same time the worker was transcribing old 535 MB backlog job `6a03b36bd6d99f6f4ad9ec66` with `transcriptionConcurrency=1`; DB snapshot still had `ready=54`, `queued_for_download=13`, `downloading=6`, `transcribing=2`. So the oldest/newest fairness work helps, but a huge stale backlog item can still monopolize transcription and block a fresh short job. I moved VOI-71 back to `in-progress` with the production evidence and required rework.