Worker is not running
Edit this pageSystem · Related pages
Missing cleanup, scheduled production, or collector progress can mean a worker is suspended, unclaimed, or failing.
Inspect worker state
SELECT worker.id, worker.name, worker.system_id, worker.stream_id, worker.consumer_group_id, worker.target_instances, count(instance.id) FILTER (WHERE instance.expires_at > now()) AS live_instances, coalesce(max(instance.attempts) FILTER (WHERE instance.expires_at > now()), 0) AS attemptsFROM {schema}.worker_config AS workerLEFT JOIN {schema}.worker_instance AS instance ON instance.worker_id = worker.idWHERE worker.name = '{worker}'GROUP BY worker.idORDER BY worker.id;Use the warning or alert’s worker name. Match the returned owner ids to the affected resource.
| Result | Action |
|---|---|
target_instances is 0 |
The worker is suspended. Follow step 1 if it should run. |
Nonzero target, live_instances is 0 |
No instance currently holds a live claim. Follow step 2. |
Live instances with attempts above 0 |
The worker is running but failing. Follow step 3. |
Live instances with attempts equal to 0 |
A claim exists without a recorded failure streak. Check SQL0040 for late ticks, or the collector completion measurement for collector progress. |
| No rows | Check the worker name and installation. Run the resource’s existing registration code if its declaration is missing. |
Restore the worker
-
If stream maintenance was suspended, resume the intended worker.
sqlstreams stream janitor unsuspend '{stream}'Use
vacuumin place ofjanitorforstream_vacuum. Vacuum starts suspended by default, so an unused vacuum worker is not itself a fault.If the system’s
managerormetrics_collectorrow was suspended through an administrative database change, restore its normal one-instance target:UPDATE {schema}.worker_configSET target_instances = 1WHERE id = {worker_id}AND system_id IS NOT NULLAND name IN ('manager', 'metrics_collector')AND target_instances = 0RETURNING id, name, target_instances;No returned row means it did not match those conditions. Re-registration preserves suspension and does not substitute for this change.
-
If background workers have no live claims, run a manager.
sqlstreams manager runFor
message_consumerorexception_consumer, start the application’sConsumecall instead. The standalone manager cannot run application consumer handlers. A suspended parent manager must be resumed first. -
If the worker has failures, read its latest tick error. Match
workerandownerin either log:level=WARN msg="could not run worker tick -- backing off" worker={worker} owner={owner} attempts={attempts} delay={delay} error={error}level=ERROR msg="worker tick backoff curve exhausted -- ticks continue at its cap" code=SQL0036 worker={worker} owner={owner} attempts={attempts} delay={delay} error={error}Fix the nested database or configuration error. Ticks continue with backoff, so a process restart alone does not correct the cause.
Related topics
- Background managers and workers — How workers start, renew claims, and take over.
- Worker logs — Ownership, suspension, and tick failure fields.
- Control stream maintenance — Inspect, suspend, or resume a janitor or vacuum worker.