0638 — The manager row is gated and Run is shared

Context. Planning [0635] exposed that its shape kept two models of the manager alive at once: SystemManager.Run as a call one caller owns (a permit, a refusal error, errors returned) and auto-run as a shared service of the client (N users, first starts, last stops). Every seam problem traced to that split. The auto path would have had to recognize the “system manager already running” refusal — a plain errors.New in an API package, which declares no named error variables, and branching on message text is banned — so the client would have needed its own bookkeeping anyway. Schedule.Schedule’s contract under sharing was unanswerable. And 0635 accepted idle reconcile polling that scales with the consumer fleet. The deeper fact: the manager is the one worker exempt from the system’s own claim gate. target_instances already means -1 no cap, 0 suspended, N at most N live instances — for every worker (the claim SQL’s count of unexpired rows, metrics.WorkerSuspended // target_instances = 0, VK0035’s diagnose query) — and the run loop’s “manager row suspended” label on every declined claim is an inference that holds only while manager rows are always -1. renewInstance requires expires_at > now(), so a lost claim cannot renew past the gate; a gated manager re-claim just gets declined. Re-declaration updates only metadata, never target_instances, so an operator’s edit survives restarts.

Decision. Two parts, built together. (1) SystemManager.Run(ctx) becomes join-and-block: a mutex-guarded count of active callers; the first join resolves the system owner and starts the reconcile loop under an internal context; every caller blocks on its own ctx and returns nil when it cancels; the last one out cancels the loop and waits for it to finish, so the instance row is released, not left to expire. The permit and its refusal are deleted — nothing is ever refused. RunManager, Schedule.Schedule, and Consume are all this same call; Consume reaches it through consumer.ConsumerConfig.RunSystemManager func(ctx) error, filled by the client unless ClientConfig.DisableManager is set (nil is the fact the start line reports as disable_manager). Errors before the loop starts return to the join that starts it; a fatal loop exit has no caller and logs the declared Error event VK0065, with no automatic restart — a spawned worker returning fatal would replay — and the next join starts a fresh loop. (2) The system manager’s worker row is declared target_instances = 1 by admin’s declarer provisioner: one process in the deployment runs the reconcile loop, every other process’s loop retries the claim at RetryDelay (30s, jittered) and takes over at TTL expiry. The loop branches a declined claim on the row’s target — 0 warns VK0035 as today, positive logs Debug and keeps waiting — which requires WorkerData to carry TargetInstances. Group manager rows and consumer rows stay -1: each replica of a group must run its own reconcile loop to spawn its own share of consumers.

Consequences. 0635’s accepted polling risk is gone — a waiting process costs one declined insert per interval — and the column becomes the deployment’s runtime dial with no code: 0 suspends upkeep everywhere, 1 is the default, N runs N copies, -1 restores every-process behavior. RunManager narrows: errors after the loop claims are logged, not returned (matching the loop’s own “degraded upkeep never stops the host process” stance), and two explicit runs in one process now share instead of refusing. Existing rows keep their -1 until drop+recreate (pre-v1; the target is an INSERT value, not DDL). Supersedes [0635]‘s clause “the in-process permit still refuses a second explicit RunManager” and its idle-polling accepted risk; the rest of 0635 stands. Rejected: a status column for suspend — 0 already encodes it and metrics already derives it; revisit only if a worker suspend/unsuspend CLI verb pair needs the old cap restored in one step; recognizing the refusal error in the auto path — no named variable to branch on, and the permit was guarding waste, not correctness (N-way was always safe across processes); counting in the client rather than the SystemManager — 0635’s “the client counts” wording was intent, and the count belongs beside the loop it starts and stops.

Amended. [0640] supersedes the clause “with no automatic restart — a spawned worker returning fatal would replay — and the next join starts a fresh loop”: the shared loop re-claims behind a backoff. The rest of this record stands, including that no caller receives the fatal. [0641] supersedes the join-and-block clause — the mutex-guarded caller count and shared loop are deleted; Run is a per-caller loop and the row is the only arbiter. [0642] supersedes the consumer.ConsumerConfig.RunSystemManager clause — the client’s own ConsumerInstance wrapper runs the manager beside Consume, and pkg/consumer stays untouched.