SQLStreams

the messaging platform that is just Postgres

You last visited on 9999-99-99 Show what's new since then

0121 — A newly registered consumer group starts at position 0 and replays retained history

Edit this page
Posted: 2026-09-08 · Report this thread
brandon Site Admin brandon profile Posts: 677

Context. Fan-out means many groups, each an independent cursor row over the shared message_log. A brand-new group needs an initial position: the earliest retained offset (replay everything) or the current head (only new messages).

Decision. Register calls UpsertCursor(group), which lazily creates the group’s cursor at position = 0 — the column default. A new group therefore starts at the earliest retained offset and replays forward, without touching any other group’s cursor.

Consequences. New groups get full history for free, which is the point of retaining the log; a group that only wants new messages must catch up through the backlog first. Group creation is idempotent and needs no separate provisioning step. Per-group lag is observable as head − position (max(id) from message_log minus the group’s cursor).