SQLStreams

the messaging platform that is just Postgres

You last visited on 9999-99-99 Show what's new since then
Posted: 2026-09-12 · Report this thread
brandon Site Admin brandon profile Posts: 677

client.Stream[T](name) names a stream under the payload type T. No I/O, no failure: the handle is the name plus the client, and every verb resolves the name when called. T is spelled once here and every handle under the stream inherits it (the type argument).

orders := client.Stream[OrderPlaced]("orders.placed")
stream, err := orders.Get(ctx)
if err != nil {
	return err
}
if stream == nil {
	stream, err = orders.Register(ctx, &sqlstreams.StreamConfig{RetentionTTL: 30 * 24 * time.Hour})
	if err != nil {
		return err
	}
}

Verbs

verbreturnsnotes
Register(ctx, cfg)*Streamdeclares the stream and creates its tables on first registration; idempotent; nil cfg is the defaults; a differing redeclaration logs SQL0061
Get(ctx)*Streamthe comma-ok read: (nil, nil) when not registered
Migrate(ctx, targetVersion)errormoves the stream’s tables to a version (migrations)
MigrationVersion(ctx)int64the version the tables are at; ErrStreamNotFound
Rename(ctx, newName)*Streamthis handle keeps its old name; registered instances keep working through the stream id; ErrStreamNotFound, ErrStreamNameTaken
Destroy(ctx, options)errordrops the stream, its messages, and every consumer group on it; ErrDestroyDisabled unless ClientConfig.AllowDestroy
Health(ctx)[]*StreamVersionHealtheach payload version’s retirement verdict, read live (schema versions); ErrStreamNotFound
CompactionHeads(ctx)[]*StoredMessage[T]every key’s current compaction head, ordered by message key, unpaged
Consumers(ctx)[]*Consumerevery consumer group on the stream, ordered by name
Consumer(name)*ConsumerHandle[T]no I/O; Consumer
Producer()*ProducerHandle[T]no I/O; Producer
Key(messageKey)*KeyHandle[T]no I/O; Message key
Metrics()*StreamMetricsHandleno I/O; Metrics
Janitor()*MaintenanceHandleno I/O; retention cleanup; Maintenance
Vacuum()*MaintenanceHandleno I/O; scheduled key-table vacuum; Maintenance
Alerts()*StreamAlertsHandleno I/O; Alerts

Get is the only comma-ok verb. Every other verb returns the not-found error itself: Destroy is one statement whose zero-rows-affected is the existence check, and a Get before it would only add a window in which the answer can change.

CLI

sqlstreams stream get orders.created reads the stream’s registration and config. sqlstreams stream health orders.created reads each payload version’s retirement verdict. Both accept --output json; health returns an array of version verdicts. Get returns the stream object directly, with duration strings such as "1h0m0s", or null with exit 1 when the stream is absent.

Use stream get --quiet orders.created for an existence check: no output, exit 0 when registered, exit 1 when absent. --quiet and --output json cannot be combined.

sqlstreams stream config get orders.created retention_ttl adds a CLI view of Stream.Get: one config key’s current value beside its default. Omit the key to show all supported keys. This view has its own JSON document with stream, stream_id, and a keys array; stream get returns the resource.

Config

StreamConfig

fielddefaultwhat it decides
PartitionSize1_000_000rows per partition; lower for finer retention drops, higher for high-throughput ingest
RetentionTTL0 (keep forever)how long a message survives before the janitor may drop it
AllowDropPastCommittedfalsewhether retention may drop rows a lagging group has not committed
IdempotencyKeyTTL24hhow long a produce-retry claim survives; zero means the default, not forever
EmptyCompactionHeadTTL1hhow long a compaction-head row with no head may idle before the janitor sweeps it; zero means the default
DeliveryLogModeDeliveryLogModeFailureswhich outcomes write to the per-attempt delivery_log: DeliveryLogModeOff, DeliveryLogModeFailures, DeliveryLogModeAll
JanitorJanitorConfig defaultscleanup timing and limits; a new stream’s janitor starts active
VacuumVacuumConfig defaultsvacuum timing and timeout; a new stream’s vacuum starts suspended

JanitorConfig

fielddefaultwhat it decides
PollRate5sdelay between cleanup passes
SweepBatchSize1000maximum rows deleted per transaction
CleanupTimeout5stime allowed for each cleanup operation, including retries
PartialSweepGracePeriod0extra retention before partial message sweeps; whole-partition drops keep the normal TTL

VacuumConfig

fielddefaultwhat it decides
PollRate2mdelay after each completed request, with scheduling jitter
VacuumTimeout1mtime allowed for one request, including queries and retries

DestroyOptions

fielddefaultwhat it decides
Forcefalseskips the in-use guard; without it a stream still holding messages returns ErrStreamNotEmpty

Gotchas

  • Every verb but the handle selectors ignores T. A stream holding two schema versions is two handles on the same rows: Stream[OrderPlacedV1] reads V1 payloads, Stream[OrderPlacedV2] reads V2, and either one destroys the stream. An operator script with no payload type uses sqlstreams.RawPayload.
  • Names starting with __system. are SQLStreams’s own streams: Register returns ErrReservedStreamName. Registering a consumer group on one of them is supported.
  • PartitionSize cannot change after registration: a redeclaration with a different value returns ErrStreamConfigMismatch.
  • Maintenance settings apply to newly claimed instances. Registration preserves suspension and instance targets. Use the maintenance handles to change whether maintenance runs.