SQLStreams

the messaging platform that is just Postgres

You last visited on 9999-99-99 Show what's new since then

The Shape of the API

Edit this page
Posted: 2026-09-12 · Report this thread
brandon Site Admin brandon profile Posts: 677

One client wraps the pool and holds the ambient config. Every resource is named from it as a handle, client.Stream[OrderPlaced]("orders"), client.Stream[OrderPlaced]("orders").Consumer("billing"), and the verbs live on the thing they act on. Every type a program spells lives in github.com/agentstax/sqlstreams/client; the per-handle lookup tables are the Reference board.

Two grammars

The client selects the noun: Stream[T](name), Scheduler(name), System(), Manager(). A stream handle selects its own: Consumer(name), Producer(), Key(messageKey). A handle uses the bare verb, orders.Register, orders.Destroy, nightly.Suspend: it already knows what it is, and orders.DestroyStream says stream twice. A plural method reads the materialized collection: Streams, Schedulers, Consumers. The act verbs on typed instances, Consume, Produce, Schedule, follow the same split.

A handle does no I/O and cannot fail. It is a name plus the client, and every verb resolves the name when called. Get is the snapshot read and the only comma-ok verb, (nil, nil) for an absent resource; every other verb returns the not-found error itself. Destroy is one statement whose zero-rows-affected is the existence check, and a Get before it would only add a window in which the answer can change. kubectl delete, sqs.DeleteQueue, and JetStream’s stream.DeleteConsumer(ctx, name) are the same shape. JetStream’s js.Stream(ctx, name) is the other design: it fetches the row at acquisition and caches it on the handle, which is why it then needs both CachedInfo() and Info(ctx). The no-I/O one was picked so a handle can never hold a row that has gone stale and a one-shot CLI verb is one round trip.

The type argument

The stream carries the message type and everything under it inherits it. client.Stream[OrderPlaced]("orders") is spelled once, and the group’s Register, the producer’s Register, and the key reads take no type of their own. Go cannot infer a type from a string, so the argument is always written, and a wrong type on a consumer becomes impossible to spell: the handler’s parameter type is the stream’s.

The admin verbs never look at it. Register, Destroy, Migrate, Rename, Health, and Metrics run the same for any type, so a stream holding two schema versions is two handles on the same rows: Stream[OrderPlacedV1] reads V1 payloads, Stream[OrderPlacedV2] reads V2, and either one destroys the stream.

A caller that does not know the type, the CLI and any admin-only script, uses sqlstreams.RawPayload: the stored JSON bytes as they are, declaring version 0. It works on every admin and key read. A consumer cannot register with it and a produce with it is refused, so no row is ever stored or read at version 0.

// an operator script -- no message type in scope
orders := client.Stream[sqlstreams.RawPayload]("orders")
health, err := orders.Health(ctx)
head, err := orders.Key("order-42").CompactionHead(ctx) // head.Message is the raw JSON

The schedule is the one handle outside the tree. Its Register takes the payload value, and Go infers the type from that argument.

What a declaration does

Newest wins, every time. Register on a stream, a consumer group, a schedule, or the system writes the declaration it was handed; a differing one overwrites the stored document and logs the change as old -> new (SQL0061 for a stream, SQL0059 for a group, SQL0062 for a schedule), and an identical one writes nothing. A declaration is the whole document: fields left unset mean the library default, there is no field-level patch, and nil declares the empty document. Start and Bindings keep their own rules: read once at creation, and install / wait.

That is right for a rolling deploy and wrong for two services that both think they own charge-cards: they overwrite each other forever, and the tell is the same SQL0059 line on every restart of either one. One declaring service per resource is a convention, not a constraint.

The act verbs also run more than they say. Consume and SchedulerInstance.Schedule run the system manager beside the session, so any one of them keeps every worker in the deployment running (Manager). A user who has learned Register then Consume gets Register then Produce and Register then Schedule with nothing new, and none of the three quietly does more than the others.

Names

The suffix says what a value does. A handle is the lazy identity plus client (StreamHandle, ConsumerHandle, KeyHandle, SchedulerHandle); the bare noun is the materialized value returned by Get (Stream, Consumer, Schedule). StoredMessage[T], Alert, and Measurement follow the same rule: values, not handles. The rest of the grammar follows the value’s role: Config for declarations, Options and Item for command inputs, Result for command outputs, Snapshot for point-in-time observability, Status for current state, Summary for an aggregate projection, Health for a verdict, Instance for running process state. A projection includes its subject when the bare role would be vague: ScheduleConsumerGroupSummary, StreamVersionHealth. Data and Info are not roles, so neither is a public suffix.

The supported API

client/ (package sqlstreams) is the supported public entry point: its exported names and every exported field and method reachable through its types, aliases, parameters, and results. Before v1 this surface still changes as the API settles. Other packages remain importable for advanced use, without a stability commitment; sqlstreams.RetryPolicy aliases a type declared in pkg/common, its methods are part of the supported surface, and calling common.NewDefaultRetryPolicy through a direct import is advanced usage. Third-party types such as pgxpool.Pool keep their upstream contracts.

sqlstreams is a large package: every config, options, error, and read-model a user can name. That is the point of it, and also the cost.

Testing your code

The library returns concrete ProducerInstance[T], ConsumerInstance[T], and SchedulerInstance[T] values. A package that needs a fake declares the small interface it consumes beside its own code; SQLStreams does not ship a second, partial definition of those instances. The client stays concrete for integration tests.