SQLStreams vs. RabbitMQ & SQS
Edit this page
RabbitMQ and SQS are the workhorses of the queue world, and SQLStreams borrows their best ideas shamelessly: per-message retries, dead-lettering, leases (SQS calls its version a visibility timeout), routing. The disagreement is about one default: what happens to a message after it’s consumed.
They delete it. SQLStreams keeps it. Almost every difference below falls out of that.
Feature by feature
Scored on shipped behavior; proposed capabilities are labeled, never checkmarked.
| RabbitMQ | SQS | SQLStreams | |
|---|---|---|---|
| Retry / dead-letter per message | yes | yes | yes, per consumer group |
| Delayed delivery | plugin | yes (15-min cap) | no — retry backoff, or a handler-requested delay |
| Routing | yes — exchanges (the gold standard) | no (SNS bolt-on) | yes — routing-key patterns |
| FIFO | per-queue | FIFO queues (with a throughput cap) | per-key order (opt-in per message, no throughput cap on unordered keys) |
| Read history again | no — consumed is gone | no — consumed is gone | yes — new groups read retained history |
| Fan-out to N consumers | exchange→N queues, declared in advance | SNS→N queues, declared in advance | any group, anytime — including one added after the fact |
| Message visibility | opaque broker state | opaque, 14-day max | rows; retention you choose |
| Transactional produce | no | no | yes |
| Infrastructure | Erlang cluster to run | none (but AWS-only) | the Postgres you already run |
The retroactive fan-out problem
Both systems can fan out — if you declared the topology before the messages flowed. An exchange with three bound queues delivers to three consumers; the fourth queue you bind next quarter receives only what’s published after it exists. The past is gone; it was deleted on consume.
This sounds like an edge case until you live it: every new service that “wishes it had been listening from the start” becomes a custom backfill project. In SQLStreams, you register a new consumer group and it reads the retained history from the beginning — that’s just what a new group does, because nothing was deleted.
Debugging: black box vs. glass box
When a message goes missing in RabbitMQ, the investigation involves the
management UI, rabbitmqctl, and educated guessing about which queue,
which exchange, which binding. In SQS, you have CloudWatch metrics and
a console that shows you counts but fights you on contents.
In SQLStreams, the message is a row and the question “what happened to
order ord_8431, everywhere?” is a query (tables are named by stream id —
1 here):
SELECT g.name AS consumer_group, d.status, d.attempts, d.last_error
FROM sqlstreams.message_log_1 m
JOIN sqlstreams.exception_queue_1 d ON d.message_id = m.id
JOIN sqlstreams.consumer_group_config g ON g.id = d.consumer_group_id
WHERE m.payload->>'order_id' = 'ord_8431';One reading note that shows the design: this query returns only trouble — groups where the message failed, is retrying, or died. A group that processed it cleanly wrote no row at all; its committed cursor past the message’s id is the receipt.
When they’re still the right call
- SQS when you’re all-in on AWS and want Lambda event-source mappings and IAM-native everything with zero libraries — and you don’t need history, routing, or transactions.
- RabbitMQ when you need its protocol reach (AMQP/MQTT/STOMP clients in every language since 2009) or fire-and-forget delivery where the broker’s in-memory path is the point.
- Either, when your messaging volume dwarfs your database capacity and you want the isolation of a separate system.
For the standard case — a product whose background work, messages, and routing currently sprawl across one of these plus schedules plus an outbox — one Postgres-native platform is less to run, more to query, and strictly more capable.