Side Effects & Retries

Transactional produce makes your statements and the message one COMMIT — both land or neither does. Two things stay outside that guarantee, and both are yours to handle: side effects, which are not statements and cannot roll back, and rerunning the closure, which only you know is safe.

Side effects don’t roll back

Don’t call sendEmailConfirmation() before the transaction is known to commit. If a later step fails and rolls back, the database will undo its half but that email can’t be ‘unsent’. The transaction covers your statements and the produced message; a plain function call inside the closure runs the moment it’s reached, and no ROLLBACK reaches it.

Here’s the multi-stream closure with an email in the wrong place. Say orderId is 4127 and the customer is [email protected]:

// ❌ the email fires whether or not the commit happens
err := client.InTransaction(ctx, func(ctx context.Context, tx sqlstreams.Tx) error {
	if _, err := tx.Exec(ctx, `UPDATE orders SET status = 'paid' WHERE id = $1`, orderId); err != nil {
		return err
	}

	sendEmailConfirmation("[email protected]") // fires NOW, mid-transaction

	_, err := receipts.ProduceInTx(ctx, tx, &ReceiptRequested{OrderId: orderId}, nil)
	return err
})

Walk the failure: the produce returns an error (a dropped connection is enough) -> InTransaction rolls back -> order 4127 is unpaid again and no receipt message exists -> Jamie is holding a “payment received” email for a payment the database says never happened.

Moving the call below InTransaction fixes that, with one catch: it must sit behind the nil check. An error return doesn’t confirm the rollback either — an ambiguous commit failure surfaces as-is (see retries) — so on error the only safe move is still no email.

err := client.InTransaction(ctx, func(ctx context.Context, tx sqlstreams.Tx) error {
	// statements and produces only -- everything in here can roll back
	_, err := tx.Exec(ctx, `UPDATE orders SET status = 'paid' WHERE id = $1`, orderId)
	return err
})
if err != nil {
	return err // maybe committed, maybe not -- still no email
}
sendEmailConfirmation("[email protected]") // the commit is known here

Now walk this failure: the process crashes after the commit, before the send. The payment is recorded, but the email never goes out. This is the dual-write failure window: database first, side effect second. Make the email a message:

_, err := emails.ProduceInTx(ctx, tx,
	&EmailRequested{To: "[email protected]", Template: "payment-received"}, nil)

An email-sender consumer group calls sendEmailConfirmation() in its consumerFunc — the relay shape from the produce guide’s external-systems note, pointed at your own mail provider. The request to send now commits atomically with the payment, a flaky provider gets retries, and a send that keeps failing is dead-lettered instead of vanishing with a crashed process.

Retries are yours to own

InTransaction never retries your closure — a transient blip or an ambiguous commit failure surfaces to you as-is, because only you know what’s safe to rerun. A retry must make both the business statements and the produced message safe to repeat. A stable ProduceOptions.IdempotencyKey deduplicates the message within its TTL; it does not undo or deduplicate other statements in your closure. Use a business operation id or an equivalent database constraint for those writes. Omitting IdempotencyKey mints a fresh key per call, so a naive rerun can produce a second message.

// any stable string works as the key -- here the order id already is one
produced, err := instance.Produce(ctx, &OrderCreated{OrderId: order.Id},
	&sqlstreams.ProduceOptions{IdempotencyKey: "order-created-" + order.Id})
if err != nil {
	return err
}
if produced.Duplicate {
	fmt.Println("an earlier attempt already landed -- nothing produced twice")
}

Duplicate == true means an earlier call — or an earlier attempt of this one, after a commit whose outcome you never learned — already produced under the same key. The key is an opaque string: one that parses as a UUID is stored verbatim, anything else is hashed to a deterministic UUID first, so the same string always lands on the same claim row. If you mint keys yourself on a hot path, prefer UUIDv7 strings — random-shaped keys cost extra WAL once the claim table is millions of rows deep. A caller-supplied key also locks that key for the length of the transaction it runs in, so keep transactions that reuse keys short. A caller-supplied key also takes the call out of batching: it runs in its own transaction, so a producer that keys every message has left the batch path under load.

The claim lives for StreamConfig.IdempotencyKeyTTL, 24 hours by default, then the stream janitor sweeps it. A retry that arrives after that window is a new message, not a duplicate, and nothing logs it. Size the TTL to your upstream’s retry horizon: a webhook provider that retries for three days needs a three-day TTL on that stream.

The key is an identifier and is logged as one: a duplicate publish narrates idempotency_key at Debug, and a produce failure line carries that narration. Build it from ids, not from the document’s contents.