A queue can deliver the same message twice. How do you make processing safe?
System designMid-level
In shortAcknowledge only after the work succeeds, and make the consumer idempotent, usually by recording each message id with a unique constraint in the same transaction as the side effect, so a duplicate is detected and skipped.
Model answer
Queues are effectively at least once: a consumer can finish the work and crash before acknowledging, a Kafka rebalance can resume from an older offset, or a retry can race with a slow first attempt. So the consumer must make a second delivery harmless:
- Ack after the work, never before (acking first loses messages).
- Record processed ids: insert the message id (or a business key like
paymentId) into aprocessed_messagestable with a unique constraint, in the same transaction as the side effect. A duplicate fails the insert or finds the row and does nothing. - Prefer naturally idempotent operations where possible: “set status to paid” or an upsert by key instead of “add 10 EUR”.
- Publish reliably: the producer writes events to an outbox table in the same transaction as its own change, and a relay publishes them, so events are never lost and duplicates are handled by the idempotent consumer.
Kafka’s exactly-once semantics cover Kafka-to-Kafka processing; side effects in your own database still need this.
What they're looking for
Why duplicates happen (crash before ack, rebalances, retries), an idempotency mechanism tied to the same transaction, and the outbox pattern for publishing.
Common mistakes
- Claiming the broker guarantees exactly-once delivery to your database.
- Checking for duplicates in a separate step from the side effect.
- Acknowledging before processing.