Key takeaway
Repeated delivery and a second business event are different cases. Prove which one your receiving system is handling.
Define the effect that must happen once
A transport can retry after an ambiguous acknowledgement. Kafka distinguishes producer delivery from consumer processing: a crash after processing but before checkpointing can cause the consumer to see the same message again. Its design also explains why an external destination needs coordination with the consumer position. An idempotent producer option alone does not establish one invoice adjustment, stock movement or API action in a receiving system.
For a data evaluation, choose the business effect under test. Is the recipient storing a source event, updating a current state or adding an amount to a total? Replacing a row with identical contents and adding the same amount twice have different consequences. Describe the expected final state before selecting a deduplication mechanism.
Use a receipt that follows business identity
This hypothetical acceptance test uses source event E-41 for a movement of five units. The receiving ledger starts at zero. The same source, business identifier and version constitute one event; a correction carries an explicitly different version. No actual seller feed is represented.
| Input sequence | Expected ledger | Required evidence |
|---|---|---|
| E-41 v1, then identical E-41 v1 | 5 units | One accepted effect; second delivery recorded as replay |
| E-41 v1, then conflicting E-41 v1 | 5 units pending review | Collision held; changed payload not silently ignored |
| E-41 v1, then correction E-41 v2 | Agreed corrected state | Version relation and correction rule recorded |
| Two distinct events of five units | 10 units | Different business identifiers; neither discarded |
Put receipt and effect inside the right boundary
PostgreSQL supports ON CONFLICT against suitable uniqueness constraints, including an atomic insert-or-update outcome. That can support a receipt table keyed by source and business identity. It does not decide the correct identity for your domain or automatically make an external service call transactional.
A proposed database-only design records the receipt and changes the ledger in one transaction. If both commit, replay reads the receipt and leaves the ledger unchanged. If neither commits, retry can process the event. Where the effect reaches a separate API, describe the destination’s idempotency capability and recovery process; do not extend a database guarantee across that boundary without evidence.
Test the awkward crash, not just the happy path
Use instrumented substitutes and invented records to interrupt processing immediately before and after the effect, before checkpoint acknowledgement, and during receipt persistence. After restart, compare the destination ledger with the business-event list. Count successful effects and replay receipts separately. A clean queue length is insufficient evidence.
Also test a key collision with different contents. A policy that discards every repeated identifier can hide a legitimate correction or a source-system defect. The illustrated hold keeps the first effect and prevents an unexplained second effect, while preserving enough evidence to resolve the conflict. Whether the final answer should replace, reverse or add is a domain decision to specify in the data contract.
Bound the acceptance statement
A useful result names the source identity rule, payload comparison rule, receipt retention window, destination, crash points and observed final states. A receipt retention window shorter than the permitted replay window leaves a gap; disclose it and test the boundary before acceptance.
Keep this record separate from a relational join audit, which tests how records multiply across tables. This test examines repeated delivery and external effects. A passing synthetic exercise is a bounded engineering observation, not proof that every failure is handled or that a real feed is licensable. Use the diligence question builder to ask for the recipient’s recovery design before agreeing to a sample.