Idempotency

An operation is idempotent when repeating it changes nothing beyond the first application. In a distributed pipeline this is not a refinement but a requirement, because networks fail in the one way that cannot be distinguished from success: the sender never learns whether the receiver got the message, so the only safe response is to send it again. A consumer that is not idempotent turns that retry into a duplicate.

Making it work is usually a matter of identity. A stable record identifier lets a consumer recognize what it has already processed, and an upsert keyed on that identifier absorbs a repeat without a second row. This is the mechanism that turns an at-least-once delivery guarantee into effectively-once processing, and it is why deduplication belongs at the boundary rather than in a nightly cleanup job.