When an agent writes into a system of record, posting an invoice, creating a sales order, issuing a vendor payment, the danger is not that the write fails. A failed write is visible and recoverable. The danger is that the write succeeds, the agent never learns it succeeded, and the agent tries again. The second attempt posts a second invoice. Over any unreliable channel, and every network is unreliable, a retry that produces a duplicate is close to inevitable rather than a remote possibility, unless the write was designed to be safe to repeat. That design discipline is idempotency, and it is the part of agent engineering that separates a demo from something a controller will let touch the ledger.
Why retries are not optional
The first instinct is to write the code so it never retries: make the call once, check whether it worked, move on. The problem is that the agent often cannot tell whether it worked. Consider the sequence the agent actually executes. It sends the request to post the invoice, the request reaches the accounting system, the system commits the row, and then the response on the way back is lost to a dropped connection or a timeout. From the agent's vantage point the call failed. From the ledger's vantage point the invoice exists. The agent now faces a choice with no good answer if the write is not idempotent: retry and risk a duplicate, or give up and risk a silent miss. Real systems retry, because a missed posting is the worse failure and because at-least-once delivery is the default behavior of nearly every queue and HTTP client. The duplicate is the price of that safety, and the only place to stop paying it is on the receiving side.
The result that says exactly-once is a marketing term
No protocol can engineer this away at the transport layer, and a classic distributed-systems result explains why. The Two Generals Problem, formalized by Akkoyunlu, Ekanadham, and Huber in 1975 and later popularized by Jim Gray, concerns two parties who must agree on an action by sending messages over a channel that can lose any message. The proof shows that no finite exchange of acknowledgments can guarantee both sides reach agreement, because the last message sent always might be the one that was lost, and the sender can never be certain it arrived. Applied to a write, the agent and the system of record can never be jointly certain, from messages alone, that the invoice posted exactly once.
What this rules out matters for buyers reading vendor copy. When a vendor says its agent delivers exactly-once writes, that phrase, taken literally, describes something proven impossible. What is actually delivered, and what is sufficient, is at-least-once delivery combined with idempotent processing on the side that applies the change. The Apache Kafka documentation is candid about this for its own exactly-once semantics: Kafka guarantees at-least-once delivery by default, and the stronger guarantee is built from an idempotent producer, which deduplicates on a broker-assigned producer id and a per-message sequence number, plus transactions that make writes across partitions atomic, not from the network promising single delivery. The construction has a price, which Confluent measured when the feature shipped: producer throughput declines by 3% for 1 KB messages with transactions lasting 100 ms, and stream processing at a 100 ms commit interval gives up 15% to 30% of throughput depending on message size. Exactly-once is an outcome you construct on the apply side, not a property the channel hands you.
The deduplication record, written with the change
The mechanism has three moving parts. The first is the idempotency key. The agent generates a unique identifier and attaches it to the request, and the receiving system records that key the first time it sees it. If a request arrives bearing a key already on file, the system returns the original result instead of doing the work again. Stripe's published idempotency-key design is the reference pattern: the client generates a unique key and sends it in an Idempotency-Key header, and Stripe saves the status and body of the first request made under that key and returns that same stored result to every retry carrying the key, so a retried charge request bills the customer once rather than twice.
The second part is where most home-grown attempts quietly break. The naive approach is check-then-act: look up whether the key has been seen, and if not, do the work and then record the key. The gap is the window between doing the work and recording the key. If the process fails there, after the external call has committed but before the deduplication record is written, the system has performed the side effect and has no memory of it. The next retry sees no key on file and does the whole thing again. Amazon's Builders' Library names the requirement that closes this gap: recording the idempotency token and performing the mutating work must together form one atomic, all-or-nothing operation, so that either both happen or neither does, never the token without the work or the work without the token. There is then no instant at which the work is done but unrecorded.
Scope the key to the intent, not the attempt
The third part is what the key is keyed to. A key derived from the transport attempt, a fresh value generated on each retry, defeats the entire purpose, because every retry then looks like a new intent and deduplication never fires. The key has to be scoped to the business intent: this specific invoice, for this vendor, for this period, gets one key, and every retry of posting it carries that same key. The discipline is to derive the key from what the agent is trying to accomplish, not from the mechanics of how it is trying. Paired with this, conditional updates using a version check guard against a different hazard, two writers acting on stale state. The agent reads a record at version five and writes only if the record is still at version five, so a concurrent change is detected rather than silently overwritten. An intent-scoped key, an atomic deduplication record, and version-checked updates together give the apply side what it needs to absorb retries without harm.
A reasonable counter, answered
A reasonable counter is that the systems agents must write into, QuickBooks, an aging ERP, a billing portal, were not built to honor an idempotency key, so this discipline is academic where it is needed most. There is real truth in that. When the target exposes no key parameter and no version field, the agent cannot push the guarantee into the system of record itself. But it can keep its own ledger of intents and their outcomes, reconcile against the target after each write to confirm what landed, and refuse to repeat an intent it has already confirmed. The guarantee moves to the agent's side of the boundary, which is the same place the rest of the write-back machinery for systems with no usable API already lives. That is more work than reading a header, and it is the work that has to happen for an agent to be trusted with the books. It is why we treat idempotency as a first-class part of how production reliability gets engineered, why duplicate-write detection sits inside the controls for autonomous actions, and why the reconciliation it depends on is part of monitoring an agent after it ships. The double posting that teams discover in production is almost never a model error. It is a missing idempotency key, and it is fixable before the first write goes out.