Replying to @Leapofcode
everyone shouting 'idempotency keys' is failing the actual MTS interview question—you specifically asked what happens when the external API DOES NOT support idempotency
the 3 distributed systems patterns for surviving agent timeouts against non-idempotent legacy APIs:
1. the fundamental rule: timeout is an unknown in-flight state:
a timeout is the Two Generals' problem: the TCP packet could have failed on ingress, OR the bank/vendor executed the mutation and only the response dropped
if an agent blindly retries a non-idempotent POST, you guarantee duplicate side effects (double charges, duplicate records)
the state machine must transition to IN_DOUBT, never straight to RETRY
2. deterministic read-before-retry (synthetic idempotency probes):
even if the vendor has no Idempotency-Key header, their payload usually accepts custom metadata or reference fields
embed a deterministic token into the request: reference = hash(agent_id, task_id, step_nonce)
on timeout, the agent NEVER retries the POST directly—it issues an out-of-band probe:
GET /transactions?reference=REF_42
- if the record exists: adopt the remote ID, mark succeeded, proceed
- if 404 after an established quiet window: only then is it safe to retry
3. the Saga pattern & asynchronous reconciliation queues:
if the vendor has neither idempotency nor deterministic reference queries, synchronous resolution is mathematically impossible
the agent must persist the intent to a dead-letter / reconciliation outbox and yield
an out-of-band worker attempts reconciliation or issues a compensating reversal (void / cancel) before unlocking the agent's workflow
the true MTS architecture answer: never let agents call raw un-idempotent third-party APIs directly. you place an internal egress proxy in front that shims synthetic idempotency and transaction locks
Replying to @Kizuno18
This explanations deserve to be a stand alone post. Thanks mate.
Replying to @adnaannasir_
what if that external api does not support idempotency?

