What you’ll be able to do
- Design retry policy with backoff and a maximum attempt count
- Make retried side effects idempotent before relying on retries at all
- Set timeouts that match streaming versus non-streaming latency profiles
- Tell a genuine failure apart from a slow-but-succeeding call
What you need to know
A retry is only safe if the thing being retried is idempotent
Retrying a read is free — asking Claude the same question twice does nothing harmful the second time. Retrying a side effect — a tool call that sends an email, charges a card, writes a database row — is a different problem entirely: if the first attempt actually succeeded and the caller just never saw the response (a timeout, a dropped connection), a naive retry executes the side effect a second time.
The fix isn't "retry less." It's making the side effect idempotent so a duplicate attempt is safe regardless of whether the first one landed:
- Idempotency keys — the caller generates a unique key per logical operation and passes it through; the receiving system checks whether that key already executed before doing the work again.
- Natural idempotency — some operations are safe by construction, like "set status to shipped" (repeating it changes nothing) versus "increment shipped count by one" (repeating it double-counts).
- Dedup on the receiving side — a downstream system that records "already processed this request ID" and short-circuits a repeat, even if the caller didn't explicitly ask for that.
Without one of these, a retry policy isn't a reliability improvement — it's a mechanism for occasionally doubling a side effect under exactly the conditions (timeouts, network blips) most likely to trigger a retry in the first place.
Backoff and a ceiling, not immediate retry to infinity
Retrying immediately with no backoff is the reliability-pattern equivalent of a misplaced cache checkpoint: it looks like it's helping, and during a real outage it actively makes things worse, hammering an already-struggling service with the full retry volume of every failed caller at once. Backoff spreads that load out; a maximum attempt count converts "silently retry forever" into "fail loudly after N tries," which is the outcome an on-call engineer actually wants during an incident.
Timeouts have to match what the call is actually doing
A single global timeout value applied to both streaming and non-streaming calls is a common source of false failures. A non-streaming call to generate a long response can legitimately take much longer than a short one — timing it against a short-response baseline produces timeouts on requests that were simply going to take longer to finish, not requests that failed. A streaming call has a different profile again: the first token should arrive quickly even if the full response takes a while, so time-to-first-token and total-duration are two different signals worth watching separately.
The practical distinction the exam cares about: a timeout means "I stopped waiting," not "the call failed." If the underlying request actually succeeded after the caller gave up on it, and the caller then retries a non-idempotent side effect, the timeout policy and the idempotency gap from the first section compound into exactly the double-execution failure both were meant to prevent.
Key concept
A retry without idempotency doesn’t fix a failure — it risks doubling a side effect. Backoff with a ceiling protects a struggling service; a timeout means “I stopped waiting,” not “it failed.”
When a scenario describes a duplicate charge, duplicate email, or duplicate write that only happens "sometimes, under load," look for a retry policy sitting on top of a non-idempotent operation — that combination is the textbook cause.
Practice scenario
Work it through, then open this
The timeout doesn’t mean the first attempt failed — it means the caller stopped waiting for a response that may have already landed. Under load, the first charge is more likely to actually complete just slowly, so the timeout-triggered retry executes the charge a second time. The fix is an idempotency key on the payment call, generated once per purchase and passed on every attempt, so the payment system recognizes the retry as a duplicate rather than a new charge — not a longer timeout, which only makes the race less frequent, not impossible.
Build exercise — Make a retry policy safe
Intermediate · 25 min
What you’ll learn
- Recognizing which operations in a system are safe to retry as-is and which aren’t
- Adding backoff and a maximum attempt count to a retry loop
- Setting timeout values per call shape instead of one global number
-
List the side-effecting tool calls in a system and mark each as naturally idempotent, idempotency-key-able, or currently unsafe to retry.
- Why: This is the check that has to happen before any retry policy is added, not after.
- You should see: At least one call in most real systems that isn’t safe to retry yet.
-
Take a retry loop with no backoff and no ceiling and add exponential backoff plus a maximum attempt count.
- Why: Unbounded immediate retries amplify load on a struggling dependency during exactly the outage they’re meant to survive.
- You should see: A loop that fails loudly after N attempts instead of retrying forever.
-
Compare the timeout value used for a streaming call against a non-streaming call in the same system.
- Why: One global timeout value applied to both shapes produces false failures on whichever shape it wasn’t tuned for.
- You should see: Either two different timeout values, or a single value you can now show is wrong for one of the two call shapes.
Exam traps
Retrying a side-effecting tool call without an idempotency key
The most direct path to a duplicate charge, duplicate send, or duplicate write, and it only shows up under exactly the conditions that trigger retries.
Using one timeout value for both streaming and non-streaming calls
Produces false failures on whichever call shape the value wasn’t actually tuned for.
Retrying immediately with no backoff, amplifying load during an outage
Turns a retry policy into extra load on a service that’s already struggling.
Treating a timeout as proof of failure and retrying a call that actually succeeded
The request may have completed after the caller stopped waiting; the retry then risks a duplicate.
Setting max retry attempts high enough to mask a systemic outage as a slow day
Delays detection of a real incident instead of failing loudly enough for someone to notice.
Building retries at every layer independently, so one failure retries N times over
Retry logic at the tool layer, the agent layer, and the orchestration layer all firing on the same failure multiplies attempts far beyond what any single layer intended.