Draft — in editorial review, not yet published. Content has not been approved by a human reviewer.
Lesson 18 / 50 · Version 2.2.0
Error Handling and Retries
- Level
- Intermediate
- Duration
- 15 min learn + 15 min apply
- Prerequisites:
- Lessons 11, 15, 16
Why it matters: Every integration fails sometimes. The difference between a minor blip and a week of corrupted data is whether you retried the right things, avoided duplicates, and made failures visible.
By the end you can:
- Classify failures as transient or permanent
- Design bounded retries with backoff and jitter
- Prevent duplicate writes
- Route exhausted failures to an owner
Learn (15 min)
1. Classify the failure (5 min)
| Category | Examples | Response |
|---|---|---|
| Transient | Timeout, 502/503, brief outage | Retry with backoff |
| Rate limit | 429 | Wait as instructed, then retry |
| Permanent (input) | 400/422, unknown field | Fix data; don't retry unchanged |
| Permanent (access) | 401/403 | Fix credentials/permissions; alert |
| Log the category, step, and a safe record reference (no secrets). |
2. Retry carefully (5 min)
Use exponential backoff with jitter: wait ~1s, 2s, 4s, 8s... each plus a random amount so many jobs don't retry together. Cap both the number of attempts and total time.
delay = min(cap, base * 2**attempt) + random(0, base)
Danger: a timeout is ambiguous. The destination may have succeeded and only the response was lost. Retrying a non-idempotent create can double-write. Use a provider idempotency key, or look up the record by a stable source ID before creating.
3. Recover visibly (5 min)
When retries are exhausted, send the item to a dead-letter queue (a review sheet or table), alert an owner, and keep enough context to replay. Document how to replay after a fix without redoing completed side effects. Never swallow errors or retry forever.
Apply (15 min)
Worked example (3 min). Creating a CRM contact times out. Flow: store source_id = booking_8841 → on timeout, search CRM by source_id → if found, treat as success; if not found, retry up to 4 times with backoff + jitter → if still failing, write to "Failed syncs" with the error category and alert Ops → after fix, replay from the queue (the search-first step prevents duplicates).
Lab (7 min): Failure matrix.
- List five failures for a workflow you use (timeout, 429, invalid email, expired token, 500).
- For each, fill: category, retry?, max attempts, who is alerted, how to replay.
- Mark which steps are non-idempotent and how you protect them.
Deliverable: A failure matrix. You're done when: Every row names an owner and a duplicate-protection method (or says "read-only").
Check yourself (5 min)
- Should invalid input be retried unchanged?
- What is exponential backoff, and why add jitter?
- Scenario: A write times out. Why is "just retry" risky?
- How can you prevent duplicate writes?
- What happens after retries are exhausted?
Reference
Common mistakes
| Symptom | Likely cause | Fix |
|---|---|---|
| Duplicate records after outages | Blind retry of creates | Idempotency key or search-first |
| Retry storms | No jitter or cap | Add jitter; cap attempts/time |
| Silent data loss | Errors swallowed | Dead-letter queue + alert |
Cheat sheet: Classify → retry only transient → cap → deduplicate writes → alert and recover.
Key terms: transient error, permanent error, backoff, jitter, idempotency key, dead-letter queue.
Interview prompts: How do you retry safely a payment-like operation? What's your replay procedure?
Go deeper: n8n error handling · n8n rate limits · Stripe idempotent requests
Next: Lesson 19 — routing leads across apps.