REST APIs and Pagination

Why it matters: When a no-code module doesn't exist, or you need volume, you call the API directly. Reading an API contract, handling errors, and paginating correctly are the skills behind every serious integration.

By the end you can:

• Read an HTTP request and response
• Interpret common status codes
• Paginate completely and resumably
• React properly to rate limits

Learn (15 min)

1. Request and response (5 min)

GET /v1/contacts?limit=100 HTTP/1.1
Host: api.example.com
Authorization: Bearer <token>
Accept: application/json

→ 200 OK
{ "results": [ ... 100 contacts ... ], "next": "cursor_8f3a" }

Parts: method (GET reads, POST creates/triggers actions, PUT/PATCH update, DELETE removes; follow the provider's documentation), URL, headers, optional body; response has status, headers, and body.

2. Authentication and errors (5 min)

Credentials go in headers or client libraries, not URLs or logs. Know the families:

3. Pagination and rate limits (5 min)

APIs return results in pages. Follow the exact next cursor/URL until the API signals completion; don't construct opaque cursors. Save progress so a failed job can resume.

cursor = load_checkpoint() or None
while True:
    page = GET /contacts?limit=100&cursor=cursor
    upsert(page.results)            # idempotent by contact id
    save_checkpoint(page.next)
    if not page.next: break
    respect_rate_limit()

Apply (15 min)

Worked example (3 min). Import 12,000 contacts at 100/page = 120 requests. Plan: upsert by stable ID (no duplicates on re-run), checkpoint after each page, if 429 then wait the indicated time and retry the same page, final check compares the destination count to the source total and spot-checks 20 records.

Lab (7 min): Read a real API.

1. Open any public API documentation (a weather or public-data API works) and find: base URL, authentication, one list endpoint, pagination method, rate limit.
2. Write the request you'd send for page 2 using the documented method.
3. List what you'd do on a 401, a 429, and a 500.

Deliverable: A "contract summary" table plus three error reactions. You're done when: Someone else could write the loop from your summary.

Check yourself (5 min)

1. Four parts of an HTTP request?
2. What does 401 generally mean? 403?
3. Scenario: Your import stops at page 37 after a timeout. How do you resume without duplicates?
4. How should an opaque cursor be handled?
5. Safe reaction to a 429?

Reference

Common mistakes

Cheat sheet: Read contract → authenticate → request page → process idempotently → checkpoint → follow returned cursor → stop at the documented end.

Key terms: HTTP method, status code, header, cursor, rate limit, upsert.

Interview prompts: How do you design a resumable import? How do you handle rate limits across parallel workers?

Go deeper: MDN HTTP overview · GET method · Microsoft pagination example · n8n rate-limit handling

Next: Lesson 16 — webhooks, APIs in reverse.