Paging and Caching Decisions That Bite Later
Making an API fast is easy. Making it fast without serving stale data or someone else's data means picking your paging shape before callers depend on it and never letting a cache key forget who asked.
The demo version returns every record on every list call and reads the store on every request. That is fine with 10 records and a store that answers in a microsecond. Make the store slow and the questions change: how do you make the API fast without serving anyone stale data, or someone else’s data?
Paging and caching are the first answers. Each one has a decision buried in it that is cheap to make now and expensive to change later.
Envelope or headers?
Many APIs return { "items": [...], "totalCount": 312, "next": "..." }. It is easier for clients to read than headers, and plenty of teams prefer it.
If your v1 list already returns a bare array, switching to an envelope breaks every caller. So it is a v2 change. That is the payoff of versioning on day one: add a v2 list endpoint, deprecate v1’s on a schedule, and nobody’s integration breaks. Without versioning, you are stuck with headers forever or you break people.
Offset or cursor paging?
page and pageSize (offset paging) are simple and let a client jump to page 7. They get slow on large tables, because the database still walks past every skipped row. And items shift between pages when records are added mid-walk.
Cursor paging (?after=<last id seen>) is fast at any depth and stable under inserts, but you can only go forward.
Large, busy collections usually end up with cursors. If you know a collection will be large and busy, start there.
What to cache, and for how long
- Cache things read far more often than written, keyed by something stable.
- Invalidate on every write you own. Expiry is the safety net for writes you don’t see: another instance, a background job, someone editing the database directly.
- Multiple instances need a distributed cache. Otherwise each instance holds its own stale copy after another one writes.
HybridCachehandles that with one more registration. - Watch negative results. Caching “not found” for an ID until it expires is harmless when the server generates GUIDs. With client-chosen IDs, you just cached a lie that blocks the create.
Output caching, when it fits
Output caching shines for anonymous, public responses: a product catalog, a status page, a public OpenAPI document.
If you ever turn it on for authenticated requests, the cache key must include the user (VaryByValue on the user ID). Otherwise you’ve built a data leak. User A’s response gets served to user B, and nothing in your logs looks wrong.
The point
Fast is easy. Fast and correct is the job. Pick your paging shape before callers depend on it, invalidate every write you own, and never let a cache key forget who asked.