Why API Response Caching Must Be Explicitly Designed

Lucas: You know that moment when you're checking a ride-hailing app during a storm, and the fare estimate is way higher than normal - but then you refresh and it drops by thirty percent? Luna: Yeah, and you wonder - did I just dodge a surge, or was the first number just wrong? Lucas: Right. Well, in a lot of cases, that discrepancy isn't about surge pricing changing.

It's about caching. Specifically, an API response getting served from a cache when it was never supposed to be. Luna: So the app is showing me a fare that's minutes old, not the live price. Lucas: Exactly.

And this is the topic I want to dig into today: API response caching that happens implicitly - the kind you didn't explicitly design for, but your infrastructure just does anyway. Luna: I've definitely seen this. A team deploys a CDN in front of their API to speed up static assets, and suddenly some POST responses are getting cached because nobody set Cache-Control headers. Lucas: That's the classic pattern.

And the problem is that caching is actually a really subtle contract between the server and whatever's in the middle - a CDN, a reverse proxy, even the browser. If you don't explicitly tell those intermediaries what to cache and for how long, they'll guess. Luna: And their guess is usually wrong for dynamic data. Lucas: Look, there's a well-trodden example I want to walk through.

A few years ago, a major e-commerce company had an API endpoint that returned personalized product recommendations. The endpoint used POST because the request body contained user preferences. Luna: And POST responses are not cacheable by default, according to the HTTP spec. Lucas: Correct.

But their CDN had a feature that allowed caching POST responses if you explicitly enabled it. Someone on the infrastructure team turned it on to reduce latency for a different endpoint - and forgot to disable it for the recommendations one. Suddenly, User A was seeing User B's recommendations because the CDN cached the response and served it to the next request with similar body parameters. Luna: That's a privacy nightmare.

And it's not just about POST - I've seen GET requests for dashboard analytics that were cached for an hour by a reverse proxy, when the data updates every five minutes. Lucas: Right. And the frustrating part is that the HTTP spec gives us really clear tools to prevent this. The Cache-Control header.

You can set 'no-store' to tell caches never to store the response. 'no-cache' tells them they must revalidate before using a cached copy. 'private' means only the browser can cache it, not shared caches like a CDN. Luna: But I think a lot of teams - especially early-stage ones - just don't set these headers.

They ship the API, it works, they move on. Lucas: Exactly. And then six months later, they have a production incident where stale data causes a customer-facing bug. I remember Stripe's API documentation explicitly sets Cache-Control: no-store on every response.

They don't leave it to chance. And that's the level of discipline that's rare but necessary. Luna: Stripe also uses ETags for their GET endpoints, right? So clients can do conditional requests.

Lucas: Yes, and that's the smart pattern. For endpoints where caching is actually desirable - maybe a list of public product categories that rarely changes - you can set a Cache-Control header with a max-age, and also return an ETag. Then clients can send if none match and get a 304 Not Modified if nothing changed. Luna: But you have to explicitly decide which endpoints benefit from caching and which don't.

That's the key point. Lucas: Right. And it's not just about correctness - it's about performance too. If you cache a response that's supposed to be fresh, you save latency and server load.

But if you cache a response that should be dynamic, you introduce staleness. Luna: I worked with a healthcare API once that had a reverse proxy caching responses for thirty seconds by default. The API returned a patient's current medication list. Thirty seconds of staleness could mean a doctor sees an old list and prescribes something that interacts.

Lucas: That's exactly the kind of scenario where you need 'no-store'. And the fix was probably a one-line header change. Luna: It was. But finding it required tracing through three layers of infrastructure.

The team didn't even know the proxy was caching. Lucas: So the lesson is: treat caching as an explicit design decision for every single endpoint. Don't rely on defaults. And test your caching headers under realistic load, because a cache that works fine with ten requests per second might behave differently at ten thousand.

Luna: And don't forget about the client side. Even if your server says 'no-store', a mobile app might cache the response locally because its networking library does that by default. Lucas: Good point. That's another layer.

So you really need to think about the full path from server to screen. And honestly, if today was worth a coffee to you, that's the link - buy me a coffee dot com slash fexingo. Luna: Yeah, it's a small way to keep the show ad-free and focused on these deep dives. Appreciate anyone who chips in.

Lucas: Alright, back to caching. One more pattern I see often: teams use a CDN that caches based on the full URL, including query parameters. So if you have a GET endpoint like /api/products?category=shoes, that response gets cached separately from /api/products?

category=hats. Luna: That's usually fine, unless the same category returns different results for different users - like personalized pricing. Then you need to either include a user ID in the cache key or use a Vary header. Lucas: Exactly.

The Vary header tells caches to key on something other than just the URL - like Authorization or Accept-Encoding. But if you forget it, you get cross-user data leaks. Luna: And there's the Vary: * wildcard, which basically says 'don't cache this for any shared cache.' That's a blunt but effective tool.

Lucas: It is. But the broader point is: caching is not a set-it-and-forget-it feature. It requires ongoing attention. Every time you add a new endpoint or change the semantics of an existing one, you should revisit the caching strategy.

Luna: I think a good practice is to include caching headers in your API design review checklist. Before any endpoint ships, someone should ask: is this data cacheable? If so, for how long? And who should be able to cache it?

Lucas: Yes. And then verify it in staging with a tool like curl or a browser's developer tools. Check the response headers. Make sure Cache-Control says what you expect.

Luna: One thing that catches teams is that some CDNs and proxies ignore Cache-Control if you have a default rule that says 'cache everything'. So even if your API says 'no-store', the proxy might override it. Lucas: That's a critical nuance. Infrastructure defaults can override application headers.

So you need to align both layers. The CDN config should respect origin headers unless there's a very good reason not to. Luna: And test end to end. Because the CDN might be caching at the edge, and the browser might be caching separately.

You need to know what the user actually receives. Lucas: Alright, I think we've covered the key pitfalls. Let's leave listeners with one concrete action: this week, pick one API endpoint your team owns, check its Cache-Control header in production, and verify that it matches your intent. You might be surprised.

Luna: Yeah, and if you find a misconfiguration, that's a quick win. Lucas: Exactly. Thanks, Luna. Luna: Thanks, Lucas.