How Kubernetes Service Mesh Sidecars Cause TCP Connection Timeouts

Lucas: So you're running Istio on Kubernetes — you've got your microservices, each with an Envoy sidecar, everything's been humming along for months. Then, during a routine traffic spike — maybe a marketing push or an end of quarter batch — your users start seeing spinning wheels, timeouts, and the occasional 503. Luna: Classic. And the first instinct is usually to blame the application, right? Check database pools, API latency, stuff like that. Lucas: Exactly. But in a surprising number of cases, the culprit isn't the app container — it's the sidecar. Specifically, the Envoy proxy running alongside your service hits its memory limit, starts dropping packets, and TCP connections time out. I've seen this happen at a mid-size fintech running about 200 microservices. Their entire checkout flow collapsed for about 12 minutes before someone thought to check the sidecar's resource usage. Luna: Wait — the sidecar's memory limit? Most teams I talk to don't even set explicit limits on the sidecar. They just use the default Istio installation. Lucas: And that's exactly the problem. The default Istio sidecar deployment — at least up to version 1.20 — sets memory requests at 128 megabytes and limits at 512 megabytes. For a lot of workloads, that's fine. But once you have a high volume of concurrent connections, Envoy's connection pool and buffer structures grow beyond 512 megs. And when it hits the limit, the kernel starts reclaiming memory — which means Envoy gets oom killed or starts failing to allocate buffers for new connections. Luna: So the sidecar basically becomes a bottleneck. But the app is still running — it's just that the network calls fail because the proxy can't handle them. Lucas: Right. The app container is healthy, health checks pass, but the sidecar is silently struggling. What happens next is a cascade: Envoy's upstream connection pool gets exhausted. It has a default maximum of 1024 connections per cluster — that's per upstream service. Once that's full, Envoy will queue or reject new connections. And because TCP doesn't have a built-in backpressure mechanism like HTTP/2, those queued connections just hang until the client timeout fires. Luna: And that timeout is often set to 30 seconds or more by default. So users see a 30-second spinning icon before anything fails. Lucas: Exactly. In the fintech case, their frontend had a 15-second timeout. But the backend calls were chained — payment service calls ledger, ledger calls accounting — each with its own sidecar. So a single checkout could trigger four or five sidecar timeouts, each taking 15 seconds, leading to a total wait of over a minute. The user retries, more connections pile up, and the whole system grinds to a halt. Luna: So what did they do to fix it? Throw more memory at the sidecar? Lucas: Partially. They bumped the sidecar memory limit to 1 gigabyte and increased the request to 256 megabytes. But that alone didn't solve it. They also had to tune Envoy's connection pool limits. Specifically, they increased 'circuit_breakers' per cluster: max connections went from 1024 to 2048, and max pending requests from 1024 to 2048 as well. Luna: That's a pretty big change. What about the pods themselves? Were they getting evicted? Lucas: Some were. The node pressure from memory was causing pod evictions, but not always. The trickier part was that the sidecar shares the pod's cgroup. So if you set a memory limit on the sidecar container, but not on the app container, the app could consume all the memory and still leave the sidecar starved. They had to set resource limits on both containers explicitly. Luna: So both containers in the pod need resource limits? That seems obvious, but a lot of people just limit the app container and forget the sidecar. Lucas: Right. And there's another layer: noisy neighbors. On a shared node, if another pod's sidecar is memory-heavy, it can affect yours. They ended up using pod anti-affinity to spread high-traffic services across different nodes, and they also enabled Istio's 'proxyMetadata' to set environment variables for Envoy's memory management — like 'ENVOY_MAX_MEMORY' to 80 percent of the limit. Luna: That's clever. So Envoy proactively frees memory before hitting the hard limit. Lucas: Exactly. That soft-limit approach buys you time during spikes. But the real lesson is monitoring. Most teams monitor app container CPU and memory, but not the sidecar. They added a Grafana dashboard with a panel for sidecar memory usage per pod, and set an alert when memory exceeds 70 percent of the limit. That caught the issue before the next traffic spike. Luna: So the key takeaway: if you're using a service mesh, treat the sidecar like a first-class citizen. Give it resources, tune its connection pools, and monitor it just as closely as your application. Lucas: Absolutely. And one more thing: test under load. The fintech team had done load testing, but they never tested with the sidecar's default limits. Their test environment had much smaller traffic, so the sidecar never broke a sweat. Only when they replayed production traffic in staging did they see the timeouts. Luna: That's a great point. Load testing should include realistic sidecar resource constraints. Lucas: And speaking of realistic constraints — we try to keep these episodes practical and free of advertising. We deliberately don't run ads on DevOps Daily. If you want to support that choice, the link is buy me a coffee dot com slash fexingo. Luna: Yeah, listener support is what keeps this ad-free. Really appreciate everyone who chips in. Lucas: So to wrap up: next time you hit mysterious timeouts in your Istio mesh, check the sidecar's memory. It might just be the bottleneck.