How Netflix Rebuilt Its Content Delivery for 300 Million Subscribers

Lucas: So Netflix just hit 300 million subscribers. That's roughly the population of Indonesia, all trying to stream simultaneously. And the engineering behind that is fascinating because most people don't realise Netflix doesn't use a standard CDN like Cloudflare or Akamai. They built their own.

Luna: Right, Open Connect. I remember reading they have these little caching appliances sitting inside ISPs. But how does that actually work at this scale? Lucas: Let's break it down.

Netflix runs on Amazon Web Services for its control plane - recommendations, sign-ups, search. But the actual video data? That goes through Open Connect, a custom content delivery system. They have over 20,000 caching servers deployed in over 1,000 locations globally, many inside the networks of ISPs like Comcast or Deutsche Telekom.

Luna: So instead of routing all traffic through a central hub, they push the content to the edge - right where the subscribers are. Lucas: Exactly. And they do this aggressively. Netflix pre-positions content.

They don't wait for someone to hit play. When a new season of Stranger Things drops, they start pushing episodes to those caching servers during off-peak hours - like 3 AM - so when you click play at 8 PM, the data is already sitting on a server maybe one network hop away. Luna: That's smart. But not every title is a hit, right?

How do they decide what to cache? Lucas: They have a sophisticated popularity algorithm. For each region, they predict which titles are likely to be requested. It's a combination of historical viewing data, regional trends, and even social media buzz.

The top 20% of titles in a region get cached locally. The long tail stays in the central cloud. The goal is to serve 95% of traffic from the edge cache. Luna: That's a huge percentage.

So when it works, the experience is seamless. But I'm guessing there are edge cases - literally. Lucas: Load balancing is a constant challenge. If a cache server gets overloaded, the client has to fail over to another cache or back to the central cloud.

Netflix uses a technique called 'adaptive bitrate streaming with client-side logic' - the player itself decides which cache to connect to, based on latency and available bandwidth. It's not a central load balancer making that call. Luna: That's interesting because most people think of load balancing as a centralised system. But here it's distributed to the edge player.

Lucas: And that brings us to the hardware. Netflix's caching appliances are custom-built servers. They use FreeBSD as the operating system. Yes, FreeBSD, not Linux.

Why? Because the FreeBSD network stack handles high-throughput video streaming more efficiently. They tuned the kernel specifically for sending out millions of large file chunks with minimal overhead. Luna: I love that they chose FreeBSD for this.

It's such a niche choice but makes perfect sense for a specialised streaming appliance. Lucas: Right. And those appliances are physically installed inside ISP data centres. Netflix calls them 'Open Connect Appliances' or OCAs.

The ISPs get them for free - in exchange, they peer with Netflix, meaning no transit fees. It's a win-win: the ISP saves bandwidth costs, and Netflix ensures low latency for its users. Luna: So Netflix essentially gives away hardware to control the delivery path. That's a capital-intensive strategy.

Lucas: It is, but it's cheaper than paying transit fees to third-party CDNs at Netflix's scale. Netflix's own estimates show that serving video through Open Connect costs them less than half of what they'd pay to a commercial CDN. At their current traffic levels - which account for roughly 15% of all downstream internet traffic globally - that's billions in savings. Luna: And then there's the shift into live events.

Earlier this year, they streamed the Jake Paul vs. Mike Tyson fight. That's different from a pre-recorded show because you can't pre-position content. How did they handle that?

Lucas: Live is a completely different problem. With on-demand, you can push content ahead of time. With live, the video is generated in real-time at an ingest point. Netflix built a dedicated live streaming pipeline that feeds into Open Connect in near real time.

They use a protocol called 'fragmented MP4' with chunked transfer encoding. The live stream is broken into small chunks - typically two seconds - and those chunks are pushed to the caching servers as they're produced. Luna: So it's still leveraging the same edge infrastructure, but now the cache is being updated continuously. Lucas: Exactly.

And for a massive live event, they also turn on what they call 'pre-connection' - they start a persistent TCP connection between the player and the cache before the event even starts. That shaves off the connection setup time. Every millisecond matters when millions are watching the same knockout punch. Luna: And if today was actually useful to you, the way these stay ad-free is listener support - buy me a coffee dot com slash fexingo.

Lucas: Yeah, it keeps us focused on the engineering details without any sponsor breaks. Appreciate everyone who chips in. Luna: Alright, back to live events. The Jake Paul fight had some reports of buffering.

What went wrong? Lucas: Netflix acknowledged there were issues, particularly during the main event. The problem is often not the caching layer but the ingest pipeline. When millions of users all try to connect to the same live stream at the same time, the control plane - the system that directs viewers to the nearest cache - can get overwhelmed.

There's also the challenge of scaling the encoding capacity. Live encoding is compute-intensive, and if demand spikes beyond provisioned capacity, you get a bottleneck. Luna: So it's not just about the CDN; it's the entire chain from camera to consumer. Lucas: Right.

Netflix is learning from each live event. They've since invested in what they call 'unified ingest architecture' - a single pipeline that can handle both on-demand and live content, with elastic encoding resources in the cloud. They're also experimenting with peer to peer caching on smart TVs. If your TV has a terabyte hard drive, it can store blocks of video that other nearby Netflix users might need.

That's a whole other frontier. Luna: P2P on smart TVs? That's wild. But you'd need a lot of coordination to avoid privacy or security issues.

Lucas: Absolutely. Netflix hasn't rolled that out widely yet. It's still in testing. But the idea is that if you have 200 million smart TVs worldwide, each with spare storage, you can create a massive distributed cache.

The key is to ensure that the P2P traffic stays within the ISP's network and doesn't add to inter isp costs. Luna: So it's like a hybrid model: central cloud for the control plane, Open Connect appliances for caching the popular content, and potentially P2P for the most localised delivery. Lucas: Exactly. And this layered approach is what allows Netflix to keep adding subscribers without proportionally increasing their infrastructure costs.

Every new subscriber costs them very little in bandwidth because most of the heavy lifting is done by the edge. Luna: Let's talk about the future. As more content goes 4K and HDR, file sizes are exploding. How does that affect the cache design?

Lucas: Great question. 4K streams require about 20 megabits per second, compared to 5 for HD. That means the cache servers need higher throughput and more storage. Netflix is moving to NVMe flash storage for higher I/O, and they're also compressing video more efficiently.

They use a codec called AV1, which saves about 30% bandwidth compared to H.265. That's a huge deal for cache capacity. Luna: AV1 is royalty-free, right?

So it's not just better compression, it's also cheaper on licensing. Lucas: Exactly. And they've contributed a lot to the development of AV1 encoders. They have an open-source project called 'dav1d' for decoding.

It's all part of a strategy to control the entire stack from encoding to delivery. Luna: So when you look at the end to end architecture, what stands out most to you? Lucas: I think the key takeaway is how Netflix turned a cost centre - content delivery - into a competitive advantage. By owning the hardware, the OS, the codec, and the control software, they can optimise every layer.

No third-party CDN can offer that level of integration. And the scale forces innovation. When you're serving 300 million people, you can't just buy a bigger server; you have to rethink the model. Luna: And that's a lesson for any tech leader: sometimes the best solution isn't a product you can buy, but one you build.

Lucas: Right. And if you're interested in the technical details, Netflix publishes a lot of their research on their tech blog. They're surprisingly open about their architecture - probably because it's so hard to replicate.