The CTO Podcast with Fexingo · 2026-07-03 · 7 min
Key moments - from our scoring
Substance score
56 / 100
Five dimensions, 20 points each
GitHub's engineering team tackled a fundamental scalability problem: Git's original design stores each object (commit, tree, blob) as individual files, which breaks down at massive scale. With 100 million repositories, many containing years of history and large binaries, the filesystem approach created bottlenecks in inode management and metadata operations - stat and readdir calls consumed more CPU than actual data transfer. GitV2 replaces this with a content-addressed database built on RocksDB, Facebook's embedded key-value store, treating Git objects as database entries rather than files. The solution maintains packfiles as single blobs with bloom filters and prefix compression for fast lookups, plus a custom caching layer for frequently accessed commits and tree objects. The migration itself was methodical: GitHub used a shadow read system where writes went to both engines while reads tried the new system first, rolling out shard-by-shard over 20 months with instant fallback capability. Performance improvements include 40% faster full clones, 30% disk footprint reduction through deduplication, and noticeably faster git log and blame operations. The organizational approach - embedding engineers from git, storage, and operations teams into a tiger team reporting to the VP of engineering - avoided the typical 'us versus them' dynamics that plague infrastructure rewrites. This level of infrastructure work demonstrates why deep technical leadership decisions require data-driven justification and cross-functional alignment.
GitHub used a shadow read system where writes went to both old and new storage simultaneously, but reads were directed to GitV2 first with automatic fallback to the old system if data wasn't found, then rolled out the migration shard-by-shard over 20 months with continuous monitoring and instant rollback capability.
GitV2 is a content-addressed database built on RocksDB that treats Git objects as database entries instead of individual files, using bloom filters and prefix compression for fast lookups and enabling deduplication of identical objects across repositories, whereas Git's original model stores each commit, tree, and blob as a separate compressed file.
Full clone times dropped 40% for repositories with deep history, disk footprint reduced 30% through better deduplication and compression, and git log and blame operations became noticeably faster due to commit graph caching, while shallow clones saw only 10-15% improvement since overhead is mostly network-bound.
Cache misses were the most common failure mode, especially for repositories with specific access patterns like CI systems cloning the same commit hundreds of times per minute; partial clone operations weren't initially optimized, requiring new index structures to quickly identify which objects belong to which paths.
The filesystem approach hit fundamental scalability limits with 100 million repositories - metadata operations (stat, readdir) consumed more CPU than actual data transfer, and projections showed routine git push operations would take over a minute by late 2025 due to filesystem contention on metadata operations per second.
Our reviewer’s read on each dimension, with quotes from the episode.
The episode packs technical substance: specific architectural choices (RocksDB, content-addressed storage, bloom filters, prefix compression), concrete failure modes (cache misses, partial clone optimization), and quantified outcomes (40% clone time reduction, 30% disk savings, 10 - 15% shallow clone gains). However, it avoids deeper investigation into trade-offs, consistency guarantees, or why RocksDB specifically vs. alternatives - there's less exploratory depth than a top-tier technical deep-dive.
Git's original storage model is surprisingly simple: each object - commit, tree, blob - is stored as a separate file, compressed individually, with loose objects eventually packed into packfiles.
They had cases where a single `git clone` would open tens of thousands of file descriptors, and the metadata operations - stat, readdir - were consuming more CPU than the actual data transfer.
The episode covers a real technical case study that most listeners won't have heard, but the framing - storage layer bottleneck → database migration → successful shadow-read rollout - follows a well-worn pattern in infrastructure modernization. The RocksDB choice and bloom filter use are sensible but not contrarian; the organizational insight (tiger team embedding) is practical but not novel.
They did a shard-by-shard migration - each shard covers a range of repository IDs. They built a shadow read system: every write went to both the old storage and GitV2, but reads first tried the new engine, and if it didn't have the data, fell back to the old one.
They embedded engineers from each of the major teams - the git team, the storage team, the operations team - into a tiger team that reported to the VP of engineering.
No guest appears in this episode. Lucas and Luna are hosts discussing GitHub's publicly reported migration; neither identifies as a GitHub engineer or insider with direct involvement in GitV2. This is a secondhand retelling of published work, not a practitioner interview.
If these conversations are useful for what you're building or running, you might want to hear this one. GitHub quietly finished migrating over 100 million repositories to a brand-new storage engine they built from scratch.
Strong specificity on metrics (100M repos, 40% clone speedup, 30% disk savings, 10 - 15% shallow clone improvement, tens of thousands file descriptors, 18-month hard wall projection, 20-month rollout, Git 2.40 upstream integration). Named technologies (RocksDB, bloom filters, prefix compression, partial clones). Some gaps: no discussion of cost impact in absolute terms, no named engineer leads, no GitHub blog post links provided for listener verification.
They had cases where a single `git clone` would open tens of thousands of file descriptors
a 40% reduction...For shallow clones, the improvement was smaller - maybe 10 - 15%
Lucas and Luna maintain good pacing and occasionally ask clarifying follow-ups ('what was the actual bottleneck?' 'how did they migrate without downtime?'). However, most questions receive full answers without meaningful pushback. No one challenges assumed trade-offs (consistency, latency variance, operational complexity post-migration). A mid-conversation ad read breaks momentum. Follow-ups tend toward 'what happened next?' rather than 'why that choice over alternatives?'
Luna: So what was the actual bottleneck? Inodes? Seek times?
Luna: What was the most common failure mode?
Computed from the transcript - who did the talking, and the words that came up most.
In this episode of The CTO Podcast, Lucas and Luna dive into the technical and organizational challenges behind GitHub's migration of over 100 million repositories to a new Git storage engine. They explore why GitHub needed to move away from the traditional filesystem-based storage, the design of their new custom engine called 'GitV2,' and how they maintained zero downtime during the multi-year rollout. Lucas shares specifics on the data structures used, the migration strategy (shard-by-shard with canary testing), and the performance gains achieved: 40% faster clone times and 30% reduction in storage footprint. Luna asks about the trade-offs: how did they handle edge cases like partial clones or large monorepos? The conversation also touches on the cultural side - getting engineering buy-in for a rewrite that touched every team, and the metrics that mattered most. A must-listen for anyone leading large-scale infrastructure changes.
Transcribed and scored by The B2B Podcast Index.
Lucas: If these conversations are useful for what you're building or running, you might want to hear this one. GitHub quietly finished migrating over 100 million repositories to a brand-new storage engine they built from scratch. They call it GitV2. Luna: GitV2 - and this isn't just a new file system layer, right?
This is replacing the heart of how Git stores objects. Lucas: Exactly. Git's original storage model is surprisingly simple: each object - commit, tree, blob - is stored as a separate file, compressed individually, with loose objects eventually packed into packfiles. For a single developer repo, that's fine.
But GitHub runs about a hundred million repos, many with years of history, large binaries, and massive monorepos. The filesystem approach breaks down at that scale. Luna: So what was the actual bottleneck? Inodes?
Seek times? Lucas: Both. Plus the overhead of managing millions of tiny files across a distributed file system. GitHub's engineering team published some numbers: they had cases where a single `git clone` would open tens of thousands of file descriptors, and the metadata operations - stat, readdir - were consuming more CPU than the actual data transfer.
They needed something that treated Git objects as a database, not as files. Luna: And that's where GitV2 comes in. What's the core data structure? Lucas: It's a content-addressed store built on top of RocksDB, which is an embedded key-value store from Facebook.
Each Git object hash maps directly to a key, and the value is the object content plus some metadata. But the interesting part is how they handle packfiles: instead of storing each object individually, they keep the packfile as a single blob, and then use RocksDB's built-in bloom filters and prefix compression to make lookups fast. They also added a custom caching layer for the most frequently accessed objects - think HEAD commits, tree objects for popular branches. Luna: So the old system was filesystem-bound, and the new one is essentially a distributed database with a Git frontend.
How did they migrate 100 million repos without downtime? Lucas: That's the really impressive part. They did a shard-by-shard migration - each shard covers a range of repository IDs. They built a shadow read system: every write went to both the old storage and GitV2, but reads first tried the new engine, and if it didn't have the data, fell back to the old one.
Over months, they enabled GitV2 for reads on one shard at a time, monitoring latency and error rates. If something went wrong, they could flip the read back instantly. Luna: Right, so canary testing at the shard level. What was the most common failure mode?
Lucas: Cache misses. Some repos have extremely specific access patterns - like a CI system that clones the same commit hundreds of times per minute. The old filesystem cached those objects in the OS page cache, but GitV2 had its own cache that wasn't always warm. They had to fine-tune the cache sizing and pre-warm it for high-traffic repos.
Another issue was partial clones - where you only fetch a subset of objects. GitV2's packfile representation wasn't optimized for that initially, so they had to add a new index structure to quickly find which objects belong to which path. Luna: Makes sense. And the performance gains?
I read somewhere clone times dropped by 40%. Lucas: Yeah, for full clones of repos with deep history, they saw about a 40% reduction. For shallow clones, the improvement was smaller - maybe 10 - 15% - because the overhead is mostly in the network, not the storage. But the storage savings were huge: a 30% reduction in disk footprint, mainly due to better deduplication and compression in RocksDB.
They now store a single copy of an object even if it appears in multiple repos, which the filesystem couldn't do easily. Luna: That's a big win for GitHub's infrastructure costs. But I'm curious about the engineering org side - how did they convince the team to undertake a multi-year rewrite of the storage layer? That's a bet the company kind of project.
Lucas: And we should say, this kind of deep infrastructure work is exactly why we make these conversations. We deliberately don't run ads on these episodes - not a single one. If you want to support that choice, the link is buy me a coffee dot com slash fexingo. No pressure, just if the show is valuable to you.
Luna: Yeah, it's a small way to keep this independent and ad-free. So back to GitV2 - how did they get buy-in? Lucas: They started with data. The engineering team ran benchmarks that showed the filesystem approach would hit a hard wall within 18 months - not in terms of total storage, but in terms of metadata operations per second.
They projected that by late 2025, a routine `git push` could take over a minute due to filesystem contention. That gave the leadership a clear urgency. Then they built a prototype that handled 10,000 repos with the new engine and showed it was faster and more reliable. That convinced the core infrastructure teams.
Luna: So it was a data-driven case, not a 'let's rewrite because it's fun' kind of thing. And the rollout took how long? Lucas: About 20 months from first prototype to full migration. But the interesting organizational detail is that they didn't create a separate GitV2 team.
Instead, they embedded engineers from each of the major teams - the git team, the storage team, the operations team - into a tiger team that reported to the VP of engineering. That way, when they needed changes in, say, the replication layer, the person who owned that code was already in the room. Luna: That's smart - avoids the 'us vs. them' dynamic that plagues cross-team rewrites.
So what's next for GitV2? Are they going to open-source it? Lucas: GitHub hasn't announced plans to open-source GitV2 itself. But they've published a few blog posts on the design, and some of the ideas are already influencing Git's upstream development.
For example, the concept of a 'commit graph' that speeds up reachability checks - Git 2.40 introduced a similar feature based on what GitHub learned. So the community benefits indirectly. Luna: One last thing - how does this affect the developer experience for the average GitHub user?
Do they notice anything different? Lucas: If you're cloning a repo today, it's faster - especially for large repos like the Linux kernel or React Native. Also, 'git log' and 'git blame' operations that traverse history are noticeably snappier because the new engine caches commit graphs. But the goal was transparency: you shouldn't have to change your workflow.
The best infrastructure migrations are the ones users never see. Luna: Right. And that's the mark of a successful rewrite - when it just works, and the only difference is speed. Thanks, Lucas.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.
How Dropbox Rebuilt Its Sync Engine for 700 Million Users85 / 100How Skyscanner Migrated 300 Microservices to Event-Driven Architecture85 / 100How Palantir Rebuilt Its Foundry Ontology for Government AI Deployments85 / 100How Netflix Rebuilt Its Content Delivery for 300 Million Subscribers85 / 100How Coinbase Rebuilt Its Blockchain Node Infrastructure72 / 100
