Token Maxxing and Local Inference: Navigating the Shift in AI Usage for Developers

Episode notes

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss token spending and the "token maxing" backlash, sparked by a GitHub Copilot budget issue and broader industry moves to optimize tokens. The hosts argues companies should focus on value and learning rather than premature token optimization, while still adding circuit breakers to prevent runaway costs. They explore shifting to local inference and open-weight/open-source models for cost, privacy, and control, citing concerns about vendor policy changes like Fable's retention and classifier behavior. The conversation expands into how AI shifts bottlenecks from coding to product management, experiments pairing product and engineers using SpecKit, and how AI may realize Agile's iterative promise. They close on distribution as a key frontier and the career opportunity of being early in AI, likening it to the early internet era.

• The Pace of Innovation: Systems Thinking for Faster Engineering78 / 100
• Introducing Skein and Beaker Stack: Using AI Agent Engineers to Ship SaaS Faster67 / 100
• Method to the Madness: A CTO Framework for Systemic AI Adoption
• There's A Pattern To Follow: An Interview with Robert "Uncle Bob" Martin
• Is Uncle Bob Right? Reading Code vs. Architecting the Future