The TWIML AI Podcast

The episode contains multiple substantive research contributions and novel framings: tokenomics as an economics problem, the 'tokenflation' concept, consumer price index applied to token usage, architectural efficiency critiques of transformers, and nuanced analysis of AI fluency and expertise. However, significant portions consist of background discussion (linguistics, swearing, DSPy history) that, while contextual, dilute the density of novel operational insights per minute.

“what does it mean to think about value in this context? Even if we focus in on people who are doing just coding with coding agents, can we agree on what it means to add value?”

“the purchasing power of the tokens in those time periods...tokens divided by goods produced is a pretty rough measure of um, the purchasing power of the tokens in those time periods.”

The tokenomics and CPI framework applied to token usage is relatively fresh thinking, and the distinction between model improvements vs. product improvements (system-level changes) is counterintuitive. However, the core architectural critiques (attention scaling inefficiencies, sparse MLPs, positional encodings) rehash known optimizations from 2017-2024. The AI fluency framework builds on Anthropic's prior work. Not deeply contrarian, but genuinely thoughtful application of economics to an underexplored problem.

“Your token is not buying you what it once did. According to everything we can think to measure here and even adjusting for models getting better.”

“even for a fixed model we could get very different outcomes for these things because they really are sophisticated engineered systems at this point.”

Chris Potts is a Stanford professor with substantial AI/NLP credentials, a BigSpin co-founder with operational involvement in product deployment, published research on tokenomics and AI fluency, and deep knowledge spanning linguistics, interpretability, and systems thinking. He has genuine practitioner experience building AI products, not just theoretical credentials. This is a high-caliber guest with real skin in the game.

“Stanford professor and BigSpin co founder Chris Potts”

“one person who's been thinking deeply about this”

The episode provides specific data points: a $20-to-$500+ billing shock example, the SweChat benchmark with ~6,000 real coding sessions, February-to-mid-April timeframe for Opus 4.6 analysis, a four-day code survival metric, and CPI calculations with hedonic adjustments. However, many claims lack precision: true token costs estimated at 2-20x range without resolution, no specific numbers on data poisoning attack scale, and limited concrete examples of architectural innovations beyond byte-level models and Julie Colini's work.

“I saw a tweet from Ed Zitron, just a screenshot from someone who was noticing that copilot was telling them that their bill last month was $500. And if they keep up the way they are with copilot's new billing, it will be $11,000 in the next month.”

“which is this switchat benchmark, which was released by researchers at Stanford, it's about 6,000 real coding sessions, all the metadata, everything you'd want.”

The host asks sharp, clarifying questions (e.g., on system vs. model improvements, on architectural diversity vs. bitter lesson scaling) and follows up productively when Potts suggests consistency across models by pressing on measurement and variation. However, the conversation often lets claims stand without pressure - no challenge on the four-day survival metric validity, limited pushback on the tight linkage between fluency and success, and extended tangents on swearing and background that don't drive toward operator insights. Good but not exceptional follow-up discipline.

“But one framework we could offer that we did in the research you alluded to is let's think about this like economists might.”

“And all of these fall victim to the standard thing that once you make it a metric, it's no longer useful to you.”