The TWIML AI Podcast

The episode delivers genuinely non-obvious ideas at a solid pace - particularly the framing that structured relational data has not undergone the same raw-data-learning transformation as vision and NLP, and the mechanics of in-context learning over graph-structured databases. Some sections repeat points or meander (the biomedical detour at the top, occasional circular re-explanations), keeping it short of elite density.

“AI has not transformed the structured data space in the same way as, uh, computer vision or natural language understanding have been fundamentally transformed by AI”

“We don't learn on raw data. We run all these SQL queries, all this etl, all this feature engineering to then come up with a set of signals from which we, let's say, try to predict something.”

The core argument - that multi-table relational structure is the missing link where real ML gains are hiding, and that in-context learning can be applied to structured databases via graph encoding - is a genuinely fresh framing not commonly circulating in standard ML discourse. The churn feature-engineering reductio ad absurdum and the agent-friendly API observation are memorable and non-recycled, though the broader foundation-model narrative follows familiar patterns.

“Just attend over the transactions and let the attention figure out what predicts the churn.”

“single table problems are, you know, are Solved. I think the differences are kind of second order effects. What is unsolved is the multi table problem.”

Leskovec is simultaneously a Stanford CS faculty member running active research (Railbench, relational graph transformer, AI virtual cell) and a co-founder with demonstrable production deployments at Doordash, Reddit, Coinbase, Expedia, Databricks, and Snowflake - a rare combination of academic depth and at-scale practitioner credibility. He is genuinely the person who built the thing being discussed.

“at Doordash it's uh, um, restaurant recommendations and the notification system...We've seen, you know, uh, revenue impacting hundreds of millions of dollars.”

“we have these models running in production at Coinbase on the entire Bitcoin blockchain network”

The episode is anchored by real named customers (Doordash, Reddit, Coinbase, Databricks, Snowflake, Expedia), concrete benchmark details (Railbench: ~40 tasks over 10-15 databases; SAP Salt), measured performance lifts (5% relative from zero-shot foundation model, 12% with fine-tuning), and a highly specific debugging anecdote about an agent aggregating data to midnight rather than current time. A few impact claims ("hundreds of millions") are unattributed and unverifiable, which caps the score.

“the foundation model improves that I think for about 5% relative uh, the accuracy. Um, and then if you further tune the model...then the performance goes to 12% uh, over the State of the art”

“when it created features for that given account, it aggregated the transactions till midnight, not till the current time”

Charrington earns credit for explicitly calling out an outlandish claim and pressing on it, asking whether Reddit's hand-engineered features are a "feel good thing," and the "no free lunch" challenge about compute costs. He also makes a productive structural distinction between the system and the model during the in-context learning discussion. However, he doesn't follow through on several openings - the "hundreds of millions" at Doordash goes unchallenged, and some follow-ups are leading rather than genuinely probing.

“I find that proposition to be almost outlandish. Like they're just numbers with some unknown relationship.”

“have you looked at like if their manual features really make a difference, like is that a feel good thing”