The MAD Podcast with Matt Turck

The middle third of the episode contains genuinely useful technical content - hybrid SSM/Transformer ratios, latent MOE's 4x expert expansion, and the counter-intuitive quality-speed relationship in multi-token prediction. However, the first 20+ minutes are diplomacy-heavy open-source vs closed-source platitudes, and the organizational/safety sections trend toward abstraction. A solid but uneven density.

“with multi token prediction the speed that you get is a function of the accuracy of your model, the more accurate your model is, the faster the inference is, the cheaper the inference is, the more accurate it is. That's not usually how it works”

“using both of these together was actually better than using either one on their own. Um and that is independent of the speed benefit.”

There are a handful of fresh framings - the kitchen/external stomach analogy for AI, a genuine singularity skepticism grounded in multifaceted intelligence, and the open-source-as-safer-because-of-sunlight argument - but the broader narrative follows the well-worn open source advocacy script and the organizational culture observations are standard startup-at-scale lore.

“we, we have an external stomach, we call it a kitchen. Now we're creating an external brain.”

“the singularity is. Although it's an attractive idea, I think that it's really a wrong headed idea because it doesn't really um, take into account these other factors”

Catanzaro is a genuine first-principles practitioner: co-created CuDNN, launched the Megatron project pre-transformer-hype in 2017, worked at Baidu Silicon Valley AI Lab alongside Andrew Ng and Dario Amodei, and now leads Nvidia's frontier model research. He speaks from lived engineering experience, not thought-leader positioning.

“I published my first paper, Training Models on the GPU. And people asked me why I was there. People said, this is not a good paper for icml. We just do fancy math here.”

“when Andrew Ng asked me to go uh, build the Silicon Valley AI lab with him, uh at Baidu, I thought oh, this is a great opportunity”

The episode provides useful concrete details - active vs. total parameter counts (3B/12B/55B), 4-bit NVFP4 pre-training, 1M token context, 10-15 teacher models, 23-of-24 DLSS pixels - but is conspicuously absent of benchmark comparisons, comparative latency numbers, or hard evidence for the 'best open weights model' claim. Numbers are architectural, not evaluative.

“nano is a 30 billion um ah total 3 billion active parameter model supers 120 and 12 and ultra is 550 and 55”

“23 out of every 24 pixels is uh, being generated by our AI model. When you're using DLSS”

Turck lands two genuinely probing moments - pushing on the distillation-from-closed-models concern and the China copycat narrative - and asks clarifying technical questions that unlock real content. But he consistently accepts marketing framings without challenge (e.g., 'immediately became the best open weights model') and the safety discussion is allowed to close on an uncontested, sweeping claim.

“To ask maybe a slightly cynical question, there is at least a part of the community that's wondering whether open source as an ecosystem, not Nvidia, but uh, in general has been progressing in part based on the ability to distill closed source models”

“Is what you just described, uh, called latent moe or is that a different concept?”