Inside vLLM: The Engine Powering Open-Source AI - Large language models - AI + a16z

Topics in this episode

Large language models Artificial intelligence vLLM LLM open weight

Episode notes

Elena Burger and Matt Bornstein are joined by Simon Mo, co-founder and CEO of Inferact, the open-source inference engine powering many of today's most advanced AI applications. Together, they explore how open-source AI evolved from a research project into critical infrastructure, why inference has become one of the most important layers of the AI stack, and what it takes to bring frontier intelligence to developers around the world. The conversation covers vLLM's origins, the rise of open-weight models, why companies increasingly want control over their AI infrastructure, and how open-source inference enables the next generation of AI applications. They also discuss model licensing, the economics of open-weight AI, Kimi K3, distillation, AI infrastructure, and why Simon believes the gap between open and closed models is rapidly disappearing. Resources: Follow Simon Mo on X: Follow Matt Bornstein on X: Follow Elena Burger on X: Follow Inferact: Check out everything a16z is doing with artificial intelligence here , including articles, projects, and more podcasts. Hosted by Simplecast, an AdsWizz company.

Related episodes across the Index

Other episodes covering the same guests and topics, from across The B2B Podcast Index.

More from AI + a16z

All episodes →

Explore the best B2B AI & Data podcasts →