Behind the Craft

The episode contains substantive technical insights about agent architecture (skills-based routing, dynamic loading, LLM-as-judge evals), but relies heavily on product demos and conversational padding that dilutes density. The core ideas about minimizing instructions, avoiding context over-loading, and using native agents to encode values are valuable but discussed somewhat repetitively.

“Give it as little instruction as possible. Give it the tools to load context. Don't give it context.”

“The biggest problem is not the agents aren't smart enough. The problem is the models are really smart, but we're just not using them enough.”

The thinking is sound but not particularly contrarian or first-principles. The skills-based architecture and routing approach are reasonable engineering solutions rather than novel insights. The observation about entry points (Slack, meetings) versus chat interfaces is practical but not groundbreaking. Much of the framing echoes existing agent design discourse.

“We kind of ended up on a uh, skills type of setup where we give it the ability a tool essentially to load skills and then it, based on the request, it'll just load up the different skills that it needs.”

“AI is um, not an end to end solution, it's a middle to middle solution.”

Nan Yu and Jacob Shumway are Linear's co-founders/core engineers who built the agent end-to-end and shipped it to production at scale. They have direct operational authority, real stakes in the product, and can speak to both engineering decisions and product philosophy from authentic experience. This is high-caliber guest material.

“I was working with Jacob the other day. I'm like, I dug this up. This is a memo I wrote.”

“The first version was really prototypy. It was like we were calling the LLM from the front end directly and it was just flagged internal.”

Good use of concrete examples: the Slack demo showing an issue creation in 6 minutes with a PR, the skills-based routing system, specific evals (e.g., ensuring 'in progress' status), the mention of French Intercom ticket translation automation, and the user feedback loop. However, missing quantitative metrics on performance, error rates, adoption numbers, or latency data that would strengthen the evidence.

“Linear made the issue took six minutes, and then it gave us a, uh, PR that we clicked through.”

“if the user says in progress, make sure it always adds the status in progress. That's really deterministic.”

The host asks solid follow-up questions (evals structure, model mixing, shipping readiness) and occasionally pushes back (e.g., on LLM-as-judge limitations). However, the conversation is often too agreeable and doesn't deeply challenge claims. Several moments lack sharp follow-ups (e.g., no hard questions on hallucination rates, failure modes, or why this beats alternatives). The demo section runs long without critical interrogation.

“How do you evaluate? Like, is that like a yes, no thing or is it like a scoring.”

“It's kind of funny, like, it's kind of. Because you probably get a bunch of feedback about the agent and then you probably have something the agent ingests the feedback and synthesizes.”