Future Ready Lawyer

The episode offers several substantive ideas about AI's role in legal systems and some concrete details about Bluejay's performance metrics (90% to <0.1% disagreement rates over 3 years, 10M+ processed questions), but much of the discussion retreads familiar concepts: the access-to-justice problem, AI improving accuracy, the need for human oversight. The 'super justice' framework is more evocative than novel, and significant portions consist of philosophical musing about education, identity, and the future of lawyering that lack actionable specificity.

“close to two orders of magnitude of improvement in the accuracy of the system. So if back in mid-2023, leveraging GPT, it would have been early versions of GPT4, um, we'd see like 90% accuracy in the tax research answers coming through”

“we've processed more than 10 million tax research questions. It took us nearly two years to get to our first million tax research questions. The most recent million questions have taken under a month.”

While the 'super justice' branding is catchy, the underlying thesis - that AI can make legal services abundant, more accessible, and address scarcity in dispute resolution - is not novel. The public roads analogy is illustrative but not original. The guest does push back against conventional wisdom about AI lacking empathy, citing blind-test evidence with chatbots vs. doctors, which is useful, but the overall framing closely mirrors existing techno-optimist takes on legal AI without offering genuinely counterintuitive or first-principles arguments.

“justice can be much more like Public roadways. I don't think many of us wake up in the morning, hop into a vehicle if we're commuting to an office or to work”

“I think I disagree with people who would make the claim that AI is incapable of exhibiting empathy or exhibiting an understanding of how people are going to take things.”

Ben Alarie is a credible and relevant guest: law professor at University of Toronto, co-author of a book on AI and law, and - most importantly - founder and CEO of Bluejay, a production AI system used by 6,000+ companies and backed by $122M Series D. He has skin in the game and real operational experience building AI for legal work, not merely theoretical commentary. However, he is primarily an academic and entrepreneur in legal tech, not a practicing litigator or in-house counsel, which limits his perspective on how front-line operators would use these systems.

“Professor Ben Ellery Osler Chair in Business Law and at the University of Toronto”

“co founder, uh, and CEO of Bluejay, a University of Toronto spin out firm M which he launched with law professors now being used by over 6,000 companies”

The episode includes some concrete data points: Bluejay's accuracy improvement (90% → <0.1% disagreement over 3 years), 10M+ processed tax research questions, acceleration from ~2 years to 1 month per million questions, $122M Series D funding, 6,000+ user companies. However, these metrics are sparse and mostly relate to Bluejay's internal performance rather than real-world legal outcomes. The 'super justice' system is described largely in abstract, hypothetical terms (the neighbor dispute example). Few named case studies, client testimonials, or specific legal domains beyond tax are detailed.

“we'd see like 90% accuracy in the tax research answers coming through”

“we are now seeing disagreement rates in the platform of less than a tenth of a percent”

Hosts Mark and Armin ask generally solid opening questions ('What is Super Justice?', 'What are the underlying technologies?') and follow up on human involvement and education. However, they rarely push back or challenge claims directly. When Alarie makes broad assertions (e.g., AI can exhibit empathy, systems will naturally remain human-centered), the hosts accept these largely without skepticism or pointed counterargument. There is one instance of productive tension - Armin raises the concern about hallucination and whether AI can truly exercise discretion - but the guest's response goes mostly unchallenged. The interview is conversational but feels more like a platform for the guest's vision than a rigorous examination of it.

“you mentioned in the introduction, Mark, that this book was co authored with Professor Samuel Becker, who's a dear friend. Samuel and I agree on a lot of things. We don't agree on everything”

“if the systems, if the underlying technology becomes so powerful and those guardrails are action, the system is going to kind of, we can't predict precisely what will happen”