Detection at Scale

A handful of genuinely useful ideas surface - writing detections as criteria for agents rather than step-by-step instructions, the Jevons paradox applied to SOC capacity, and the 'closing the loop' vs. 'closing the alert' distinction - but these are diluted by a lengthy origin-story segment, repeated AI-hype validation loops, and vague macro-level claims that add little for practitioners.

“I'm not telling the model how to do those things, I'm just saying, hey, if these things are true, then I would gauge this as risky. And then what the model can do is it can use all the tools and data and context it has available to it”

“it's kind of like Jevons paradox in a way, where it's like when you have a capability that could replace something, you're doing more of it”

The reframing of detection engineering for agent consumption (criteria-based vs. deterministic runbooks) and the 'closing the loop' concept are fresh and worth hearing; however, the bulk of the episode rehashes standard 2024-2025 AI adoption narratives - skepticism-to-trust arc, cloud-wave analogy, alert fatigue - that circulate widely in security podcasts.

“when you're writing for agents... you go from, um, hey, check 1, 2, 3, these exact steps to, well, here's the criteria for what I would gauge something as bad”

“And then on the other side of that you have the security platforms getting into SIM and data pipeline...they have to evolve, everyone has to”

Jack carries real practitioner credibility - Yahoo incident response at scale, early Airbnb security hire, co-creator of StreamAlert - which grounds the conversation; however, this is structurally an internal marketing interview where the CEO is questioned by his own CPO about their own product, limiting independent external perspective.

“I'm actually going to be interviewed today by my colleague Julian. Julian is the Chief Product Officer at Panther, which is the company that, um, I'm the CEO of”

“I joined as one of the first hires on the incident response team. And my mandate was very similar to at Yahoo”

Named technologies (StreamAlert, KQL, MCP, BigQuery, Elastic, Snowflake) and concrete architectural choices (Python detection-as-code, cloud data warehouse as centre of gravity) provide some substance, but virtually no hard customer metrics, dollar outcomes, or independently verifiable data points are offered, and the one stat cited is openly vague.

“tens of millions of dollars and you have two months of retention”

“a hundred lines of a prompt that helps it understand what's different about the data that's in Panther”

Julian asks a few well-framed historical questions ('what assumptions were you running on two years ago?') and occasionally presses on timelines, but the format - CPO interviewing his own CEO - produces consistent agreement, sentence-finishing, and zero productive pushback on any product or market claim throughout the episode.

“What assumptions were you running on? What were your hopes and dreams from two years ago and versus where we are, what's materialized and what's misaligned”

“So it's going from like hesitancy, this, this bolt on feature that can like accelerate a little bit to oh, this is taking the manual work out of the equation for me”