Security Cryptography Whatever

The episode is densely packed with concrete technical insights about LLM capabilities in vulnerability discovery - oracle design, constraint propagation in fuzzing, checksum generation, protocol-aware exploitation, and the asymmetry between finding and patching bugs. However, substantial portions involve methodological explanation and clarification that, while necessary, dilutes pure insight density.

“We have, I don't know, let's say, 10-line Bash script plus Docker container. I just sort of point it at the thing and be like, I've compiled this program with ASan. Please run against it, read the source code, and try to find a bug.”

“The thing that we've been finding most recently is you don't really have to try very hard.”

The work itself is original - demonstrating that minimal prompting (no special harnesses, no domain-specific scaffolding) on production models can find real zero-days at scale is genuinely novel. However, the framing around LLM capability improvement and the observation that 'scaling works' is now relatively well-trodden territory; the originality lies in the concrete execution and measurement, not the conceptual framework.

“the models have gotten good enough that you can do security, meaning use the models to help you with security”

“You don't really have to put in a huge amount of work, which is both good and bad.”

Nicholas Carlini is among the most credible guests possible for this topic - he has published extensively on adversarial ML and security, led this exact research at Anthropic, and demonstrated it in practice at scale. He speaks with hard-earned authority grounded in months of real-world experimentation, not abstract theorizing. Few people on earth have done this work at this depth.

“I used to do pen testing stuff and this was like the thing that got me into security in large part”

“I have walked through the trace of myself. I spun up the thing because I'm still very paranoid the model is just going to lie to me. I don't want to be the person generating AI slop.”

The episode is rich with named examples (Ghost CMS, Firefox, FFmpeg, Linux kernel NFS daemon) and specific metrics (122 crashing inputs to Mozilla with 100% true positive rate, 22 CVEs, 500 zero-days from OSS-Fuzz). However, some claims lack hard numbers - exact iteration counts vary ('20 times or something,' 'maybe 5 or 10 times'), token costs are dismissed rather than quantified, and generalization curves are acknowledged as unmeasured ('I don't have a very nice scientific plot yet').

“we sent Mozilla 122 crashing inputs and like they confirmed all of these are bugs. Like 100% of the things we crashed them, perfect true positive rate, all bugs”

“It found a SQL injection that goes, nice, an unauthenticated user who has literally no perms, who can compromise the admin database, mint themself a new admin account”

The hosts ask solid, probing follow-up questions (why stop iterating? how many passes before moving on? what changed between 4.5 and 4.6?) and push back on claims (asking about the generalizability of the approach, comparing to prior work). However, they occasionally let Carlini drift into lengthy explanations without interrupting for clarification, and some conversational threads peter out ('I thought I had more to say about that'). The dynamic is warm but could be tighter.

“I'm stuck on like, so first of all, I said there is like, there were some screenshots of the presentation”

“Do you have your own homegrown spidey sense of being a vulnerability researcher about which files you're like, all right, you've given it a pass or two, but like, I really want you to do like 5 or 10 passes”