Weathering the AI Vulnerability Storm with Gadi Evron, Rob Lee and Ed Skoudis

Welcome everyone to another special edition of the Sands Cyberleaders podcast. I'm James Lyne, and I hope you're here listening because you've recently enjoyed our episode on a round with the great Tim Conway, another topical example of cyberleaders. But I'm sitting here in rural England, and a few miles away, there's a train station that connects straight to London. And like that city's famous bus system, you wait ages for one and then two come along at once.

And things are so frenzied in the cybersecurity world right now. Here is our second special edition of the Cyberleaders Podcast. That's right, James. I'm Kieran Martin.

Welcome to the show. Thanks for joining. So, what's brought us to this point? Well, this episode is about what is it about?

That's a good question. Is it about Mythos Preview, the new model from Anthropic they judge too dangerous on cybersecurity grounds to release? Or is it about AI tools as a whole? After all, OpenAI came out with their own announcement about their quite different approach to their new model security.

And warnings about the hacking capabilities of AI models are not entirely new. But this warning is a lot louder and has, as they say in media and political circles, its cut through. I think Kieran, it is all ultimately about the rise of Skynet. But anyway, look, regular listeners will know, of course, always.

I start normal episodes on this podcast with the announcement that I've been, well, hacking and breaking things since as long as I can remember. And according to some interpretations of the last month's developments, now AI is gonna do all of that for me. I can sit on a beach and sip a cocktail. So before I have to come up with a new introduction, let's set out what's happened, why it matters, and what it means for the cyber defense community.

So let's start quickly with what's happened. On the 8th of April, Anthropic, the AI giant that runs the series of Clawed LLM capabilities, announced that its new model, Claude Mythos, was going to be delayed. Why? Not because of some engineering challenges or some excuses.

Well, basically because it was too good at hacking. Now, you have to grossly oversimplify these things to summarize them, I admit. But basically they said two things. One was that it was really good at finding vulnerabilities.

Zero day, scary stuff that would be incredibly useful to attackers. Stuff that no one has ever found, or at least never reported, on a scale we haven't seen before. In the flagship example they reported in their press release, they said they'd found one vulnerability in core open source internet infrastructure that was 27 years old. It's actually probably older than some of the analysts that are going to be tasked with deploying patches.

And as a result, the time taken from discovery of vulnerability to exploitation is going to collapse even further. It's already gone from 28 months to 28 hours in the last eight years. And now we're heading down to minutes. You've basically already said it.

I'm very impressed. Nope, I've l I've lost it already. Have I? What was the other bit?

The bit about it being really good at hacking. That's it. And I suppose in some ways, and perhaps this is surprising. The bit received less attention, but to cyber defenders out there, it really matters.

Essentially, you know, Mythos preview was very good at the full attack chain, getting through to actual exploitation. You know, whether you're using zero days or all the existing weaknesses we already know about, but people don't always defend, being able to leverage them and quickly with potentially less expertise at scale is a big deal. So to cut a long story short, for those two reasons, anthropic delayed. To do what?

Maybe they were trying to do what I normally do when I've got a problem, and that's to ignore it long enough in the hope that it will go away. If only Kieran. No, no, they announced something called Project Glasswing. That's a partnership based on privileged access to a few dozen big American companies, tech giants, cybersecurity companies, banks, you name it, to try and work on mitigations.

And they do this with privileged access to the model before it comes out. So, James, is everybody else going to do this? Well, sort of, but not quite. I mean, OpenAI have said that they will also delay their next model, but they want a much more open partnership based on a trusted accreditation model, which I'm sure we'll get into with our guests shortly.

And of course, the new Chinese models will inevitably come in. I mean, history suggests they'll be very powerful, and we've no idea how they'll approach that release necessarily. So what does it all mean? Well, it dropped in an unsuspecting world at the height of one of the most serious wars in recent history.

So it's a lot to take in. But as people across the world have started to take it in, there are some points of consensus and some ongoing points of debate that are emerging. First, this is a big deal. Now, James, when I was writing this up, I hit the wrong key on my keyboard.

And I promise you I'm not making this up. You know me well enough. I don't have that imagination. But I typed this is a bug deal.

I and you are beside each other in the keyboard. I thought about leaving that. So anyway, it's a big bug deal. A new t-shirt, and maybe that's going to be the title of our podcast today.

Very punny. But hey, look, get to the point, my subordinate. I I mean friend. Haha.

Thank you. Yes, boss, it's a big bug deal. It's a serious acceleration of something that was already on our radar, the hacking power of new AI models. Now, how much of a transformation is it really?

That's hard to know. As you've said, James, for understandable reasons, anthropic have kept access to the model on a privileged basis. So when the news first came out, we only had the analysis they did with their partners to go on. There's only been one independent assessment of Mythos Preview based on having actual access to the model.

And you and I, James, I think we can take a bit of pride here, because it was done by the UK's very own AI Security Institute. That's a remarkably good bit of progressive government there. Building a government agency capable of winning the trust of an AI giant based in another country to do an independent analysis of their tool and publishing it within a week. Well, frankly, I'm flabbergasted, Kieran.

Well, well done them. And I'll gently suggest they've set a standard for the rest of government and hopefully for other governments too. But the AI SI work was a markedly perhaps calmer assessment than some of the more headline-grabbing stuff reported when Anthropic's release came out. It focused, I think, on the second part of your two points, the general hacking capability rather than the discovery of zero-day vulnerabilities.

And it concluded that Mythos Preview is really, really good at hacking. And it was the first AI model to fully complete its full range of attack tasks across the whole chain. But it said it had some shortcomings. And if you look at their charts, it was once again a notable acceleration of something we've already known about.

For a while, I'm struck by what Heather Atkins, that brilliant SISU at Google, has said about the clock started ticking months ago. And they also caveated by saying that no one, including them, has really tested Mythos preview against actual cyber defences. That's right, Kieran. So look, what to make of it all?

I mean, clearly this is a big bug deal. Definitely a t-fit we're making. But there's different, you know, versions of this. And, you know, some are saying it changes everything.

Some are saying it's marketing hype and a good way of trying to cover up for recent issues over Anthropic on the security side. And few have lost money in our industry by predicting the apocalypse. It's certainly been an attractive habit for marketers over my 20 odd years. And then some are even saying it's actually a wonderful opportunity to fix some of the serious bugs in the internet and more broadly the technology we're allowing every day, we've never been able to fix before.

So we've got everything from dire straits through to wonderful optimism and apocalypse mixed in the middle. Some are saying it's a combination of all of these. Well, what does it mean for cyber defenders for this community? And that's where we're here to help.

Indeed. Well, look, folks, we sometimes talk about cyber defense as a community. And that's a real thing. It's who this podcast is aimed at.

And key parts of the community have raced into action rapidly. Experts from all sorts of different backgrounds with different perspectives arguing through and trying to figure out what really matters here or the line I use very often, what works and what doesn't. And they came up with a paper that was published jointly by us here at SANS and our wonderful friends at the Cloud Security Alliance, who let us into the party to provide some expertise. And it gives a balanced, informative, and actionable account as we could make of this remarkable development.

Yes. Thank you, everyone involved in that. And that brings us to our guests finally. Now it's a new format today, so we had to set out the scene.

And we have three guests, and all of them are involved in this report, the AI Vulnerability Storm. We're hugely grateful to them. Now let's introduce them. One is our very own Rob Lee, chief AI officer and chief of research at SAMS.

Welcome back to the show, Rob. Hey, thank you for having me here. And we're also extremely privileged to have Gaddy Evron, founder and CEO at the AI agent security company Gnostic and CISO in residence for the Cloud Security Alliance, who led the charge on this wonderful paper, and well indeed, lead author of the paper. So welcome, Gaddy.

Thanks for making some time for us today. Thank you. And I appreciate the accolades, although honestly, I just herded cats. It was everybody, truly it.

250 people working together over a weekend. I am happy to take credit for the herding of the cats, but not for the whole paper. I'm excited to hear how you might have accelerated cat herding with Mythos, but that's a question for later in the podcast, isn't it, Kieran? Indeed.

And speaking as a cat, last but not least, it's great to welcome Ed Scootis of Sands. Now, when you mention Ed when discussing penetration testing or incident response, everybody knows who you're talking about. So for the first, but not the last, I hope, time on the show, let's welcome Ed Scootis. Ed, hello.

Hello, thank you. I'm honored to be here, looking forward to this discussion. Fantastic. Well, look, let's get into this.

Kira and I have done some scene setting, but it's time to get to the experts. So, first of the two things I highlighted in the introduction, that the Mythos preview, it's very good at finding bugs that we don't currently know about. That seems to me to be pretty proven at this point, and a theme that frankly exists even beyond frontier models. But what are the implications of that?

Gaddy, why don't you start us off on this? I believe the first implication really is about the model itself. This is now the Mythos problem, a big bug Mythos problem, because Mythos is what people know. That's what broke through, breached, if I'm to use that term lightly, the New York Times and the CNN barrier.

So the problem itself has been around for a long time. You don't need Mythos to be able to find bugs or exploit bugs. It is indeed more capable. It didn't do a lot more than previous models, but you can do a ton with what we already had.

So now this is the Mythos problem, because this is how it's now known. This is how we discovered it. As to the implications, I believe that we'll get into that throughout today. But the most important thing to realize, which you already touched on, is number one, there is marketing here, obviously.

And there are a lot of skeptics. And let me promise you, there is hype, but the truth is larger than any hype. We have all been in this industry for a long time. We survived how many apocalypse, however you pronounce the multiple of apocalypse?

Many. All the way back to Y2K and beyond. With that said, it is also real, and we must recognize that to start moving on this because it shifts so many of our assumptions in cyber defense that we have to start working on this now, even if it takes us a while to get there. That makes sense to me.

And I agree. We've seen quite a few of these, certainly over the tenure of my career. But Ed, if I might come to you here, you've been around even longer than me, one of the titans on which this industry was built. Ah, shucks.

Are you watching this play out thinking, oh, here we go again, another overhyped capability? Or, you know, like Gaddy, do you think underneath that hype and marketing, there really is a significant trend here? How are you thinking about it? I think there's a real there-there.

Like Gaddy said, there is some marketing happening here on the anthropics part and others, but we've been doing source-assisted pen testing with AI for 15 months, me and my team, putting aside mythos. Using current models, we have found massive vulnerabilities, you know, major issues in not only open source code bases, but also, you know, closed source, where our customers give us the source code and we start combing through it. We've had things that we've had human pen testers test year after year after year for five, seven, 10 years.

And then we apply these techniques using current models, you know, nothing mythos or or beyond. And on the first day of the pen test, we'll discover five or 10 critical vulnerabilities that we've never seen before. Because the AI is able to find subtle flaws. And this is not something like traditional SAST would find, you know, source assisted and software analysis, which would be like cross-site scripting or SQL injection.

Sure, these tools can find that, but they're finding subtler bugs like strange authentication bypasses or authorization flaws. Or one of the big things we're finding is in SaaS applications, cross-tenant access. So we're able to go in as one user of the SaaS application and through manipulation of a bug found by the AI, we're able to start accessing other parts and other customers of the SaaS application. So there's a whole raft of bugs that current models are really good at finding.

And my worry is that a lot of our listeners here will say, Well, I don't have access to mythos. So that's a future problem for me. But the fact is you can use current models, turning them against your own code bases to find vulnerabilities today and eradicate them so that when future attackers get mythos or beyond capabilities, those vulns are away from you. They're already gone.

You need to kind of drain the swamp now. We've been given this little reprieve and warning, and we need to take advantage of that. I do love that point. And it's funny, we've got to admit, we're talking out of both sides of our mouth a little bit, aren't we?

We're kind of saying, oh, look, there's a big new piece of technology. We've got our eyes on mythos. And then we're kind of simultaneously going, and a lot of it was already capable with prior models, and there's a bigger trend, and probably ought to get on with solving this, even though there's a lot of marketing hype. Rob, if I might come to you quickly on this, you spent a lot of time playing with AI.

You know, I'd say you're an AI pragmatist, but certainly on the optimistic side of it as well. You know, I have my bouts of cynicism and I need a dose of Rob to lift me out of my frustrations as a grumpy curmudgeon with AI. How do you think about this impacting the cybersecurity profession? I mean, is it the case, as I said in my intro, that I just need to be able to type good prompts now and I can go and find as many zero days as I like?

Gone are the days of needing skilled humans. Where do you sit compared to Ed and Gaddy on there? Similar perspective or different? I'll put it this way.

So one of the things that Gotti and Heather Atkins, yeah, I hope I get this right, Gotti, which is the cataclysm, you know, where you guys wrote that paper was about six months ago. What was the official name of that paper again? So I believe we called it something like preparing for or surviving through positive notes because we called it a cataclysm. We made three choices, right, that we thought about deeply.

Number one, we called it a cataclysm, an AI vulnerability cataclysm. Number two, we put a number next to it, six months. And stopping there, making these two choices means fad, fear, uncertainty, and doubt. And in our careers, we really tried to both learn and strive to avoid that, to speak with metrics, to speak with risk, to speak to executives, really.

But the warning had to come out, and we really tried to get the point across of this is already here. And that was the risk we took back then, and that's why we used words like cataclysm. Today, we're no longer using this teronology, but the problem is no less urgent. It's just insecurity, you are often Cassandra.

You're blessed with foresight, but you're cursed for nobody to ever believe you and then blame you after it happens, right? That's one of the things I loved about that, which was the way you're describing it, but also the foresight, because I mean, in terms of where you guys were pretty crystal balling that down the line. And even with what Ed said, you know, I think his team and using it for so long highlights a couple of things here. And I'll go back and, you know, when we're writing the paper, something that really struck me, which I loved, you know, describing a new capability inside organizations that they should develop called vulnerability operations.

So vulnops for short. So when you end up taking a look at what Ed's doing and the potential, whatever people, you know, calling the apocalypse or, you know, people are overworked trying to patch all these things, it's hard to predict exactly what the workload change is going to be. A couple weeks ago, actually I think three weeks now, I watched a really good documentary called AI Doc, which highly recommend everyone watch this. But there's a key word in there which kind of describes me and how just this situation is described.

The subtitle of that is How I Became Apocalyptimist. Definitely a t-shirt, just putting it out now. The combination of both you believe in the apocalypse, we became really optimistic simultaneously. So I've I've adopted that word in terms of my own viewpoint with this, which is right now, everyone's feeling the worry, the stress, the, you know, there's still a lot of FUD that's surrounding this.

But the positive angle that I look at is number one, Anthropic's given us a gift, which we're on this podcast and were able to talk about something that Gaddy and others were referencing six months ago. And you were talking to me about it. And now Josh Wright, you know, did his keynote about it. There's this influx of people knew this was coming and now it's here.

And Anthropic's given this gift because we were struggling and getting the word out. You know, Gaddy, how many people asking you in the press about this? Now we're talking about this, and now we have a chance to highlight this to other executives and to the board. You know, should there be an additional investment?

Sure, it's marketing hype, but now it's a focus and now you're able to make these asks. I'm also optimistic on what is this going to change for code development in the future when you have these capabilities at the front end before you deploy your capabilities, that you'll be a lot more aggressive, you know, in Volnops working directly with Sec DevOps and the ability for coding to potentially find more flaws before they're hit to production. I think that is likely going to happen.

We just need to ride out this initial storm to get to that point where I believe there's going to be a lot more secure code, not 100%, but a lot more accurate secure code that's going to be released. You know, Jenny Easterly has said this again and again and again. It's not a cybersecurity problem, it is a bad code writing problem. And with that in mind, you know, and a hat to everyone that's kind of in here and kind of like bringing my own thoughts together.

That's why an apocalyptimist here. Right. So hold those thoughts. Just as I was trying to work out what the plural of apocalypse was, you've now taken it a step further and given me a bigger challenge.

But we are coming back to vulnerabilities. We are coming back to those points that you raised. We need to go into detail in them about what defenders should do, what organizations can do. And we will come back absolutely to the optimism.

But as an optimist, it's my sad duty to take us back to the other part of the storm before we do that, so that we can get our full picture of what we're up against here. So I'll move us on to the second bit. I used to head up a public authority, as you know, in the UK for cybersecurity. We always told people, zero days, yeah, don't get overexcited about them.

It's all the existing weaknesses. And so I want to ask you, maybe we rob go first, about Mythos is a very good hacker, the three out of ten simulations in the UK's AI Security Institute that managed the full attack chain pretty quickly. No AI model has done this before. Again, we've seen this coming for a while.

What does this mean? Well, first of all, I think most of us in here are looking at a capability that we've not directly put our hands on. So, you know, I go take a look at the capabilities that currently exist and really emphasize that mythos is likely a change in logarithmic speed, going from, say, we're traveling at 50 kilometers an hour, you know, you're now breaking the sound barrier. Yeah.

But I really want to highlight that 50 kilometers an hour for a lot of things, it's really fast. Yeah. And even in my own development and working with the previous models, um, we're able to get extremely accurate reports, even pointing at compromise systems. And so when you end up taking a look at it holistically, we need to remind ourselves that cybersecurity, you know, we do have this capability and it has launched the hacking capability forward.

And Anthropic noted that in their report, GTG 1002 back in November. That not only are we looking at the vulnerability exposing at a really short window, yeah. We're also looking at autonomous hacking that is extremely uh sped up. You combine that peanut butter and chocolate to make your Snickers bar.

That is what I'm really concerned about. It's not just finding zero days, it's the speed of attack in the chain that would be able to be accomplished. All right. Well, look, the listeners can't see, but I saw it because it's on video for me.

Sorry. Audio podcast for everybody else. Ed, you were nodding vigorously when we started talking about this. Give me your perspective.

Almost aggressively, I would say. With a smile, with a smile. I think it was pretty aggressively, yes. So that's uh the really interesting area of this.

You know, I had mentioned earlier, let's not get obsessed with mythos' ability to find zero days because current models are really good at it. That's good. That's true. We have found in applying current models to our active pen testing that they're okay at it.

They're not great. We have created some internal tooling, we call it sidekick, where, you know, it rides alongside a pen tester and the pen tester compares notes with it and so forth. And we found some stuff. We found some medium vulnerabilities, uh, some low risk stuff.

We did have our little sidekick find a critical vulnerability a few weeks ago that the pen tester working on the thing didn't find himself. And that's good. Mythos could really be a game changer here, though. So I wouldn't discourage anybody from using current models to augment their current pen test team.

But mythos, I think, is the real game changer here. You know, another thing, if we're uh emphasizing some more of the pessimism before uh the optimism later. Before the optimism, yeah, I think that's the right order. Yes, exactly.

I'm a little concerned with, you know, the promise that mythos and current models are gonna find a lot of zero day vulnerabilities, because I think it's gonna force those who have an arsenal of the zero day vulnerabilities, especially the ones that they know are easy to find, to use them now. Use them or lose them because the swamp is gonna be drained over the next three, six, twelve months. And this is kind of completely independent of mythos and next generation models, but it's being pushed by them.

That is, take your current zero days and use them. The ones that you may have spent tens of thousands, hundreds of thousands, millions of dollars on, use them now because they're probably gonna evaporate when mythos and other models turn their attention to finding them. Brilliant. So, Gaddy, I think we're at the border checkpoint between optimism and pessimism.

So you can stay where you want for a while. But I really love your perspective on this. If you like the non zero day part of this and how you evaluate it. When it comes down to it, we're security professionals, we're risk professionals, we're technologists.

Depending on who we are, we might be none or neither. But as we started, we survived the multi apocalypse. I don't know if that's the way it would be announced, but you'll update me, right? Love it.

And it comes down To optimism because we did survive those. People will come together, the sun will rise, but don't make it any less serious. Yeah. And up to now, we covered the specific risk of vulnerabilities in code.

And now we can turn that capability of attackers as they kind of hit their singularity moment, maybe a micro-singularity. The singularity is not evenly distributed, whatever you want to call it. And the fence hasn't. We have seen a lot of AI technologies go into the fence.

None of them are yet mature. And we're still imagining at the level of we can use AI for something we've done before as opposed to something completely new and different. So attackers are there. They can find vulnerabilities, they can exploit them, and they can run autonomous operations.

Malware is now coming in. Nothing is perfect. So the first thing, just in summary, is take the technology, build your vulnerable ops capabilities, and start. Start by just pointing an agent at your code and saying, find something just to get started.

Then start using uh whether it's commercial tools like from anthropic codec security or closed code security artwork. You can use open source, which we actually released to not compete with them from Gnostic, which is called OpenAnd, ANT. You can use anything but start. But then there is the second point.

And there are three. Always three. That's big. Always three.

I was in the military. I don't know if there will be three, but I'll always keep it to three. So the second point is we don't really have these other defensive technologies yet. We are borrowing from the attackers.

What we do have is our people. And our people can be accelerated today. We can't allow or forward ourselves to move at human speed anymore. We have to run at machine speed.

And if there is one thing that helps everybody across the board, and attackers for sure, whether you're in GRC or audit or incident response or threat hunting or threat intelligence, and of course coding, is coding agents. These are agents, but the vast majority of agents out there, the best agents in the world right now are coding agents. Cloud code, cursor, copalt, whichever one you like. If we don't ask, suggest, encourage, make it mandatory and force people to start using agents now to empower themselves to be two times to 100, 200, 300 times, depending on what, better, more capable, more efficient, able to understand faster as they did before, we're behind.

This, of course, needs to be done securely, and I, of course, biased on that, but I truly believe that. But use agents. I am on the optimistic side because I believe that is truly the number one thing we need to do as we develop these new capabilities to a defense. I love that.

Gaddy, there's something in there I want to pick up on, and I want to bring Rob into as well, because it's a little discussion we had over the last couple of weeks. This moving to machine speed. Now I've I've seen a bit of this movie before. I was there in the early days of malware, where we used to kind of reverse engineer and try to write identities or signatures, as they might have been called back then, by hand for each piece of malicious code.

And then attackers started generating them using generators, you know, programming solutions. So we had to build kind of machine learning rigs and expert systems as the number of samples went from thousands a day to hundreds of thousands a day. And it changed some of the makeup of the skills that were needed of the analysts, but we fundamentally needed more analysts to deal with the scale of the problem. Gaddy, Rob, up to you who goes first, I shall allow first mover advantage.

I'm curious when you talk about machine speed, how you think this might change the shape of cybersecurity professional skills over the next few years in accomplishing, Gaddy, what you were just describing, which I think is going to be the right approach to defense. So I guess my point, we have to go back and yeah, I'm a big fan of history and how we landed in our careers here. When the cybersecurity, quote unquote, industry began, sysadmins were the cybersecurity professionals.

It was a side job, you know, in many cases early on, it wasn't even a side job. They couldn't even describe it. They were just thrown at problems that were related to security. And then late 90s, early 2000s, you started seeing very slow, you have to go back and ask, you know, when did the job security analyst and even forensicators and so forth, instant responders, pen testers, and I think you even helped coin the term pen tester, if I'm correct, came to exist.

And I see that currently going on here is that the skills where people are dabbling and trying to learn is not unlike what we saw at the beginning, is that you end up having this separate skill set that is being developed. Now, whether those become full up AI, cybersecurity analysts, machine learning, I think that'll metastasize over uh the next few years. There may be even a split of cybersecurity. It would be almost like how IT split from cybersecurity in two different functions.

That, and again, if a little bit of a reach, which is almost a debate for another day, is will you have AI secops and security operations separate from cybersecurity capabilities? And it sounds like that is impossible, but given how different AI is, especially once you get beyond, you know, like in the singularity world, working and utilizing AI becomes a lot different because it, you know, you're reasoning with it, you have to train it, you have to do a lot more introspection, but we still need someone focusing in on the core cybersecurity stuff.

So I don't know. I think it's skill-based, James, but I also think there's room to make the analogy that it could go separate. It could be, you know, clear career fields. I think we're in the sysadmin days where people are kind of figuring out, hey, I'm a cybersecurity person.

It's really interesting whether we'll end up with AI folks who are bilingual with cyber or, you know, cyber with AI merged in, or what spectrum of separation. Ed, you are no doubt gonna have some thoughts here, having seen a few of these movies over the years and the evolution of our industry. Exactly. And I've been thinking a lot along the lines of what Rob just said.

You know, 30 years ago, we saw people, and you could think of it as skill sets if you'd like, but I've been thinking about it as workflows. So if you go back 30 years ago, people started putting down the workflows of intrusion analyst or incident handling, or Rob was hugely involved in digital forensics, and other friends of ours did cyber defense. And maybe 25 years ago, we formalized the workflow of what a penetration test is and what's a web app pen test versus a network pen test and so forth.

And there's others, right? Mobile, cloud, ICS, and so forth. But what I think the industry needs now is to define what the workflows are, the AI-centric workflows. And they may be very different from the current workflows for given jobs.

One example I'm very familiar with is this whole AI-enabled source analysis and assisted penetration testing. It's a completely inverted workflow where you start with the AI looking at the source code, as opposed to a traditional, say, web app pen test, where you have a human looking at the target environment. Also, this whole discussion about vulnops. I think what vulnops fundamentally is is a new workflow around which there are skills, but I would love to see more people who are very focused on vulns define AI-centric Vulnaps workflows so that then we can figure out what the skills are around it and build up those skills in ourselves and in our workmates.

If I had to bet on whether AI splits out so that there's sort of a cyberized AI versus traditional cybersecurity, my bet is it won't split. I understand, you know, James and Rob's comment that it split because it did in the sysadmin days. However, I don't think there will be enough for traditional cybersecurity people to do without AI, so that they're all going to move over and have AI-centric workflows and provide the unique capability that the humans can provide in those workflows.

So what I'm really interested in is AI-enabled workflows. In fact, I don't want to sound too, you know, haughty about this or anything. But if you're just sprinkling AI on an existing workflow, maybe that'll help, but I don't find that that interesting. I want to see AI workflows built from the ground up.

Dan Guido of Trail of Bits did an amazing presentation at UnpromptedCon about six weeks ago or so. And it was all about how do you build AI-centric workflows around your business and not just AI enable existing workflows. And I've been thinking a lot about that and how it applies to penetration testing, cyber defense, digital forensics. I know Rob's been thinking about it for digital forensics and incident response.

Heather Barnhart has been thinking about it a lot for digital forensics and incident response. But I think creating those workflows and then the brand new concept of Vulnaps, creating a workflow for that. That's fascinating, Ed. And I do buy what you're selling.

I have to say, I think, you know, I've been using this expression that over the next few years, as AI is adopted by the good guys and the bad, and we figure out the best ways to supercharge our various efforts and reinvent workflows, you know, when everyone has AI, the edge, again, is human, critical thinking, humans with interesting ideas and training and problem domain expertise. And that would suggest that people that understand AI and their particular cybersecurity problem domain are likely going to be best placed to be able to define those new workflows, processes, and systems.

Ed, if I might come back to you on a related question, I suppose. So now we've reassured the cybersecurity community that, you know, it isn't Skynet. We might need some people. It's all fine.

No. Sorry. What happens now? I mean, this has been accelerating for a while.

Gaddy covered that in his opening remarks, that this has been a really effective way to draw attention to an underlying trend that's been swelling. OpenAI have just come out with their own delay and a different model for how the community handles the challenge. We then are going to have the wave of Chinese models and open models and so on. And then the whole cycle starts again.

So, for security leaders and practitioners listening, what should we be looking out for months down the line or 12 or 18 out from here, do you think, Ed? This is where I get really excited. I think we're going to go through some rough times in three months, six months, nine months, maybe a year, as we adapt to new attacker capabilities augmented by AI and we implement things like Volno ops and improved AI-centric workflows. So that might happen over the space of the next year or so.

But what I see two, three, four, five years out, I'm highly optimistic about. I saw Phil Venables quoted, he put something on X about after he watched the videos and such from Unprompted. He said he's short-term pessimistic and he's never been more long-term optimistic. And I agree wholeheartedly with that idea.

I think three or four years out, we're actually going to drain the swamp of many major vulnerabilities. We got to get there first. And, you know, I say this to a lot of my friends who, you know, look at this with pessimism over the short term, or even what happens to jobs in the long term, job loss in cyberstreet, what am I to mean? And I say this: I've been doing cybersecurity work for 30 years now.

And I've worked with some of the most amazing people in the industry, and we've given the best years of our lives to try to make the world safer and more secure. At best, we've barely treaded water. I mean, it is true that we're more secure than we were then in absolute terms. If you look at the security capabilities of Windows 11 versus the security capabilities of Windows 2000, oh my goodness, we've come a long way.

However, the attackers have gotten so much better that at best, we've gotten better at the same rate they have, or maybe they've gotten a little better than we have. So we're just treading water. I see with these new capabilities that are coming online and the significant lowering of vulnerabilities in our software base, it's possible that three or four years from now we can actually have a fundamentally more secure environment so that attackers will figure out other ways to attack us on top of it.

But the baseline will be more secure. And that's the first time I think that's fundamentally happened in my career. We have a chance to get a little bit ahead here. So, Ed, just to help amplify this for our listeners, we all know he's a household name in our industry.

We all know who the Grateful Venables is. But we have, we know, some listeners who aren't full-time cybersecurity professionals. We believe most of them are serving community orders, and this is part of their judge-mandated sentence that they have to listen to us. But for people not steeped in cybersecurity, tell us about who Phil is.

And also we'll put his comments. We'll link that to them in the show notes. And then I'd love to bring in Gaddy. I've been following Phil for a couple of decades, and he's sort of a CISO's CISO.

Yeah. Very well steeped in financial services CISO, kind of where I think he cut his teeth years ago. He has very much influenced my thinking over the years. He's certainly one worth following on X and his other postings and blog articles.

But he's where a lot of very serious CISOs look to for guidance and influence of the industry. Great. Gaddy, we're in an optimistic mood now with Ed speaking for himself and quoting Phil. What's your take on the whole what's next?

So who the hell knows, man? Yeah. But now that now that I said that, I have opinions, right? I'm a science fiction.

And I can go as far as say there are levels of science fiction to reality. Six months ago, people would have disagreed on other things, but I can now comfortably say what six months ago people would have looked at me very oddly instead of oddly. For example, are we gonna be able to wait for vendors with their binary, not code, to patch their problems, refine their problems for them before we do it on our own? But before I go that far, two points that matter.

Yeah. I love that we went to the people element because I think that's our only moat. There is no moat. Code is nothing.

Everybody can develop whatever they want. So Skynet, I choose to believe as a belief system. Skynet, what happened? I can't do anything about it if it will, but I can be around people.

I can't take care of community. And looking at how jobs become redundant, not might become redundant. Yeah. It's no longer 2025 where we thought uh I will not be replaced by AI, I will replace by a human using AI.

I think some of us will be. My fiance is a vulnerability researcher and a pretty good one. She's looking at should I become a washing machine technician or something? Like, I'm not sure many vulnerability researchers will still be here.

So looking at people, looking at how we implement this in the future is critical. But also strategically, yeah. When we are now as CISOs, keying off what you mentioned about Phil, who is also an author of this paper, are we, for example, we have communicated to the board and the risk committee and the CIO and the chief legal officer and finance officer that we are at a certain risk level at red, yellow, and green. A lot of the reds are now yellows, a lot of the yellows are now greens, a lot of the greens are now yellows.

This has shifted drastically. It's outdated. We need to consider what our risk metrics are, both to communicate with the board, maybe even with shareholders in my reports to Nasdaq. So a lot needs to be considered about communication amongst stakeholders.

And the second thing is governance-wise, we have to move faster. Okay. We have to be able to bring on vendors, new technologies in a decent way, but also think about our organizations. Because while we want to be positive, we need to understand this is just the first wave.

And as we build resiliency for this wave, let's make sure we build security programs that are resilient to the next one. That's fantastic, Ganning. Thank you. I get that a lot.

Thank you. It is fantastic because that nuance is what we need. This is really complicated. I loved your first answer, who the hell knows?

Because anybody who predicts with confidence where this is going. I'll cut you off and double down on the complication. Fine. But how are we supposed to give advice when we tell people, for example, patch faster?

When our advice now is often wait on your patching, cool them down to avoid supply chain issues, right? This is complex. And we came up with this paper. Actually, everybody here contributed to the paper.

Yeah. And we need to realize the industry came together, 250 people, CISOs, others, wrote this together to be ready with information, to educate, to have an external document, somebody can put on the table, say the industry says so, not just me, to be able to establish some guidelines for tomorrow morning so we can build our programs. But this is just the start. We need to go for this mode, the community to move forward and adjust.

So this is perfect. You should actually host the show. You're much better at it than I am, because this pivots us perfectly to, I think in the US you used to call it the $64 million question. But in AI terms, that'll buy you about half an hour on some model or something.

So let's call it the $64 trillion question. And I'm going to turn it over to Rob. So Gadi set out beautifully how the community came together. You're central to this.

And it says a lot of stuff we've already talked about. We don't need to go into the diagnosis of what these things can do. There's some great stuff about what organizations need to do to prepare for this wave. Summarize it for us, Rob.

I think one of the things, you know, going back to both what Ed and Gatti are referring to, organizations need to take a look at this is a people thing. You have a great team. You're going to be working through a significant challenge over the next six to 12 months. But I also lean back to what was just proven to us, something I alluded to in, you know, talks I've given over the past couple of months, is that a very optimistic way of looking at this is everyone is talking about asymmetry between the attackers and the defenders.

And it exists, it will exist. However, we do have a structural advantage over the attackers, which there's a hell of a lot more of us than there are of them. And what we've shown over the past week is when you have a call to action, you know, this is where, you know, with what you just said, what's your organization look to? It's the community, is that we can come together and do great things and come up with great ideas and the thinking using our structural advantage, I think is going to empower us and enable us much more than before.

And that is, you know, from coming up with new ideas and developing them. My theory is that we can quote unquote open claw solutions by having a lot of us go hands-on keyboard, rapidly produce things that will potentially get in front of some of these offensive capabilities as quick as they're able to develop them. These teams are developed in stove-typed environments. You don't really want to share your capabilities.

I've worked in one, you know, limited number of people that you would expose to the really good onslaught that you developed on the defender side. As soon as we get, you know, a little hint of what they're doing, there could be a call to action across the community. And that is something that Daddy, you know, hurting cats or whatever you want to call it, demonstrably shown over the past week. And it should not go unlooked.

To me, it's the path of the future. We as a community can do this, and it's going to take a lot of smart humans. AI is not going to coordinate to all the agents together to figure this out. Smart people, community, structural advantage.

That's the path forward. Well, on that wonderful trio of staccato sentences there, I know we're charging through time here. And this feels like a topic we could probably spend or many days on, honestly, with this group. But maybe let me give the last question here to Ed.

You said something interesting before that I think gels with what our other guests are sharing. There's going to be a lot of change, a lot of disruption. And that could lead to some real pain for the cybersecurity community in the short term. Actually, quite worried about some of the misinterpretation from hype causing some of that.

And then you have reasons to be optimistic in the long term. I also agree with that view, having seen similar versions of this movie before. If you were advising security leaders right now on the things they should do in the short term to try to make the long term better, what types of things would you suggest they do? Aside from obviously reading the paper we keep referencing.

Yeah, I think there's three areas to focus on. I mean, there's more, but you know, you can't really focus on more than three things. One is this whole vulnerability discovery, patching, et cetera, and optimizing that as they're calling it vulnerops and looking at your vulnerability discovery and patching systems today and figuring out how you can radically improve those. And I think as a community, we have to flesh out what this vuln ops really means and do it quickly, Gaddy.

No pressure. Um we got to get together and figure that out. The second thing is limiting the blast radius to segment your network where you can, to do detection quicker where you can, so that if something does get owned with the zero day, it doesn't hurt your entire environment. That's, you know, nothing fundamentally new.

You should have been doing that for a while. But if you haven't gotten around to it, now absolutely is the time. And then the third thing is on the incident response side of things, doing tabletop exercises to make sure you're ready for a single incident. But we're recommending now to folks is what if you had two or three simultaneous incidents of different kinds?

What would you do with that? Are you prepared to even contemplate that? Those are three things in the order that I think you should focus on them in. And I'm sure there's something we could debate about all of that.

But uh, those are the three things that really bubble up to the top for me in what I see in my enterprise customers. And all of them are covered in more detail in that uh Cloud Security Alliance Sans unprompted paper we've been talking about. Love it. If I may, please do.

Here is my call to action slash question slash challenge to people and then to the community. And maybe Sans can lead that forward if somebody can figure it out. The first one is for ourselves. Many people feel outmoded, many CISOs feel outmoded, we feel old, we feel we'll never catch up.

I think that's yes. We can. All we need is English. Kick yourself in the bum, go download a coding agent right now.

It's all about coding. You don't need to know code, cursor, clod code. Use it, ask it a question, do one task each day different with it, learn it. It will empower you.

That's my challenge to you. Do it now, please, and take somebody with you for the journey. Don't leave people behind. And the second thing is speaking about people, people are being left behind right now, not just because disappearing jobs, but rather because the disappearing junior and sons in the expert in training this industry.

How do we allow people to come in? What do they need to do? What do they need to know? How do they find a job?

We were already an aging industry before this. It was so hard to find a job before this. So that's kind of a challenge I would like to leave out there that I think matters a lot. Please go kick yourself in the button, start now, download an agent and try, talk to each about something different every day.

And two, let's think about the future of who is in the industry. That sounded like a takeaway to me, but you've already had two goes at the final question, James. Neither of them were, because I'm gonna ask the final question and there is a twist. We have three guests.

That's right. So three guests, three takeaways, right? Nope, not gonna do that. Three takeaways is too many.

Gaddy said it's always three, but he was whittling it down to two. And you put on weight if you have three takeaways. We all know that. Well, I do like three takeaways, I must admit.

But what are we gonna do instead then, Kieran? That's easy. You're gonna do it. Punishment for all those times you've hinded our poor guests.

They're exhausted at the end of a long recording. They've said everything you want to say, and you just say to them, oh, go for it and describe everything in 30 seconds. So, James, James Lyne, CEO Sands Institute. Sir, this has been described as the most important event in cybersecurity for years.

Organizations have to act. Tell them everything they need to know. You have 30 seconds. Go.

Oh, that is just so brutal. I understand why our guests hate and love this now. And I feel like Daddy just did such a wonderful job of it. But okay, all right, okay.

Here we go. 30 seconds. Let me comment in macro here for security leaders. There's gonna be change.

There's gonna be overhyping. Don't worry about that. Like you're hearing from World-class practitioners here that there is something real here. There's short-term substantial risk and long-term reasons to be optimistic.

So go read the paper. It's in the show notes. Watch Ed and his team's wonderful webcast doing a live demonstration of the actual practical utility of these things we're talking about. The dynamics of cybersecurity, they're changing in front of us.

We're going to have whole new areas like volnots. Roles are going to compress, they're going to expand, skills are going to change. And if we throw our hands in the air, declare BS and ignore it, we're going to be weaker for it. So make sure you make time for your teams to spend real time here and learn.

Just like Gaddy said, most security teams are running at 100% capacity or more. They're not getting to do enough here. This isn't the end of cybersecurity. It's not solved in time to go home.

It's not Armageddon. So shooting down the middle, we've got to be ready. And as leaders, you need to make space for your team to be ready, to keep upskilling and following this for more developments. It's going to move fast.

So bet on people and ignore AI's cybersecurity impact at your peril. Well, that was wonderful. The zero-day clock may be collapsing. Your 30-second stopwatch is going in the opposite direction, but I'll let you off because that was genuinely excellent.

But I think we have to thank Gaddy. We have to thank Rob. We have to thank Ed. Absolutely wonderful guests.

Thank you all. And thanks to Sands for stepping in, helping with this document last minute, taking the risk to work on this over the weekend. All of you were writers and authors of this and supported us. Kieran, James, Ed, of course, Rob will stepped in and did all the work.

Thank you. Absolute pleasure. Thank you. Thank you for herding those cats.

Although I did want to make a cataclysm joke at some point here. I'm sure we can use AI to generate a new graphic there. But you guys have been wonderful, and I suspect we'll be back. This is not going to be the last discussion on this topic.

But I do believe we've been very useful to security leaders today. Don't you think, Kieran? Well, I hope so. So if you did find this useful, please leave us a rating wherever you got this podcast.

People who understand modern communication technology tells us that that sort of thing helps, especially if it's a good rating. And if you have any suggestions more prosaically, longer form feedback or follow-ups on our show, you can email us on cyberleaderspodcast at sans.org. And with that, thank you very much for listening.

Thank you for listening. Keep cybering in the AI world. For me, Kieran Martin, and me, James Line. It's goodbye.

And avoid the cataclysm.