Risky Business #844 - China closes AI vulndev gap as USA lifts Fable ban

Speaker A: Foreign.

Speaker B: And welcome to Risky Business. My name's Patrick Bray. We've got a great show for you this week. There's been a whole flurry of, like, cyber AI activity over the last week. Models are getting banned, models are getting unbanned. People are using them to find huge tranches of o day that they're just dumping onto the Internet without telling vendors. Uh, so much going on. And we'll get into that in just a moment with Adam Boileau and James Wilson. Uh, then after that, we'll be hearing from this week's sponsor. And this week's show is brought to you by portswigger, the makers of Burp Suite. Portswigger, product manager Katie Warren, and, uh, also the founder, DAF Studded, will be joining me a little bit later on to talk through how they've created an AI based, uh, burpsweet product. And you know, what they've had to consider when it comes to human in the loop safety, all those sort of things. Right? Uh, James Kettle, their director of research, uh, famously has created a tool called the HTTP Terminator. This does not have happy corporate procurement vibes. So we'll be talking to them about what does have happy corporate procurement vibes when it comes to AI based pen, uh, testing products. Uh, that's coming up after this week's news segment, which starts now. Now, Adam, James, uh, we've just seen this absolute crazy flurry of activity and indeed, uh, just, you know, a very short time before we hit, we've hit record on this podcast. The US Government has agreed that Anthropic can, uh, start shipping, uh, you know, Mythos, uh, Fable, actually, for general availability. Mythos is back to being sort of active for a, uh, small number of organizations. But basically the US Government is unwinding these restrictions. We've seen a note being passed around on social media, a letter from, uh, Howard Lutnick, the Secretary of Commerce in the United States, addressed to Tom Brown, who's the chief compute officer at Anthropic, notably not the chief executive. Right. Dario. They've stuck him in a basement somewhere. You know, you know, he's got the tape over his mouth, uh, and it looks like Tom Brown has been the one, uh, taking point with the government now. But this letter basically says, look, Anthropic has agreed to proactively detect and address security risks associated with the models to work diligently with the US Government on protocols and, uh, standards of releases, releases for Mythos, Fable and future models and blah, blah, blah, blah, blah. So it looks like, uh, basically Anthropic has promised it will do what the government wants it to. And. And, uh, the government, as a result of that, has said, okay, very well, you may ship products, uh, out there to the market. I mean, that's about the long and the short of it, right, James?

Speaker C: It is a long and the short, Pat. And like, all I can say is thank goodness, because we have been in a terrible spot. We, as in the people that rely on these coding agents over the last week or two, because as we know, when there's a new model on the horizon, the current one gets a bit lobotomized. And so far that's been okay, right? When Claude goes bad, you switch to Codex. When Codex goes bad, you switch to Claude. For the last week, we've had literally 5.6 on the horizon from GPT and Fable 5 on the horizon for Claude. So both of them have been pretty useless and hard to use. Um, so very glad the government got. Got this sorted out. But, like, even what you just said in that statement there, it's not clear what the difference is, other than perhaps the work diligently with the government as opposed to sort of ignoring.

Speaker B: Vague is vague af. Right? Like this is that. And look, I will note too, that that letter that's been passed around, we have seen wired referrals to this letter, um, but we haven't been able to confirm that the letter that's being passed around on social media is authentic. I suspect it is, but I just want to add that caveat there, you know, like, after all, who would make up such boring and bland and vague language? You'd think if someone was going to fake it, it'd be a bit spicier. But it really does just look like, okay, they are sufficiently sorry they have put Dario in a basement. We're going to let them do business again. I mean, that's what it feels like, right?

Speaker C: Yeah, it's really all it seems like. And it was actually reported, I think, last week that there was a real turnaround of, uh, relations with the government. It was directly attributed to them stopping, uh, or no longer sending Dario to meet with White House staff and sending Tom Brown instead. So, clearly, that is the thing that is different here. I'm really excited to get back in and play with Fable. As I mentioned before, last time I used it, it felt like the guardrails were about a five statement switch statement that looked for you saying, cyber security exploit chemical weapon, biological weapon. And if you said any of those, you were ban. Um, let's see. Hopefully there is Actually some real guardrails around this. And, yeah, uh, I'm keen to see what it'll let me do and what it will not.

Speaker B: Well, meanwhile, we might actually owe the Trump admin, uh, an apology because we have suggested that they were singling out anthropic, uh, but it looks like they're also imposing similarly bizarre bans on OpenAI's models as well. Uh, GPT 5.6 was rolled out to a limited subset of customers. I know one of them who was in the middle of doing some work when, boom, access revoked. And it turned out that this is also some sort of export restriction. Well, there was no export restriction applied to the OpenAI's, like GPT 5.6, but the government asked them to restrict the, uh, rollout, and they did. Yeah, I don't even. What is going on.

Speaker C: Yeah, look, as you were saying that, I was thinking there's an interesting parallel here between GPT and Claude. Right. Claude's always been the model that's in a bit of a mood. Right. It will just gives you pretty stern responses and it'll tell you when it doesn't agree with what you're doing. GPT has always been the super sycophantic one. You know, you're absolutely right. And it kind of feels like that's mirrored in the way that they're interacting with the government here. You know, Claude's a bit standoffish.

Speaker B: These attitudes are a product of their corporate culture.

Speaker C: It's clearly been trained into at least one layer of the model to, uh, you know, to take on the corporate culture. Uh, but I read this more as just OpenAI, um, being vastly better at reading the room and being on the front foot with the government and saying, you know, okay with this, you know, can we release this? Who do you want us to release this to? And, uh, clearly that's going to hold them in much better stead. Short, medium, and potentially longer term as well.

Speaker B: Now, meanwhile, uh, Alex Stamos, good friend of the show, uh, he is all over, uh, Twitter, slash X, whatever you want to call it these days. I still call it Twitter. He's, uh, all over Twitter these days. Uh, basically saying he feels like he's taking crazy pills because, like, while the US Government is slapping restrictions on these models, the Chinese keep shipping better and better and better local models.

Speaker A: Right?

Speaker C: Yes, uh, amazingly good. Uh, a couple of interesting ones that have come out in the last week or 2. Obviously, GLM 5.2 is the real huge flavor of the month, and that, in fact, is showing better scores in cybersecurity benchmarks than even the, I uh, think the Codex 5.5 cyber models. And so they're streaking far ahead. Um, there's also just, I don't know, I mean like for all the attention that OpenAI anthropic gets and they ship these new models and they you know, new great model release for everyone. There's a lot more exciting stuff in the very specialized models that are coming out of China and other labs. Like I was looking last night at this one, the uh, uh, owner of, I think it's called it, it, you know, it's like a model that creates its own harness as it's working. Right. We've, we've gone from just having chat interactions and providing these things, tools and now they're really branching out to creating their own harnesses, their own workflows. It's uh, the autonomy is ramping up and up and up and that's, that's coming out of these Chinese models, not out of the US based frontier models at the moment.

Speaker B: Well, we'll talk a little bit more about local models uh, in just a moment. But um, there's been this. Look, there's been this other story that's happened over the last week or so which is Anthropic's come out and said Alibaba, uh, the Chinese company has been distilling uh, Claude's, you know, Claude AI basically and that we need, we need action on this and it's not on and blah, blah, blah, blah, blah. Now you pair that uh, with a piece that has come from China talk which is just fascinating, which is about the underground like token economy, uh, where Chinese people are able to actually get access to Claude through all sorts of like weird API routers and things like this. And they're getting, there's this whole ecosystem where people sign up en masse for like free credits with Anthropic and then they're monetizing that by selling the tokens, you uh, know, through this huge uh, ecosystem, which is fascinating. So we got, we've got two stories kind of in one here which is Anthropic, yeah. Is complaining that Alibaba is distilling its model. But, but then we've got this absolutely amazing deep dive uh, from M. Chinatalk and if you only read one thing this week, you should read that. It uh, is called how to Buy Cheap Claude Claude Tokens in China. And it is absolutely fascinating. Adam, I want to get your take on this because what I find, yeah, I mean I just find the Whole economy here that has sprung up that enables these sort of model distillation attacks to occur that ain't going to get dismantled easily. Right. So I think distillation is just going to be a thing.

Speaker A: Yeah, I mean this particular, that story, um, the China talks one, uh, really kind of crystallized for me a little bit more about how that whole mechanism works. Because like, you know, you think about, you know, you read these stories about distillation attacks and you can imagine people, you know, crafting prompts to get the specific data they want to help tune their model. Uh, but ultimately just getting access to a whole bunch of inputs and a whole bunch of outputs is kind of really what you need for distillation. And that pairs very well with operating a router, you know, front end proxy to let people get access to models because you get access to all of that data in the middle.

Speaker B: Yeah, they're selling, they're selling the logs to people to do like training with. It's amazing.

Speaker A: Exactly. And like the, the piece starts by saying like you can buy CLAUDE tokens at a fraction of the uh, like you know, street price in China. And part of that's because there's some other funny stuff going on. For example, some of these router layers will shim in cheaper, less good models. So you think you're using the latest high end anthropic model, but you're actually getting some open source piece of trash and they're just relabelling the output because you're doing business with people who, it's not like this is a legitimate business reselling stolen access to CLAUDE or fraudulently obtained. Uh, and all of the tricks they have to do to bypass kyc, getting people in third world countries to complete the humanness checks to get CLAUDE service. Um, but that the real reason it's so cheap is because they're collecting the logs. And that's the real business here, not the selling you five bucks a month worth of claude.

Speaker B: That's the old joke, right? It's like you are the product.

Speaker A: Well, exactly. Right, exactly. Um, and seeing that whole ecosystem like it totally makes sense. Right? I mean if you are a developer in China right now, are you going to sit on the sidelines while all of your Western counterparts have access to these amazing coding models while you're stuck with attachments? Or are you just going to let the free market sort it out? Because China loves the free market and this is exactly uh, what you would expect. Like all of the Chinese developers miraculously just coming out of Singapore to use Claude and the usage stats for uh, anthropic services out of Singapore are like through the roof because everybody be proxying. Uh, so yeah, it's such a great write up. Totally worth a read.

Speaker B: Yeah. And the Chinese really just don't care. The government doesn't care that people are using Claude because it gives them an edge. It's going to accelerate software development and whatnot. Now James, what's interesting here, right, is you might have some of these companies trying to develop their own frontier hosted models, right, uh, in China doing these sort of distillation attacks. But the focus in China seems to be much more on these open weights models. What's really interesting though is you're seeing stuff now that's like being advertised on hugging faces. Like we've totally distilled Claude into this, right? So they're like openly bragging about having distilled, you know, I prefer like, you know, they stole its soul, they extracted Claude soul, uh, and stuck it in a, in an open weight model. I mean this is just, this is going to be a problem for the Americans eventually. Like, and I mean a real one.

Speaker C: Yeah, I think it is very, very soon going to be a very, very, very, very big problem for them because yeah, it's, it's just, it's brash and it's open. Like you go to hugging face and you look through the top models at the moment for coding agents or even general purpose and there's just like, not only is there models that are, that are fine tuned with, you know, people saying, oh, uh, Gwen, with 500 million tokens from fable exchanges, uh, fine tuned on top of it, right? It's just, it's out there. But not just the models, the training sets are there as well, right? You can actually go and download these huge volumes of chat logs between, uh, people using the agents. But the thing to remember here that points out why this is such a huge problem and I think virtually an unsolvable problem is first of all kind of to what Adam said, right? It's, it's not that you need really specific prompts, right? A distillation attack against a large language model is not a back and forth of tell me how you reason, tell me how you would approach this, tell me what is different about how you would do this thing.

Speaker B: It's just inputs and outputs, right? It's just collecting off of them.

Speaker C: But even then, there's a fascinating paper co authored by Nicholas Carlini, who's been on Risky Business Features before, showing that you can actually just throw gibberish at it. And the response you get is just as telling because at the end of the day you're trying to match the patterns in the token distribution, not even the words itself, right? So massive problem. But the other thing to highlight here is it's almost like the protocol stack and the implementation of this product is designed in virtually every way to enable this to happen, right? There is no concept of like I was thinking about how we did DRM in itunes, right? There's various libraries and serpent and things like that that will encrypt headers on the request. So you know it's only a legit itunes account. The DRM is specific to itunes, right? So even if you could sit on the wire and get the packets, you can neither forge a request nor can you do much with the resulting things. Unless you've got the decryption keys. No such thing. In large language models, there's no end to end attestation. There is no real end to end encryption in the sense of that would prevent uh, a proxy sitting in the middle, right? There's no like end to end cert pending to prevent this. The clients are just APIs because people

Speaker B: are using legit like you know, request routers are used, they have legitimate use cases like a lot of businesses now are using them because they can gauge, they can look at a request and go, well, this one we can probably handle over here on a local model or this one can be handled over here. Or this is a premium uh, query, you know. Well, we will send that one off to Anthropic. So like if you, you would enrage uh, your customers by doing, by imposing a restriction like that. And I also think that you'd run into capacity constraints as well when you think about the number of like just the volume of little queries that are probably being routed away from the frontier models, which are slow and expensive. You know, like if you tried to then route all of them back to the frontier models, like it would be A pointless and wasteful and B, would probably cause stuff to start breaking. Like if not now, then eventually that's what's going to happen.

Speaker C: I agree with that latter point, but just to the point around alienating the customers. This is exactly what Anthropic did when they said you cannot use any third party harness with your CLAUDE code. So subscription, right? This, this was sort of the uh, the fallout of the, the openclaw thing. They said, look, if you're going to use openclaw with our um, subscription plan and not use our CLAUDE code, binary itself on the, on the endpoint, you will now eat into your extra usage instead of your subscription charges. But that's ineffective because then users just go either, well, you know, forget you guys, I'll just go and use GPT instead. Or they look at the ecosystem of many, many other, you know, um, shells and harnesses and eight coding agents that will look and feel like anthropic to either get around this or just provide a like, for like service. It's like the, the client itself is not high value enough to sufficiently disrupt or displease the user base if they are forced to only use that one. Right. There's just so many other opportunities out there.

Speaker B: Yeah, I mean, so why I say this is going to be a problem for the Americans. Right. Is like so much money has gone into developing these models and now we've got all these Chinese knockoffs that are actually pretty good. Also the hardware is American. Right. So this is just like total like theft of American innovation, which I would definitely feel a bit salty about if I were like, you know, in the US Government. Right. So I get the animosity here, but you also get the sense that this is completely and utterly uh, inevitable. Um, so you know, you did a podcast with Carsten Knoll James where he looked at using, you know, we spoke about his research last week where he was using open weight models to do a whole bunch of code review but he was using the frontier models to like manage the local models and whatnot. And that was a really effective approach. So you've published a, ah, podcast on that, um, which has gone into the Risky Business Features channel. For those who are not subscribed. Go subscribe to that. That's like an hour plus conversation with Carsten about that. Very interesting research. Um, but meanwhile we've got some, some other like research to talk about this week. Uh, some, someone unknown has just gone and dropped something like. So the, the headlines say 130 unpatched exploits. But we've looked at this. Catalyn looked at this. He thinks it's more like 15. It was just like split across 130 files or whatever. So a bunch of the headlines are wrong, but someone has gone out and dropped a bunch of oda, uh, just straight onto the Internet with no attempt to disclose to vendors. And this was all AI discovered stuff. They use GPT in this case though, right?

Speaker C: Yes, they do say they used uh, GPT 5.5. Um, but you know, even then you've got to take them at their word. We don't know who this person is the GitHub repo is a bit of a shambles in terms of some of the comments and the way it's uh, sort of got this sort of sprawling, rambling read me in there. So look, if we take them at their word, Yep, it's um, GPT 5.5. Um, but yeah, um, it's such a uh, volume and sort of all over the place. It really just feels like uh, why not let our agents run wild and we'll just yolo the results up into a repo and git push and see what happens.

Speaker B: Well, Adam, Adam, you actually noticed that it looked like the LLM thought it was doing some sort of CTF challenge or something, right? Why did you, how did you determine that?

Speaker A: I think it was just um, like in one of the bugs. The kind of the headline bug that's been getting coverage here is one in levssh two. Uh, and I was reading the AI generated readme of how it had discovered the bug and learning about the details. Uh, and yeah, you just uh, is producing a proof of concept and it says like this isn't quite good enough, but it will probably get you there for a, uh, you know, demonstrate uh, the pattern sufficient for a capture the flag or a hack the box service and uh, it will need some real work for use in the real world. So that's the kind of thing that felt a little bit like that's the lies they had told the model. Um, but yeah, there's all sorts of interesting bugs in here. Some are more so than others. I mean the libssh one is pretty real. Uh, there's a couple of other things that are um, things that would be useful in the real world. Like um, there's an ImageMagick bug where you can supply uh, a binary alongside and through like path confusion, it'll run that binary instead of the system provided one. So there's a few, you know, things that are actually useful.

Speaker B: Ah.

Speaker A: And then some other bugs that are really, you know, just kind of not filler.

Speaker B: Filler bugs man, let's be honest. Right.

Speaker A: Yeah, well, I mean you throw a whole bunch of target sets and models in you, you'll get some good ones, you'll get some trash ones. Um, but yeah, dropping them just, you know, straight up posting them on GitHub, you know, no vendor disclosure, no nothing, you know, in the year 2026 is a little bit rude. But you know, at the same time disclosing all of these bugs to their respective software projects would have taken longer than finding and writing up all of these bugs, uh, so you can kind of see why the researcher might be like, eh, I'll let the Internet do that for me.

Speaker B: I'll just let you know, kill them all. Let God sort them out. Uh, Cyber Edition, basically the thinking. Um, meanwhile, this next piece that we're going to talk about again, it's LLM Driven Research, it's from Spectre Ops, which disclosure, they are a, uh, risky Business sponsor and I'm an advisor there. Um, they published this blog post that was all about, um, accelerating EDR evasion with LLM Driven Analysis. It's cool. What's really funny though is I put it in our run sheet. Uh, thinking, oh, I got to put this one in because Adam's here this week and he's going to find this one really cool. He'll be into it. And your note on it is, this is cool, I'm into it. Which is exactly the words I imagined you using it. Uh, so please discuss, uh, Adam, if you would, this paper from Spectrops or this blog post from Spectrops where basically what they're doing is they're using LLMs to extract, um, the detection logic from EDRs so they can better formulate evasions. Basically. That's the pitch here, right?

Speaker A: Yeah, yeah. Obviously SpectreOps does a lot of red teaming. Uh, they have a big interest in understanding how security products work so that you can evade them. And everyone who's a red teamer has some tricks for dealing with antivirus, dealing with edr, uh, dealing with the kind of host based protection mechanisms that you run into every day. You know, application whitelisting and all that kind of stuff, like whatever things you end up running into in the wild. So in this particular case, as you said, they fed uh, some EDR products, you know, built a test harness that EDR product could be evaluated by the LLM. Uh, and the goal is extract all of the interesting bits of logic. And much like with, you know, firewall bypasses, if you can actually read the firewall rules, then you can usually find a way to thread the needle because the intent of the rule often doesn't match the actual specific implementation. And so that's kind of what they've been doing here with the edr. Find ways to figure out how it detects that a suspicious command line process is executed, like CMD M with funny looking behavior or something like that. Figure out exactly what those logics, those logic rules are. Uh, and once you can get them out of the binary, out of the plumbing of the uh, EDR platform, then you can figure out ways to, to circumvent them and letting a model loose, uh, on the binaries having it reverse engineer. How do the detection rules work? How are they encoded, how are they loaded into the detection engine so that you can enumerate them and find ways past. There's a really great application of a thing that LLMs are very good at. Um, and they've got some examples in here of uh, things they came up with and ways they can pass. And they said here that this is uh, they're talking about one particular, um, I think it was the Cortex XDR uh, product. But they've done the same kind of thing with all of the other products that they have access to and with similar kinds of results. So it's just a great methodology. And if you're you know, a ah, red team of these days, this is exactly how you would fill your tool chest, uh, with the things that you need.

Speaker B: Yeah, I mean it's just like every week now we see really interesting work being done with LLMs. It really does feel like it's been this year. This year is the year that it all sort of landed uh, on cybersecurity. I think next year is the year where it all goes to local models. Right. And I think that's the discussion we've had so far is looking at all of these export controls and all of this panic about things like Mythos and whatever. The headline we gave for that interview that uh, James did with Carsten, um, Knoll is Mythos on your desk using local alums to um. But you know, besides vulnerability, uh, uh, uh, discovery and exploit development, there is this stuff around like EDR evasion and whatever. There's just so many use cases here. It's uh, it's a great time to be in the industry. And I love it that people are using these things to find 15 bugs that they're just doing like full disk. Like most of the people listening to this won't even understand that reference, uh, because the full disclosure mailing list is like long defunct. Right. But they're going full disk just dropping o day on the Internet. I was explaining to James this morning that this is how it felt 20 years ago.

Speaker C: Right.

Speaker B: Which is why all the old heads are walking around with a spring in their steps. It's like it's all chaotic again. We love it. Uh, speaking of, uh, apparently, and this is bad man, like you know, there's this Persona out there, various rumors about their identity, possibly someone who worked at Microsoft, but there's this Persona out there called Nightmare Eclipse who dropped oday in Windows. Uh, Defender. I mean, it's long since patched now, but apparently ransomware actors are using are exploiting this Blue Hammer flaw. Which is no mean feat actually, considering that the bug is actually quite fiddly to, uh, exploit.

Speaker C: Yeah, it is. And you know, it's funny, um, when I looked at this headline, I had to actually go back and look over the catalogue of Nightmare Eclipse's work to remember, hang on, which one of these is Blue Hammer again? Because there's just been so many novel ways that they've published to get around Windows, uh, Defender. But this is the one that it's a race condition. So, uh, I guess, you know, the ransomware actors are kind of really up against, uh, two challenges here. Yes, it's already been patched, but also trying to exploit this requires quite a bit of patience and going over and over and over again to eventually get it to pop. Um, and just so folks are aware that the particular race condition here is it exploits just some really fiddly machinery in the update mechanism. There's like this brilliantly small window of time when Defender gives a file a privileged attacks. There's all the kind of things you see in an update process. Right. Update processes are privileged and do dangerous stuff because that's what they've got to do. And there's just this window of time when you're going to exploit it. But yeah, you know, it gives them the ability to pop short. That's got system privileges that'll get them access to the security account manager database. And so it's tidy if it works.

Speaker B: Yeah. Now, meanwhile, Sean Lingus has been doing a great job over at CNN actually, uh, filling in some details on this story we've spoken about a couple of times where we had, um, uh, people physically turning up to law firms in the United States trying to plug in malicious, you know, malware riddled, uh, devices trying to, uh, you know, get ransomware attacks going. So, you know, our frustration there was, there really wasn't much detail, uh, on how they'd been doing this or who these people were. And you know, there's just a lot of color in this story about how, you know, there's a guy turned up with like, smart glasses who's like, speaking Russian into them while looking around the room and, you know, another one where they like, you know, got someone to call a guy so he'd step away from his computer so they could plug it in and whatever. So there's just like a Bunch more color there. It's, uh, it's a great piece. Adam, what did you think of this one?

Speaker A: Yeah, it's great to see the detail and, you know, you can kind of totally imagine these things working. Like, I mean, it makes sense that, um, you know, people would resort to this kind of tactic because, hey, we're getting marginally better at computers, which is like, this is a success story, right? This is. We have gotten good enough at defending our computer systems that it's worth paying someone on telegram 500 bucks to go blag their way in, you know, to a law firm and plug stuff in so it feels successful. Although I guess plugging a USB stick and getting temporary physical access ought to be safe these days. You would hope. And there were some, uh, improvements clearly, um, to be made in physical security for making it safe to use devices, uh, with untrusted USB sticks or that kind of thing. Um, but, yeah, the thing that I find really great about this particular story is the amount of times as a red teamer, uh, we would see some people in the security testing industry that would do physical entry as their initial access mechanism, uh, on red teams. And those of us that preferred to do actual computer hacking instead of just talking our way in felt like we kind of poo pooed that, like, looked down on people who have to talk their way in because they can't do the hacking. But now we have life imitating art where cybercriminals are actually doing it. And so the things that we were testing as pen testers, when we would actually go walk into a data center or walk into an office, actually do matter now in the real world. So joke was on us, uh, snobs that thought, you know, hacking computers was the only way to do it. And, you know, maybe there was some value, uh, to testing people's office speed gates or receptionist protocols for, you know, for visitors coming in. So, yeah, I guess, you know, that's ultimately a good news story for all of us in the industry, even though it probably does suck a bit when you're the lawyer, you know, being targeted or law firms being targeted or their clients.

Speaker B: Such a computer person, uh, uh, sort of critique there, right? That'd be like someone from nsa, like, looking down on CIA because they, quote, unquote, out hanging, have to talk to people, you know, human, human, humans. Yuck. Uh, so what else have we got here real quick? Uh, Microsoft has extended its, like, you know, Windows 10 patch support until October 2027. It was supposed to end this year. Uh, so, you know, they'll charge you for, for patches and whatnot for that. They always do this. You know, James, uh, had a note in here wondering if this was a hat tip to the bug pocalypse. And it's like, no, they always do this. They always announce that it' then they always extend it because people ring them up freaking out and uh, you know, they just eventually have to. And then they just gradually ramp up how much it costs to keep getting your patches, uh, for your hideously out of date, uh, uh, software. And then eventually it all winds up on VMware, although you can't do that anymore because, uh, you know, Broadcom is going to charge you $1 million per endpoint with the way their licensing is going. Uh, what else have we got here? This is interesting. Um, so this one, you know, stimulated uh, a bit of conversation around the office. I actually sent this one off to Daniel Shell at Airlock because I figured he'd find it interesting and his reaction was, oh, interesting. So I guess I was right. Um, but yeah, so some threat actors are using like a malicious edge extension to get persistence. I mean it's not really getting them much that they can't get with malware. But I guess what is interesting in this case is it looks like they're kind of turning an edge extension into like they're kind of treating edge like a lobby in a way and trying to be a bit more stealthy by using this. Adam, why don't you start off by explaining to us like, how this all worked.

Speaker A: Yeah, so this is uh, a set of malware that's being dropped, uh, it seems through like click fix style, you know, click here to uh, you know, fix your computer kind of things where it will drop a malicious edge extension and then a harness like some scripts and awesome Python or some um, Visual Basic or whatever else to kind of bootstrap it where it will start a whole nother instance of Edge, so with separate settings, um, that is headless so you can't see it and uses this extension. So in your regular desktop edge that the actual human is using, if you go and look in the browser extensions, you're not going to see this because

Speaker B: it's actually, I got to say, I love the concept of ah, an extension for a headless browser. That's wild. We're going to strip down all of the functionality into a headless browser and then let you whack, you know, bloated extensions on top of it. You just think, what, who is that for?

Speaker A: Why not? Modular architecture is the future. Um, so uh, and then they build so in Edge and it's you know, uh, basis in Chrome there is a mechanism for communicating with external processors. So if you're using Chrome M like in the style of Discord or uh, you know, signal, you're using them as a host for you know, rich applications that need to interact with the rest of the operating system. There is a mechanism for doing that. And so this malware ships with like a native component that will do stuff the browser can't. A browser component as an extension and then the harness to kind of build this and have it run and then it acts as kind of regular common garden malware. But a significant portion of the functionality is running inside Edge and that seems to be a counter detection mechanism rather than you know like um, some. I didn't see any other particular reason you would be in the browser. Like the separate browser instance doesn't immediately get access to you know, cookies or session tokens or keyboard uh, input to the other real human used browser. So it seems to be mostly a lol bin style sort of glorified process hollowing I guess. Um, which seems like a lot of work and I do wonder whether versus just running a Python script because they're already shipping a Python process with it anyway.

Speaker B: But you know how EDR is man, you wonder like you don't have to do as much with your Python script script I guess is.

Speaker A: Yeah, I guess that's the thing. Like it's detection evasion in an interesting way. And I do wonder how far you could push this concept like what else you could do in the browser. Because as browser engines become more full featured and less inspectable by things like Airlock or other host based security techniques, maybe there is some value. But much like we've seen people use uh, some of the virtual machine engines or the Windows Linux subsystem or whatever else to you know, uh, obscure their stuff. So yeah, interesting, interesting work, you know a little bit. Why though?

Speaker B: A little bit why a little bit. Why? Uh, then we've got this blue kit phishing kit which is, this is funny. So they're basically thin clienting and remote browser into your browser like yo dog, I heard you like browsers. So I put a browser in your browser kind of vibes uh, for phishing which is like, I mean, you know, sure. Uh, what's interesting about this one though is like it, it's actually pretty clean, it works well. Uh, so I just wanted to include this this week because the engineering around these phish kits is like they're pretty slick these days.

Speaker C: Yeah.

Speaker A: In this case, they're using an open source tool called RR Web, which, uh, lets you take the DOM state from one browser, serialize it, shovel it over a websocket to another browser, and then kind of rehydrate it at the other end. So you have a user interface rendered in one browser that's actually being driven by a JavaScript engine in a different browser somewhere else. And then in this case, one's the phishing victim and one's the attacker operating as kind of like a man in the browser. And it's sort of the end game, I guess, of this kind of like man in the browser, because it used to be we would hard code JavaScript for particular sites and try and steal tokens or shim particular bits of the login process. This is the nuclear solution of just shovel the entire DOM back and forward and uh, push differential DOM state updates. Like some kind of like, you know, almost Google Docs style, you know, incremental update over the wire. Like, it's cool engineering, but it doesn't really buy you much. That regular man in the browser phishing, you know, didn't already accept being kind of like pixel perfect and engineering. Interesting. Um, but, you know, Yubikey's Web authentoken is still going to take care of it. So, you know, it's everything else around the edges that's the problem.

Speaker B: It's all the fallback things that happen when you click the button that says I left my Yubikey at home. Um, that's the, that's going to be a problem there. Uh, James, let's get your take on this one. Uh, there's some Mac OS malware out there where it's been called Gaslight. Uh, because one of the things that the person who created this malware has done is stuff it full of like, strings that are trying to indicate to AI agents that might be analyzing it that there's been some sort of unrecoverable error and whatever. They're just trying to gum up the works. We talked about a similar approach last week where it's like, you know, uh, someone annotated their malware with like, this is the routine that helps us build a biological weapon. And they're making the LLMs go, Whoa, buddy, no, you know, can't safety trample export banners. Right? Um, so, yeah, I mean, this is just a different approach to the same thing. I think we're just going to see more and more of this. Although Sentinel One I think did the work Here, the uh, analysis work here and they say, well, we don't actually have any evidence that this works.

Speaker C: Yeah, it's more of a, uh, interesting to see attackers gravitating towards this and just having a shot at it. I think the difference here is that the one we covered last week was, um, comments in source code being scanned. This was actually a rust binary. And so if something is, you're analyzing that binary, probably one of the first things it's going to do is extract the strings out of there, see if there's anything interesting. And that in theory could trip up an LLM. Like, I'm highly skeptical that this would work in anything other than the most naive prompt of, uh, hey Claude, I've got uh, this binary here. Can you tell me if it looks bad? Um, but ultimately the thing to remember here is, and I've seen this myself, is that a large language model has no concept of a deterministic way to say, look at this binary. Extract the strings. Now these strings can't possibly be real error messages. That sort of logic, you just kind of can't encode into something that you're sending to an LLM. You can try to.

Speaker B: This is the problem when the data and code channel are the same thing. Right? Which is, uh, it's a pretty fundamental issue that we keep coming back to.

Speaker C: Yeah, fundamental issue. But let's not forget it's also enabled all the wonderful things that we see out of LLMs when you do actually combine code and data. So that door swings two ways and I think it ultimately has unlocked more good than bad. And look, there are already things like various ways to do markup within, uh, an LLM transcript that can help give the model a really strong thing, signal that, hey, this is, you know, a prompt here. This is not a prompt. This is reasoning. This is not. But you know, these things, they still trip up on the basics. Like, you know, I was talking to you guys about the example of, you know, if you get the, you know, an NLM transcript has a dedicated space where you describe what tools are available to the agent. I've seen examples where in that chat, if you then chat to the agent about a tool that you're trying to create, it trips itself up and tries to call that tool that you've talked about, not the one that's actually available just because, well, it's an inference engine. It's inferring. So that's, you know.

Speaker B: Well, I mean, your joke in our notes, in our news management system is, uh, your idea for a startup is to replace LLM inference with deduction. A billion dollar idea. I would posit that that's a $10 trillion idea if you can pull it off. Uh, James.

Speaker C: I've already got some funding meetings this afternoon, Pat, so if we can wrap this up quickly. I need to go and get to talk to some VCs.

Speaker B: A billion dollars will be the. Will, uh, be the series A, uh, at a million. That's the seed round, uh, on an idea like that. Look, one thing I did want to float, though, is that, like, look, if you really wanted to take this to its logical conclusion of stuffing things into malware, that would really gum up the works when it comes to AI analysis. And I hate to even invoke it, but you would put child sexual exploitation material, uh, you know, child sex abuse material into the malware. Not only is this going to cause you legal problems as a threat actor, because imagine if you get caught, then you've got all of these additional charges of distributing that sort of material, which I imagine, if you're Russian, not so concerned about that, but you will be causing problems for the victims of that malware because now they're all of a sudden in possession of that material. Uh, you would also be causing huge problems for anyone doing this sort of analysis because they're going to have to kick, uh, off all sorts of procedures for what they do, uh, when this sort of thing happens and the, uh, models themselves. Can you imagine Claude's reaction to finding something like that in a code base? So, like, it's awful. I hate to invoke it, but I think it's coming. I think it's something that's, uh, that's going to happen.

Speaker C: Well, Pat, I think the one that's even more scary there is. It's not just about shipping the CSAM as the means to defeat this. It's shipping injected prompts to generate the CSAM on demand so that there is no actual way to filter this. And, like, that really does make it a problem for the researchers and attackers, less so for the people creating it. So it's diabolical and horrible. But, yeah, that's. This is. This is, as you said, this is a logical conclusion of where it ends up.

Speaker B: Yeah, yeah. I mean, we saw people playing funny games around, uh, blockchains, like, putting that sort of stuff on blockchains and whatever, just to, like, make life hard for everyone. But I think this would be even worse because by its nature, it heads out to different endpoints and whatever, like, you know, not looking forward to covering that when it eventually happens. Um, moving on. And yeah, uh, we've had a couple of incidents here involving online wagering. Because, like, online wagering is a thing. Online betting, online gambling is a thing in the United States now, that's relatively recent. Uh, as a result, we've seen a guy, uh, being sentenced to 18 months in prison for hacking DraftKings accounts and then like, draining money out of him somewhat, somehow. This was back in 2022. Uh, but it pairs nicely with a story here that said polymarket actually lost a few million bucks of user funds. They're making their users whole. What's interesting here though, is there seems a bit of confusion out there about whether or not this was a phishing attack or some sort of supply chain incident. I know. James, you looked into it. Did you find anything?

Speaker C: Can't find anything. Inconclusive. Uh, Catalan's thoughts was it might have been like a CDN poisoning. Um, there was an article I read that said, no, it was a JavaScript supply chain component. And then another one said it was a phishing attack. And so no one really knows. I think the only strong signal here is people looking at the blockchain and seeing these funds move where they shouldn't move. And that's.

Speaker B: I mean, it could have been all three. Someone phished a developer who provided a JavaScript component that was distributed through the CNN CDN.

Speaker C: Poke no, lost. Why not Both. Yeah, exactly.

Speaker B: Poke no. Los Tres, I think is what you see. Uh, what else have we got here? We have a warning from Mike Burgess, who runs ASIO here in Australia, which is the, uh, you know, Australia's domestic intelligence agency. Um, Burgess also used to run asd, so he knows a thing or two about cyber. And he's given a big speech in which he's talked about, um, how foreign, uh, threat actors, country, unnamed. China. China. China, uh, have been pre positioning in, uh, critical infrastructure. This seems like an acknowledgment of Vault Typhoon like activity in Australia, uh, which, you know, it's obvious. But it is interesting to see, uh, a senior government official talking about that. And further, it's interesting to see them not, uh, naming, choosing not to name the country. So this is just, um, yeah, worth, uh, worth noting on now we've got a report from the New York Times that says in fact its headline is Russian hackers were behind $2.5 billion hack of Jaguar Land Rover. Um, this is interesting because the Times being. The Times is pretty light on details here. They're just saying that they spoke to like five people close to the investigation who said it was Russians and it wasn't, you know, the comm kids who had been sort of claiming it. But there's just no detail in this report, so it's really hard to understand how seriously to take it. Does this mean that those kids got some initial access and essentially acted as initial access brokers and just gave shells to the Russians? Were they just pranking the world in claiming the attack and the whole thing was end to end? Russians. If it was Russians, was this some sort of grey zone thing where the threat actor was operating with the tacit approval of the state, or was it state directed to harm, uh, you know, the uk, UK economic interests because of its support, uh, for Ukraine. And, uh, you know, the Russians are so paranoid about the Brits, it's actually funny. They just see like British conspiracies everywhere. So the whole thing's a bit crazy, uh, in that we don't, you know, it could mean a lot or it could mean a little, and we just don't really know. I think the interesting thing though is like, this is why Russia loves this gray zone stuff, right? It's a spectrum of attribution all the way from, well, the, these guys operate with a bit of top cover right through to this estate directed. And us not knowing is, would kind of be the, the desired outcome, uh, here a la little, uh, green man in Crimea in 2014. Right. Uh, let's see what else we got here. We got an Iranian national, uh, who was arrested. Apparently, uh, an Iranian apt operator has been arrested in Montenegro and they were arrested at a place called Kotor. And I thought, okay, I'm gonna Google this place to see, like, why would someone be there? Is this a case of a foreign, uh, apt operator not realizing that their identity had been discovered by the Americans, turning up for a holiday somewhere nice or do they live there or what's going on? I googled this place and it looks. Oh my God, it's amazing. It's on my bucket list now. Kotori Montenegro. I want to go sailing there. Google image it, people. Uh, go have a look. Like, I actually got mad with Catalan Kimpanu because he lives like, even though it's a 20 hour drive, I'm like, that's so close. You've got to go there. Why haven't you told me about this place? Absolutely gorgeous. But yes, uh, it does look like a typical Disneyland trip gone wrong for this guy. James.

Speaker C: Yeah, I mean, look, uh, a break well deserved. Uh, $3.4 billion apparently of damage between 2013 onwards. So the guy's been busy, right? So you got to give him a little bit of a break. Clearly he was due for this. But my joke that I was talking to this morning was, you know, clearly might need an apt aligned travel agent. Because when you're going to take a holiday, let's maybe not go to somewhere that is both a NATO ally, a US Ally and part of NATO, because that's not gonna. You're not getting home from that one, buddy.

Speaker B: I mean, I don't know that there's too many places for Russians to go, uh, where they're outside the reach of the Western law enforcement agencies.

Speaker C: So that's why you need a specialized travel agency, Pat. That's what I'm saying. There's an industry here, niche.

Speaker A: This is a niche opportunity right there.

Speaker B: Yeah, yeah, but then you're just going to wind up doing camping trips in Kazakhstan every time or something like it's not, I don't know that there's a full time travel agent job in that, uh, you know, you're going to wind up of one of three package options. Now look, just before we go to, uh, I just wanted to mention that Unprompted, uh, the Unprompted Conference is happening in Australia in Sydney on the 18th and 19th of September. The, uh, Australian one is being organized by Mark Dowd. Risky Business is going to sponsor the conference. I'm going to be there. James, you're going to be there because you already live in Sydney, so that's nice and convenient. Um, yeah, there's going to be amazing speakers, some already locked in. Uh, but the CFP is open. So if you want to speak at Unprompted Australia, uh, I've linked through in the show notes uh, to a, uh, link where you can go and uh, submit a CFP. Uh, the CFP is open until the 31st of July. And uh, yeah, we all hope to see you there. Uh, but that is it for this week's news segment. Gentlemen, uh, thank you so much for joining me to talk through everything that happened this week. It's been a lot of fun. I'm on vacation for the next two weeks. It's school holidays here in New South Wales. So, yeah, I won't be around for a couple of weeks. But I'll, uh, catch you all, uh, when I'm back in three weeks from now. But, uh, yes, gents, thank you for joining me and I'll look forward to chatting to you again soon.

Speaker A: Thanks very much, Pat. We'll see you next time.

Speaker C: Yeah, thanks, Pat. Amazing. We're only halfway through this year.

Speaker B: That was Adam Boileau and James Wilson there with a check of the Wood Week's security news. Big thanks to them for that. It is time for this week's sponsor interview now and today we are chatting with Katie Warren and Daft Studded who are uh, from uh, portswigger. Portswigger of course makes the famed Burp suite, uh, web application security testing framework or product, uh, and now they are actually pushing more and more in an AI enabled direction which shouldn't be too surprising. Uh, we had uh, uh, James Kettle, who is portswigger's uh, director of research into an interview with James recently which we published, published to YouTube and I think into the Risky Bulletin feed where he was talking about HTTP Terminator, uh, and this was a tool that he developed which would apply his research methodologies in uh, a with AI and just you know, he could cut it loose on targets and get it to go and find crazy stuff and report it to him. Now obviously that's fun research but something called the HTTP Terminator doesn't exactly give people in uh, corporate environments the warm and fuzzies. Right? So this conversation uh, with DAF and Katie is really about like how do you build a security testing uh, product for uh, the enterprise in a way where safety has been considered, where this thing isn't going to run right, and do all sorts of unpredictable things. So here's my interview with DAF Studded and Katie Warren from portswigger, all about uh, building AI security testing tooling. Enjoy.

Speaker D: So I think the question in the past was maybe can AI hack? Can AI find vulnerabilities? And that question has been very clearly answered. Of course it can. Many people have demonstrated that. We've seen all kinds of examples of AI doing crazy tricks on vulnerability discovery. So the question moves on to what AI or what usage of AI can I trust to test my systems or the systems that I'm responsible for testing? And that opens a whole bunch of other questions, questions around uh, the safety and the governance of that process and how the human can bring their judgment to it, uh, but also questions about uh, what are the right tooling and research, uh, techniques and the right scaffolding around it for it to be maximally uh, effective.

Speaker B: Now Katie Warren is also with us. She's a product manager, uh, over there at portswigger. And I uh, want to bring you in here Katie, because I'd imagine that a part of like trying to scope out the development of a product like this is to go and Talk to customers and see what sort of things they want in the mix. And I'm guessing things like Kettle's research project. Do you know, would you be interested in buying a product called the Terminator? Um, probably less so. Right. Uh, and they're going to be more interested in the sort of stuff Daft's talking about, like, you know, sensible approaches to governance and controls and things like that. I mean, like, what's, what's the general attitude out there in buyer land when it comes to, you know, agentic security testing stuff? I'd imagine people are still a bit skittish.

Speaker E: Yeah, definitely. I think, um, something that we've seen quite a lot of, um, especially over the last year or so, is a kind of change in adoption, uh, around buying AI, but also just utilizing it a lot more. A year ago we were probably a lot more skeptical around what it can do. And as DAF said, it's been proven that it can do a lot. A lot of it now is around trust and kind of that human, the loop element, something that kind of practitioners and all the way through to enterprises really want. So governance by default has been something that we've built in from the get go, um, really understanding what users need, what they need to see and have complete control. So we have got, um, very strong governance and guardrails in throughout all of our kind of our product. But the really crucial bit is being able to adapt as things change so as AI can do more and agentic platforms can do more is how do you actually make sure enterprise scales and practitioners alike really trust that it's doing what they want it to do and what they need it to do, and not kind of taking away a lot of their, kind of their, um, empowerment and autonomy over it. Um, so it's kind of a dual hat kind of relationship of giving uh, the kind of platform enough that it can kind of go off and do some really cool stuff, but in a really trusted, visible, auditable way that um, is what people really want to see.

Speaker B: Well, it's interesting that you say human in the loop too, because this is a big conversation right now in security and AI, which is there's a lot of advice coming out saying you need to do everything at 100 times the previous speed and you need to use AI to do that, but you also need a human in the loop. And this is not really practical advice. Right. So I wonder, like when we say human in the loop, the human is only in some of the loops, right? I think that's the, that's really what's interesting here is because obviously you build an agentic product, there's going to be some loops that the human isn't in where the tool just makes decisions and it goes off and does stuff. But then there's other loops that the customers want the humans to be in. Like, so is that part of the conversation out there with customers which is like, well, where do you actually want the human in the loop? Like, where do you want the control?

Speaker E: Yeah, definitely. I think what we've learned is it's very dependent on what you're trying to do and where you are and kind of um, in your, on your test itself. So it's been an interesting learning curve to kind of go through of how are people needing to build that efficiency in and go that 10x100x that people assume you can go, but having that full control from a setup. So we have been looking at how we can build that in automatically through our guardrails and kind of smart kind of approvals type of framework through to full human in the loop for when it becomes more sensitive and you're in the kind of, um, the scarier parts of your workflow where the human actually has to be in the loop and you're in complete control of what it's doing and why.

Speaker B: I mean, this is a bit of a technical challenge though, right? So Daf, I'd like to bring you in on that. Um, you know, I've often described some of these agents and it's a moving target, right. Like they do. They are getting better, the models are getting better. But I've kind of described them as like infinity, ah, variably, as having access to infinity script kitties or infinity work experience kids as well, because they're like really eager to do stuff for you and they just. Absolutely. But they don't kind of understand what's normal. You know, like, uh, the great example of someone asking it to update a wiki and it didn't have creds, so it like found an ODA in the wiki software so that it could hack into the wiki and make the change that the person asked for, uh, just because it's eager to please. So, you know, how do you go about building a product that, you know, where do you like, how do you try to put deterministic controls on it and have it still be useful? Like, I imagine that was a big part of the challenge of building this thing.

Speaker D: Yes, uh, absolutely right. I mean there are two sides to this. So one of them, uh, as Katie was describing is around the kind of deterministic harness that provides um, the safety and control. And that just means very clear architectural separation between what the agent.

Speaker B: Don't give this thing if it asks to resolve a domain outside of this allow list, don't let it. Is it that sort of thing?

Speaker D: Absolutely. So things like scope control and which endpoints you can hit are uh, just fully altered, school boring, deterministic. And there's no way that it can just crank out a curl request. But the other side of it, and this is what was really interesting in James Kettle's research and in our product development, was that the amount of total processing and compute that happens is 95% plus fully deterministic using the specialized domain tooling that BURP suite is built on. So the agent will think and reason and decide what strategies to pursue and then it will invoke very surgically and cleanly those tools which will then run and do uh, a job like a focused intruder attack in that really controlled way where there isn't any scope for hallucination or making things up or ignoring instructions because it's just running exactly what a human would have told it to do.

Speaker B: You know, I describe the infinity script kiddies, or the infinity work experience kids. Another way that another term I've used to describe some of these agents or certain use cases is you can use them as what I'd call a self sourcing bash script. Right? Like it's like a bash script that writes itself so you could stick in front of a toolchain like burp. I mean it seems like that's kind of the idea here.

Speaker D: Well, I think part of the creativity of the models is that uh, non deterministic determination to go and do something and break any rules that they can. And I think we see that that's a feature, not a bug. And that's what does unlock some of that creative power. The challenge is just to expose the tools in the right way that um, you know, whatever the model thinks it's going to be able to do, we fully uh, do control what it's actually able to do. And we're able to give it the tooling that makes it work really well. Because in the real world's full of edge cases that a model out of the box won't anticipate and it will try something a couple of obvious ways and it won't work and it'll give up and then try something else, just carry on like brute forcing, uh, trying to achieve its goal. Because our tooling has solved a lot of those edge cases over couple of decades, uh, it's much more likely that the tools will work and will achieve what the model was actually trying to achieve. So it doesn't need to kind of try and bust out.

Speaker B: I mean, it's almost like we've evolved pretty quickly from that copilot model, right. Where if this were two years ago, you'd be saying, hey, we've baked like a copilot into, uh, you know, into burp suite, because that's AI and that's what we're doing with AI. And now we're like, well, we got to kind of have a human in the loop for some, some stuff. Right? You see where I'm going with this? I wonder, I wonder how long it's going to be before the humans out of the loop. Um, before that's a. Before that's a checkbox that people are just like, yeah, just go, go. Do you know, I can imagine for certain internal teams, you know, internal pen test teams, internal red teams. Right. Once they are comfortable enough with the product, they're just going to let it crawl around in their network and let them know what's up. Like, you know, Katie, do you think that we are headed to a situation where the market is going to be comfortable with that? I mean, personally I do, but you're the one out there talking to customers, you tell me.

Speaker E: Yeah, we're seeing a lot of kind of as people are trusting it and getting used to it in their workflow, letting it do more for them. Um, there are still core elements where you can see and we've heard from users of, I'm still here, please don't go and do this for me. Um, but a lot of the kind of repetitive and the repeatable kind of things that have been going on in their workflow. Yeah, we're seeing a lot of our users on our beta kind of, um, going on. One of our users said, um, something that would have taken me four hours to set up. I kind of just open a session and off it goes. Um, and I don't have to worry. And by the end of the day I'm already finding something useful. So, um, to see that kind of as that trust builds and working out as the kind of individual, um, expert where I want to go and um, when I want to intervene has been really interesting. Seeing the kind of progression and the growth in that. I do think, um, as you kind of give more context and you've seen it in kind of other areas, you know, engineers and developers are, uh, more autonomous with Things like Claude code now, but they're still engineers, they still are writing and making sure that they're building the right platforms and kind of infrastructure. So it's a similar kind of concept as you build it, build it for yourself, your own operating system, your own context, you can do more and also really dive into the hardcore stuff yourself and not really worry about it.

Speaker B: Now I've got one last question. Uh, this one goes to daf. I often talk about how threat hunting, detection, response, thanks to AI, it's all kind of collapsing in slow mo into the same thing. And I do sort of wonder at some point if stuff like Burp Suite winds um, up kind of becoming like a vulnerability scanner in some ways. Because the level of automation gets so high that you can effectively use it as one, it becomes less of an expert tool and more of a point and shoot and go do the thing. Is that where it's heading?

Speaker D: I don't quite know if that's where it's heading. I think some parts of that kind of the entire value chain through from uh, vulnerability detection, evidence remediation, I think the automation is going to eat a chunk of that. And certainly at the end of static code analysis, if models are generating the code, they're largely going to follow the patterns they've been trained on. So having the same kind of technology audit its own work looking for those patterns is going to make less sense. So I think the interesting part of the spectrum of testing is that on the dynamic side, it's where you're deploying an application and interacting with it and seeing what its behavior is to find the validated uh, vulnerabilities and spurious behavior. I think um, the difference, the product category between what's a vulnerability scanner, what's a specialized tool, what's a toolkit for humans is going to become much more blurry in the same way we've seen other domains become blurry, say between product design and engineering kind of converging. But I think effectively what this agentic tech means for every single use case you do have uh, intelligent reasoning just as if you had a domain expert human, uh, doing the work. And if the tooling is right, it does allow that domain expert to be supervising and guiding and bringing their expert judgment and control into what it does. So I think the possibilities for automation are huge. This is a huge force multiplier for domain expertise. But um, we don't see this just fully commoditizing and becoming something that you just point and shoot and it finds everything.

Speaker B: I don't know, man, I don't know. But look, ah, you know, we've still got jobs for now, and that's a wonderful thing. Katie Warren Duff, Studded. Thanks a lot for joining me for that conversation. It's very interesting stuff.

Speaker D: Thanks, Pat.

Speaker E: Thank you.

Speaker B: That was Daft Studded and Katie Warren there from Portswig. A big thanks to them for that. And that is it for this week's show. I do hope you enjoyed it. We'll be running a soapbox edition of the show while I'm away on vacation. That one is with Damian Luki from Nebulok. Ah, very interesting stuff. Uh, but otherwise, I will see you all in three weeks when I am back. Thanks for listening,

Speaker A: Sam.