LLM Kiddies: The New Script Kiddies? A Veteran SOC Analyst's Take on AI's Security Revolution

Speaker A: Dylan Williams, thanks very much for joining me today. How are you?

Speaker B: I'm doing great. Thanks for having me on David.

Speaker A: It's always amazing to see people that are um, in the cyberspace but also keep with the program. Meaning that ah, you turn in from a ah, practitioner. Traditional uh, practitioner to somebody who specialize in LLMs and AI. So why don't you uh, just kind of walk me through memory lane. Just so for people that are uh, unfamiliar with what you've done and what you're doing today.

Speaker B: Yeah, yeah, totally. So I mean majority of my security career if not all of it has been security operations right. So pretty much everything. Detection and response, it's your typical blue teaming stuff. So I work for um, Enterprise Security Team, you know for internally for a company. Um and that's where I, my specialty lived. Right. Most of my stuff was around um, and so uh, you know probably the past couple years most of the work's been around detection, um, writing, research, threat hunting. Right. Uh, really cloud, cloud native stuff. That's what I found the most fascinating. That's probably one of the newer things that folks in that specialty are exploring right now. Um, but yeah and then it really uh. So we just passed I think the second anniversary of CHAT GPT, the Gen AI explosion. Right. Um but yeah probably a year and a year and a half ago ago um, I had my CHAT GPT moment. People talk about. I'm sure you're familiar with that but um. That was a whole uh, thing for me. But the second piece I guess that really got me into the rabbit hole of large language models in general was like bringing that um, and automating significant portions of your personal life or your professional life and that's kind of where you see the impact. Um, it doesn't happen like that. Right. Um, it's one of those technologies that's more of an art than a science and it comes with like six months of experimenting. Like that's what I think is really cool about this. It's, it's really being a ah, healthy mixture of an engineer but a scientist too. Right. With this technology specifically. So yeah. So um, that's how I got from a practitioner to you know, building with LLMs. Um, you know um, uh, a lot of the stuff that I talk about is taking that idea and applying it to our domain of cybersecurity which is like a really unique form of expertise in itself. Right. You know AI is been around for you know it could be medical, lawyers, uh, all these high level specialties. Um, but I kind of Stick it to our domain. There's so many unique specialties in there that this technology can be used to, um, augment a lot of our. Our skills. Right. Um, so that's kind of what I'm working on now. So, um, uh, we just started a cybersecurity venture, so just follow me on LinkedIn. We'll be sharing a lot of stuff we're doing shortly. Uh, pretty pumped. Um, so, yeah, that's kind of where I'm at now.

Speaker A: So what attracted you to the kind of the cyberspace cybersecurity Operations Center? Was it incidental? Uh, you just kind of fell into it. Walk, uh, me through, like, the early journey and for the people that are not familiar with it, um, because it's. A lot of people want to get into this space, and it's a little bit of enigma to them, so it's really interesting. And I feel that the whole LLM space and AI provides an additional opportunity for these people that want to get in because they can specialize in both cyber and, uh, you know, the application of AI.

Speaker B: Yeah, 100%. Yeah. So I started out as an intern in, uh, information technology and it, and I think the story is literally like, I saw, um, like, Craig's credentials go in raw text over the wire, and I was like, that's wild, man. Um, and I was like, you got to see. So that kind of like, opened the can of worms for me. I was like, what's going on here? Um, but yeah, I mean, if we divvied up into. You got the blue side defense red time.

Speaker A: Hold on a second. Is that. Do you have inherent curiosity? Uh, we always curious because that's, you know, I find that's a common thread with a lot of practitioners.

Speaker B: Oh, yeah, 100%. Yeah. I mean, you could be the type of person, you're like, you know, whatever. I saw that, but I was like, don't you want to know how that worked or why someone's password, like in clear text over the thing? So, yeah, it's totally a curiosity thing, 100%. Um, I think that's a good way, uh, you know, for folks like, I'm a big, um. I think the term is auto. Autodidact. But it's just a fancy way of saying, like, you like to go out on your own and learn and teach yourself things. Right. And it's great. Today especially, you know, overlapping with the whole AI stuff is, um, you know, having YouTube at, uh, your disposable on your phone. I mean, there's so much you have Accessible to you. And then now having tools with AI, we have such a big lever to like accelerate the rate of learning or it's just like an access thing. Right. So times have changed quite a bit that way. Um, but yeah, I started there and I think I went totally down the path of, um, I do think certifications do have a certain place at a certain skill level because it's like a really affordable way to get, uh, credibility. Right. Whether it's on a resume or you just having to chat with somebody, it shows that you care. You're motivated too. Right. But yeah, if you, this is like you starting fresh out of the cyber field or IT or network in general, I think that's a good starting point and also will expose you to the whole umbrella of this world. Right. And you'll have visibility into all these different specialties and maybe one will, you know, be pique your interest. Oh, let me go go check that out. Right.

Speaker A: Um, yeah, yeah. And there's no excuse today. It seems like first of all, you've been giving back by contributing, uh, information and articles and so on. And it's really interesting because there's no excuse. As you mentioned, a lot of it can be just acquired by going to GitHub, looking at repositories, looking at, uh, YouTube videos, uh, Telegram channels. In uh, fact, now with AI itself, I can go to a Tropic plot or ChatGPT and ask it to teach you cyber or teach you how to do things. It's almost like it's your own personal tutor. There's really no excuse these days to not do it. But yet, you know, some people still looking at, you know, boot camps and so on. You can have Claude run you through a boot camp just by prompting.

Speaker B: Yep, exactly. Yeah. It's almost to the point where we have the opposite problem where like, you have to start creating an information diet because there's so much stuff out there, it is borderline overwhelming. So, um, and a big part of that too, I feel like it's becoming difficult to sift good quality stuff, uh, from the rest of it. And like, I'm totally a victim of, I don't know what that phenomena, uh, is if you have too many choices, like you're going down the grocery store, there's like 70 different cereals. I'm going to be there for hours, man, trying to pick that cereal. So, yeah, you gotta. But no, a big part of that

Speaker A: I think is so you can't go, you can't go hungry to the grocery.

Speaker B: Right? Right.

Speaker A: You can't go hungry to the grocery store.

Speaker B: Exactly.

Speaker A: Yes.

Speaker B: That's, that's a good hack. Exactly. Um, but yeah, a big part of this is knowing where to go. Like, you know, I totally five to ten, uh, resources that I think are like vetted, you know, reputable, whether it's from people you follow online or you heard through word of mouth. Right. Um, as a starting point and going there for learning and stuff. But yeah, there's, there's, there's so much stuff out there. So, so much stuff.

Speaker A: And, and isn't it amazing? Like, I mean, you know, this, this whole prompting an AI, you know, real AI, not like the, the one where you used to use with quotation mark, uh, only like two and a half years ago. Two years ago. It's relatively new. But there's still. So first of all, a lot of people say, well, I don't know, I don't remember my life beforehand. Like how did I do my work before using these tools? And then on the other continuum, uh, scale, there are people that are not embracing it. I mean, how many people do you think use? I mean there's quite a few, but how many people.

Speaker B: Whoa.

Speaker A: I would like to reverse question how many people do not use it on a regular basis and are completely oblivious. The fact that there are tools like this and what's amazing is the speed of which these tools are getting better and better. I mean it's six months ago prompting did not give you the same answers today. Uh, these models are getting better and better.

Speaker B: Yeah. The range, how many people do you

Speaker A: think are not embracing? Yeah, go ahead.

Speaker B: Yeah, I think it's pretty split down the middle. I'd say majority are not. And uh, we're totally victim of being on our own bubble bubbles of people who are in a technology industry. So we're like learning, touching all this stuff all the time. So it comes very natural to us. Right. Like as far as first adopters to this. But I think outside of that bubble, a majority of the world is probably not even uh, privy to this, this sort of thing. And it's still been two whole years after we had LLMs at this uh, caliber come out to the public. Right. You have a phone, you got a computer, you can use these things. Um, I don't think it's far fetched to compare this to when like the Internet or the search engine came out. Right. I imagine a lot of the same experiences were we had. People are like, look at this thing. It's incredible. How are you not using it? And the person to left or Right. Of you is like I never heard of it. What are you talking about? And you kind of got to show them. And so you might have this experience with like family or friends or even at work with my co workers all the time. I think is, is the thing. But um, that's. I think a majority of people are not uh, are very naive to this. We have this piece of technology here that's pretty incredible.

Speaker A: So and, and where do you think in a high level. Where's the promise with LLMs to, to help the defenders in the whole cyberspace?

Speaker B: Yeah. I um, mean you know at the same time I do think there's, there needs to be a healthy dose of skepticism as with any new technology. Right. Um, for us as defenders, um, I really like to look at it as enabling us to become power users. Um, this could be the blue side, the red side. But I just think it's very equivalent to um, doing something manually. Maybe you're tailing logs. Uh, which everybody used to do before we had all these fancy tools like a SIEM or a data lake. Um then now we have another piece of technology that's another abstraction.

Speaker A: Log is a term Dylan before because I have to. For these people are not familiar with it basically at using. For you it's natural because you're like you know, but you basically it's a tail command to show, to go back and to see what the last like 50 lines of logs. Right?

Speaker B: Yep, yep. Yeah. Just manually on the command line. Yeah. You like imagine you have a really large text file and ah. You're just looking at it. Yeah.

Speaker A: These days seems like it's like who does that today? Right.

Speaker B: You're causing more pain. Yeah. You're causing more pain than good. Yeah.

Speaker A: But then fast forward SIM was all the rage basically these data lakes storing all that and then doing commands to extract I guess some of the knowledge associated with it being captured. So operating on that data. So that was all the rage for a little while but that didn't.

Speaker B: Yeah.

Speaker A: Really pan out I don't think.

Speaker B: Right. Yeah. Like I, I think we have um, these new backbone technologies that either make their way from another industry into ours. Right. Sim's a great example of um, uh. And then taking it a step further like you know we, we have the, the snowflakes, the data bricks, the warehouses where you know I try to keep it simple. Stupid. It's like it to us as operators we're doing one thing. We use this tool to interact with the data. That's really it. Right. To me it could Be no different if that's my day job. Whether it's a data warehouse, a splunk, a sim or some other bespoke thing that you built your company. Sure. Um, but I think LLM is a search.

Speaker A: I've heard sometimes.

Speaker B: Exactly. Yeah, exactly. Super popular, super big one. But yeah, the LLM angle for defenders is interesting because um, it takes a certain level of expertise to know one, uh, if not more of these tools. Right. And there's dozens of them. You could totally uh, as a defender be using like easily 10, 15, 20 tools on your day to day depending on your company, how large the organization is. Right. But LLMs are a great way. It's like I uh, talk to my buddies about this and compare it to um, you know, you could go, let's say you run into an issue or you have a question you want to work out, you naturally you're going to go to search engine like Google. Right. Maybe browse hours through stack overflow, you know, stuff like that on the forums, trying to figure it out. Well now we have this piece of technology that listen, you know, they're pre trained models. There might be some hallucinations. It's not perfect 100% of the time. Absolutely not. But on the other end we have this thing that I could probably get to answer in less than five minutes. Why wouldn't you use that? So I think that we're teetering on that edge of uh, people are going to start to go here and then I'm sure we'll talk about. There's a lot of ways to get around that hallucination problem too. Right.

Speaker A: But yeah, we definitely, we're going to double click a lot of the issues uh, that are eventually going to get solved um, with, with all kinds of other means. But um, talk to me about for example there's this dude like co pilot by Microsoft that was supposed to be again all the rage, you know, for $25 a month, whatever. It be like your own personal uh, co analyst I would say. Right. So you, you know, you run it, you have a ability to ask questions but that really take off. Didn't take off as much as people thought. Right. I think it had some limitations.

Speaker B: Yeah, yeah, that's been a um, tough thing. Like a big thing that I try to help people with is to sift kind of the, the, the, the snake oil or a lot of the vendors or extremely large enterprises who have these large flagship products and just slap AI or LLM or gen AI whatever onto the product. Um, yeah, so I didn't use Copilot a whole lot. I know that people were not happy with that experience. But yeah, I think um, it really, you should be skeptical of things that unfortunately probably a large portion of the market is people using chatgpt wrappers or just uh, putting a gen AI chatbot onto some component of their product. Um, that's cool and all. I think it just comes down to what's the impact. Was it useful for you? Like, I'm sure there's a portion of Copilot users out there that were really happy with it and that's, that's great. Um, um. But yeah, I think some of the more interesting products out there that are building uh, you know, LLMs into the product or building around LLMs on a new product. Um, it's going to be um, a lot about how it incorporates with that person's workflow. Right. Like I, I've been a huge fan of um, cursors, like an AI, uh, IDE integrated development environment. Right. We probably have a lot of folks who use VS code. Um, they just um, forked that and it's like. And that you know, a lot of the, the way that we use um, AI chatbots today, it's a lot of copy and pasting going back and forth. Pictures, screenshots, uh, messaging, um, the products that I think are uh, interesting, it's going to be bringing that closer to the product that you're using. So like even we talked about the defender thing. I know there's been a lot of copilots or chatbots like text to uh, uh, query language. Right. To elastaserge to uh, spl, um, stuff like that. Um, but yeah, I think that the two things that stand out to me, I think about execution around bringing AI to a product. You can call it a copilot, you can call it whatever, um, is the user interface and the user experience I think are pretty important. Um, and the second it's the system around the LLMs, which a big part of that is, uh, the LLM is not going anywhere. It's going to be part of the software stack probably forever now. Right. But it comes with its own, ah, uniqueness, own interesting things when you're building around it. Um, but if you nail those two things, um, I think you're going to set yourself up for success. Right. But yeah, I like to think of it as copilot. Um, might be a term that like we have Chatbot now we have Copilot. I do think it makes sense to think of it um, as like a junior uh employee, junior analyst, an intern. Right. That you can kind of offload or work with where a lot of the tasks that you're doing to speed yourself up increase uh, the quality, consistency, accuracy of your work. Right. Um, but yeah it's a similar thing to like, like I said earlier like the search engine or Google. Right. Or what are your other go to tools that you use when you run into issues for your day job. Right. You don't have to be a cyber security person any, it could be any type of job. Right?

Speaker A: Yeah. And these tools are getting better and better and the integration with other tools. So in Entropic example they came out with a integration to desktop. So essentially you can now have a control anything that's on your, on your laptop or desktop completely uh, natively. So you can essentially have it uh, do your tasks as you may mentioned, workflow, automation where you decide okay let me frame it, what do I do in a regular basis? And then let me let have Entropic uh Claude do it for me completely even using the same enterprise tools or whatever because it has access to now I don't know how risky that would be to rely heavily. Maybe you could just have it do it automatically and you sit down and say well just let me know when you're done and I'll take a look at it for a few minutes and let it run in the background. Uh but essentially you can automate a lot of this work. Don't you think that essentially eventually we'll get rid of all the analysts and have it. If the engine is good enough and we remove some limitation from hallucinations and some of the things is not correct uh then eventually it will be completely automated.

Speaker B: Yeah, yeah I think it was Google put out this um, paper uh, or this guide on the levels of autonomy with respect to LLMs. And I know people compare it a lot to like there's the five levels of autonomy for self driving vehicles or cars. Right. Um, but, but they basically started with um, assistant. Right. It's kind of like how we use ChatGPT or CLAW today. Right. Um and then more to like a copilot or semi autonomy. Right. Semi autonomous. Uh and then the last step would be full autonomy. Right. There's no humans in the loop. It's going to do it autonomously, uh whatever way it thinks best to complete the task. Um, I think we're for uh, most tasks especially in cyber that it's a high impact, high risk environment. We're probably not going to see that for a while. I um, don't think human in the loop, um, is going to go away for a while and I think that's great too. And even, um, LangChain has a great article where they talk about human on the loop. And it's like imagine, um, there's all, you know, the LM's doing all these tasks for us. Right? Um, but we're going to pick and choose where does a human expert come in and approve or deny or review and change the work product that the LLM is making for us. Right. Just like you would, um, do it with a coworker or an employee. Right. I think that's a great way to think about it. Is um, you know, the difference between, you know, now, in five, ten years from now is we might have a lot of this work being done behind the scenes. Um, the heavy lifting is done by the LLM, but at certain points, um, we'll have a human on the loop. So like imagine you have a, you're a cybersecurity defender, you're an analyst, you have a really complex task. Maybe it takes you an hour. It's 20, 30 different tools or steps. I think a significant portion of that will be automated. But we, it's going to be up to us to strategically pick which ones we want to have a human. And the task can't finish until we go in and say yes or no. But I think that healthy mixture that's the orchestration of that, um, is going to be pretty critical. But yeah, I don't think, I think of it more as getting augmented. It's a humans plus AI, not one or the other. And I don't think it's going to be one or the other for a really long time. Yeah.

Speaker A: And let's talk a bit about the limitations today, uh, with these LLMs. So first and foremost, I just saw a couple weeks ago there's a quote from Snowflake CEO say that they, these LLMs are 5% wrong. And the issue is we don't know which 5% that is. So that's number one. Number two, they're, they're fairly slow. A lot of time it takes, When I say slow, it might take 20, 30 seconds for a query. And uh, depending on uh, you know, what it is you're doing, maybe if you need like automatic detection, you know, it's too slow for that. Uh, secondly, uh, from an energy consumption, I, I made a comment to a LinkedIn post and somebody like immediately went on my tail saying that, oh, you know, we shouldn't worry about the green Planet Uh, you know, when it comes to, to progress. And I said my response was, well I don't necessarily, I mean I'm all for green, but it's not about that. It's about the inability for us to support these massive uh, operations. Hence Google investing in nuclear power. So there's quite a few. And then the context. Right, so meaning that you don't have enough uh, context associated with it. So I have a certain capacity to jest, uh, you know, the type of information. So that's also limited in nature. You cannot, you know, potentially, you know, just tens of thousands of data points. So there's quite a few of these uh, limitations right now for the technology. So let me, I have another follow up question to that, but let me kind of let it, let it linger, uh, for a second.

Speaker B: Yeah, yeah. Um, yeah, I mean if we look at the rate of change that's happened since um, you know, was it GPT 3, 3.5 came out uh, two years ago. Right. Um, the models are clearly uh, getting marginally um, better, faster. And when I say better, that's pretty ambiguous but we could say like um, the quality and accuracy of the results are better. The odds of it hallucinating are going down. Right. That's what we define as better for us. Right. Um, I know recently I've seen a lot of people talk about uh, plateaus, technological plateaus.

Speaker A: Yeah, you beat me to it. I was about to, but I was about to not stop you because you were on a roll. But thank you for bringing this up.

Speaker B: Yeah, yeah, yeah, no, I mean maybe you can educate me. I don't know like too much about the devils in the details so to speak of what that means. But it's like any other technology, right?

Speaker A: Yeah, I think it was due uh, to the fact that there's limitations associated with the number of the size of the model. You know, it doesn't grow when you do, you know, to infinity and then it gets exponentially better. There's certain plateau meaning that there's, you know, if you go to several billion data points or you uh, know artifacts, then if you go to several billion times 10, it doesn't, you know, the models become 10x better. So I think that's where the limitations are.

Speaker B: Yeah, like um, yeah, like. So I can't speak to, I don't know. I mean I think humans are notoriously bad at predicting technology in general. So I, I don't think this is going to be any different from that. Who knows if um, you know, the next five years are going to be insane or it's just going to be, you know Claude 3.6 will be a little bit better but I mean we're in a place where the technology for my uh, you know, anecdotes is pretty incredible already. Like I'm, I think a lot of people are really, really happy but I don't think it's necessarily um, going to be just the models getting better, faster, stronger. I think it's really going to be an engineering developing thing. It's like the models are already so, so, so good. Um, it's going to be people who have spent and done the homework. It's been so much time understanding the minutiae of building a ah, product or a system around it and I think it's going to be most of that. Doing that really really well is going to bring up the bar super, super high even if the model hasn't gotten um, a little bit better. Does that kind of make sense? That's where I think it might be interesting. Yeah.

Speaker A: And now you're seeing a lot of um, there's a lot of promise in using rag, multi agents, you know agents. There's, there's quite a few startup actually that are uh, looking at automating SOC, uh, you know, threat management by using um, using LLMs, but in the agent uh factor meaning that they're running parallel and then uh, you know using RAG to automate some of the, the data ingestion and so on. What's your take on that direction in the market?

Speaker B: Yeah, um, I guess we can totally compare it to different levels of automation. So in this case if we pick on ah, alert triage which is just like a human's job as an analyst is they get um, a batch of alerts every day and they're supposed to label them, investigate them as benign or bad. It needs to be escalated. Right. Um, and we consider that like the low hanging fruit more or less is. And when I say that I mean with respect to the time and the effort and the level of expertise required to do that job. I think that's a really natural, good starting point. But if we go back and look at our initial attempts to automate this right. Uh, could be basic scripting like you know, I'm a human having to manually go and do these things to figure out if the alert is good or bad, that's still meaningful. It saved me time. Uh, it did things correctly as I would. Right. We can measure that too. Um, yeah, I think, I think starting there and then you know we get into soar Playbooks. Um, I think a lot of folks probably have already achieved um, the level of automation that a lot of the AI SoC tools will have. That's not everybody, but these are people who've been doing it for years are already really, really good at it. Um, but that's not to say that uh, you know, introducing ah, AI as a uh, means for automation can have the same efficacy if not better. I think that's totally doable. But I think it just comes down to um, evaluations. Like we have uh, you know, three or five team of analysts. It takes them this long to uh, triage these alerts and percentage of them were accurate or not. Right. Having those numbers. Um, and I think a really cool way to compare it to the um, the AI or the autonomous SoC, uh tools out there is they'll do like a retrospective. So go back a year worth of the alerts that a human expert has already triaged and just look at the results. And I feel like it's going to be difficult uh, for somebody to say hey we can you know, triage, you know, 50, 60, 80% of your alerts with the same efficacy as your team, um, if not better. I mean that's, that's, that's a win, win. That's a great. And that's not talking about uh, the you know, accepting that risk of. We're talking about full autonomy now as opposed to semi autonomous. But I think yeah it's, it's, it's going to be just measuring the impact. You uh, know, try to be objective about it. But um, like I, when looking at these sorts of tools, whether it's in the software, other parts of security. Right. I know a big popular one now is for the GRC folks. Um, you know, trust is like they have a lot of controls, uh, long list of controls and questionnaires they have to fill out and stuff. Right. Um, I think of it as like ah, like a kind of like a 3D box and you know, one angle we have complexity or a better way to put that is like requisite ah, subject matter expertise. So like that would be lower for like a tier one analyst in a soc versus like a tier three, five, whatever you want to call it. Um, that's one. Uh, and then uh, uh, labor, like how, how much labor is it costing you resource wise? Like uh, is it taking hours and hours and hours per human, per team, per day for this function, this capability in your org. Um, and the last is, is speed, uh, or cost really. Right, sorry. Um, let's. No, I think what was I trying to say? Uh, risk, risk or impact. So like if you think that in a 3D box, I think a large portion of that uh, is really good opportunity um, for using this technology to automate. But you got to pick along those, those axes where you feel comfortable, uh, you know, with the risk and the impact. Right. So like one, I think good uh, compromise is like uh, the triaging the low hanging fruit. Typically more low risk activities might be really, really good for an LLM to step in and do it semi autonomous, autonomously if not fully autonomously. But um, the higher you go and the risk, we're just going to loop humans in appropriately. Um, but that all goes back to that idea of us being AI with humans augmenting each other's work. Right. Uh, but yeah, I think that's probably a good way to start to think about it. Yeah.

Speaker A: And it seems like there's a whole new uh, area of cyber now which is the, the protection against uh, LLM poisoning and be able to put some guardrails around AI. You know, it's super interesting because again coming back to the limitation of these models, there's a lot of time cases where we don't know who actually trained the model. Uh, we don't know the. It's very much like a black box. There's no expandability. Uh, so explainable AI, uh, and transparent AI, something that uh, a lot of companies are struggling with. So what's your take on that in terms of be able to A, understand what the model did, uh, B, making sure that the model is not biased, um, due to the other training or poisoning or whatever the case may be?

Speaker B: Yeah, 100%. Uh, I think that's the best way to go about it is we need to have explainable and transparent AI, um, that makes the people who are using it way more comfortable. That shouldn't be the only reason. It's because we just want to see, um, the LLM is a black box. Right. Um, and a lot of these products that go off that model is they might have AI somewhere in there. I'm, um, putting information in, I'm getting it back. Um, you know, like, show me your work. It's just like a human. Right. Um, you know, we're just automatically trusting that the work product that comes out of it is, is good or okay. But we should always have the opportunity to open up and take a look. Right? Um, I think, yeah, explainable and transparent, that's the way to go. I think the best way to go about this is to have um, quality assurance, it might be a very mundane feel. I know people like to say that qa, but it's totally necessary, totally necessary for so many things. It's so important. Um, and I think a great way to put this is when you vet or hire people for your team, uh, you know, you're trusting them to do the job up to a certain standard. And I think we can borrow some concepts for that for when we're incorporating AI into these workflows. Right. Um, but a big part of that is like, show me your work always or give me the option to go in and take a look. Right. Um, you know, hopefully as granular as we can get. Right. Um, yeah.

Speaker A: And should somebody, let's say you're running a SoC, and as an MSSP or as a company, should you just wait for some of the existing tools to just slap on AI? On top of that, I'm assuming that there's Quite a few EDRs in SIM and whatever tools. I mean, everybody's adding AI capabilities, so maybe as a practitioner you just have to kind of stay back and wait for it. Maybe you don't have to do anything.

Speaker B: Yeah, yeah. I mean, I think it's going to be like any other technology if we like pick on EDR vendors. Right. Endpoint, uh, protection, you know, in the early days it was just, you know, uh, even before edr, av, like signature based. Right. Um, and then they moved to more fancy stuff, so to speak, like heuristics or pattern matching. And then when EDR came out, we get full blown, um, you know, machine learning. But it's just a different means to an end of surfacing alerts. Right. Um, but I think the whole point is regardless of what vendor you are, this new technology comes out, everybody starts adding ML or if you're in the sim, everybody started doing risk based alerting, um, or user behavior analytics. Same concept. Right. Everybody's going to start adding, uh, uh, LLMs, whether it takes the form of a copilot or chatbot. Uh, we'll see. But, um, yeah, I mean, I think some of the most valuable stuff is, you got to ask yourself, um, it's still going to be siloed to that one tool. Um, I think some of the stuff that we've seen that is a real pain is like I mentioned earlier, the average defender has probably got, uh, 10, 20, 30 tools. Right. Well, now I have 10, 20, 30 AIs. It's not really helpful. So I think you should come at it from an angle of is it more impactful for you to spend the time or effort uh, to go procure a product or tool or research and build it yourself. That is really more of an augmenting thing because the value to me is um, one use case is I have all these different tools, these different steps, uh, that I have to do. Um, having one central place which I think is a really good candidate for AI to help out because that's what it's really good at. Right?

Speaker A: Yeah. Maybe like a AI orchestration tool. I don't know if you've seen that video of this. I uh, think it was a airport robot that went ahead and convinced all the other robots to quit their job. So something like that. But on the positive side where it would be like the orchestration of all the other tools, just tell them hey, you gotta step it up and figure out uh, how do we make this better. Maybe there's a startup idea right there,

Speaker B: right there, right there. Yeah. If it takes physical form as marketing I think kudos to you guys. That would get people pretty excited if they saw physical robots walking around using a saw or something.

Speaker A: So while we're doing this and speaking about all these eight uh, tools, the adversaries are doing a phenomenal job at embracing, embracing the technology. Uh from uh, morphing existing like zero days to exploiting vulnerabilities to real fake. I mean the list goes on and on and on in terms of how AI has been exploited and even I think in the nation state level meaning uh, again they talk about, I don't even know if quantum computing is real. I'm assuming it is but be able to break ah, encryption. So what's your take on where things are going on the other side of the uh, of the aisle?

Speaker B: Yeah, I mean, I mean it's very similar to um, you know a lot of the, the ethical hacking or red teaming that, that we do to test our defenses before the bad guys even get a chance to do it. Um, a lot of those tools and techniques and methodologies um, can get leaked over or you know, if they're public. A lot of open source tools can be used for bad too. I, I think it's um, just like anything else. Um, the difference here I think is that um, AI, you know, Gen AI this past two years is such a powerful lever, it's such a big lever. Um, and I think the rate of change is going to be much greater than some of these um, than our predecessors. Right. And so if you think about what we can do um, with, with LLMs or incorporating them into tools today um, what's, why can't a bad guy or a group of bad guys, uh, use these? It's absolutely the case. And it's like the barrier to entry has been lowered dramatically. That is why I think this has had so much impact and why it has potential for much more impact. Right.

Speaker A: Like, and Dylan, it started with, I think that the original barriers to entry went down. If you remember the. They used to talk about script kitties.

Speaker B: Yeah, exactly.

Speaker A: So now it's like LLM kitties.

Speaker B: LLM kitties. Yeah. I don't want to know what that's. Yeah, but think about it. It's like. And people, you know, assume that um, you know, a lot of the, the AI companies, uh, like OpenAI and Anthropic have done really, really good job at uh, and Meta and llama. Really good job of um, building uh, models that are secure and trusted and you know, they can't be used for, for bad reasons. Right. Um, it's a whole, a whole science in itself. But there's totally ways out there just like you could, yeah. Jailbreak or crack software. You could jailbreak or crack an LLM and. Exactly.

Speaker A: So yeah, you download, you go to, you know, you download your like local instance, like you know, using glamour, you install it and there's a hugging face where it is thousands of these, these models and you run it. And uh, it's actually some of these models even tell you to begin with that there's, you know, they don't have any rails.

Speaker B: Yeah, yeah, there was um. Yeah, I mean I think LLM kitties, we're out to see if we can coin that term. I haven't heard that yet, but that's. Yeah. Like if you can sit at home and you know you're gonna have uh, LLM write polymorphic malware for you, you don't need to be an expert in, in malware writing. Like. Yeah, I mean we're getting there. Right. So um, there's a really, really wild paper by OpenAI, I think it's from earlier this year or last year about you know, they have like these experts come in and red team the model. Right. Um, but one of them was like, if you think about it, um, what's like the real world impact, somebody can leverage this thing for bad. Right. And we're talking about the inference providers, OpenAI. It's a third party service sitting out on the web. I pay for it, I can use it. Right. Uh, and now that we're getting into the agentic world. Right. I think 2025 is going to be, that's what it's going to be about. We're giving the models access to take action. Uh, maybe on behalf of a human or not. Um, but like we live in the Internet age, right? Like there is, the Internet is tied to the real world and stuff. But in the paper they go over, they wanted to test and see could someone like um, since they have these remote wet labs, right, to do experiments, stuff like that, could you have an AI register and do it. But I mean it was scary but it was quite fascinating because I like think of the lengths that people can go to, how creative they can get with these sort of things. Right. So um, I'll send you that one. That was a really wild paper.

Speaker A: And then, and then on the other side even, even like using uh, AI to generate like unbelievable uh, honeypot like for example fake uh, world uh, you know there's just, I just saw just the other day like uh, where you take a photo, uh, a two dimensional photo and you convert into three dimensional world. So the possibilities of using AIs are endless. And both from, from the you know, attacker and a defender. And then as you mentioned some of these tools can be exploited in both ways. So you, you know, red, um, like a automatic pen tester can be potentially be sold, you know, rebranded and resold on the other side. And it reminds me of the, the good old days of, of email security years ago where you used to uh, send, you used to sell anti spam engines and then the same tool was like a very fast, high frequency, high capacity mailer. Okay. Yeah, exactly, the sending span. So in this particular case you have the automatic pen testing uh, and then on the other side you'll have basically just the same tool but being utilized um, completely on the other side.

Speaker B: Yeah, like I think um, yeah, I think it comes down to uh, just like always we have to do our best to have a really good understanding of adversary tradecraft. What are they doing, how are they doing it? Right. I don't think it's any different that way than threat intel. It's just that the rate of change is going to be much faster. They have a really powerful tool at uh, their disposal just like us. Right. It's uh, going to make things more difficult. Right. I would be curious to know because I haven't looked into this too much. Like, like you mentioned our nation state actors are you know, criminal syndicate groups. Are they leveraging AI for their tools? Is it more of an LLM kitties thing? I don't know. Like what's the Frequency, um, that people are actually using this in real world attacks. But from what I've seen, there's quite a number of them already that have caused some serious incidents. Right?

Speaker A: Yeah, absolutely. And then are we risking, uh, creating almost like a monoculture because everybody is using essentially the same LLM tools and the same technology. So you can see today when you receive an email and the first line says, I, uh, hope this email finds you. Well, it's an indicator. Somebody used the I'll open it.

Speaker B: Yeah, and I'll open it.

Speaker A: Yeah. No, but they're using the same. So the reason you recognize right away because there's certain patterns, there's certain words that the LLM uses. So now you're creating a monoculture where all these security tools are using essentially the same LLMs to defend. And then all you have to do is know these patterns and be able to exploit that and say, okay, I'm just gonna circumvent that because I know exactly what these tools are using.

Speaker B: Yeah, exactly. I think an interesting place for people to research and explore is like, I think this is kind of like a game theory thing. But if we have red team LLM or we know that the bad guys are using in the wild and we have our defensive LLM or system, you know, maybe it's an automated soc, Right. Why don't I just stick those things together and run it all the time continuously. And it's like this one's writing really good social engineering emails and I'm actually sending them. This one's job is to catch those LLM written social, uh, engineered emails. Right. And use that to kind of stay in touch with the speed of these things that, that might get really complex. But I think it's no different than a lot of the times we like to test our defenses, whether it's a pen test, a red team, or we use some type of like a breach and attack simulation tool. Right. But now I think what we'll have is like LLM, uh, emulation with the goal of can we catch those things too. But I think using that conceptually will work, right?

Speaker A: Yeah. And do you think that eventually, uh, these, these tools will make it things cheaper? Uh, so running a sock today is very costly. Uh, you know, there is almost like a bare minimum of a million dollars. You know, it's like how do you, how do you uh, how do you run a, how do you create a two, like a, you know, a million dollar sock business? You start with two.

Speaker B: Yeah.

Speaker A: So 2 million, uh, do you think these AI tools are going to make it like, you know, more efficient or inexpensive. And then should, should these providers, these, all these MSSPs are using, um, you know, providing this as a service, SOC as a service, which is really popular. Should they disclose the fact that they're using AI tools or should they just keep it completely black box to the customer?

Speaker B: Yeah. Um, so two questions there I guess for the first one. Yeah, I mean the way I've seen it and look at the real world impact is because LLMs or AI are doing, helping us offload a lot of the manual work that us, uh, experts will do. Like the cost savings there. I guess if you want to tie it to the bottom line, it's going to be um, you know, if I have two full time engineers or employees that are doing this work a month, um, I have both of them plus an AI right now and it saves them 70% of their time. Okay, that's, that's great because it's, it's 70% extra free time for them. Um, it's not really saving you money in the sense that um, you know, you're, you're, it's, it's going to be tied to labor I think is what I'm saying. It's like um, by augmenting your, your workforce, you're trying to make one or two people the equivalent of three to five. Right. So it's giving you know, everybody a lever that they can use to increase the scale and the speed of their work. Um, um, but you, I would be curious to know if you tie that back to, hey, we got a Soc. It's 10 people in there. They spend this amount of hours after we've got this AI assisted, uh, tool or gave them, gave it to them to use. They have 50% extra free time now they can go work on other projects or help out in areas, other areas of the organization. Right. But I think that's what it's going to come down to.

Speaker A: Or Dylan, maybe they're 50% better. Maybe they provide a more accurate prediction and much better recovery or uh, better to protect your organization. So you're 50% better.

Speaker B: Yeah, exactly, exactly.

Speaker A: Remember, is intel, intel inside. They should have LLM inside.

Speaker B: Yeah. Um, but yeah, I mean it's, yeah. If MDRs, MSSPs are using this, uh, yeah, I mean I think you, I mean you got to treat it like any other third party. Like you got to want to know, are they using anthropic or OpenAI? Are they using Bedrock or you know, are they running their own models? What models are Those. How were they created? Right. I think it's like any other third party, uh, trust. Right. Um, but yeah, I don't, I think everybody should always know, especially because if you're getting a work product back from them and it was assisted or entirely created by AI, you, ah, want to know that. Right? And I think we still need to keep the human in the loop, especially, uh, for things like that. Right. So.

Speaker A: And are we risking a whole new generation of security practitioners that don't know what, you know, tail dash is, and they only use LLMs, and so you're, you're risking the fact that they, you know, that's the amazing piece. You still have to have expertise when you're using these, but then you're, you're creating a whole generation of kind of lazy practitioners where they rely heavily on, uh, that and they don't actually have the underlying understanding. Uh, I, I think I, I think I saw like some, some, I don't know if it was a meme, but somebody said, oh, you know, I'm a prompt engineer. And then they ask, okay, well, you know, what's, what's GitHub? Or like, did you know? And they didn't know because they call themselves Engineer, but they're just strictly like LLM users.

Speaker B: Yeah, yeah, I think that's going to be tough, uh, and hard to avoid. Yeah. Like if, if you like, imagine somebody who enters, uh, the security field, you know, today or 10 years from now, uh, and, you know, I imagine their whole sock must heavily augmented with all these AI tools. That's, you know, that's great. But like, just like a funny example, like if one of them breaks one day and it's like, oh man, you got to do this investigation manually, bro. Good luck. And you're like, I have no idea. But I, I think we need to be careful with that. Um, especially as, uh, yeah, like a perfect example is you might use Claude, uh, or ChatGPT to do something. Right? And we're going to get to a point where we kind of like mindlessly take that information as good or good enough. So I'm just gonna use it, I'm just gonna copy paste it. Um, but there's no one forcing you to go and examine those results. Slow down, read it, whether it's code or whatever. Right. Uh, I think we need to be careful with that.

Speaker A: Do you remember that lawyer came out with that story?

Speaker B: Right.

Speaker A: They were not accepted. Well, he came to the court and I guess he uses m legal cases to were hallucinations. There were, there were no auction legal cases like that. So it was basically the making these up and they came to present it to court.

Speaker B: So it was a perfect uh, example.

Speaker A: It was a. Yeah, perfect example. So, so what's the, I guess the knowing that's the risk and that's a, I think a pretty substantial risk of, of creating a whole new generation of people, uh, defenders that are relying heavily on these tools but don't have the underlying knowledge. And, and just like I remember growing up they would not allow me to use a calculator. And you know, now, now they do. I mean most, most schools do. So maybe like we'll just say okay, let's embrace it and just use LLM. So that's it.

Speaker B: Yeah, I mean I'm gonna try to be not, not biased and partial here in my, my Pro AI card. But yeah, like I, you know, I, I'm trying to think like. Yeah, I mean the last mile is with us, right? Because we're using the AI, we're taking it, we're going to do something with it whether it's at work or personal. And you're the one who's going to be at risk for the hallucination. Like the difference I'm trying to think is if you go out and like even with the lawyer example, you're looking up real world cases or you're using a search engine like Google and you're pulling up articles or pieces of information that humans have written, I guess like it's, it's not the same as hallucination and that it's just factually not correct at all versus like some sources on the web or some sources in the lawyer's library, whatever might be more reputable or accurate than others. But you're the one, the human who's making that thing. So yeah, I mean as far as responsibility goes, I think it's gotta end with the human. Right. So like we have this great technology at your disposal. I totally think people should uh, should use it. Um, but we do have to be careful about, it's like going to be up to your judgment. Um, and then that gets the relationship of the organization, like are we going to block chat GPT? Because um, you know, we don't want them to be using a third party inference provider period. Or are we going to give them a local one, encourage them to use it. But I think there just needs to be guardrails, paved roads in place for like where people are using this technology that's going to have the likelihood of impact could be pretty risky. So I Think that's how we navigate. This is like, uh, but ultimately it's got to be the responsibility of the human. Like I could be here doing my day job all the time and everyone's like super happy. It's great. I was like, oh, jokes on you guys. I was using AI the whole time. You know, six months later it's like, oh man, we found out that, you know. But yeah, you got to be the one who's determined. You're the human, you're the expert. This is hallucination, you got to check. But I think guardrails will help, stuff like that.

Speaker A: Yeah, and speaking of guardrails, uh, what about the fact that if using public LLMs, anything that you input may show up somewhere else and it's somebody that's close to me said, well, you know, the LLMs are like, Ah, a baby, like a toddler, they're always listening and then they may spit out the information somewhere else completely randomly. Because you can ask a LLMs to give you a sample of a credit card, credit card number and they may potentially just give you a real credit card that they, that they grab from somebody else's input. And so this risk associated there. How do, how do companies and how do you think we can solve this, this particular issue?

Speaker B: Yeah, I mean you got to treat it like any other third party. I mean I would ask someone, uh, you know, do you use Gmail, do you use, you know, Chrome on your phone, whatever app? Right. Um, what information are you giving them? I think we just have to treat it like that. And I know like a natural step is um, you know, knowing where your data is going and why. And so like perfect example is. Yeah, I mean, maybe don't use uh, the third party inference providers like Anthropic or, or Gemini. Okay, let's go do like Amazon Bedrock or I think Google has Vertex or whatever the Azure equivalent is for OpenAI. Right. So now we're down to the trusted, uh, infrastructure. They're hosting the model and I'm pretty sure the way it works is they'll take like the anthropic or the OpenAI model and they'll do like a, um, one way write or they'll at least place it and host it in the trusted infrastructure. So now your data is going to that trusted infrastructure. But uh, you know, the model is immutable in the sense that it's a black box. The weights don't change. We're not tampering or touching the model. Right. It's data going in and out. But um, then you got to look at, well, in your company who's using the data there, right? Like, did the HR person put their thing in there? So there's still issues and then, but as far as like we keep a simple example, like the organization is going to get complex. Enterprises are complex. Lots of different data, lots of different places. I think there's already a lot of work that's been done there. But the personal sense, like you don't want to put in, um, you know, your personal information and ChatGPT on your phone or your family or your friends or whatever. Um, the local models I've heard are pretty good. They're getting better and better. Like Olam or the one you can install on your laptop. I'm sure there's a way you can do it on your phone, but then the data is never leaving your device. Right. And then we could talk about encryption at rest and all that stuff. But yeah, I think going through those motions and then um, I know I try to push uh, people towards like Groq, Gro Q. Uh, that Groq, um, they're an inference provider. So like um, they, the text or the prompts, the data that you're sending the model and the data you get back is ephemeral. So it's thrown out. Right. Uh, they're not storing them. I think Bedrock uses that as well. But at the end of the day it's kind of similar to hey, am I trusting this VPN provider says they don't save my logs. You're like, you know, so, yeah, if you want to, you can totally. Um, something I want to try is like host like one of the open source models, like Llama on bare metal or like on uh, an infrastructure provider. See how much it costs, see how performant it is. That way you're controlling your data. Like imagine you have your own private network and just host it there. But yeah, going through those steps and figuring out which one checks all the boxes or makes you more, more comfortable. Right?

Speaker A: But yeah, and if you remember the, the this used to sell or still do sell these bracelets that says in case of death, please delete my Google search history. Maybe, maybe they should have one of these. Like please delete my, you know, ChatGPT project.

Speaker B: Yeah, yeah, yeah, 100%. Yeah.

Speaker A: You heard that?

Speaker B: Yeah. If, yeah, I mean the way that we use it, all the chats are saved in there. And I, I know when they released their memory feature, I don't know if you've had experience with this. I think it was default Checked on. But, you know, some of it was cool. It's like, oh, remember what I was talking about two days ago? Now I don't have to retype that whole paragraph. Great. But then you go and read it, you're like, you're like, wow, it knows my weight, it knows my, you know, everything. I don't know what kind of question

Speaker A: you can ask it. Right, yeah. And you can ask it, what does he know? And in fact, I think some of the tools encourage you to tell it, uh, a little bit about yourself so it can tailor results associated with, depending on who you are, what you do and what you're expecting. And it's super interesting. So, Dylan, we're running out of time, but what exciting new stuff you're working on? Because I know you're very active on LinkedIn and you're posting a lot of stuff and you're giving back to your community. Tell me a bit about the stuff that you're working on and maybe, uh, how the best of people get involved or get in touch with you.

Speaker B: Yeah, yeah, sure, yeah. Um, if you're interested in the chat, just message me on LinkedIn. That's typically where I hang out. So I guess a lot of the stuff that I write about or post is around applying LLMs to cyber security to our domain, right? Um, yeah, I guess if you put it in like two buckets, there's security for AI, right. And, um, then there's AI for security. So I'm more of the latter. Not our option. But those are kind of the two big spheres that, um, our worlds are operating in right now. But yeah, um, we, yeah, we just, we just uh, started, um, early stage cyber security company. Um, it's going to be tapping on a lot of the same theme, themes that we've talked about here. Um, I'm pretty excited. So, you know, my background's being a SEC ops practitioner, detection engineer, threat hunting, all that stuff. Right. Um, it's going to be, uh, in that space, a lot of the problems in that space. But yeah, just stay tuned on LinkedIn. You know, we're going to release a lot of stuff on what we're, we're working on shortly, make some announcements.

Speaker A: Um, but yeah, super exciting. You know, the whole space has always been exciting and I think that now it's like a hockey stick excitement. Yeah. Uh, because there's just so much going on and almost like a daily basis. Um, so Dylan, thank you very, very much. It's been really phenomenal conversation. Thank you for joining me today. Thanks for all those who joined me. And, uh, until then, I'll see you in the next episode. And be safe, uh, online as well as offline.

Speaker B: Awesome. Thanks, David.

Speaker A: Cheers. Thank you very much.