The B2B Podcast Index
Index
All categories
MarketingSalesSaaSFinanceHROpsLeadershipCustomer SuccessAI & DataProductStartups & FoundersRevOpsEngineering & DevTools
MethodologySubmit
Best of:MarketingSalesSaaSFinanceHROpsLeadershipCustomer SuccessAI & DataProductStartups & FoundersRevOpsEngineering & DevTools
An independent project byFame
SearchBest episodesGuestsInsightsMethodologySubmit a podcast
Index/Engineering & DevTools/Ship It Weekly
Ship It Weekly artwork

Ship It Conversations: Kat Traxler of Vectra AI on AI Security, the Zero-Day Clock, IAM, and Cloud Risk

Ship It Weekly · 2026-06-28 · 43 min

0:00--:--

Key moments - from our scoring

Substance score

76 / 100

Five dimensions, 20 points each

Insight Density16 / 20
Originality14 / 20
Guest Caliber17 / 20
Specificity & Evidence14 / 20
Conversational Craft15 / 20

The conversation explores AI's dual impact on security research and attack speed, grounded in real data from RSA, the Unprompted conference, and Rapid7's vulnerability research. Traxler explains the "San Francisco Consensus" - the belief that as LLMs become better at finding code-level vulnerabilities like SQL injection and RCEs, attackers will weaponize these discoveries in minutes, creating a doomsday scenario. However, she pushes back on catastrophizing by examining what actually causes breaches: credentials, misconfigurations, known CVEs like Log4j exploited repeatedly, and IAM failures. LLMs excel at narrowing search space in large codebases and finding specific vulnerability classes (injection issues, IDOR, XSS) but fail at understanding context, threat models, privilege levels, and whether flaws matter in deployed systems. Traxler discusses using multiple models strategically (Claude, Gemini, Gemma, ChatGPT) rather than maxing out one, leveraging AI for writing support when given strong direction, and why expert knowledge remains essential - models hallucinate confidently about things like VPC flow log capabilities. She frames IAM as fundamentally a people problem with an API: credentials, authentication shortcuts, fatigue, and attackers taking the lowest-friction path. The episode cuts through hype to examine what actually changes when both defenders and attackers get faster.

Key takeaways

  • →LLMs are excellent at narrowing the search space in large codebases for specific injection-class vulnerabilities but cannot understand deployment context, privilege models, or threat models - meaning expert judgment remains essential.
  • →Teams are still getting breached primarily through credentials, misconfigurations, and known unpatched vulnerabilities (Log4j, etc.) far more often than zero-day exploits, even as the zero-day clock shrinks from years to hours.
  • →IAM failures persist because they're fundamentally a people problem colliding with technology - human fatigue, credential reuse, authentication shortcuts, and attackers taking the lowest-friction path will outweigh zero-day sophistication in the near term.
  • →Using different LLM models strategically for different research tasks (Claude for code problems, ChatGPT for spatial awareness, local models for efficiency) yields better results than defaulting to a single expensive model.
  • →The San Francisco Consensus assumes zero-day exploitation will become the primary attack vector, but evidence suggests attackers will continue exploiting known, patchable vulnerabilities because they work reliably at scale.

Guests

Kat Traxler

Topics in this episode

OWASP Top 10SQL injectionProject GlasswingZero-day clockSan Francisco ConsensusUnprompted conferenceVectra AIAnthropic Frontier Red TeamProject MythosXSS

Questions this episode answers

What is the zero-day clock and why does it matter for security?

The zero-day clock measures the time between a vulnerability being announced and attackers weaponizing it in the wild - it has compressed from two years in 2018 to hours now, driven by LLMs finding code-level vulnerabilities faster. The concern is that this acceleration will eventually enable attacks on unknown flaws at scale, though current breaches still stem from known, patchable vulnerabilities.

What types of vulnerabilities are LLMs actually good at finding?

LLMs excel at finding code-level injection issues (SQL injection, XSS, command injection, RCE), IDOR flaws, and other specific vulnerability patterns in large codebases by narrowing search space from thousands of lines to dozens. They struggle with context-dependent issues like confused deputy problems, privilege analysis, and understanding whether a flaw actually matters in the deployed system.

Why do teams keep failing at IAM if it's so important?

IAM is a people-and-process problem with technology layered on top - it fails because humans get tired, reuse credentials, take shortcuts, and attackers exploit these lowest-friction paths. It's not purely a technology or configuration issue; it's the collision of human behavior with systems.

Which LLM models should you use for security research and bug hunting?

Claude and its variants are strong for general coding challenges, ChatGPT excels at spatial awareness, and local models like Gemma are efficient for token usage; using different models for different tasks and letting them catch each other's gaps is more effective than defaulting to a single expensive model like Claude Opus.

What's the biggest overhyped AI security narrative?

The catastrophic scenario that reaching a zero-day clock of zero hours will collapse infrastructure is overstated - while the data on shrinking zero-day windows is real, attackers will continue exploiting known, reusable vulnerabilities because they're lower friction and work reliably at scale, especially against mid-sized companies.

What our scoring noted

Our reviewer’s read on each dimension, with quotes from the episode.

Insight Density

16 / 20

The episode delivers substantial, non-obvious insights about AI's role in security research without relying on platitudes. Kat provides concrete distinctions between what LLMs are good at (finding injection bugs, XSS, IDOR in codebases) versus what they cannot do (understanding context, privilege, threat models, deployment specifics). The framing around the 'San Francisco Consensus' and zero-day clock provides valuable critical distance from hype, and the insight that AI is excellent at narrowing search space but not understanding system-wide implications is repeated across multiple domains (bug hunting, writing, IAM). However, some sections devolve into broader philosophizing rather than dense technical detail.

They can find them like no other tooling has found before... it's amazing for these known problems... It's not so great at understanding context or the threat model around where it's deployed, who the users are, what data those users might be accessing, under what conditions.
Credentials, authentication, access, human behavior, shortcuts. Fatigue and attackers taking the lowest friction path... it is not just a cloud service problem it is a people problem with an API.

Originality

14 / 20

The episode avoids typical AI-security doomism and cheerleading by landing in an honest middle ground. The specific framing of 'San Francisco Consensus' as a term for uncritical zero-day-apocalypse thinking is fresh. Kat's distinction between code-level vulnerabilities and insecure-by-design flaws, using AirTags as a case study, is a useful counterpoint to typical vulnerability discussions. However, the core arguments - that LLMs narrow search space, that IAM is hard because it's a people problem, that attackers take low-friction paths - are not entirely novel observations. The originality is more in tone and framing than in fundamentally new thinking.

So I'm kind of catastrophizing a little bit, but that's the natural conclusion of the zero-day clock... I think we should, as a community, spend a little bit more time filling out the 'and then what', as opposed to just kind of letting our minds go wild.
The article that I published about the San Francisco Consensus is probably the biggest one that I take exception with... I think that if you're a technologist, in the space, deeply embedded in the space... it's very natural to be alarmed.

Guest Caliber

17 / 20

Kat Traxler is a principal security researcher at Vectra AI with demonstrable expertise in cloud security, IAM, and contemporary AI security research. She was a CFP organizer for the Unprompted conference and clearly has hands-on experience with bug hunting using LLMs, not just theoretical knowledge. She speaks with the authority of someone actually doing the work - switching between models for efficiency, understanding model-specific strengths, conducting real research projects on RCE and VPC flow logs. She avoids the trap of being a pure pundit; her skepticism of doomsday narratives is grounded in practical experience, not contrarianism for its own sake.

I was one of the organizers. I was one part of the CFP process...
I've done a huge project on just trying to find remote code execution... I'm using a lot of different public models, but I switch between a ton of different ones.

Specificity & Evidence

14 / 20

The episode includes several specific examples (Log4j, XSS, SQL injection, IDOR, confused deputy, RCE, prompt injection) and names concrete models (Claude Opus, Gemini, Gemma, ChatGPT) with specific comparative claims (ChatGPT is good at spatial awareness; Claude is good at coding). The RAPId7 report is cited with the claim that 2018 showed two-year exploitation windows versus current minutes. The AirTag stalking example is concrete. However, the transcript lacks precise metrics - no specific CVE numbers are given, no exact timelines for recent breaches, no concrete dollar figures or company-specific case studies. The zero-day clock claim lacks quantified evidence beyond 'hours,' and broader claims about where teams 'actually get hurt' cite the RAPId7 report but don't pull specific numbers from it.

So I'll use Log4j, but just those kind of like reused over and over and over CVE exploits that are known and patchable and are just tried over and over and over again.
In 2018, it was two years to exploitation. Now it's minutes.

Conversational Craft

15 / 20

Brian Teller demonstrates solid interview technique throughout. He asks clarifying follow-ups ('Tell me a little bit about what have you seen lately...'), presses for specificity ('Is it primarily certain languages or certain query formats?'), and builds on Kat's answers with genuine curiosity rather than scripted questions. However, some moments miss opportunities to push harder: when Kat admits 'I really couldn't tell you why it's good at that,' a sharper follow-up on the mechanism would deepen the insight. The discussion of prompt injection as an 'ops problem' starts awkwardly ('Tell me more. What are you thinking more about that?'), and Brian doesn't always challenge claims - for example, the zero-day clock timeframe ('hours') could have been pressed for specificity. Brian does effectively draw out nuance around the people vs. technology problem in IAM, and his final monologue demonstrates he's genuinely synthesized the conversation, but the live conversation itself is friendly and exploratory rather than rigorous or probing.

I probably didn't ask it well... Does that answer your question?
Is there a specific model that maybe works better for prompt injection-type research, or is it more nuanced than that?

Conversation analysis

Computed from the transcript - who did the talking, and the words that came up most.

Most-used words

security27code24level16vulnerabilities15teams14injection14zero13finding12different12problem12credentials11model11design11cloud10writing9conversation9

Episode notes

This is a guest conversation episode of Ship It Weekly , separate from the weekly news recaps. In this Ship It: Conversations episode, I talk with Kat Traxler of Vectra AI about AI security, the zero-day clock, IAM, cloud risk, AI-assisted bug hunting, and why the scariest future security problems may still start with the boring fundamentals teams already struggle with today. Kat is a Principal Security Researcher at Vectra AI focused on abuse techniques and vulnerabilities in the public cloud, especially around the intersection of cloud security, AppSec, IAM, managed identities, and insecure-by-design flaws. We talk about the current AI security mood, from the excitement around faster research and bug hunting to the fear that AI could shrink the window between vulnerability disclosure and exploitation. Kat explains the “San Francisco Consensus,” why the zero-day clock is getting so much attention, and why she thinks the facts may be real while some of the conclusions are overextended. The bigger theme here is that AI is absolutely changing security work, but it does not erase the fundamentals. Attackers still take the lowest-friction path that works.

Full transcript

43 min

Transcribed and scored by The B2B Podcast Index.

AI security is in a weird place right now. On one side, there is real excitement. People are building faster. Researchers are moving faster.

Bug hunters can narrow huge codebases down in ways that would have felt impossible a few years ago. The blank page is less scary. The first draft is easier. The first pass through a massive repo is not quite as painful.

And honestly, that part is cool. But on the other side, security teams are looking at the same acceleration and asking a much less fun question. What happens when attackers get faster too? Because this is not just about AI writing code.

It is also about AI finding bugs, AI helping weaponize vulnerabilities, AI making it cheaper to search for weird code paths, injection bugs, RCEs. And the kinds of flaws that used to take a lot more time and patience to find. And somewhere in the middle of that, we still have the same boring problems. Credentials, IAM, misconfigurations, unpatched systems, source code secrets, people getting tired, teams moving fast, systems nobody fully threat-modeled because the feature had to ship.

That is really what this conversation is about. Not AI will destroy security. Not AI will save security. More like what actually changes when AI speeds up both the people building software and the people trying to break it.

I'm Brian Teller from Teller's Tech, and this is Ship It Weekly. Welcome back to Ship It Weekly, where I filter the noise and focus on what actually matters when you are the one running infrastructure and owning reliability. Most weeks, it's a quick news recap. In between those, I do conversation episodes with people who are building platforms, running infrastructure, finding security issues, and thinking through where this industry is actually headed.

Today is one of those conversations. I'm joined by Kat Traxler, a principal security researcher at Vectra AI. Kat focuses on abuse techniques and vulnerabilities in the public cloud, especially around cloud, AppSec, IAM, and those uncomfortable places where everybody kind of assumed the layer underneath was safe. And I like this conversation because it lands in a more honest middle ground than a lot of AI security takes.

There is definitely hype. There is definitely fear. There was also a bunch of practical reality sitting underneath both of those things. Kat had just come out of RSA and the Unprompted conference, so we talk about the current AI security mood, the excitement from startups, the nervousness from security teams, the San Francisco Consensus, the zero -day clock, and this growing belief that AI is changing how quickly vulnerabilities can be found, understood, and potentially weaponized.

But we also talk about where teams actually get hurt first. And the answer is not always the flashiest new AI attack. A lot of the time, it is still credentials, IAM, misconfiguration, known vulnerabilities, unpatched systems, the same boring fundamentals that have been ruining everyone's week for years. That tension is what makes this conversation useful.

Because AI is absolutely changing security research. Kat talks about using models to narrow the search space in bug hunting, why they are good at finding certain code-level issues, and why they are not as good at understanding context, privilege, deployment shape, threat models, or whether a flaw really matters in the system where it lives. We also get into prompt injection, AI-assisted writing, using different models for different research tasks, and why being an expert still matters if you do not want the model to confidently lead you off a cliff.

There is a really good thread in here around IAM too. Why do teams keep failing at it? Kat's answer is pretty simple and pretty true. IAM is where people and technology collide.

Credentials, authentication, access, human behavior, shortcuts. Fatigue and attackers taking the lowest friction path it is not just a cloud service problem it is a people problem with an API. And toward the end we talk about overhyped AI security narratives what actually happens if the zero-day clock keeps shrinking whether we are headed towards more insecure by design flaws and why some of the hardest problems left maybe less about bad lines of code and more about bad assumptions.

So if you work around cloud, DevOps, SRE, AppSec, IAM, security operations, platform engineering, or you are trying to separate useful AI security thinking from panic and vendor fog machines, this one should be worth your time. All right, let's jump in. Today, I'm joined by Kat Traxler. She is a principal security researcher at Vectra AI focused on abuse techniques and vulnerabilities in the public cloud, especially at the intersection of cloud, AppSec, IAM, and “we thought this layer was safe”.

Kat, thank you for joining me. Thanks for having me, Brian. I'm excited to talk about the new accelerated world we find ourselves in. It is.

It's definitely a new accelerated world for sure. Tell me a little bit about what have you seen lately in the last couple of weeks or months as far as AI or security in general? I'm coming off the heels of RSA. So I spent a four-day pilgrimage out to the bay.

The vibe was definitely rational exuberance. Exuberance? Yeah. That's not quite a word.

But you know what I mean. Everybody was incredibly optimistic. A lot of two-pizza teams. Somebody breaks off from a big company with...

A set of like AI LLM skill sets and are like racing to the future. So in the startup space, people are jazzed. I haven't seen this level of excitement in a while. You contrast that with middle America, where I live and folks are much less enthusiastic about AI and its role in our life.

Then you contrast that with, you know, the security sec op folks. And people are very, very nervous about what's to come. So you just have so many different experiences depending on where you are at in this world. That's fair.

I think I hold all of those in my head as well. I mean, just as a person, right? Like, I mean, there's excitement. We talked a little bit before we started.

Like, you're able to iterate more. You have all these projects that you wanted to start that you couldn't start. Now you are. But then there's also the security side where you see the RCEs and attack vectors that are more prevalent than they were maybe a year, two years ago.

So, yeah, it's a different world for sure. The security reporters are busy, right? It seems like there's a new massive major story that they're covering weekly. A new, you know, critical zero-day, a new supply chain attack.

The frontier model companies, Anthropic and OpenAI, have showed us that these models are really, really good at finding these specific kinds of vulnerabilities. Not all vulnerabilities, but these specific kinds of code-level vulnerabilities. They can find them like no other tooling has found before. So we're just having an onslaught of these vulnerability announcements now, and it can be really overwhelming.

That's fair. So recently you wrote about, you had a right, you had an article, the San Francisco Consensus. Can you explain that in your words? Sure.

That was coming off the heels of another pilgrimage to San Francisco, going to the Unprompted conference, which I was one of the organizers. I was one part of the CFP process and had a blast communing with all of the fantastic AI researchers, but there was definitely a consensus amongst folks that coalesced and peaked with Nicholas Carlini's, I almost called it speech, but talk. He's with Anthropic's Frontier Red Team. If you haven't seen that talk, please go see it.

He really lays bare the issue of the zero-day clock, which is the idea, not the idea, I mean, the proven phenomenon that The time between a zero-day being announced and it being exploded in the wild had gone from two years. A couple of years ago, it was like two years it would take. You'd have this announcement, and then it would take attackers literally two years to weaponize it. Now it is just hours.

You have the announcement, they can weaponize it. Nicholas Carlini's talk at unprompted late is over there. And I'm telling my dog to not bark at everybody. Can you hear that?

I'm sorry. Of course, right in the middle of what I'm talking to people. This captures the zeitgeist and the, you know, kind of pure, the pure terror slash pure excitement in everybody's eyes that LLM can find code-level vulnerabilities like no tooling has found. Attackers are weaponizing it in minutes.

Therefore, and people don't really explain what the “and then therefore” is, but the and then therefore is kind of assumed that and then therefore systems will crumble, our global technology infrastructure will no longer work. We will have a complete collapse of critical infrastructure. We will go to the Stone Age, move to the woods. So I'm kind of catastrophizing a little bit, but that's the natural conclusion of the zero-day clock.

And that's the consensus, is that we are months away from this scenario occurring. So is it kind of a take on the doomsday clock, where we're counting down to... Yeah, yeah, yeah. Although I think the doomsday clock is sort of like...

arbitrarily set by some people. Yeah, this is, I think this is backed by some real data. And I've read a report from RAPId7 recently that was really good that has a lot of backing data. They never say zero-day apocalypse, but they have this like 26-page report that backs up all of the observed data points in the wild that just kind of come down to.

In 2018, it was two years to exploitation. Now it's minutes. And the big question is, and then what? And I think we should, as a community, spend a little bit more time filling out the “and then what”, as opposed to just kind of letting our minds go wild.

And everybody has a different idea of what and then what means. So if you had to rank where teams actually get hurt first, what's at the top? Like, where should we start? Yeah.

I mean, historically, it's been credentials. Historically, it's been mishandling of credentials, reusing of credentials. It's been committing them to source code, having them be static, not having efficient rotation in them, and then misconfigurations. In the new RAPId7 report, I believe that they had said moving to the top of initial access was...

just a small suite of known code-level vulnerabilities. So I'll use Log4j, but just those kind of like reused over and over and over CVE exploits that are known and patchable and are just tried over and over and over again. So by the numbers, those are still the things that are biting people when it comes to initial access. Going forward, the zero-day clock and the San Francisco Consensus says They don't necessarily say that those things aren't going to matter.

What they say is that there's a whole new attack surface that's going to be open and that's all the zero-days that nobody knows about. All of the code-level vulnerabilities within your exposed on-prem stack or any other piece of infrastructure that hasn't been exploited anywhere that because finding a zero-day is now just costs and tokens that's what you have to now protect for and i'm just saying like that's that's the san francisco consensus is like that that attackers will shift to that exploitation I doubt it because i think all the other mechanisms just work so well that I do not think that that will be, at least in the foreseeable future, the exploitation route that attackers will leverage, at least initially.

It might be a concern for some of the large institutions, Google, IBM, nation-states, but for your average mid-size technology companies, it's still potential reuse on patched servers. That kind of stuff. So what's your take on prompt injection as an ops problem? An operations problem?

Yeah, over just like an app problem specifically. Tell me more. What are you thinking more about that? Like the different layers it interacts with or like how it affects the infrastructure?

Yeah. Where that may be. Or if you see it as an issue going forward, more so in the future, operationally. Yeah, I'm not exactly sure how to answer that.

I mean, there are frameworks now to protect against a significant amount of prompt injection. They're notoriously difficult to implement and operationalize. Ultimately, there are bypasses. So, I mean, it's always some level of a WAF.

So it's always going to be some whack-a-mole that you're playing with the prompt injection defenses. But I don't see that folks are really even using the most basic level of prompt injection mitigations out there that are available to them. Here in 2026, there are... A significant amount of prompt injection frameworks that could help.

Again, you know, it might come out to be 96 % effective, 97 % effective, 98 % effective. There's always still going to be a bypass. Yeah. Does that answer your question?

Yeah, it does. That's good. I probably didn't ask it well. I don't know if you wrote about this or I heard you talk about this.

We're talking about AI helping narrowed the search space in bug hunting oh yeah how are you using that in practice well i can't tell you I am using it to narrow my search space uh no i mean okay so for example, for code-level vulnerabilities you have these like huge massive gnarly codebases that you know, to some level or another, you're like just grepping your way through looking for bad patterns. And then maybe you're using, this is the old way, maybe you're using a static code analysis tool to kind of help you with some of that work.

But ultimately, you're just grepping your way through a problem. But the LLMs love this sort of problem, this sort of like injection issue, which... A large portion of the OWASP Top 10 just comes down to like injection issues. And they love this sort of problem of taking this massive interwoven complex blob and being injected with the concept of like, you know, bad input in results in bad input out.

And where are the potential mitigations? How can they potentially be mitigated along the way, doing all the heavy lifting for you and being able to get down from 500 lines of code to maybe 100 that it's going to flag for you as potentially bad. I found that it's amazing for these known problems. These issues that you can come down to, this is this bad line of code, and you can then insert this particular type of sanitization or parameterization and fix it.

It's something that it's really great at. It's not so great at understanding context or the threat model around where it's deployed, who the users are, what data those users might be accessing, under what conditions, those subtleties that are going to change. That's a whole different problem set that they're not good at yet, and which makes me realize that I still have a job. But those code-level things that are like needles in a haystack, they're good at finding needles.

They're not good at finding, I would say that they're not good at finding all of the needles. If you want to find a needle, you'll find one, but you won't find all of them, if that makes sense. Is it primarily certain languages or certain query formats? Like, are we talking like SQL injection?

Are we talking like app function calls? Or specifically Python, maybe, because the LLM has that training data? You know, I really don't know what it comes down to. I've done a huge project on just trying to find remote code execution.

And then ensuring that it's tracing back to some sort of untrusted input from a user. So being very clear about where... What inputs it is allowing, you know, from where. SQL injection is great at finding.

XSS is great at finding. Not as great as finding things like confused deputy, which are, again, contextual. Like, what's privilege? That's always, like, a squishy thing.

You have to be really explicit with it. I mean, the people who are doing bug hunting. On the client side client-side issues it's great finding IDOR um like it's really great at these very specific not things that don't require a ton of context issues and going from this massive code base to these handful of things and then proving exploitability yeah i i really couldn't tell you why it's good at that why that puzzle it it turns on really well as opposed to it doesn't understand the nuance of privilege.

Are you using public models or are you training like your own data? No, I'm using a lot of different public models, but I switch between a ton of different ones. Obviously the Claudes, I'm switching between the various versions of Gemini. I'm switching between also some local models like Gemma.

So I try to be efficient with my token usage. Folks that are wanting to get into bug hunting, I think there's a lot of advice that just says like run it all on Claude Opus 4.6 and just go to town. I'm much more, I'm just much more delicate around, which model I use when.

Because I think that they can do... They play well off each other. You can use one to catch the bugs of another or to help catch the faulty process of another. One might have a glaring gap that you just aren't aware of that another one can catch.

Yeah, I know. ChatGPT is really good at spatial awareness, whereas Claude's very good at generally coding -type challenges and problems. Is there a specific model that maybe works better for prompt injection -type research, or is it more nuanced than that? I mean, all of the bug bounty people will just say, turn Claude on to your target and go to town.

I don't think you're going to go wrong with that. It just depends on how efficient you want to be with your token usage. I think it's overkill to use Claude on the vast majority of tasks. Now, if that's your profession, if you are a full-time bug bounty hunter, you're going to scale up to as many Claude Max subscriptions as your budget will allow and just set them off.

Me, not so much. I'm going to be a little bit more delicate with what I use when. And I think that's kind of fun because you get to learn all of their intricacies. And I don't know if you heard, but just today there was the revelation from Anthropic on Project Mythos, or sorry, Mythos, their next iteration of models and Project Glasswing.

which is their private preview of Mythos to about 40 different companies. Particular researchers within AWS, Azure, Google, IBM, Cisco, Palo Alto, CrowdStrike, kind of like all the big ones, have private access to this incredibly powerful model, more powerful than the current iterations. And they're not planning on releasing it to the public. They are wanting it to only be in the hands of hand-picked organizations to allow them to shore up their infrastructure for potential onslaughts.

I wonder how the U.S. Government will react to that. They weren't on the list.

I want to see that list of MAGIC 41, you know, Cisco, IBM, Red Hat, Kat Traxler. That is what i'm looking for that'd be a cool list to be on for sure yeah that would be a great list yeah what parts are AI good at versus what do you still need a human for like is there any part where you think that like AI is not quite there yet i mean we talked a little bit about context and not having like contextual awareness around like large monorepos or large monolithic apps but are there other areas maybe where i mean for example For example, writing.

I do a ton of writing. And I have had LLMs just generate an article for me and it's crap. It sounds like garbage. I can't handle it.

But I can go through and put together a decent rough draft. Write it as if I'm just writing for myself very casually. I do not concern myself too much with... Spelling mistakes and word order if I mess up here and there.

But make sure I have like the needs of my argument. Sometimes I'll even do it in bullet points and then send it to my LLM of choice. Have it with some very direct instructions on like the tone and how much to shift away from my tone. Have it be very constrained and then iterate on that from there.

So I think it has a great place in writing. We just, I just personally, use it somewhat sparingly. So just getting your voice out there I think is great. It's such a barrier to entry for some people to blog, to put articles out there.

They see that blank page and they don't really know what to do. If they could say, you know what, just write a table of contents, a list of bullet points, sentence fragments, paragraphs here and there, and then start. I think that can maybe unlock a lot of knowledge that people have and they can publish their ideas. But if you don't give it enough meat, it's really going to come out pretty poorly.

So that's with writing. And then, you know, obviously you need that expert level knowledge because it's going to, it's not, the models aren't truth seeking. They're not going to write you research that's necessarily true. And when you challenge them on things, they'll be like, oh, I'm so sorry.

That's not at all what I thought. So it can supercharge an expert's workflow to really get your ideas out there. But if you're not an expert in the space, it can really lead you astray. Garbage in, garbage out to a degree.

Garbage in, garbage out. Yeah, garbage in, garbage out. I was iterating on something now. And there was a ton of instances where there was an LLM just hallucinating on what you could really do with cloud flow logs, VPC flow logs.

And at some point I had an idea that it would have some really like packet-level revelations in it. And if I wasn't an expert in this space, I would have just assumed, okay, they're telling me I can do X, Y, and Z with these VPC flow logs. And as an expert in the space, I can challenge and say like, this is impossible. This is impossible.

This is impossible because these are the, this is the. These are the fields you have. And this is the kind of inferences you can make based on those fields. But it is not truth-seeking.

It's going to try to make it fit. Yeah. And so if you don't challenge it. Okay.

So let's stay in cloud a little bit. I'm curious your take on this. So if most breaches still start with boring fundamentals for the most part. Why do we keep failing at IAM?

Why is that still such an issue? It seems like. Hard. IAM is really, really hard.

I mean, IAM has this thing where it's actually just about people and people's uses of things and people, process, technology. The people part is the hardest thing. So even though like IAM is another service, just like computer S3, it's actually just like the intersection of people. And technology, which, you know, how people use their credentials, how people authenticate, all of that is about human behavior.

That's just the worst. I mean, I don't know about you, but I'm the worst. So I don't know if it's a technology problem or it's just that within those three pillars, people, process, and technology, it's always the most difficult. I feel like we all think we're not susceptible either to phishing attacks or whatever, like not having strong credentials or some sort of injection that happens that we think that we would catch.

But everyone gets tired. Everyone gets complacent to a degree. We're all people just trying to do our best. And conversely, attackers, threat actors are all people just trying to do their best.

And what are they going to do? They're going to take the easiest low-friction route, which is going to be credentials. Yeah. Is there any overhyped AI security take that you disagree with?

I think it's like the... The article that I published about the San Francisco Consensus is probably the biggest one that I take exception with. Once we do get to the point where the zero-day clock gets to actually zero, that's not actually a catastrophe for the lives of average people. I think that if you're a technologist, in the space, deeply embedded in the space, and you see all of these crazy advancements within LLMs, and you see their capabilities for offensive research, it's very natural to be alarmed.

And then you're, you being alarmed, alarms other people. And then there's this like, there's a ferocity that goes around. And so I think the take is that we are on the precipice of a catastrophe. But if we step outside of that and we say, what actual catastrophe is going to happen?

I don't think that we will see large-scale, system-wide availability, confidentiality problems that will affect the vast majority of humans on Earth. I think it's going to lead to some no good rotten days in the SOC. But that's not the same. Where do we go from here then?

What do we do? What do we do? Where do we go from here? Keep on, keep on, right?

Like one foot in front of the other. I think I take this maybe more middle of the road approach because I do live in middle America and I don't live on one of the coasts. And if you're in that bubble, it can be really easy to drink the Kool-Aid and live in the zeitgeist. But when you, Live elsewhere where the vast majority of people don't work in tech.

The vast majority of people have other concerns within their life. You can kind of see the full breadth and scope of humanity and you can kind of say like, yep, this problem is coming and we should absolutely be working on it. However, it actually won't lead to worldwide system collapses. So what do we do?

Show up for work tomorrow. That's all we can do. That's all you can do. That's all you can do, Brian.

Make sure you get your PTO in. Don't fall into the fallacy that you and your skills are so important that you can't spend your hard-earned time resting and recharging. I think that's a common fallacy in IT and security folks that this is all going to come crumbling down if I step away for a minute, and it won't. So, okay, closing thought.

Are we headed towards more insecure-by-design cloud flaws given that? Maybe. Maybe, Brian. Maybe.

You know, I am always struck when people talk about all of the vulnerabilities that LLMs can find. And I'm always asking people to define what they mean by vulnerability. And it always comes down to like a code-level vulnerability, not necessarily a design flaw. So these insecure-by-design problems, these like “I did not threat model appropriately” ones, I'm not sure if they're going to increase as a result of LLMs, but I do think that they might be some of the last ones left on the table.

The best way to think about insecure-by-design is to think about Apple AirTags. And, you know, when they were released, nobody thought about the threat model of somebody of stalking. And yeah, right. So and it took somebody writing a series of articles about.

Women being stalked by their partners with Apple AirTags before Apple took it seriously to then build in both Android apps and newer capabilities on your phones to then know if you're being followed by an AirTag. So that's kind of the classification of, that's like the best way to describe an insecure by design issue. And often they are the most fun to find because they require some amount of convincing. Like convincing somebody that, no, you just didn't think about this correctly.

So that's why I love looking at those specifically. If we're going to be relying more on LLMs to build our software and have less human oversight, maybe it does mean that we're going to have more of these insecure-by-design flaws. Hopefully not. Hopefully it just means that I still have a job.

Either way, I think they're not going to go anywhere anytime soon. I guess the problems just change over time, right? I mean, the problems don't go away. The shape of it might change.

Yeah, and it's all about expectations. What does your consumer base expect from this product? Maybe 50 years ago, if AirTags came out and they could be used for stalking abused women, maybe 50 years ago, it wouldn't have been considered a vulnerability. But in 2020, it certainly was.

So it's also just about the expectations of your customers. If this was publicized, would there be an outrage? Yeah. Awesome.

Well, where can people find more of your work and find you online? I'm chronically online on Twitter, as NightmareJS. Also on Bluesky and then LinkedIn. Awesome.

I will put links to all of those in the show notes. Thank you so much, Kat, for coming on. Really appreciate it. Thanks for having me.

All right. That was my conversation with Kat Traxler from Vectra AI. My biggest takeaway from this one is that AI security is not one story. It is a speed story.

It is a context story. It is a fundamentals story. And it is very much still a people story. Because, yes.

AI is changing vulnerability research. It can help narrow massive codebases. It can find certain code-level bugs faster than older tooling. It can help researchers move from where do I even start to here are the suspicious paths worth looking at.

That matters. But it does not magically understand the whole system. It does not automatically know the threat model. It does not always understand privilege.

It does not know what your customers expect. It does not know which weird business rule turns a harmless-looking behavior into a real abuse path. That is where humans still matter. And honestly, that might be the more interesting version of this whole AI security conversation.

The model can help find needles, but the human still has to know which needles matter. That came up a few different ways in the conversation. Bug hunting. Writing, prompt injection, IAM, insecure-by-design flaws.

The pattern is pretty consistent. AI can accelerate the work. But if you do not have the judgment to challenge it, constrain it, validate it, and understand where it is guessing, it can absolutely make you faster in the wrong direction. And that is not just true for researchers.

It is true for platform teams, DevOps teams, SRE teams. AppSec teams. Anyone using AI to generate code, review code, write policies, inspect infrastructure, or explain some terrifying IAM graph that nobody wants to look at manually. The other thing that struck me is Kat's point about the boring stuff still being the place teams get hurt.

Credentials still matter. IAM still matters. Misconfiguration still matters. Known vulnerabilities still matter.

Attackers are practical. They are going to take the lowest friction path that works. And a lot of the time, that path is not some cinematic AI-generated zero-day chain. It is a leaked key, an over-permissive role, a reused credential, a server that should have been patched, a service account nobody remembered existed.

That does not mean the AI risk is fake. It means we need to hold both ideas at the same time. AI may compress the time it takes to find and weaponize certain classes of vulnerabilities. That is real, and security teams should care.

But if your IAM is a mess, your secrets are everywhere, your patching is inconsistent, and nobody owns the exposed attack surface, then the scary future problem is not an excuse to ignore the thing already sitting on fire. I also like the discussion around insecure-by-design flaws. Those are not always easy for tools to find because they are not just bad code. They are bad assumptions.

The AirTag example that Kat brought up is a good way to think about it. The issue was not just whether the device functioned as designed. It was whether the design accounted for how real people could abuse it. That is a different class of security work.

It requires context. It requires thinking about users, incentives, misuse. Power dynamics expectations and what happens when a feature gets used by someone with bad intent and if AI removes some of the easier code-level bug hunting work or at least makes it faster i would not be surprised if more of the interesting work shifts towards those deeper design and abuse questions not is this line vulnerable more like what does this system allow someone to do that we did not intend that is harder and probably more important so my takeaway is not panic about AI security it is also not relax everything is fine it is more like keep your head use AI where it helps let it speed up the annoying parts let it help with first drafts first passes code search query generation pattern matching and research flows.

Do not outsource judgment. Do not let the AI output become the security conclusion. And definitely, do not let the excitement around future attacks distract from the fundamentals that are still hurting teams right now. Credentials.

IAM. Patch the known stuff. Understand what your system actually promises. And when the model sounds confident, remember that confident and correct are still not the same thing.

I'll have links to Kat, her work, and anything else we mentioned in the show notes. If you enjoyed this conversation, follow or subscribe to Ship It Weekly wherever you listen to podcasts. It helps the show and it makes sure you get both these conversation episodes and the weekly DevOps, SRE, platform, cloud, and security news recaps. You can also find the show notes and links over at shipitweekly.

fm. Thanks for listening, and I'll see you later this week.

Related episodes across the Index

Other episodes covering the same guests and topics, from across The B2B Podcast Index.

  • AI Finds Vulns You Can't With Nicholas CarliniSecurity Cryptography Whatever · on SQL injection100 / 100
  • Mythos And The Disappearing Patch WindowAI Proving Ground Podcast · on Project Glasswing96 / 100
  • Weathering the AI Vulnerability Storm with Gadi Evron, Rob Lee and Ed SkoudisCyber Leaders · on Project Glasswing89 / 100
  • Chris Pogue: Digital Forensics in the Modern Threat LandscapeKitecast · on SQL injection82 / 100
  • Mythos is not the AI ApocalypseThreat Talks · on Project Glasswing80 / 100
  • An AI Just Out-Hacked 2 Million Humans. She Decides What Happens Next | Nidhi Aggarwal, CPO HackerOneCXO Spotlight · on Project Glasswing80 / 100

More from Ship It Weekly

All episodes →
  • Amazon Q CVEs, Hijacked npm and Go Packages, AWS WAF HTTP/2 Issues, Lambda MicroVMs, and Why Execution Is the Boundary Now88 / 100
  • containerd CRI Vulnerabilities, Datadog PostgreSQL HA on Kubernetes, AWS DevOps Agent with Datadog MCP Server, EKS Control Plane Egress, and Why Users Feel the Wait70 / 100
  • Ship It Conversations: Guardsquare’s Joel DeStefano on Mobile App Security, Runtime Protection, App Hardening, and Why Scanning Isn’t Enough55 / 100
  • PeopleSoft Zero-Day Exploited, npm v12 Install Script Changes, GitHub Agentic Tokens, Anthropic Model Risk, and Default Trust Breaking48 / 100
  • Ship It Conversations: Meta’s Francois Richard on AI Incident Response, SLOs, and Reliability at Scale96 / 100
Explore the best B2B Engineering & DevTools podcasts →
All Ship It Weekly episodes →