Your Sales Team is now a Developer

Help. My sales team is suddenly developing. Welcome to Threat Talks. My name is Lieuwe Jan Koning.

And here from the headquarters at Venable during the RSA conference, we bring you the next episode. And the subject of today is vulnerabilities in software, hacking the Air Force, And what do we do about agentic AI? Let's get on to it. Welcome to Threat Talks.

Let's delve deep into the dynamic world of cybersecurity. So I'm really thrilled to introduce our next guest of today. His name is Jack Cable, and he found a little time in his very, very busy schedule here at RSA to talk to us. Jack, welcome to the show.

Thank you for having me. Excited to be here. So I mean, to do you justice, in young career still, we have to spend a whole episode of Threat Talks. And so I have to summarize.

I apologize for it, but you are first and foremost an ethical hacker. And you hacked the Air Force in the Air Force challenge, but you didn't stop there. I mean, you found 350 vulnerabilities. That's a lot of work.

But not small ones and not in small companies; Google, Facebook, the US Department of Defense. You were nominated by time magazine as one of the 25 most Influential teens. You don't get that title for nothing. You went to CISA to work on Secure by Design, probably, well, we'll find out why in a minute.

And now you decided to found your own company, so. Well, I'm really thrilled to learn today on why and what all your experience, how that led to this company and why this one and what how would you contribute to a safer world here? So once again, welcome. Likewise.

Excited to dive in. Yeah. So let's start at the beginning, because you were this teen and you were starting to hack. What did you do?

Can you tell the audience? Yeah, I grew up outside of Chicago, right. And started coding when I was in middle school. Just was very curious and saw that I could create my own websites, apps, games, those sorts of things.

And over time, I somewhat accidentally found my way into security. So this happened when I was 15. I was, you know, as many 15 year olds do, was looking at some cryptocurrency websites and noticed that I was able to send a negative amount of money to other people on the site, which would in turn steal money from their accounts. So I didn't know much about security at the time, but knew that was pretty severe.

And fortunately they had a bug bounty program, right? So I was able to report it to them. They paid me. I then really got the desire to start teaching myself a lot more about security, really, just by hands on poking around, right.

Companies who had bug bounties. I was fortunate to grow up in a time right where there were many bug bounties readily accessible. So I was able to kind of go down that path, right. Eventually got into lots of these live hacking events that HackerOne and others would do, and led to some of the work with the Department of Defense as well.

So what did you do there? So that was funny, because when I was 16, I get an email from HackerOne asking me if I wanted to hack the Pentagon. And that was a little strange for me, right? I was a sophomore in high school and was wondering, okay, why would the Pentagon want me, of all people, to hack it?

But I soon learned, right, this was the start of the Hack the Pentagon program. Right? So this was the first time the US government was inviting outside hackers like myself to try to find vulnerabilities in their system. So of course, I didn't pass on that opportunity.

I took them up on it. The first one around- So how does this work? So you get an invitation. Do you get a starting point or is it just like: the Pentagon.

Good luck. So you get a scope right. And this was a bug bounty program out of HackerOne. So there were a number of websites that were within scope for it.

But after that it's basically like yeah, just see what you can find. A URL. A list of URLs that’s a good start. Exactly.

Yeah. This was right. And it was at the start of my journey. So I think I found a couple things, nothing super interesting.

But then I started doing more of these, and particularly a year later with the Hack the Air Force challenge was when I found many, I think 30 vulnerabilities, wound up winning the competition, found all sorts of critical issues. You know, being able to get remote code execution on various Air Force websites, for instance. Issues, right, that if you want to you can use to presumably pivot to compromise other systems. So that was pretty eye opening, right?

Because here I was, high schooler who was able to to find all these really critical issues and also then showed me, right, what working in government could look like. Before that, I'd never really interacted with anyone in government, right? The perceptions are that government is slow and boring and not not very fun to be in, but I had a pretty opposite experience, right? This was a team called the Defense Digital Service, was sort of known as the SWAT team of nerds that came into the Pentagon.

Right, was part of the- You wanted to be in there. Exactly. Yeah. Part of the US digital service movement where they brought in a bunch of technologists.

This was after, if you remember, I think it was like 2014 or so, the healthcare.gov incident, where, you know, governments spent hundreds of millions of dollars. This was President Obama's flagship program. And then all that money later, they couldn't make a website that worked, right.

The day they launched, it went down and all that money went down the drain. So they realized they need people who actually understood technology in government to make it better. Right. And Defense Digital Service was an offshoot of that.

And I had the privilege of going to work there after high school and seeing firsthand what working in government was like. Yeah, you didn't need any more education. You found that quickly. Well, yeah.

And then I went to Stanford soon after. But yeah, it was actually a fun journey to get hired by the Pentagon. Right? They wanted to hire me.

I wanted to work there. I took them about nine months to actually work through the bureaucracy to get me hired, in part since I didn't even have at that time a high school diploma where at one point they were told that I would need to work at Best Buy or RadioShack selling computers first before I'd be qualified. And they were like, does RadioShack even exist anymore? Yeah, that's an interesting start of your career then there.

Yeah. So is there anything, I mentioned in the introduction 350 different abilities that you found. Is there anything to say about what kind of vulnerabilities, what is the cause of this, are these, for example, super sophisticated tests that you needed to do or was it surprisingly simple? What's your opinion on what you found back then?

Yes. Which year are we talking about? Roughly, which time period? So this was, when I was doing bug bounties most was maybe 2016 through 2020 or so.

So particularly when I was in high school, right. Would basically devote all my free time to poking around on HackerOne, seeing what I could find. If I had to categorize it, most of them weren't anything complicated, right? They were pretty straightforward vulnerabilities, many of them authorization or business logic type flaws.

Right? Issues like an insecure direct object reference, where the number of times I've been able to change a number in a URL and get access to someone else's data is pretty high, including on a lot of government websites, believe it or not. To be clear to the audience, for those of us who don’t... so you have a URL that says /invoice/number/view or so, you change the number and you get another invoice, but it doesn't belong to you but to someone else.

That's what you mean. Exactly. Very simple things. Yeah.

So issues like that. Right. And you might think well how are those still present. Right.

Part of the element is at least up until, you know, two years ago with the advent of LLMs, there really wasn't any way to scan for those. Right? Because the existing scanners, they would look for known types of vulnerablities, SQL injections, cross-site scripting, those sorts of issues. But, you know, authorization issue is subjective, right?

Does someone have access to that information? It requires knowledge of the logic of the application, it's not obvious to a computer. Yeah. Whether it’s allowed or not.

Yeah. That's right. So a lot of the vulnerability were like those there were some more complicated attack chains or issues that related to details in logic. Think about authorization flows.

For instance. One of my favorite vulnerabilities actually was surprisingly trivial. This was in a site owned by Yahoo! And they, of course had a button where you could sign in with Yahoo, right?

And right. This goes through the OAuth flow, right? A way of signing in through single sign on. But what they would do is when you went through that, they sent a request saying your password was not provided right because there was no password.

But the issue was that it sent the password. It was literally ‘not been provided’. And it turns out that you could log into anyone's account to sign in with Yahoo, with the password ‘not provided’. Right.

So it was sort of this mismatch in how they obviously didn't intend for that to happen. But nonetheless, that was the effect. I found that at lunch one day over a high school internship. And that was pretty fun.

Nice. So and then you went to CISA because secure by design is something you dream, right? Can you... Is that a direct result of finding these kind of vulnerabilities?

So it was interesting right. So my path in government began at the Pentagon. Was Defense Digital Service right. Was there after high school and then stayed on advising.

And then my first foray into CISA was in 2020. I was registering to vote a couple of years before, for the first time in Illinois, my home state, and stumbled across a vulnerability in the voter registration system. It was a SQL injection, again, quite basic. It also turns out this was an issue that if you read the moro report the Russian government had exploited several years before.

So I had found that was present in another place and went across a lengthy process to try to report that to them involved CISA helping to disclose that. So in 2020, I went in focusing on election security, building out tools to help stand local governments better identify vulnerabilities on their network. We had build out an open source attack surface management product. So that was my first kind of step at CISA, but always knew I wanted to return, right.

And I was given the opportunity to a couple of years later. And right. This was when Jen Easterly was coming in, was a really exciting moment to try to redefine how CISA operates right. It's at this point, I don’t know, a six year old agency or so, but that's when it was named CISA.

It's been around in some form for the better part of two decades. Right. And as part of that, really, we're looking to see, okay, what, as the nation's lead cyber defense agency how can CISA assume that role and actually help build more secure systems. And a lot of credit to people like Bob Lord, right, who really have this vision to enable a safer future for everyone.

Right. And that starts with the technology that's underpinning everything we use. Right? So asking questions like, okay, why do we just accept when a vendor releases a CVE and all of their customers have to scramble and patch it, some get hit with ransomware, shouldn't it be on the vendor to do a better job to ideally not have vulnerabilities like that in the first place?

So that was really a- So it’s a software problem, is what you're saying. We need to make better software. Yeah. And I had seen this firsthand, right, through bug bounties, right, time and time again with very basic vulnerabilities that could be exploited to to cause harm.

So that was what I was really motivated to try to- You thought, developers need to do a better job. Developers or the companies behind it. Right, and put in place systemic means of preventing entire classes of vulnerabilities. That's why at CISA, Secure by Design was your way to advocate this, better software.

Yeah. So and software writing has changed recently, you have experienced that first hand. Can you summarize a little bit how how was the developers job like five years ago, and today? Because it’s completely different.

It's a different job right? Entirely different I think this is the, you know, greatest change in the history of software development, potentially since the compiler. Right, where it is almost right. I think drawing parallel.

Right. Where before, you know, in the early days of computing, you would have to program punch cards or write an assembly, right? Instruct the computer literally what to write to each register, how to do things at a very atomic level. We found with programing languages that, okay, we were able to make this more human understandable, right.

Have greater abstraction. And I think what's really wild about this latest wave with AI coding, right, is we are now moving to English being the language that can then get compiled into code. Right. And this has never been possible before.

Because as a developer, you have to teach yourself all these languages - Exactly. - like Golang or PHP or whatever your go to language is. And now it's English that can get you there. Yeah, yeah.

And this really has happened within the span of 18 months or so. Right. We're about a year and a half ago was when Cursor came out. And I think they really were the first to show that okay.

It's ...Initially there were some uses of autocomplete, which kind of interesting but nothing revolutionary with AI. But Cursor came around and showed that, okay, if you give an English language prompt, AI, it was quite good then, it's incredibly more powerful now, at turning that into a fully functional program. So we're at the point now, right.

And I encourage folks listening to go around and try whether it's Claude Code, Lovable, any product out there really just to see how powerful AI is for coding. But what that means, right. The day to day life of a software engineer is entirely different, right? The engineers at our company are almost never touching code by hand.

Right. Because the agent can do it for you. And of course, right. And we'll get into this.

There's security implications of that. Still have to make sure the code that's going through is secure. But I think, right, it's reasonable that a year from now humans won't ever be writing code by hand. You just, you can even with text to speech, you can simply say, listen you if you go to this page where my I don't know if your CRM software you have that and you say, listen, I want an extra field there and there you just describe it.

I want the field to do the such and such and so, it needs to compute like this and that, it will actually, the AI will actually write an epic, make the issues, then implement the issues, make the pull requests, validate the codes, ship it, deploy it, and it's really nothing... Well, we're just there to tell the computer the next step, right? Exactly. [ ] Because we are a little bit afraid of of saying do everything because who knows what happens.

Yeah. The job is turning into a mix of being a product manager, a QA tester to kind of guide agents to know what to do. And I will say, right, important caveat the people who can make the best use of coding agents are those who understand the fundamentals of computer science and software engineering, because you need to be kind of the critic to the coding agent, right? To point out where it's going wrong because, right, they're really good, but they're going to be missing context.

They're going to do things that aren't totally in line with what you want. So you still need to understand how everything is working under the hood. But I think, right, you can be exceptionally more productive. You need to babysit the agent, sometimes say, okay, this is the wrong turn.

And the earlier, in my experience, the earlier you detect where it's going in the wrong direction. You steer a little bit in the right direction, and then it goes on and it finishes. So that's the job of the developer today. But like you said, tomorrow maybe in a couple of months, it's really hard to predict the future...

Yeah. Yeah this technology's been around for a year and a half, it changed the entire software industry. Yeah, it used to be a very good job perspective if you would teach yourself Golang or Python or for that matter. But it's no longer the case.

So you need to be like more of a product owner or architect. Exactly. Today, at least. People who know how to use agents well.

But now we get to a point that I mean, there is AI models are trained on open source projects, on code that's out there. Does it get any better than what the humans create? Because you figured out first hand all those vulnerabilities that are there, how secure is the code that this AI is actually making? It's a great question.

Right. And I think there's a couple of ways to view it. Right. I am, I would say overall optimistic that in the long run, AI can be used to produce code that is significantly more secure than we've had in the decades before.

Right? Part of it is, and you can make an analogy to self-driving cars where, right. Self-Driving cars aren't perfect. They do get into accidents.

But right. Human drivers are notoriously not very great either, right? I think there is something like 40,000 deaths a year in the US due to automotive accidents, and you have to be perfect, right? If you're building a self-driving car and your goal is to improve safety.

You just have to be better than a human. And right. That's where we see all the- Do you think there's a ceiling in this so that it doesn't get better than the best input we train it to? So I think there's elements of how we can kind of help give coding agents the right boundaries to operate in.

Right. But what I'd say overall right is that the like on a per line basis today, I would bet that the average LLM is significantly better than a human. But it's not just a line to line comparison, right? Because a human developer might produce a couple hundred lines of code in a day, LLM might produce tens of thousands of lines of code.

So the absolute amount of bugs makes is much bigger. So yeah, we are seeing that number skyrocket. Right. And there's benchmarks out there.

There's one called BaxBench that was put out by some researchers and found that, right, the average model, even very smart models like Opus 46 or GPT Codex, are introducing vulnerabilities 20 to 40% of the time when they're writing code. Right. And if you think about the scale this is moving at, right, this just means that we are going to have to radically rethink how security is done, right. Again, I think that one of the advantages now, right, is that instead of having to train every developer on how to write perfectly secure code, and that being the main path to increasing a company's security.

Turns out the models are very knowledgeable about security, right? They're far more knowledgeable than the average software engineer. But you have to ask them the right question. Exactly.

If you don't ask it, then they won't... So yeah, you have to give them the right context, right? And a coding agent isn't going to know that your financial services company, has to follow X regulations and do specific actions with data, they really need to know the boundaries to operate in. And when you give them the right information, specific instructions on how to do it, they do it quite well.

But in the absence of that, they start making assumptions and that can lead to mistakes. Yeah. So you have a solution to this. Before we get into that, I want to talk a little bit about the what I said in the introduction.

My sales team is now I mean, you mentioned it's becoming more you need to be more of a UX-er, like a designer of the application. You need to know your business very well. You need to be able to speak English, which is very convenient. So everybody can be a your non engineers are writing code now.

So there's in itself maybe a risk in there, because I mean your dev teams are governed by all kinds of four eyes principles, code reviews and whatnot and testing and all that. But yeah if you are installing Cursor or Antigravity or whatever or using a... everybody can write software today, it's really true, right? That's scary.

At least to developers or to CISOs, right? So what should be the response? I mean the natural response maybe’ let's forbid this from happening. Sales team, don't write code.

What do you think? Yeah. We're seeing this increasingly among companies right where now anyone can build anything really. Right.

And this is an incredibly powerful tool. Right. We're using this internally to, for one, build better UIs on top of all the tools we're using. And previously would have been just resigned to accept those.

Now we can in ten minutes use Claude Code to build something that actually gives us what we want. So- And we have to, because the competition does the same. Exactly. So if we don't, then we will be outperformed.

Yeah. Yeah. So I would say right. It's not an option to block this or prevent it from happening.

Right. Because you will fall behind. These are incredibly capable tools that allow every team to scale in their specific tasks, right? They can build software that gets to exactly what they need.

So my advice would be right really to embrace it and figure out how to give your teams, right, these kind of paved roads to build securely, right. To deploy in a sanctioned environment, all of that. Right. Because otherwise they're not, not going to do it.

They're going to set up, you know, use one of the, you know, tools like Lovable or v0, deploy things out there. Right. We've seen, you know, tools getting created that use customer data and have the potential to leak it. So my advice would be to figure out a sanctioned way to allow teams to build and deploy these tools in a way that has some constraints around it and some kind of- What does that mean?

You have like a git repository that you connect, make a like a how to guide or a sales guide you deploy, I don't know, one of the tools you mentioned, for example, or CS code, even if they're a little bit more savvy and some plugins and you provide them with a GitHub repository with some kind of infrastructure. It's getting a bit technical right now, but the company, like the IT development department provide this to all your employees. Yeah. And what we've done internally for instance.

Right. We've made super easy to we have a unified infrastructure for deploying internally developed applications and ways to enforce right that we have SSO deployed, enforced for everything so that only our employees can access it. Right. So it's about kind of making it easy to deploy, but also having it in the sanctioned environment right where- So then we get the quality a little bit up, but what immediately springs into my mind is you have this famous xkcd comic where there's this building build and there's one crucial pillar, like if it would break, then the whole building would collapse.

Maintained by this, I believe it's a disgruntled employee or no, an employee from Arizona. The single person that hasn't gotten a raise in ten years or so. So what could possibly go wrong? Yeah.

Yeah. I mean, despite the fact that we then have all kinds of guardrails, etc. in place, it's hard to solve that I think. Yeah.

And right, that's where it gets to towards like these tools are very good for prototyping building simple applications. Right. It doesn't absolve the need to maintain systems. Right.

And if you want to build these and can seriously deploy them. The cost isn’t zero, right? And kind of gets back to the, they can help with every task. But there still is that non-zero cost of maintenance and keeping up to date all of that.

That doesn't go away. Yeah. But that's probably, that's a different subject you might argue, let’s first focus on how to get this whole thing secure. You mentioned systemic guardrails.

What does that mean in practice? Let's get a little bit technical here, because we have to, because it's an operational thing that needs to be fixed. And what did you invent for that. Yeah.

And right, a lot of this goes back to the work that I was doing at CISA, with secure by design. Right. And our big push was for companies to put in place mitigations that would prevent entire classes of vulnerabilities. Right.

To give maybe a couple examples. Right. Memory safety. One instance.

Right, where about two thirds of all vulnerabilities discovered in code written in unsafe programing languages C, C++, are memory safety vulnerabilities that can be prevented by using a memory safe programing language, Go, Rust, etc.. Right. So there's these systemic controls. Similarly for issues like SQL injection, right, where you can use what's called parameterized queries that make it so you just cant- Something that an AI can detect.

I mean, if you tell an AI listen search for programing languages that are not memory safe or search for non parameterized anything actually, whether it's HTML or SQL queries, it can detect this. Exactly. Right. So there's these systemic mitigations that if you have them in place, it just makes it a lot easier to not introduce those vulnerabilities in the first place.

And at CISA, and a lot of this was pre the AI coding boom. But I think it is even more important now right where you want to give the AI coding agents the boundaries within to operate. Right. If you and what our project Corridor does is it makes it super easy to define what those guardrails are.

So it's okay, for database queries, we use this framework and for authorization, here's our internal authorization framework how to use it. Right. And then we inject that context in so the coding agent knows what those secure boundaries are. Yeah.

So this means that your development department, well the OPS guys create this infrastructure, development department gives you like the framework of an application with the authentication with maybe a couple of programing languages, the tools to build it. And then everybody can go off to focus more on the more functional part. Exactly. Yeah.

Yeah. And that's where right, we integrate in with all the coding agents folks are using. And we help both provide that context up front. We're scanning code as it's being created, giving the coding agent advice on how to prevent vulnerabilities.

So we do all of that in order to really again, a lot of how we see this is as a context problem, right, where especially for large companies with large code bases, policies around secure coding that might exist in a PDF somewhere might exist in someone's head, right? We can get that all in one place and enforce that from the very start, right? So to be clear, if someone in the company who is not tech savvy, they cannot deploy any software if it doesn't pass these guardrails that are codified somehow in an AI product or like a pre-commit hook in technical terms, right?

Yeah, yeah. And really we see this as an acceleration, right, we don’t want to slow down development. We want to make it possible to build and do everything right. Most of the time, developers don't even notice that Corridor’s there, because it's working behind the scenes with their coding agent to help their code be more secure.

So what if someone in the sales team makes something that's wrong and then your tool says no. So it's interesting, right? There's a balance between like what the developer is asking for. And again it's right that balancing friction with speed of development.

Right. And we really don't think that that has to be a trade off. Right. And sure, there can be more extreme cases, right?

Where someone's like, I want to expose all of my customer data on the internet and we'll be like, you should add authorization behind this. Ensure they can still, if they really want to do that can go and make that happen. Right. But the minimum we want to make it so the default path is a secure one right.

And then the second part is giving the security team visibility into what's happening. Right. Because right now there's really a lack of insight there. And we can help them see which coding agents are being used, how they're used.

Right. As well as put in place these guardrails that then get applied across the board. So you mentioned visibility to the CISO team, so what kind of things... Would that be like a report or a dashboard or what do you, what does the CISO need to know when this whole ecosystem of homegrown I don't want to call a shadow IT, because we're trying to avoid shadow IT and make it like more of a production state thing.

That's the mission. Yeah. So what kind of information do they need to know? What should they ask for?

Right. So there's a couple of things, I would break it roughly into two parts. One is around the security of the code itself. Right.

And knowing what guardrails are enforced there. Right. And then the second is around the environments in which code is being developed. Right.

So how the coding agents are configured. Right. One thing we see often is that people will install a coding agent and use their personal license, right. Many of these by default have provisions to train on your data, which if you're an enterprise, you almost certainly don't want.

Right. So those sorts of elements where we can help them see both how the coding agents are being used, which licenses they’re on get everyone over on to the sanctioned plan. And then things like which MCP servers are called, tool calls, all of that. So that is all under how the coding agent is being used.

And then we can also- Is this runtime information? Or can you already see this when the developer is creating the application? So if you're not engineer make something apparently because the AI told them so uses an MCP server. And to explain to the audience, an MCP server is like a standard think API.

But now in a way that an AI agent can understand. Right. So you're connecting different systems to each other and well, there's maybe a risk there because who knows what could possibly go wrong. Right.

Yeah. So I'd say right. It's like by default. Right.

The companies don't have this data. Right. You get fairly little information. For instance, if you're just looking at what's like checked into the code base in GitHub because you don't know how that was built, you don't know which tools were used to build that, right.

So the first step is really to know what's being used and we deploy an extension to Cursor and plug in for Claude Code and support for the major coding agents. So it's about getting that preliminary visibility right. That can then help them understand okay what guardrails should we be putting in place. What sorts of applications are even being developed?

Maybe there's something that was built in because some of these it's not even checked into source control, right? You just go and create a website and can deploy it without having any trace of that. It’s the dream of every CISO. I'd say.

To have this kind of a visibility. And what you're saying is this has to be mandatory, this kind of thing. Right? Yeah, I think it's right, a starting point again, if the goal is to not block these tools but enable and accelerate development, right, then it's a prerequisite to understand how they're being used.

Great. Yeah. I also want to talk to you about agents, building agents and all the risk associated that, but we’re kind of out of time. So let's get back to next week and talk about that.

But for now, because there's a lot of unwrap there as well. Because that's the next episode I think that will happen. What's already happening in the real world. But for now, thank you very much for all these insights.

It's really clarifying. And to our audience, yeah. Think of this kind of infrastructure and how you in a secure way, enable your complete staff to build AI. That's what you're saying.

And then in the meantime, avoid those basic mistakes and AI's can help you there. To summarize what you're saying. Yeah. Exactly.

Thank you so much. To our audience, thanks very much for tuning in today. If you thought it was really useful, like us, it will help us spread the word more, but especially today. Very important.

There's a subscribe button there and if you press that, then next week, that next episode about agentic security, well, that will also be in your inbox. So thank you for today. I hope to see you next week. Bye bye.

Thank you for listening to Threat Talks, a podcast by ON2IT cybersecurity and AMS-IX. Did you like what you heard? Do you want to learn more? Follow Threat Talks to stay up to date on the topic of cybersecurity.