Nate Lee: Building a GenAI Security App for Fun (and No Profit)

Speaker A: This is the Cloudace podcast, bringing you the latest in cloud security through captivating chats with fascinating cybersecurity experts who are leaving their mark on the industry. Cloudace is brought to you by the SANS Institute and hosted by Sans fellow Frank Kim. And now prepare for departure. We're cleared for takeoff. Here's your captain, Frank Kim.

Speaker B: Hello and welcome to the Sans Cloud Ace podcast. I'm very excited to have on the show today Nate Lee, who is the CISO at Tradeshift. Nate, thanks so much for joining us.

Speaker C: Hey, thanks, Frank.

Speaker D: Glad to be here.

Speaker B: All right, well, hey, as we like to do, let's go ahead and start, uh, the show here with a little bit of personal history. So Nate, can you tell us a little bit about your family, where you grew up, things like that?

Speaker C: Sure, sure. So I'm originally from Minneapolis, uh, so Midwest, um, grew up there. My, well my dad was a bus driver and my mom kind of had started lots of her own businesses. So a non tech family let's say. But, um, uh, my parents had got me an Apple Iie when I was five, uh, and uh, I spent many nights, uh, stuffing kind of clothes under

Speaker D: the crack of my door so they wouldn't see that I was kind of

Speaker C: hacking away at stuff, uh, from a very young age. Uh, so really got my start in computers, uh, probably far earlier than your average person. M and that kind of carried me through, uh, into diving into tech and uh, eventually of course led out to

Speaker D: Silicon Valley, the promised land.

Speaker B: Now, did you always know, I guess because of that experience that you did want to get into tech as a career or in school.

Speaker C: I don't know if I ever thought about it as something for a career. It was just always so fun. Um, I remember trying to write adventure games in basic, uh, as a kid. So, um, lots of go to statements going back and forth and I'm fairly

Speaker D: certain that never worked well.

Speaker C: Uh, but it was just super fun, um, being able to kind of build,

Speaker D: uh, with something you could take you away to another world.

Speaker B: Yeah, I remember all those go to statements. Lots of spaghetti code for sure. Uh, so then you're in school and then you've got to find a job at some point. Now, did you start in security?

Speaker C: I didn't start in security. I started doing, um, kind of more general system admin work, um, network engineering. I spent a lot of time doing. But with all of those, right, I

Speaker D: mean, security is kind of part and parcel to doing any of those, uh,

Speaker C: in any reasonably sensible, uh, way. So I mean, I'd Always been, I

Speaker D: would say security adjacent and very security aware.

Speaker C: Um, but really I think over the

Speaker D: last maybe 15 years is when I started uh, focusing on that.

Speaker C: I did a CIS SP while I was working at Target, um, because I was focusing on a lot of uh,

Speaker D: sort of security bits there on the network side.

Speaker C: Um, and then coming out to the Bay Area working at software companies.

Speaker D: I mean it's.

Speaker C: You're exposing your services to the Internet. Uh, you better be pretty security aware

Speaker D: or problems will arise very quickly.

Speaker B: Mhm. Now you mentioned ah, being a Target, I believe. Hey, when the breach occurred, that was uh, a year or two after you had left.

Speaker C: After I left? Uh, yeah, yeah, that's, I'm, I'm uh, I remember many discussions around some of

Speaker D: the systems affected beforehand. But yeah, I wasn't, wasn't uh, there when the, the fan was hit.

Speaker B: Well, hey, lucky timing. So you know, hey, you got to give me some, I don't know, stock advice or something because it sounds like you got a little bit of a uh, crystal ball there.

Speaker C: Uh, well, I still had my stock

Speaker D: at Target when that happened, so maybe you don't want the advice from me.

Speaker B: Hey, so you're in these various IT technology roles and then you know, how do you get into security full time?

Speaker C: I mean I would say I sort of fell into it, but again I think I was really already headed in that direction just based on um, a

Speaker D: lot of the work I was already

Speaker C: doing um, when I started at Tradeshift. So the company I'm at now, we um, didn't have a security program. I was running the platform operations team, um, IT site, reliability, engineering, a few other teams, M. And then it really

Speaker D: started to become a blocker for sales.

Speaker C: Um, and it's something that I was already fairly familiar with. And I think part of me wanting to jump into the role was because I'd worked in places where it was very GRC heavy, um, and people would

Speaker D: put in controls that you know, were what I would consider the wrong ones.

Speaker C: Um, and I really didn't want someone

Speaker D: telling me what to do that wouldn't uh, make sense.

Speaker C: Uh, and I figured it would be

Speaker D: better if I was in charge of that and you know, could make sure things were done properly.

Speaker C: So uh, started from there, not um, necessarily a ciso, but really just building the security program. But then uh, as the company grew and the role grew, um, started to

Speaker D: focus more and more time before, uh, eventually just taking on the role full time.

Speaker B: I like that advice, that approach. Hey, you don't want somebody else to tell you what to do, tell yourself what to do.

Speaker C: I'm not sure I've heard anyone else with that story of how they got into it, but that really was it for me.

Speaker D: I just had some fear of someone trying to make me rotate my password every 30 days and it was too much.

Speaker B: Now, so you've been at Tradeshift, I think a little bit about uh, nine years now. Right. So that's a long career. Different roles as you've said. Yeah. You said ten years?

Speaker C: Ten years, uh, next week.

Speaker D: Ten years, yes.

Speaker B: Wow. All right. Congratulations. Yeah. And so, you know, can you tell us a little bit about the company, the business, you know, how did it change from when 10 years ago to today? And along the way, you know, what did you kind of technology it looked like from a kind of, you know, cloud perspective and how did that mature? Similarly, you know, what did security look like then versus now?

Speaker D: Sure, sure.

Speaker C: I mean I think it's been really exciting. Um, when I started the platform itself

Speaker D: maybe had uh, 10 services and that, that might even be high, might have

Speaker C: been something like five.

Speaker D: It was fairly monolithic as far as how the back, backend looks.

Speaker C: Um, so it's been really great, um,

Speaker D: being a part of that growth. So we've been kind of breaking off into microservices, moving into containerizing stuff, kubernetes, you know, all of the, the, the

Speaker C: cloud native stuff that, that we take for granted now. Um, for companies that started 10, 15

Speaker D: years ago, you actually had to transition to that. You didn't get to build that way natively.

Speaker C: So m. It was really great getting

Speaker D: to work with the engineering teams on, you know, how do we think about where we slice this up, how do we talk about uh, authorization authentication between services?

Speaker C: Um, and you can't necessarily tackle all that up front. So really thinking through what can we

Speaker D: do now, how do we make plans for the future to, to build this in a more more secure manner.

Speaker C: Um, and just developing uh, the engineering

Speaker D: culture along the way.

Speaker C: I mean definitely had great relationships with

Speaker D: the engineering team and support from, from

Speaker C: leadership and the rest of the engineering

Speaker D: org, um, as far as building ah, process and culture into how we build things.

Speaker C: So over time, moving to general microservices, um, we've always been uh, AWS, uh, as far as 100% of the platform runs there. Um, I think one of the interesting things was originally we were like, hey, we're going to do EC2 only because we want to be in control of our destiny.

Speaker D: If we want to move to Azure

Speaker C: someday or Google Cloud, we want that freedom. Uh, which always seemed like a great idea.

Speaker D: Uh, and then I think maybe like

Speaker C: five years later we're like, why are

Speaker D: we doing this to ourselves?

Speaker C: Uh, we're managing so much operating systems

Speaker D: and databases and this and that.

Speaker C: Um, so over the last several years we've had kind of a market move

Speaker D: to using more managed services, um, just to let us focus on delivery of the actual business differentiators as opposed to, we're really good at, uh, hardening our OS or something like that.

Speaker C: Um, so, yeah, it's been a very interesting journey.

Speaker B: That makes a lot of sense. Anytime in the past, if I try to think back of where we tried to be technology or vendor or platform agnostic and try to architect the system in such a way where we could, at the drop of a hat, move from one platform to another. Uh, never really worked out that way. Right. From a practical perspective, I mean, I

Speaker C: think eventually you would hit enough scale where like, okay, maybe it makes sense

Speaker D: to have our own data center and you move things, uh, on prem as opposed to being in the cloud, but you, you really need to be at a fairly large scale before that starts making sense. That you have network engineers and people racking servers and kind of managing all the physical bits.

Speaker B: Yeah. Um, how much of the security capabilities you mentioned, working with all the other teams, platform team, DevOps team and so on, how much of it is kind of security, building those capabilities with an engineering capability inside the security team versus working with those other teams, the other engineering teams, to have them build them into the product from scratch.

Speaker C: I think it's a mix, but it really ends up being that no security team is well staffed enough that you

Speaker D: can build all of this. Right.

Speaker C: So, um, working with the other teams to help educate them, I think first on why it's important. Um, because a lot of times too many of these teams, they know security is important, they want to build secure platform, um, but they don't necessarily understand how some of the vulnerabilities and threats

Speaker D: could materialize in what they're building.

Speaker C: Um, so helping kind of of educate them on, hey, here's how, uh, some of these systems may be exploited, here's what the implication could be, um, and just really building relationships with them where

Speaker D: they trust that you're also looking out for the best interest of the company and their team and the architecture and whatnot.

Speaker C: Um, starting there is always great because

Speaker D: then you can start having those deeper

Speaker C: technical discussions and they'll bring you in when, um, if they're building out a

Speaker D: New authentication layer or um, exposing some

Speaker C: new services, they'll reach out ahead of time to say, hey, can you help me take a look at this? You know, is there anything you think

Speaker D: we should be doing?

Speaker C: Um, because if you start off from the standpoint of you know, the ivory tower where you're just telling people you

Speaker D: have to do this, here's your list of controls and you're more of a

Speaker C: kind of the police officer going around,

Speaker D: people won't come and talk to you

Speaker C: because they know they can get it almost out and then it's ready to release and yeah, they'll tell you about it and you might not like it

Speaker D: but you're not really going to be able to do a lot.

Speaker C: So um, I think for us it was a lot of just making sure

Speaker D: we work with teams listening to their concerns.

Speaker C: Um, as an example, when we released

Speaker D: static, um, code analysis, so we started

Speaker C: putting checks in place, um, so that whenever developers were checking in code it

Speaker D: was looking for things like um, cross site scripting, um, SQL injection, all of

Speaker C: the OAS bits, um, making sure that we were super hyper reactive, uh, as

Speaker D: far as any false positives or blocking,

Speaker C: um, you know, starting in listening mode, tuning stuff. If it's like hey, this is alerting but is this really an issue? Um, being really sensitive to that, I

Speaker D: think that built a huge amount of trust that just paid off uh, as we went forward as a security engineering program.

Speaker C: So I mean with any of this

Speaker D: I think that's where I would start. Mhm.

Speaker B: You mentioned a really good example of a tool category of tool with sast. And so was it really, hey, making sure that the SAS tool is running in the pipeline, in the CICD pipeline itself and as you said just hey, in non, uh blocking mode, uh, to start. And so you actually get the findings repetitively. And we, I think we all know from experience that SAS solutions you run for the first time gives you a uh, gazillion findings. How did you work get to the point of tuning down those rules to a manageable level so that the pipeline still actually runs decently fast.

Speaker C: I think we started with just turning down a lot of them that we knew would be kind of not terribly useful. Um, and then we left it on blocking for uh, I believe just critical, severe level, kind um, of major security issues, not so much the, you know,

Speaker D: hey, you used spaces, you should have used tabs. Uh, didn't want to kind of wade into that mess.

Speaker C: Um, so we started from there and then you know, we would start turning on more things but Again just making sure that we were super um, able

Speaker D: to react quickly if there was problems

Speaker C: uh because we have a fairly distributed engineering team. Uh so across there's folks in Asia, there's folks in, in the, in Europe and uh, in the United States. Um so being able to, if someone

Speaker D: has problems regardless of where they're at, you know be being able to kind

Speaker C: of tune the, the alert and if we can't just disabling it for now

Speaker D: and, and making sure we follow up uh, quickly afterwards. Um, I mean that was very key. Mhm.

Speaker B: Are there any other categories of tools that you found most useful as you're building the security program, especially related to cloud, like how much did you use the cloud native, the cloud provider, security related services versus a ah third party off the shelf server tool.

Speaker C: I mean and this is more reactive but a ah, guardduty has been super helpful just when there are problems to kind of alert us. Um, we've started using some CSPM which is good in some ways but I think a lot of them are just developing very quickly.

Speaker D: And so uh, we've had some usability problems and uh, just general bugs we've been running into as we try to

Speaker C: automate uh, some of that usage. Um, I think one of the other big things and again it's a bit more reactive but um, having a bug bounty program was really great and I think obviously from being able to close

Speaker D: some of the vulnerabilities you didn't find otherwise.

Speaker C: But what it did is it made it real. Um, again for developers you're kind of like hey, if this happens. And they're like well that's not an issue. And you're like yeah but if this happens, this happens together. And they're like yeah but could that really happen? And then you get a report where

Speaker D: someone found it and actually did it.

Speaker C: It makes it much more real. And again that changes how people think

Speaker D: about it because they don't want to

Speaker C: write insecure code but they also don't want to waste their time if they

Speaker D: think that this is an impossibility.

Speaker C: And then um, just the diversity of people who are hacking away at your

Speaker D: system when you're in a bounty program

Speaker C: is, I mean it's just really something

Speaker D: you're not going to get from a pen test from somebody if you have them coming, have two people poking at

Speaker C: your system for a few weeks. Um, and I think that was really

Speaker D: helpful again to set the culture of like okay, these are things we need

Speaker C: to worry about and really made people

Speaker D: want to kind of shift Left and get ahead of things.

Speaker B: Mhm. Yeah. So you know, you mentioned cspm and you know that's kind of to me like becoming the vulnerability management scanning solution for the cloud. Uh, you know, any misconfiguration, any issue. And a lot has changed in the last five years or so with that category of tool. But you mentioned some of the challenges and what are some of the things that typically come up? Is it too many findings? Deduplication, you know, not being able to track it to the root cause? What, what's been the, the most cumbersome things and how did you get, get around those?

Speaker C: Yeah, I mean it's, it's been kind of a mix of all those. There's, there's lots of findings. Um, and I think this is something

Speaker D: every security program has. Right? You have a lot of findings.

Speaker C: Uh, you turn on something that's going to find vulnerabilities in your software, especially if you, you start digging into dependencies and uh, suddenly you're like, oh, if we turn this automation on, I'm going

Speaker D: to throw 10,000 tickets at a team. That's not helpful.

Speaker C: Um, so some of it was that figuring out from the findings, like yeah, there's a vulnerability. But where does this fit in our

Speaker D: list of uh, priorities and actual risks?

Speaker C: I think that's been, uh, still not something that we've been able to solve there. There's just been some general bugs as far as within the UI and APIs

Speaker D: as we build out, uh, integrations and automation, where it's just been, I think, frustrating for the team a bit to um, try to sort out why something isn't working. And I mean general things when you have a space that's been uh, kind of come out of nowhere and built up so fast. So I don't necessarily fault the vendor

Speaker C: but uh, it's been, I know, a

Speaker D: bit frustrating for the team.

Speaker C: Uh, but at the end of the

Speaker D: day I still think it's a tool class that will be super valuable because

Speaker C: as uh, your company's cloud footprint grows, you can't wrap your brain around all of it anymore. There's just different services, um, you have

Speaker D: things deployed all over the place, different

Speaker C: regions, um, and having something that can

Speaker D: make sense of that and tie policies together across services and is this exposed because it chains through whatever the load balancer or something like that? Um, I think there's still a ton of value there.

Speaker C: Mm mhm.

Speaker B: Yeah. And speaking of policies, you know, one of the ways that you build customer trust, get them to eventually buy your product is by having third party uh, attestations, third party reviews and related to policy. You know it's got your SoC1 type 2, your SoC2 type 2, you've got your ISO 27000 and, and so on. And I think you uh, know at Tradeshift you are the one that uh, are responsible for getting all of those things in place. Right. What was the, what was that journey like? How did you start with that and how was it, where were the elements that were cloud related? Where was there a disconnect with kind of cloud, especially early on in terms of making the auditors feel comfortable that you were doing the right thing?

Speaker C: Yeah, and I think it's changed over time because I think uh, like you said, um, if we look back seven, eight years ago when we were first starting, we uh, had to do a

Speaker D: lot of educating the auditor on what

Speaker C: the systems meant, uh, and figuring out how do we phrase controls in a

Speaker D: way that, that makes sense for uh,

Speaker C: both our customers as well as the auditors.

Speaker D: Uh, because you know, people are like

Speaker C: where's your three tier application?

Speaker D: You know, do you have it?

Speaker C: Where's your dmz? And uh, you have to kind of explain that uh, we don't have a dmz, but it's you know, there's per

Speaker D: instance firewalling and um, and then explain how security groups work versus uh, you know, host ACLs, things like that.

Speaker C: Um, so originally there was, there was a lot of that I would assume

Speaker D: now with most auditors they're going to be uh, fairly familiar with, with various cloud uh, implementations.

Speaker C: So it's, it's probably less of a

Speaker D: uh, an uphill climb there.

Speaker C: But um, I think the big one with, with having these right is especially if you're uh, a cloud provider and you're not so you know, a major household name, uh, having these is, is

Speaker D: just you need to. Because people don't know about your security program.

Speaker C: And this is a way where you can describe the controls you have, you

Speaker D: can describe how you think about security.

Speaker C: Uh, I'm not sure how many people actually read my reports that we send to them. I'm always surprised when someone comes back

Speaker D: with a question or has feedback. It's kind of gratifying to know that someone actually read it instead of, I assume 90% of customers kind of get it. They check their box and uh, move

Speaker C: on with their day. Um, but being able to describe the program uh, for those customers that actually

Speaker D: do care is super, super valuable.

Speaker C: Um, and being able to work with your auditors and guide Them, Uh, because I think a lot of people take what the auditors say as, oh, hey, you need this.

Speaker D: And then, you know, they'll put it

Speaker C: in without really thinking about it. Um, being able to take feedback from the auditor and kind of synthesize that

Speaker D: with what you know about security, best

Speaker C: practices and what you know about your, your systems, ah, risks, threats, so forth. Um, and being able to work with the auditor then to say, well, hey, maybe that doesn't make sense, but I think this is the concern you're talking about.

Speaker D: What about if we change this control or add a control, um, into something sensible?

Speaker C: So I think that's, that's one of the key things to really making sure you have a successful compliance program that adds value to customers and also doesn't alienate. You know, your engineers are going to be the ones, uh, especially sensitive to this because if you're putting in controls,

Speaker D: uh, kind of blindly because your auditor told you to, you're probably putting in some things that, you know, aren't adding so much value and are causing a big hit to productivity.

Speaker B: Yeah, yeah. Even though obtaining those third party certifications might be kind of the headline, kind of the magic is in how you actually get it done, not just blindly checking the box. Yeah, that makes perfect sense. What did you start with? Did you start with Soc 2, Type 2? Did you start with ISO or PCI?

Speaker C: Uh, we started with Soc 1. Oddly we had that, uh, before I even started. I think it was a couple years

Speaker D: old and hadn't been renewed.

Speaker C: Um, so we had that beforehand and that really, for those that haven't gone through a SoC one, it's focused much more on the financial controls, um, because our platform is handling, uh, supply chain

Speaker D: finance, uh, virtual credit cards and all this.

Speaker C: So it's. What are the technical controls in your platform? Both at the underlying infrastructure layer, but

Speaker D: also at the application layer.

Speaker C: What are those controls that um, ensure

Speaker D: customers can have faith in, in how it could affect their financial reporting?

Speaker C: Um, and this has kind of been a challenge for my team because as a security team, people just say, oh, SOC report here, you should work on this. Um, but really it's much more of something for the product team to own.

Speaker D: Uh, as far as, uh, what features do you have in place to make sure tax is calculated correctly or that the invoice and the purchase order are reconciled correctly?

Speaker C: Um, so we started there, didn't have much to do with my team, but

Speaker D: we ran it because it started with the letter S and I think, uh, it had an audit and people just

Speaker C: Kind of threw it to us. Um, and then we went with Saw2

Speaker D: as our first security, uh, related, um, certification or attestation and we did it.

Speaker C: Um, I wanted to go straight to a, ah, Type two because type one is not necessarily super valuable. Um, so we had kind of put things in place, did a quick gap assessment, um, and then I talked to our auditor about doing uh, just a three month SoC2. Um, so we looked at the previous three months which is not super uh,

Speaker D: common unless you're doing quarterly.

Speaker C: Um, but we did that just so we could get something out in the hands of customers quickly and then we

Speaker D: moved to annual thereafter. Um, and then this year I think we just switched to doing biannual, uh, so we can kind of have it in a way that aligns with various customers fiscal years.

Speaker B: Mhm. And so then you went from the SoC2 and then it was Ah, ISO. Was it easier to get 27001 because you already had SoC2 type 2 in place? You know, I've always uh, in the past I've always felt like, hey, ISO 27000 is a little bit more rigorous, a little bit harder. So I've always started with uh, Soc 2 type 2.

Speaker C: Yeah, yeah.

Speaker D: I mean we did.

Speaker C: It was fairly easy because we used the same auditor. Um, so they'd already been gathering evidence

Speaker D: for a lot of the stuff they,

Speaker C: they already were very familiar with M,

Speaker D: the different controls we had in place, how the platform works, um, kind of the back office systems.

Speaker C: So um, I think in a vacuum it probably would have been more difficult to start with with 27,001. But because we already had kind of

Speaker D: the familiarity with the auditors and they had familiarity with us, it uh, made a much lighter lift I would say.

Speaker B: How many, how long did that take? How many months did that take to,

Speaker C: from beginning to end to get it for the 27,001? Um, I'm not even sure off top I had probably six months from when we said we were going to do it until um, we actually had them come. I think it was we'd finished up one year's audit cycle and then somewhere in the middle we um, we realized

Speaker D: we wanted to add this as well. Um, so we added that on and started the prep work and then they were able to come through and um,

Speaker C: kind of do those additional checks uh,

Speaker D: when the audit time comes.

Speaker C: We ended up combining all of our audits, um, which saves a bit of

Speaker D: cost but also saves a bunch of time on our side because like you said, we have whatever it is four

Speaker C: or five of them. Um, a lot of them are going to have kind of overlapping checks like are you offboarding users correctly, are you

Speaker D: reviewing firewall rules, whatever it might be.

Speaker C: Um, so you can kind of kill

Speaker D: all those at once, uh, across all of the reports. So that was fairly handy as well.

Speaker B: Yeah. So you're also uh, involved in some customer facing items. For example negotiating customer contracts. Certainly it helps to say, hey, here's our third party certification reports. What are some of the important points that come up in those customer negotiations?

Speaker C: Um, I mean I think the points that come up are just general things. They want to make sure you're dealing with, uh, vulnerability management, uh, there's the

Speaker D: standard bits of are you encrypting data

Speaker C: and things like that. Um, generally I think one of the important things when dealing with customers, um, hopefully you can get it on your own paper where your contract is kind

Speaker D: of customized to your platform.

Speaker C: But often, um, when it's sort of,

Speaker D: um,

Speaker C: it's not symmetric. Right.

Speaker D: Your customers sometimes can be multiples larger than you and they have all the leverage in the world. You want to close this deal, you're

Speaker C: going to use our security addendum. Um, and often in those times it's really helpful to make sure you're having

Speaker D: a conversation with them as you go through, not just crossing off all the

Speaker C: things you don't want to do or

Speaker D: won't do or can't do.

Speaker C: Um, because there's just a lot of things that show up in some of these contracts where they've um, been in

Speaker D: there probably for the last decade or two decades.

Speaker C: It doesn't make sense. Um, including my dreaded, you will rotate

Speaker D: all of your passwords every 60 days or something.

Speaker C: Um, and it really helps to sit down with those customers to understand what's the risk that they're actually looking to address with this. Um, because when you dive down into it, most of them are like, oh,

Speaker D: this has just always been in our

Speaker C: contract, you know, uh, it didn't really

Speaker D: think too much about it.

Speaker C: Um, so being able to have that conversation in a non confrontational manner, um,

Speaker D: and being able to just listen to

Speaker C: their concerns and then talk through how you've addressed it in other ways and be able to come up with some mutually agreed language. Uh, from that, I think that's probably the most important thing when it comes

Speaker D: to going through those customer contracts.

Speaker C: Because otherwise it can be fairly easy to uh, end up sort of in an adversarial uh, position or where you

Speaker D: get frustrated because you're just.

Speaker C: If it's kind of a larger, not uh, software focused company. It can just seem like they don't

Speaker D: understand what they're even asking for.

Speaker C: Um, sometimes maybe they don't but uh, I look at the role kind of also as uh, you know, one where you're helping to educate them which will help them you know, at their jobs

Speaker D: and going forward to assess risk for, for their other vendors as well. Mhm. Yeah.

Speaker B: And uh, like you said it's uh, in terms of uh, those, those requirements, you know the contracts are typically custom, um, they want it on their paper. A related element of that is you know all of the different customers usually have their own long laborious security questionnaire that they want you to, to fill out. Have you found a decent way around that and having to fill out different questionnaires? Numerous, uh, you know, every single time.

Speaker C: Yeah, I mean this is probably the bane of anybody who ever is tasked with that job. You get the 300 question, uh Excel spreadsheet and God forbid you get the one that someone's had too much time to add macros and kind uh of field validation or something to it. What um we did originally because the team was really small, we didn't have

Speaker D: a lot of time to just be

Speaker C: stuck doing a lot of these questionnaires. So we worked with um, our solutions architecture team so like the, the pre sales engineers um and help them kind of populate an RFP database with all

Speaker D: of the right kind of questions and answers, things like that.

Speaker C: And then um, had them take the first dab at it so that it

Speaker D: would be more distributed across the sales team. Uh and then you know of course happy to help uh, address any concerns that weren't covered there.

Speaker C: Um this year though one of the things we did um, just as kind of a side project I was hacking things together. Um had wrote a security white paper which is something I put together um just so we would have you know

Speaker D: something I, that marketing team made pretty.

Speaker C: But it kind of described our security program modeled um, it after the folks

Speaker D: at Confluent had a really nice one um, which I was jealous of.

Speaker C: So uh, kind of took some of that format and wrote it to cover our platform.

Speaker D: But kind of plain English right? Describing what you do and how you address various concerns.

Speaker C: Um and then I realized that hey

Speaker D: this, this covers a lot of the

Speaker C: stuff that would be um, in a security questionnaire. So um, used uh, just some, some Python libraries, created embeddings out of that um and turned it into a Slack bot that you could ask questions and it would Kind of query the document, uh, to find the relevant phrases, send that off to OpenAI. Uh, they would respond with an answer based on whatever I popped out of the um, uh, that white paper is relevant. Um, and then would, would provide the

Speaker D: response as a uh, just a Slack message back to whoever was asking.

Speaker C: So again for, for our solutions architects,

Speaker D: or maybe it's a salesperson or, or

Speaker C: support whoever they um, could ask a

Speaker D: question of this Slack bot and kind

Speaker C: of get security answers back, which I

Speaker D: think also helps on the, the questionnaire side. I mean you could kind of harness

Speaker C: that uh, directly into, into spreadsheets as

Speaker D: well and just have it be a function that gets called.

Speaker C: So I think there's a, there's a lot of opportunity there and I assume

Speaker D: in the nearest future this is going

Speaker C: to be a product someone makes and

Speaker D: you know, having to fill out security questionnaires will be um, one of those old war stories people tell that uh, young people won't understand.

Speaker B: Hey, that is very cool. Which is a perfect segue to talk a little bit more about AI and gen, AI and large language models. You mentioned that embeddings and you know, maybe in your case there was a vector database involved as well. And for our listeners who, you know, who are just starting to get familiar with all of this, can you kind of describe maybe the high level LLM application architecture? What are the different components? Certainly anybody using ChatGPT knows that you've got to provide a prompt and then back in all of this magic will happen. But how is this functionality being integrated into. Well, like the application you built or the applications that many other companies, SaaS providers are building as well. What is that high level application architecture?

Speaker D: Sure, sure.

Speaker C: And I think, I mean this is where it's, it's really exciting. Um, because the, the architecture blueprint, I mean there's, there's some general um, things out there. I think Andreessen Horowitz has a fairly nice LLM blueprint, uh, that they have out. Um, but at a high level you're taking user inputs and whether that's a prompt or maybe it's a document you

Speaker D: want summarized or a question, you're asking

Speaker C: about a knowledge base. Um, you're going to take that and at least for the use case I just mentioned, you want to be able to provide the extra context because those LLMs are trained on whatever's on the Internet. But it's also fairly general.

Speaker D: It's not going to know specific controls that I have within our program.

Speaker C: And even if we had that public and it was in the training data. It's not something, um, that would be

Speaker D: repeated across the Internet enough that it would probably be able to verbatim come back correctly. You'd end up with some hallucinations and it would say things that we're probably not doing.

Speaker C: Um, so, uh, the way around that is the uh, retrieval, augmented generation. Um, and so basically, like you said, M, we take some embeddings, uh, which is, uh, if we just take like

Speaker D: a Word Doc or something as an

Speaker C: easy example, um, you run it through. In this case I just use some simple Python libraries. It ingests the Google Doc, takes chunks in whatever size makes sense, um, and then converts that to vectors, which is the kind of numerical representation of whatever that data is. Um, and so it's just a list

Speaker D: of, uh, you know, however many.

Speaker C: I think it's 20, 48 different numbers. Um, and basically when someone asks a question now it's going to do the same thing with the question they asked. Create, uh, these embeddings, uh, create these vectors. And now it can just do a similarity thing.

Speaker D: So it compares all of the numbers,

Speaker C: sees how close they are, and they can say, okay, well this is the chunk from the white paper that looks closest to what this person's asking. Um, and then all it is. And this is where I think, um, when OpenAI released their GPTs, it probably killed half of the new AI businesses, because this is all a lot of them were doing. Um, you're taking the user's input, whatever their question is, you're taking the relevant context that you've sniffed out of the documents you've processed into vectors, and then you're sending that to OpenAI with a system prompt that says, hey, you're a helpful security answering bot. Here's the user's question.

Speaker D: And then you answered the question question,

Speaker C: and you're like, here's some relevant context.

Speaker D: And then you insert your snippet out

Speaker C: of your, um, your document you've created. You're like, give them a concise answer that's three sentences long, and then you

Speaker D: send that off as an API call to OpenAI.

Speaker C: They shoot something back and that's your answer. Um, I think there was a lot of companies that were built around basically doing that.

Speaker D: And, you know, now it's just a kind of a very easy thing to set up, uh, within OpenAI.

Speaker B: Yeah, yeah, that makes a lot of sense. That was a really good, uh, overview. And you mentioned kind of that newish feature that OpenAI recently announced called GPTs. And uh, a lot of the manual work, if you will, that you were doing to create the application and tie these API calls together and figure out what data should get sent. A good chunk of that now is, as you said, kind of uh, uh, made easier by the uh, GPT functionality that OpenAI has published.

Speaker D: Yep, yep.

Speaker C: Yeah, it's super interesting just uh, watching how fast they're releasing things and watching

Speaker D: what this enables people to do with all the new tools. I mean it's uh, exciting times.

Speaker B: Yeah. Now you mentioned kind of the example of a Google Doc or a Word Doc and the security white paper that you had written and that is uh, input into um, the retrieval augmented generation. What other types of information did you feed it? Was there other configuration information, other specifics about certain tools and so on?

Speaker C: No, that was, I mean in this case this was really just me hacking stuff together because I wanted to see how it worked. Um, obviously you can do lots of

Speaker D: different things though, right?

Speaker C: You could um, in theory you can

Speaker D: tie it into whatever it has the potential to understand, uh, different codes.

Speaker C: You could uh, I assume tie it into um, your infrastructure as code repos if you wanted to be able to ask questions about how X, Y and

Speaker D: Z is set up. I don't know how well that would work.

Speaker C: It would probably take uh, a bit of tuning to figure out how to

Speaker D: best route it and provide the right

Speaker C: context, but it really can take in anything. Um, and if you think about uh, well like Google just had their new release of, or maybe it's just soft release, but they have their Gemini model coming out now, uh, multimodal from the start, so able to.

Speaker B: Supposedly better than uh, GPT4, right?

Speaker C: Supposed to be better than GPT4, especially in code and answering complex questions and doing uh, reasoning. And I think as these models get better and better it opens up more and more doors. Um, and as they're all competing, um,

Speaker D: the foundational models will start becoming commoditized, which is great for everyone else.

Speaker C: Um, they're also having different tiers so you can have the ultimate super performance one, um, where it can do crazy

Speaker D: advanced uh, logic and reasoning.

Speaker C: Um, it'll probably be more expensive, but then all the way down to models

Speaker D: that are small enough to run on phones, um, you know, laptop, whatever.

Speaker C: Um, and that's really just going to enable so much more, um, as far

Speaker D: as functionality things you can do the

Speaker C: speed, you can do it with, um,

Speaker D: the security around it. Because if you're running the model locally on your, your phone or computer, uh, very different than sending it off to the Cloud and having to trust somebody else with it. So yeah, lots of, lots of interesting things happening there.

Speaker B: Yeah. So you just now started to segue into the security, uh, consideration. So when you're fine tuning a, ah, foundational model like you're talking about through this process, what are the things that security, uh, teams and development teams should be thinking about as you're sharing some of this data with the foundational model provider?

Speaker C: I mean I think a lot of it is really stepping through to understand how all these bits work. Um, because like what I was doing was the retrieval augmented generation.

Speaker D: Right. So it's pulling specific data and sending

Speaker C: that as part of the prompt, uh, versus fine tuning which maybe it's giving it a bunch of examples of. Here's a question, here's how I want you to answer. Um, and that's where you could train

Speaker D: it to answer like a pirate or

Speaker C: something like that or in whatever tone you want it to. Um, and less about feeding specific data

Speaker D: back into the model.

Speaker C: Um, and I think it's really worth it for people that are, especially if you're going to be working on security related projects for anything generative AI related to make sure you understand how do the models work? What does fine tuning actually mean? What's the effect this is going to

Speaker D: have and what sort of threats and risks? What's the likelihood?

Speaker C: Um, because there's a lot of, I mean there's a lot of different things out there.

Speaker D: Right.

Speaker C: And with LLMs there's a lot of ways that you can break them that

Speaker D: we haven't discovered yet. I mean there was the type in whatever the word is and just repeat

Speaker C: it for forever and then it spits out training data. Uh, that happened last week. Um, so I think really diving into, make sure you understand how these work.

Speaker D: You don't have to be a data

Speaker C: scientist, but really when we talk about

Speaker D: fine tuning, what is that doing?

Speaker C: And when we talk about using fine tuning to adjust the weights, what does that actually mean? Um, I think that's probably one of

Speaker D: the most helpful things people can do.

Speaker C: And it sounds intimidating because the people

Speaker D: working on this are all PhDs with advanced math degrees.

Speaker C: Um, but you really can get a fairly solid understanding um, of how they work, at least to the point where you can understand, okay, what are the

Speaker D: actual risks and how can we prioritize them in a, in a sensible manner?

Speaker B: Yeah. So you know, if we take a step back, there's kind of two buckets, there's kind of what are the, what's acceptable use for users using things Like Microsoft, Bing or ChatGPT. Right. Are they potentially sending sensitive data in the prompts and so on?

Speaker D: Right.

Speaker B: That's always a concern for potentially any system. But then what we're, what you're talking about right now is, hey, more deeply understanding what are the nuances, what are the differences between retrieval, augmented generation, fine tuning, or maybe even training your organization, training its own model from scratch with some, uh, proprietary data that uh, may be in house, which we haven't uh, exactly covered. So security wise, right? What do you, what would be the guidance in terms of kind of thinking about that? Is it uh, associated with the risk level? Is it, uh, you know, hey there, you've got an engineering team that's uh, building out this capability. You know, for all our listeners, what would you say, hey, what should the security team, what should the CISO be doing right now, uh, once they've kind of better understood what the organization is doing?

Speaker C: Yeah, I mean, I, again, start with the, just making sure you understand the workings because I think for both sides it's helpful.

Speaker D: Right.

Speaker C: Even with, uh, how users are using, if they're pasting something into ChatGPT, I mean, I know there was a huge, uh, bunch of noise made around. There's somebody pasted in X and now

Speaker D: it's in ChatGPT's training data and it'll pop back out. That's kind of the fear.

Speaker C: And I mean, is that really something that would happen? Probably not. Um, but making sure you understand why or why not, because the more you, you raise kind of a false alert, that you lose credibility.

Speaker D: Right.

Speaker C: So it's making sure you understand the

Speaker D: level of risk there.

Speaker C: Um, but when it comes to, if your engineering teams are looking to build something, um, really I think it gets to starting to think through how does authorization take place, where is it taking

Speaker D: place, what does the system architecture look like?

Speaker C: Um, like we were saying that the LLMs are going to be broken in ways that are going to be new and novel and they'll probably be coming

Speaker D: out on a weekly basis.

Speaker C: Um, so while you should have layered defenses, you know, you want to have

Speaker D: protections against prompt engineering. Um, so you put in, uh, you know, there's different libraries that can protect

Speaker C: there or external services, um, but you have to assume those will break as well. Like someone can trick the uh, you know, the filter by having it reply

Speaker D: in a different language or in parts of base 64 encoded or whatever kind of crazy thing people come up with.

Speaker C: Um, so I think it's something you want to make sure you're Talking to them about how can we at a

Speaker D: more fundamental level protect the data.

Speaker C: So you don't want the LLM making

Speaker D: a decision on should this person be

Speaker C: able to see X, Y or Z. Because the person might tell the LLM that it's critically important that they know and someone will die if they don't

Speaker D: and the LLM will feel bad for them and give them the data.

Speaker C: Um it gets to how can you segment the LLM separately uh, so that the data is never accessible to the

Speaker D: LLM unless that person has the rights to view it.

Speaker C: Um and you want to be able to have kind of that strong security story so you can answer customer concerns

Speaker D: with a fundamental this is not possible.

Speaker C: These are the controls we have in

Speaker D: place and this is why makes a

Speaker B: lot of sense in terms of kind of tooling have you seen any trends in terms of most popularly used tool, uh chain to uh build these LLM based applications? And similarly on the security side have you seen any tooling that is so far seems to be helpful?

Speaker C: I mean LangChain is the one that most people are using I think to start taking inputs and outputs and tying um, them together. There's stuff with, with agents so you have like auto GPT uh where people are making agents that can kind of do things. I think that that's still sort of problematic in its, its current uh, iteration.

Speaker D: Um, again I think all of these will, will get better and better over time um, and probably in the near to midterm but, but right now they can get easily stuck uh in different loops where you know things just, just

Speaker C: sort of break and fall apart as

Speaker D: you get kind of more and more complexity and need to have multi step sort of actions.

Speaker C: Um, on the security side I'm not totally aware of you know, any specific tools. I mean I think it really would

Speaker D: be using these general tools that are available.

Speaker C: It's kind of thinking through um, you know, what are the use cases that you can do. Um, you can integrate it certainly into

Speaker D: two different tools to help maybe parse some of your data.

Speaker C: But I think even without building anything there's a ton of value in just using like chat GPT, right and coming through um, if you're uh, especially for smaller shops where you don't have dedicated teams, right, like maybe you don't have an IR team or it's a very small team, um, being able to um, kind of say hey here's our situation, here's what our systems look like. Can you give me a prioritized list

Speaker D: of things I should be Checking, um, obviously you want to still have contracts

Speaker C: in place, you're not sending all of this stuff off and whatever. But I think there's, there's a ton of value in being able to use

Speaker D: it as your sidekick or your assistant or co pilot, whatever, whatever we're using these days.

Speaker C: Um, but to guide some of that. Like, hey, are there things that, if I'm looking at lateral movement, how would I know on whatever system maybe I'm not fully familiar with, how could I look for uh, indications of compromise here or there?

Speaker D: What other things should I be looking for?

Speaker C: Um, so I think doing like tabletop exercises where you can uh, kind of have people get used to using some of these AI augmentation bits, uh, can be super helpful and especially if you're

Speaker D: kind of a smaller team where everybody's

Speaker C: got 47 hats on, um, can be really, really helpful there.

Speaker B: Yeah, very cool. Now, hey, so before you built this in house, ah, LLM application security answering, uh, bot, you had to figure out how this works and figured out how to write the Python code. So how did you start? And by extension where should people start when they're first learning, Learning about all of this?

Speaker C: Sure, I mean, I think especially on the security side. Caleb, uh, Sima wrote a really awesome post uh, a couple months ago, um, super in depth but digestible, um, where he kind of documented his journey, um,

Speaker D: through learning about LLMs and taking on all of this.

Speaker C: Um, and I think that's as far

Speaker D: as a primer just to get you up to speed on how the things work, the different threats.

Speaker C: Um, that one's super helpful. Um, he also had put in um, some of the resources he used to learn and one of those was a

Speaker D: fast AI course which was kind of

Speaker C: building like using Pytorch to build, ah,

Speaker D: your own model and train your own model.

Speaker C: Um, and it was super interesting and eye opening for me because within a few hours I'd built a classifier that

Speaker D: could tell a cat from a dog or something.

Speaker C: Um, the libraries just do so much of the heavy lifting now.

Speaker D: And it was uh, I mean, I

Speaker C: guess it makes sense now when I think about it, but I wasn't really aware of how easy and straightforward some of this was. And it also gave me uh, kind

Speaker D: of a deeper understanding again of how things work, which when you're trying to secure it.

Speaker C: Right.

Speaker D: It's very helpful to understand how the pieces fit together and uh, what they do.

Speaker C: You don't have to know uh, exactly how stochastic uh, gradient descent works, but knowing that something's there it's kind of figuring out, uh, if a weight moves this way and it's good, then it keeps moving it. I mean, uh, kind of just that high level understanding is super helpful for

Speaker D: you to understand, um, what the different

Speaker C: threats that face this are and if the controls you're proposing are going to

Speaker D: make sense and are going to be helpful or not.

Speaker C: Um, so yeah, when it comes to getting started, uh, definitely that's helpful. Um, and then even if you don't write a lot of code, um, it's

Speaker D: getting to the point with GPT4 that it can help you quite a bit.

Speaker C: And if you don't understand something, it's now you can ask it. And it's not like you're asking the

Speaker D: senior developer at the office who's now going to think you're an idiot because

Speaker C: you didn't know, uh, why this did that or whatever. You can ask it and ask it

Speaker D: to explain to you. You can ask it to explain it

Speaker C: to you like a 12 year old. Um, uh, it's super helpful just to learn. Um, so if you're not someone with a development background, it can be super

Speaker D: helpful for people to just start getting that right, like start building some basic scripts that automate, you know, whatever, whatever

Speaker C: easy problem you might have and then just make sure you understand what it's doing. And if you don't know what it's

Speaker D: doing, I mean, probably shouldn't use it,

Speaker C: but uh, you should be asking it to explain it so you now can understand and use that as a learning

Speaker D: experience for yourself as well.

Speaker B: Very cool. That's why tech and security is so fun, because there's always something new to learn about your journey.

Speaker C: This is just wild with how much

Speaker D: there is to learn.

Speaker C: I mean, I think in the last like 30 or 45 days there's like 2200 papers that have been released on uh, LLMs.

Speaker D: I mean it's just like you can't possibly keep up.

Speaker C: Um, so it's been super fun to

Speaker D: try I guess on that front.

Speaker B: Yeah, that's great. Hey, so before we close out the show, we're going to switch gears here and give uh, you an opportunity. We move to the Reverso round here, give you the chance to ask me anything, uh, that you might want to ask.

Speaker C: Yeah, I was curious. I was thinking about this. I'm like, what would be a very insightful question I could ask?

Speaker D: Uh, I'm not sure I came up with one but.

Speaker B: Well, I won't have an insightful answer, that's for sure.

Speaker C: Well, I was kind of wondering if historically, because you've had a lot of experience both being a CISO and talking, uh, with other, uh, organizations, but for you personally, has there been a time where, um, you know, you're running the program and there's a particular threat or control or something, and you've invested in it and driven towards it, and then, you know, in hindsight you're, you're thinking, oh, maybe that, that wasn't as, as critical as I initially thought. So similar to what we're talking about, you know, on the AI side, right, where I'm like, I'm worried about some things and then the more I read about it, they're like, oh, maybe, uh, I should rethink this. Uh, you know, is there an interesting story you have about a time where you've, you've had that?

Speaker B: That's a very good question. And uh, you know, this is probably makes me think back to quite a number of years ago, a long time ago. And you know, by nature of the fact that we're, you know, in a, in a CISO role kind of building out the security program, we're inundated a lot of times with uh, different vendors wanting to sell us stuff. And sure, there's probably been, you know, only in hindsight that once or twice there's been a time where I've spent a moderate amount of money on some vendor solution and I'm always careful to not buy something we don't need because then it becomes shelfware. But then it didn't really do what it said it was going to do and, or there was more overlap with other capabilities, other products that I thought, uh, that we, I only realized in retrospect, and you were giving the example earlier about, you know, all of the third party certifications and really working with auditor of, hey, what is the intent of this control or this thing that you're, you're asking for and can we solve it in a different way? And uh, yeah, I think more. So that issue you're describing was more for me of implementing a new capability, buying a new solution that maybe it incrementally made us 10 or 20% better, but was it really worth all of the effort and the time and not only the cost, but the people time to actually do it when. Oh yeah, you know, hey, it's actually, you know, our other solutions, our other three solutions that we've got, um, do this, um, you know, could have handled it if we approached it a little bit differently. And you know, there's been studies or surveys that have been done. And you know, every large organization has dozens, scores, if not hundreds of different security tools. And that's always been one of the hardest things for me is to kind of rationalize and think about, hey, well, what tools are being used? Are the right features turned on? Do some, do some features, do some products need to go away? Is something get superseded or not? Yeah, that's always been very difficult in terms of mapping that to the process and where there might be process gaps or inefficiencies in the program overall.

Speaker C: Yeah, and that's, I mean, I think it gets to um, kind of the general challenge that we have in, in the CISO role.

Speaker D: Right.

Speaker C: I mean it's, there's a million things you could be doing. They will all, like you said, incrementally move the security program to be a better place and you know, in a vacuum.

Speaker D: They're all great ideas.

Speaker C: Uh, but then it gets to, you know, is this the most important thing

Speaker D: we can be doing? Is this where we should be spending our time?

Speaker C: And that's a, it's such a, a hard question sometimes.

Speaker D: I mean, I, I know I've been

Speaker C: guilty, like, especially when we flipped on, we were mentioning, um, um, doing, doing scans and finding vulnerabilities and you realize all the, the third party libraries that are out of date and um, you know, thinking through what like, hey, we need to fix all of this. And then eventually you're like, well, where

Speaker D: in an ATT and CK chain would this be? Like, is.

Speaker C: Attackers have budgets, they have, you know, time that they need to, to account for. Are they really going to be trying to find some obscure library here, here you. Or are they going to try to phish someone and you know, get their

Speaker D: credentials and should we be putting better preventative controls there?

Speaker C: So I mean, I think it's, it's a problem that hits all of us.

Speaker B: And that's why, you know, things like mitre, ATT and CK are super helpful. I mean, before mitre, ATT and CK existed, I remember years ago, um, you know, me and the team creating an in depth, detailed spreadsheet of all of the threat actors and different things they can do on, and trying to do some elaborate calculations of, you know, where we might need to invest in certain controls. And to be honest, in retrospect, again, it's easy to say that that was way too over engineered. You know, the basic simple approach was directionally put us into the right direction as uh, as much as doing that complicated spreadsheet did actually.

Speaker C: Yeah, yeah. And I mean, I mean, I, I. Because I actually still like that idea of, like, thinking through what are the things you're going to protect? And then, like, okay, what are the easiest ways to hit it? And then if they couldn't do that,

Speaker D: what would they do next?

Speaker C: And, like, as an exercise to help prioritize. But, I mean, it's such a difficult challenge, especially at a larger org where

Speaker D: you have super complex systems and then you have people and you have the

Speaker C: corporate infrastructure, and, uh, there's a lot

Speaker D: of different things that need to be balanced.

Speaker B: Yeah. Well, great. Nate, that brings us to the end of the show. Thanks so much for joining us today.

Speaker C: Oh, thanks for having me. It was great to chat.

Speaker B: All right, appreciate it. Talk to you next time.

Speaker D: Bye.

Speaker A: It.