Project Discovery | CEO Rishi Sharma on AI Disruption in Software Exploitation

Speaker A: Foreign. You're listening to Secure Ventures, the show that follows cutting edge founders in the cybersecurity space to understand their plights, glories and revolutionary products. I'm your host, Kyle McNulty. I'm also a full time investor at In Q Tel, a strategic investor for the US Intelligence and Defense communities. This show is not affiliated with In Q Tel in any manner and all comments, opinions and sponsorships are wholly independent. With me in this episode is Rishi Sharma, co founder and CEO of Project Discovery. Project Discovery is an autonomous software testing platform with a focus on remediation rather than purely vulnerability identification. Before starting project discovery in 2021, Rishi worked as a security engineer for almost a decade, if you include the two years while he was still in high school where he was bringing in $50,000 a year from bug bounties. In the episode we discuss how Project Discovery emerged from an open source labor of love. The wave of AI, uh, capabilities in software analysis, the future of bug bounties and penetration testing, the limitations of existing AI models, Mythos included, and a lot more. Rishi, thanks for coming on the show.

Speaker B: Yeah, thanks for having me. Really appreciate for inviting.

Speaker A: Uh, absolutely. Take me back a little bit towards the start of your career. So we start with pretty much all of our guests. You were working in what stuck out to me as a rather interesting role. I saw this healthcare marketplace for like Indian medical services. You were working in product there. I mean, just tell me a little bit of how you ended up working at this role and what the experience was like for you.

Speaker B: Yeah, uh, so uh, originally my career started in bug bounties in 2012, like way before when Buck Crowd and Hackeron came in, uh, I was into hackings. I was like doing random stuff and someone told me that, oh, you can actually earn money by reporting vulnerabilities to these companies. And that was the first time I got really good motivation that, okay, I can turn some of my skills into uh, finding some vulnerability. It took me six to eight months to score my first bounty. But that was like, that was a lot of dopamine right after I got the first bounty. Uh, and I think I did bug bounties for the next two years. So I think until the end of uh, 2014 I was like doing uh, hunting on different programs, um, like trying to report once. And um, after 2014 I started doing pentesting or appstag engineering here and there on random work. Uh, m. And I was also uh, like interested in building stuff. Um, so the first healthcare company I worked was Actually my first job where I basically worked as product or slash engineering uh person who was like figuring out how to build ah a system and since it was in healthcare we had to comply on a few stuff uh within um Indian uh requirements and that was the time when I got the firsthand experience how to build the products. What is the process looks like but it wasn't like fully about cybersecurity so I did that very briefly. But that was like kind of my inception on working uh within the healthcare. But yeah after that pretty much all my career has been around um AppSec Engineer or uh product security engineering roles because that's the one area that I really passionate about. And then there was some blend of me uh wanting to figure out how to build cool stuff or tools within security. So throughout my career as I was um like doing all these AppSec work I was also trying to build open source tools or tools for myself uh throughout uh, throughout that career.

Speaker A: M let's dive into this bug bounty piece a little bit more. What was the most lucrative bug bounty that you were ever awarded?

Speaker B: Um so uh I got my highest bounty for I don't think I was at that period of time. Neither the bug bounty was like that mainstream enough and neither I was expert at that level. So I was kind of like medium to like noob level. So I wouldn't say I would have scored much higher bounties but $6,000 was my first bounty and it was a combination of stored um xss, uh stored cross site scripting and uh csl. That through which I combined and uh reported the finding. That was my first time uh getting that ah bounty. Uh and after that I kind of got boundary ranges like after that.

Speaker A: Yeah, I have heard that before. That's a fairly common uh like technique within uh bug bounty researchers right Is you kind of find one specific uh like bug type, attack type that works and then you just try to jam that into all these different uh companies that are in the program and see if they work for any of the others which makes a ton of sense.

Speaker B: Yeah and it's also very important for like for you as a bug bounty hunter to show the full impact so you are in the full mode to figure out what is the chain. I can design and can define the full impact of the uh vulnerability that I'm finding. So um, most of my findings and I would say even the ones through which I learned through they also followed this pattern of how to chain different vulnerabilities together to create a full impact uh and also figuring out how to automate those pieces, uh, within the uh, within these chains. So when you are sending the report there is a one script or something that they just run and they figure out, oh, this is amazing. You could just uh, simulate what an attacker would do in the end. Uh, so you basically score bounties much higher than that. I don't think I was doing that earlier, but I got to learn from some of the really good friends of mine that this is how you need to operate. You just don't uh, become like a robot on reporting random one. Nowadays you need to consider the full impact from organization's perspective. And I was also young so it took me a bit of while to figure out how to kind of like think through from organization perspective, not just like me being selfish on just scoring any random ones.

Speaker A: Yeah, there's probably some good parallels to come back to a little bit later. You mentioned the highest that you won was $6,000, but it was still a year and a half of doing these bug bounties before you decided that you needed a quote unquote real job. So what were you earning on like an annualized basis? How much were you able to pull in in your Best year? Best 12 months of working on these bug bounty programs essentially full time?

Speaker B: Yeah, I, I wouldn't say I was like disciplined enough initially. So I was pretty young. So when I started I was like 15 year old.

Speaker A: Wow.

Speaker B: At that point of time whenever you got the bounty, you are like, you feel like that is the end of the world, like you are doing a lot. So I, whenever I used to score boun, I used to get at the same time pretty lazy for maybe for next three or four weeks. Then I would get the motivation again that okay, I need to uh, I need to kind of like focus again and do it. So I was doing around uh, in total around that year that I remember like between 50 to 60 thousand dollars. But I was fairly young, um, but I quickly understood that okay, now I need to be a, uh, bit consistent because in uh, Pugmod, this is not like uh, reliable passive stream of income that you are getting. You need to be super active and proactive. Uh, so it took me a while and that's something again I learned from a lot of my community friends, uh, that oh, this is how you need to be disciplined. They were like one of the top hunters. Uh, so they were like inspiration for me that um, uh, in order for me to also be in the uh, game, it's not just me getting dopamine on just finding A vulnerability. I also need to be super consistent. Um, so I don't think I did fair um, consistent job, but I think I did score pretty well high to critical findings. I used to not just spam on reports, I used to find code reports. So my average bounties used to be decent enough for me to kind of like score um, um, well within the, within a given year.

Speaker A: Yeah, I mean well 50 to 60 thousand dollars for a 15 year old or even 16 year old is incredible. Right? Like I can only imagine how you're spending that money too at that age and where, where that's going towards. I mean let's, let's use this opportunity quickly. What do you think the future of bug bounty programs actually looks like?

Speaker B: Yeah, it's, it's, it's, it's a uh, it's one of the topics that is like I would say trending across all the Twitter. Um, the one problem that I totally understand that program programs uh, are facing is the influx on these incoming prognostic reports. Uh because pretty much all the researchers are using AI agents or their own scaffolding of automations to report these vulnerabilities. And if you are multiplying across a large set of researchers, the inbound is just looking drastically, very intense. Um, and that is why a lot of programs are shutting down their uh, bug bounty, uh, uh rewards as well as they are thinking through how to process it now. So they are much slower to uh, react in this situation. But I think very um soon they will figure out how to handle the incoming inbounds. Uh because at the end of the day you need uh, these really expert security researchers to hunt across all your different endpoints. And it's not just like, like okay I will find everything. This is like their creativity plus AI is just um, a uh, completely different level. And even out of like 100 reports if 10 are valid, uh even in this AI slope that is still pretty big deal for you as an organization to consider remediation because if you aren't doing it, uh, attackers are also kind of like spinning up that game much much faster than people realized last year that oh how it will uh happen in uh, with AI in the few years it's in 2026, it's already at its peak. Um uh, the time to exploit is all time high. Uh the time to attack on different endpoints as you are spinning up new uh instances are all time high. So you need the crowd of um, bug hunters who are actively hunting vulnerabilities. Uh so I don't think that is going away. I uh, think those who are mature programs who know um, how to consider uh, different scenarios of vulnerabilities, how they understand this whole ecosystem of AI with security, they are going to figure out how to manage the inbounds of uh, their reports in maybe like 6 to 12 months and then uh, it will uh, get out of this whole bottleneck of uh, all the AI slope and all the noise that they are getting in there in bonds. I uh, don't think it's going away. Um, the attack surface is growing so uh, so fast. Everyone in a given organization where we work are deploying um, like all different teams are deploying all random uh, things and the growth of attack surface is just all time high. And then uh, your type of vulnerabilities that you used to see is also very high. So hunters are going to be most required uh, in this uh, coming future as well as even in the further future in my opinion.

Speaker A: Yeah, and when you say all time highs, just to be clear, right, you're talking about record speed. So in some ways it's actually kind of like an all time low in terms of the amount of time to develop uh, some of these exploits and like the time from reconnaissance to exploitation within an organization.

Speaker B: Yes, the exploitation part is all time high. Um, so even in our open source, uh, just purely from CVE's perspective. So when we started in 2021 with nuclear and other projects, we were seeing weeks to months uh, from a given CVE to exploitation period. Now in some cases attackers are even reverse engineering all the releases of popular repositories to figure out whether they are about to uh, talk about a CVE or advisory of CVE in a week or two. And they are in, I would say in the negative, not even in the, in the hours or days period. They are in some cases able to figure out that okay, they are going to announce a cv, um, and even before that they um, they start to exploit much faster on the Internet. So attackers have evolved their methodologies in the recent years where every time they are aware of a, ah, given uh, let's say next JS or any popular tech stack which is a, which is vulnerable from a particular CV or about to be announced for a cv, they spin up these instances and react way too fast to attack across the Internet, um, in a speed that is unimaginable. And it's like we notice that it happens within hours just like okay CV got announced and across, even on our honeypots that people are attacking across different instances, uh and it's so wild to see it. And that part looks very scary to me that not only these uh, CVs that we are talking about, these are also uh, AI agents finding first party vulnerabilities like zero days and then exploiting in a loop. Um, so that part becomes much more complicated um, in this exploitation phase which is scary.

Speaker A: Yeah, I want to get into your thesis around this a little bit more. It's something you obviously live and breathe on a day to day basis. There's this kind of interesting uh, trend chart over time which we'll see how it plays out. But right now you have the AI capabilities that are uh, being developed and starting to be made available to attackers even despite the efforts of like Glasswing, uh, before these different security organizations have the time to actually fix all the vulnerabilities. So in the short term at a minimum you would expect an uptick in uh, like exploits and just overall exploit activity, attack activity. Um, but if we go a little bit further down the time horizon, my question for you, if we think about two years from now, you talked about the expanding attack surface because folks are just developing much more, uh, many more applications. Do you think there's going to be more vulnerabilities that are available to be exploited or less? Because you have now these AI tools that are being used from a uh, security lens as well that are fixing so many of these previously kind of low hanging fruit vulnerabilities.

Speaker B: Yeah, I would say uh, I hope that this remediation or the amount of vulnerabilities that are exposed go lower over the time within the two years. Only challenge that teams have to figure out is that across all their vulnerability management programs, um, they have 30 or 40 different tools or scanners running across different parts of their organizations and they have to run those, they have to consolidate the results, they have to run the process and then work through the remediation process. So even if you detect the vulnerabilities, the remediation process right now is a bit of a bottleneck. Uh, and teams, I don't think, I am not saying that teams have figured out how to act on the remediation cycles and like shorten that out. And that's my hope that we uh, like from cybersecurity we all figure that out in the next six to eight months. If you don't then it is going to be a bit wild to face the reality in a year or two. Um, ideally, um, um, I don't think there is a single button through which oh I run the Mythos or some XYZ and all my vulnerabilities disappear in a single moment. That is unfortunately, unfortunately not, not the situation. You have to run these, even these AI harnesses or security tools across uh layer of your organizations. And these are thousands of different instances that are being updated every time um and like being developed all the time. And you are not even talking about uh the classic injection vulnerabilities. You are talking about business logical ones that are, that might be uh due to the human errors on the fly um, that ah could not design the system well and you are exposing the vulnerability. So you have to systematically integrate these security tools and processes across uh your organizations and then act on remediation faster. And uh, so if that is the kind of goal um, in order to kind of consolidate findings that's something we always had, that's been, always been the process. But it's never been a case where we went to a zero sum or uh, like um, low number of findings of vulnerabilities in given organization. Um, it's usually um, much harder in a remediation process to control your risk at the organization level. So I am hopeful but I'm not seeing a path right now where within two years the number of findings are going to be lower. Even if we have security tools integrated in the time of writing the code, post writing the code going into the deployment then your bug bounty programs, all these layers are required in order to cop or manage the remediation. But wish we had a button where we could just press and it would remediate all. But unfortunately that is not what we have um and neither that process is going to be substantially different in terms of engineering to the actual security. So that uh, then that looks a bit scary to me too that if teams do not figure out that is going to be a bit of bottleneck for all of us uh versus like um there is going to be like less than 10 ones in any given time in any given organization. That's going to be very hard at least in two years.

Speaker A: What do you think the bottleneck is for doing remediation with these models? Right. Because if I think about the different types of vulnerabilities that need to be um, actually remediated. If it's like injection or uh, memory overflow, uh, error like those are really simple to fix in code. If you just say hey the model found this vulnerability, hey, same model, go ahead and look at the code and add in a new safer handler. It's like boom. Okay, problem solved. Um, what do you see as the bottleneck.

Speaker B: So even at the detection piece that like even before we talk about remediation if you put AI across different layers of your organizations to find these vulnerabilities, usually um, uh AI is not yet good at uh reducing the false positives so they will hallucinate and they will come up with 20 different vulnerabilities for you that you need to patch. And we have, and this is something we will also publish out very soon. But we have seen a common um, um like a negative behavior where the suggestion or remediation of um a vulnerability which is a false positive first and then you are remediating uh a vulnerability which will actually um like like create a problem within your feature. Like we had ah findings if we actually remediated based on AI consideration then that feature will not work because it hallucinated all in its own. And then uh, it figured out something to fix and that fix will actually impact your feature and it's like a uh daunting for your product. So you can't even run this auto remediation magic that we all have been talking about that oh AI finds the vulnerabilities and you just keep running in the loop. That is unfortunately very risky right now. So uh, you can't even have that world. And even before that you have to work so hard in order to reduce the false positives um and then actually getting to a further stage of remediation. Um and when we talk about remediation, remediation is uh the security aims are usually smaller and uh, the vulnerabilities through which different sources they find are usually spread across different teams, spread across different instances. And that is a process um that requires a lot of human involvement, human steps in between uh to get to the remediation phase. Um and that part becomes um, much more time consuming than just like filtering out detections or the ones that you care about. Uh we have noticed that um LLMs are very good at findings that are non business logic ones. Um even for false positive perspective you have much lower false positives on classic vulnerabilities. But the moment you take uh account into authorization privilege escalations or like business logical ones, uh they still do a very poor job. And and um, they will also like struggle on coming up with these ideas like how it will um, like impact in your organization. And that's where I would say the researchers, the bug orienters excel at or the security engineers working in organization excel at and they can take the power of AI and like come up with much better way to uh, uh, kind of like filter out those results or push AI much in the right direction than in uh, not um. So yeah, that's kind of like what we are seeing.

Speaker A: There's a lot in that from the false positives to uh, the remediation limitations of potentially breaking a feature which you potentially address with human review on both sides. But obviously these models are going to get better. Uh, but let's just transition this into Project Discovery and what you've been building. Right. So you started Project discovery back in 2021. I think at the time you were doing more traditional uh, like static scanning, is that right? Tell me about kind of what you started with and then how that's evolved over time.

Speaker B: Yeah, uh, so even before 2021 we all co founders so I was working as security engineering company and I was using this really good open source tool that I used to love and, and the author of that repository was also pretty active every time I used to file issues, send any feature, uh requests he used to like fix or ship every second day. And I was like pretty impressed with him, uh, like how much he was uh contributing in that report. I used to love how he, how he designed the tool, how he was like crafting all the innovation within it. So I got a chance to connect with him uh over the DMs and we uh, I now got to realize that there are, there are three top contributors of the same repo that are actually friend and I could be in touch with them and they all were working as security engineer. All the, all of the, all of the um, like these were my co founders but all of them had the same philosophy as I, as I had was like we as a practitioners have a lot of work in Frontline and the products that are designed for us, cybersecurity products that we use in commercial settings, things are not designed for us. They are maybe sold in top down and they um, you struggle with customization of it. They are not fast, they are clunky, they are not uh actually doing the job that we know that it should be doing. So you are struggling with the process and everything. Uh so this is like all we were feeling through that. Oh uh, we really hate these type of cybersecurity products. We have compared to like dev and infra companies. They have such a really high quality products and we were like super jealous about it. So we got in touch. We became really good friends uh for four or five months we were like doing in the same repository and I would say after that we thought like okay, we can also build new stuff together. And that was the time uh, we picked a username on GitHub called Project Discovery. We had no plans to make any company or anything out of it. We were just focused on um, okay, what we can build and what we can open source. So I still remember it's going to be different from the current time because it's while now but when we first open source our uh, repository we got like 100 stars or something in a week or two and we were so, we were like so blown out like oh we got 100 stars. We had like two or three comments on GitHub and uh, Twitter. Uh, and like we, we are like really loving it. Let's open source one more. So we were like in this loop of putting out tools that people were loving but that started very very small. And then within like six months like community grew uh, in a pretty substantial way and we had a fairly decent website called projectdescape IO where we were like putting out why we are doing it. Open source tools and our vision and everything. But that was an intention of uh, a company. We just wanted to um, talk about uh, us and VCs thought that we are actually a real company. So when we were getting the traction we started receiving the emails from VCs and other CISOs that oh, uh, are you selling a commercial product or are you raising the funds? And we had no plans at that time. Neither we had even a company registered. Um, so it took us uh, I would say six months because we all were completely remote. To be honest I did not even uh, know the names of my two of my co founders. I just remembered their username and it was pretty, pretty uh, hard for us to even think about this going full time. But since we had, since we started struggling with our uh, full time job and with Project Discovery because we had like uh, at that period of time we had like more than 10,000 GitHub stars. We had like pretty uh, active contributions feature requests coming in. Uh, we thought it might actually be a good idea to work full time in Project Discovery and uh, actually work it. And since we all love probably might be a good uh idea but we all were nervous to be honest at that time because we never thought it as a company. Um, and uh, I think it took us a month or two to get all four co founders aligned that we are now co founders and we are thinking to go live. We registered our company, we've built our first pitch deck in a month or two and then we started pitching to all the VCs. And finally we got uh, uh funding in 2021 to go full time. Um, at that time, uh, like with um, Project Discovery, uh at that period of time we had released two categories of tools and that was the kind of foundation on which we started. One was um, the tools that allows you to find the exposure within your organization store. Uh, as your teams are deploying different host applications, APIs, different cloud uh, um, uh services. We build these tools that will crawl for you, enumerate for you and create an inventory out of it very very fast and you can run it on a loop all the time. And the second category of tools uh we created was that okay, once you have understanding of your organization's exposure now you might require uh, custom vulnerability assessment on different type of assets and you need high signal to ratio so you don't like uh, get into this whole false positive game. Um, so we uh, open source this repository called Nuclei in which you can compose a vulnerability, uh, steps, uh and you can then run across your organization. And it was just an example repository that this is an engine, this is how you can run. Then immediately within a month it became a uh community repository. People started writing these templates and these were not just static templates, these were dynamic templates. So, so let's say you have laravel um instance running and um, you could write a template that will co probe your laravel instance, perform step by step that I'll send you this payload first, then I will wait for the response and see whether the vulnerability is actually working or not. Um, and that was like kind of like the engine that we designed at that period of time and that became popular for pretty much all the trending CVs that used to drop. People started using this repository to communicate that oh there's a new next JS vulnerability that we need to remediate. So I will go and file the templates. Everyone can benefit. And there was such a, I uh would say like there was such a great moment for us because we never thought about it. It was just an example repository. People made it more like a database, a community collaboration engine, uh that uh, not only was growing fast but also people were maintaining it themselves. Uh, still at this period of time we have around more than 10,000 detection templates but 80% of them are written by community. Wow.

Speaker A: I mean it's one of the things that I've always really liked about the security industry is there is this kind of collaborative nature. I think a lot of times it's uh, something we could tap into More as an industry in terms of how organizations are really sharing more directly with one another. But the idea that hey, if one person develops this detection capability, like there are thousands of organizations in the US that would love and worldwide right beyond just the US but that have the ability to benefit from that same capability. And it's kind of the defenders collectively working against the attackers. And I think that's where the whole dynamic of security can, can change in terms of leveraging that numbers advantage. So this is the kind of start of Project Discovery like you said over five years ago at this point, but it's evolved a lot and certainly you all have evolved to this new kind of AI world right in terms of how that's being used in your own scanning and your own uh, like vulnerability detection and uh, remediation. So tell me a little bit more about how that's come about and how the features have evolved as a result.

Speaker B: Yeah, so we started experimenting with um, AI in M 2023. Uh, all the tools we've built um so far in Project Discovery open source ecosystem and even internally were designed on based on UNIX philosophy. Like these are modular tools that are easy to uh, run, easy to use and easy to customize. Uh, and luckily when we started seeing AI getting into like especially LLMs getting into a ah, bit of a ah reality um, we quickly started experimenting okay, how we can leverage um, LLMs to either um, empower these tools so they can do a lot of tasks or further tasks that you would do manually or you would contextualize them. Um so for nuclei ah repository we started auto generating the templates based on incoming reports. So the first use case I remember was every time a lot of bug bounty programs used to pave bounties for if you submit a vulnerability and if you submit along with a NUCLEI template they will give you a bonus of $200 or $500 each for your report. Uh, and that was the first use case we saw people reusing LLMs that okay, I already have context of my report, all the steps I need to codify it, uh, into a template that I can then run across my organization. So if you, if you send me an instance of a vulnerability I can then probe and check that oh whether there are similar instances vulnerable or not. When we started getting a bit of a traction on or even the belief on LLMs that okay, they are really good, we did a bit of fine tuning on top of uh, I don't know what was the OpenAI earlier models that we used it on top. Um, but that Became uh, kind of like first inception of using it and we started seeing people finding for each report they used to find eight or ten net new instances vulnerable from the same vulnerability. So it became a net uh benefit for them. Um and then as LLMs evolved we started seeing one more pattern I think last, earlier last year when they were started getting better at tool calls uh that okay now they can reliably call the tools, they can process the tool calls and they can also run longer and they have a bit higher context uh than they used to be. So they had like earlier like 4,000 to 8,000 tokens and now you add like much higher degree of token. And that's when we thought that okay now instead of looking very narrow ways to um optimize for a uh tool with LLMs we can even think about automating bunch of workflows from the vulnerability detection to all the way to remediation. And what are the manual steps in between that we can think of automating them. Um and that's when the inception started of us building uh a harness on top of LLM where we got like much more confident that okay LLMs are much more reliable on tool codes, context carriness as well as windows so we can design these loop of work. And that's when we felt that not only we can optimize uh LLMs with these really well structured tools to find complex um vulnerabilities but we can also automate triaging process. We can automate the false positive reduction process, we can even automate the uh owner routing, retesting uh regression workflows. So that's when we started getting excited that okay now it is now feeling good that you are reducing the remediation cycle. Um but yeah that was the first inception and luckily since we were having this unique uh Unix philosophy from the uh start on um uh LLMs and we were also open source, all these LLMs were already trained on our data that how to use this X tool of us and um, how to uh kind of carry forward any acronyms that are underneath our tools uh and that became super beneficial for us to ramp uh up on building this infrastructure of AI on top of like these security tools.

Speaker A: So you talked about the uh development of your harness and just the improvements with uh tool calling as a whole with these models. I want to tie this back to something we were talking about a little bit earlier right? Some of the challenges that the models themselves have today. What do you see as like the primary function of the application layer around the models and how sustainable do you think that differentiation really is as opposed to uh, anthropic or OpenAI just developing their own lightweight application that allows you to run the underlying model. How much weight would you assign on each of those? Another way of framing this, right. How much weight would you assign on each of those? The model versus the application layer around it.

Speaker B: Yeah, I would say the hardness, um, the question between the harness and the raw model capabilities are something like being questioned for good amount of period. And this has been even at the Mythos level. Uh, and there is a, ah, recent post by Karl author, how he kind of ran the first Mythos scan, uh, and got the results and how he compared the results with other AI security tools he was using. Um, it is still pretty amazing to see that even though these models are compounding in a way, these smaller models that we got six months back, uh, were powerful enough because at the end of the day you want to nudge the models, you want LLMs to get into the right direction and there are ways or strategies within your harness to like maneuver them in the right direction all the time and keep running them in the verification loop, uh, to be effective. Uh, and that becomes very important for uh, you to be, especially if you are in organization, if you have like thousands of assets, you need to be cost effective, you need to be super reliable, uh, at the scale that you need and uh, even like many months before and across like a lot of uh, uh researchers within the community, they built a lot of harnesses um, that allowed them to extract much higher degree of value that even like um, on par from Mythos from the raw capabilities perspective. Um, now in terms of cost effectiveness, in terms of um, the scalability, those become kind of a bottleneck. And hardness, like as you optimize more on the application layer, you can scale all those two dimensions much more effectively uh, than just like wrapping it up and expecting that oh, go find the worlds, um, it's most of the time it will first uh, give you a long list of false positives for which you will need to build your own harness. And then once you figure out that okay, I built this uh, harness, there are like six different steps that you need to consider from security engineering perspective to automate. So it's not just about finding vulnerabilities. There are like bunch of different things uh, that you have to do and you also have to carry forward the regression cycles, um, that okay, previous known vulnerabilities are being contextually useful for us in the future scans. Um, so yeah, uh, when we consider A full cycle of secure engineering. It looks very different than you running with a claw code or with a row LLM scaffolding. The outputs are way different. The quality of automations are very different. And that's why I would say even like in the coding world we have like five or six mainstream coding tools uh built on these LLMs because that's why the application layer is so important because that is so close to the user expectation and users day to day life. And that's where you need to spend a lot of time just beyond LLM and you need to tune all the time. You need to kind of like integrate all the time and that takes a lot of energy and that is a bigger gap um, even in the contrast of whether you have um, much expensive and larger models coming like mythos or 5:5 um in, in this context the second like I do like we really um already love these models that are already released out and as they are ramping up it's actually been useful uh because we, we don't have to worry about that. Oh these LL models are going to be just good at coding but not good at our cybersecurity areas because we also wanted to automate same way as software engineers are doing. But it's kind of like becoming um, um a kind of uh bottle like that. Oh LLMs are uh, we need to scaffold way too much around the model's reasoning capabilities. At least we don't have to push too much hard on those areas. We have to push much harder on other part of the areas. So that's actually useful as these models get better um so we can design much, much um higher degree of automations and kind um of like workflows around security engineers. So I don't see even like we have um, we are like in gna we use cloud code all the time. Even we know that the gap between making it work is just so immensely high between just raw model and the hardness around it.

Speaker A: What do you see as the most existential threat to project Discovery in the couple of years coming?

Speaker B: Um I don't see the existential threat I don't have right now. AI psychosis like I'm not getting into this psychosis world that okay, uh this is the end of world because that's something we have been failing for the last two and three years and it's uh actually going in much positive direction than how people, maybe some people who are maybe not from security who consider uh the apocalypse happening or something like something as an existential crisis that's something I'm not getting. I would definitely um, like consider more from the innovation perspective with other new startups coming in, younger startups coming in. Uh, that's where I want like we want to make sure that we are innovating and bringing the value faster too, uh, versus getting concerned, uh, with AI or we are going away from jobs and sector. That's not the concern. But I definitely get uh, concerned on whether we are doing or putting value enough in the community or not, uh, whether we are innovating ourselves or not with the others. Security startups compared to just AI or uh, other uh, other pieces.

Speaker A: Yeah, that's very fair. And in some ways that's um, the most classic just like chart challenge as a startup, right, Is how do you continue to out innovate new entrants? And isn't something inherently unique to AI, but maybe uh, more exacerbated just because that uh, like innovation, uh, frontier is being pushed so much faster than it has been a long time for tech companies as a whole. So I think that's well said. Well Rishi, I really appreciate the time. I think it's a great conversation on something that's relevant and on top of lots of people's minds right now, whether they have AI psychosis or not. So I really appreciate you taking a few minutes to chat through your journey and how you're thinking about some of these challenges.

Speaker B: Yeah, thank you. It's been a great discussion.

Speaker A: Thanks so much for listening to this episode. You can subscribe wherever you get your podcasts and you can write to me at Kyle@ SecureVentures IO. Uh, I'm Kyle McCulty and you've been listening to Secure Ventures.