AIUC-1: Building trust in AI agents

1 00:00:01,760 - > 00:00:05,440 Narrator: Welcome to the Practical AI Podcast, where we 2 00:00:05,440 - > 00:00:08,160 break down the real world applications of artificial 3 00:00:08,160 - > 00:00:11,840 intelligence and how it's shaping the way we live, work, 4 00:00:11,840 - > 00:00:16,325 and create. Our goal is to help make AI technology practical, 5 00:00:16,325 - > 00:00:19,365 productive, and accessible to everyone. Whether you're a 6 00:00:19,365 - > 00:00:22,325 developer, business leader, or just curious about the tech 7 00:00:22,325 - > 00:00:25,685 behind the buzz, you're in the right place. Be sure to connect 8 00:00:25,685 - > 00:00:29,040 with us on LinkedIn, X, or Blue Sky to stay up to date with 9 00:00:29,040 - > 00:00:33,120 episode drops, behind the scenes content, and AI insights. You 10 00:00:33,120 - > 00:00:35,760 can learn more at practicalai.fm. 11 00:00:35,840 - > 00:00:37,440 Now onto the show. 12 00:00:41,360 - > 00:00:45,225 Daniel: Welcome another episode of the Practical AI Podcast. I'm 13 00:00:45,225 - > 00:00:49,865 Daniel Whitenack. I am CEO at Prediction Guard, and I'm really 14 00:00:49,865 - > 00:00:53,945 excited today to have, an amazing guest that, I'm 15 00:00:53,945 - > 00:00:57,220 personally interested in asking a bunch of selfish questions too 16 00:00:57,220 - > 00:01:00,340 because I'm I'm so interested in the topic. But we have Emil 17 00:01:00,340 - > 00:01:04,020 Lassen who's the standards lead at the Artificial Intelligence 18 00:01:04,020 - > 00:01:05,940 Underwriting Company. Welcome. 19 00:01:05,940 - > 00:01:06,820 How how are you doing? 20 00:01:06,820 - > 00:01:08,500 Emil: Thanks, Daniel. Thanks for having me. I'm doing great. How 21 00:01:08,500 - > 00:01:08,900 are you? 22 00:01:08,900 - > 00:01:12,020 Daniel: I I'm I'm doing well. Actually, you know, today in the 23 00:01:12,020 - > 00:01:15,705 Midwest, everyone's concerned about tornadoes and talking 24 00:01:15,705 - > 00:01:19,625 about hail and and some things like insurance and other things. 25 00:01:19,625 - > 00:01:23,305 So it's a whole other other world, of course, but, 26 00:01:23,305 - > 00:01:27,750 obviously, the AI underwriting company, way way more than 27 00:01:27,750 - > 00:01:31,190 thinking about insurance, but thinking about standards, 28 00:01:31,430 - > 00:01:37,030 certification around AI and agents. I'm wondering how, you 29 00:01:37,030 - > 00:01:41,075 personally, just to give the audience a little bit about you 30 00:01:41,075 - > 00:01:44,275 personally, how did you end up at this intersection of 31 00:01:44,275 - > 00:01:48,915 standards, certification, AI, agents? How how did that come 32 00:01:48,915 - > 00:01:49,315 about? 33 00:01:49,315 - > 00:01:52,515 Emil: Yeah. So I I don't think I had as clearer path as as the 34 00:01:52,515 - > 00:01:56,160 classic standards lead where you work as a say, security engineer 35 00:01:56,160 - > 00:01:59,280 for ten years and then learn the technical craft and then come 36 00:01:59,280 - > 00:02:02,160 together. My journey has been very entrepreneurial always. I 37 00:02:02,160 - > 00:02:05,680 started my first company with actually the CEO of the 38 00:02:05,680 - > 00:02:09,575 artificial intelligence company, Ruininkvist, ten years ago. It 39 00:02:09,575 - > 00:02:11,815 was a nonprofit back then helping students from low income 40 00:02:11,815 - > 00:02:13,655 backgrounds get into top universities. 41 00:02:13,815 - > 00:02:16,295 And I think what I took away from that was both the very 42 00:02:16,295 - > 00:02:20,135 entrepreneurial journey, but also desire to move fast on some 43 00:02:20,135 - > 00:02:23,410 of the challenges that society is facing. I then moved in and 44 00:02:23,410 - > 00:02:26,930 had my first interaction with standards at my second company, 45 00:02:27,010 - > 00:02:30,130 a real estate company back in Denmark, where we developed an 46 00:02:30,130 - > 00:02:32,930 impact management system that both had to navigate a lot of 47 00:02:32,930 - > 00:02:36,450 national legislation, local legislation, EU legislation, 48 00:02:36,795 - > 00:02:40,715 voluntary frameworks, investor demands. And so had my first 49 00:02:40,715 - > 00:02:46,955 interaction with one of these quite complex markets of 50 00:02:46,955 - > 00:02:50,475 different measurements and targets you wanted to get to, a 51 00:02:50,475 - > 00:02:53,650 very technical sector as well where like building codes, for 52 00:02:53,650 - > 00:02:55,970 example, require a lot of thinking through how you do 53 00:02:55,970 - > 00:02:58,930 things the right way. So I spent four years building up a real 54 00:02:58,930 - > 00:03:01,890 estate company that today is managing about 400,000,000 55 00:03:01,890 - > 00:03:06,765 together with four other co founders and took from that this 56 00:03:06,765 - > 00:03:10,125 desire to go in and standardize when we know what the right 57 00:03:10,125 - > 00:03:14,925 answer is and try to push the sector in that direction. The 58 00:03:14,925 - > 00:03:18,605 way I got into the AI space was taking a step back from the real 59 00:03:18,605 - > 00:03:21,330 estate company now a few years ago and going to Cambridge, 60 00:03:21,330 - > 00:03:24,050 Massachusetts, so Harvard University, where I spent two 61 00:03:24,050 - > 00:03:27,730 years as a fellow at the Kennedy School, really getting into the 62 00:03:27,730 - > 00:03:30,530 emerging tech and geopolitics of all of this as well. 63 00:03:30,690 - > 00:03:35,170 And left the Kennedy School with a very sheer ambition of just 64 00:03:35,205 - > 00:03:39,365 getting under the hood of the pace of AI, the safety and 65 00:03:39,365 - > 00:03:41,845 security aspects, and clearly just acknowledging that the 66 00:03:41,845 - > 00:03:45,685 technology is going to profoundly change society as we 67 00:03:45,685 - > 00:03:48,885 know it today. And in many different ways, I have 10 68 00:03:48,885 - > 00:03:52,640 nephews and nieces, have five sisters. And seeing a 10 year 69 00:03:52,640 - > 00:03:56,720 old being comfortable using AI the way they are is kind of 70 00:03:56,720 - > 00:04:01,360 scary. And I don't see yet that we've codified the principles we 71 00:04:01,360 - > 00:04:04,885 want to see when it comes to how kids use AI. So that's one 72 00:04:04,885 - > 00:04:07,205 direction where, oh, maybe we should develop standards for 73 00:04:07,205 - > 00:04:07,685 this. 74 00:04:08,645 - > 00:04:12,885 You also can read news every week and see new incidents. So 75 00:04:12,885 - > 00:04:15,845 there's clearly a big security angle to this as well. You said 76 00:04:15,845 - > 00:04:19,350 that the Midwest is facing tornadoes today. I think being a 77 00:04:19,350 - > 00:04:22,470 CISO at a company adopting and deploying AI right now feels a 78 00:04:22,470 - > 00:04:24,390 little bit like you're in a hailstorm and it's only a matter 79 00:04:24,390 - > 00:04:28,550 of time before you're hit by some of that. We just keep 80 00:04:28,550 - > 00:04:29,030 seeing that. 81 00:04:29,030 - > 00:04:31,515 So clearly there was a element to that as well. And then 82 00:04:31,515 - > 00:04:33,995 there's the even bigger picture around what this will do to our 83 00:04:33,995 - > 00:04:38,715 job markets and so forth. So left the Kennedy School being 84 00:04:38,715 - > 00:04:42,315 very interested in just using the public policies toolbox that 85 00:04:42,315 - > 00:04:45,380 I brought from that with the standards toolbox I brought from 86 00:04:45,380 - > 00:04:47,940 my real estate company and then this desire to actually just 87 00:04:47,940 - > 00:04:50,980 work on societal challenges that I think has been with me since I 88 00:04:50,980 - > 00:04:55,220 started working. And that became my way to the artificial 89 00:04:55,220 - > 00:04:57,995 intelligence underwriting company. I've since then spent 90 00:04:57,995 - > 00:05:01,915 all my time building a network of people, a consortium, to help 91 00:05:01,915 - > 00:05:04,715 them figure out how we get the right practical and technical 92 00:05:04,715 - > 00:05:07,115 insights into the standards we develop as well. 93 00:05:07,115 - > 00:05:09,675 Daniel: Yeah. That's that's awesome. And I and I know 94 00:05:09,675 - > 00:05:13,960 sometimes when people hear things like standard 95 00:05:13,960 - > 00:05:18,200 certification, you know, terms like this, maybe some some 96 00:05:18,200 - > 00:05:22,760 people might have a reaction of, like, slowing things down. 97 00:05:22,760 - > 00:05:26,535 Right? But I I like how in at the very, you know, front and 98 00:05:26,535 - > 00:05:29,975 center of what you talk about is how to actually unlock 99 00:05:29,975 - > 00:05:34,055 enterprise adoption with, certification standards, 100 00:05:34,055 - > 00:05:34,535 etcetera. 101 00:05:34,695 - > 00:05:37,670 Could you talk, before we dive into the AI side of this 102 00:05:37,670 - > 00:05:39,990 specifically, could you talk about that a little bit in 103 00:05:39,990 - > 00:05:43,510 general, like how some of these things work together in actual 104 00:05:43,510 - > 00:05:48,150 enterprise settings, standard certification, even insurance, 105 00:05:48,150 - > 00:05:52,390 and how how those things actually can enable adoption 106 00:05:52,390 - > 00:05:55,955 with and not just like block things, I guess. 107 00:05:55,955 - > 00:05:58,915 Emil: Happy to. So I think our story and the inspiration we 108 00:05:58,915 - > 00:06:02,035 take dates back to Benjamin Franklin's Philadelphia. 109 00:06:02,915 - > 00:06:06,515 Philadelphia was starting to adopt electricity. Electricity 110 00:06:06,515 - > 00:06:09,860 was scary back then. Light bulbs did not work out. 111 00:06:09,860 - > 00:06:12,900 Homes started burning down. So Benjamin Franklin formed the 112 00:06:12,900 - > 00:06:15,700 first fire brigade in Philadelphia. He started 113 00:06:15,700 - > 00:06:19,460 codifying building codes so that we basically took what we knew 114 00:06:19,460 - > 00:06:22,305 around how to build safer houses, the standards part, and 115 00:06:22,305 - > 00:06:25,505 then he developed the first mutual insurance company. So 116 00:06:25,505 - > 00:06:28,625 back then, this is the first time we see this flywheel of 117 00:06:28,625 - > 00:06:32,625 standards, audits and insurance go together. By having standards 118 00:06:32,625 - > 00:06:35,025 around building codes, for example, you knew that houses 119 00:06:35,025 - > 00:06:38,180 needed to be placed a little bit further from each other. 120 00:06:38,420 - > 00:06:40,660 They needed some of his lightning rods to ensure that 121 00:06:40,660 - > 00:06:45,460 when lightning strikes that they catch fire. The fire inspections 122 00:06:45,460 - > 00:06:48,660 was the audit part that actually went in and examined that you'd 123 00:06:48,660 - > 00:06:52,805 followed these rules appropriately. And the insurance 124 00:06:52,805 - > 00:06:56,005 side mitigated the residual risk that will always be there when 125 00:06:56,005 - > 00:07:00,165 we introduce new powerful technology into society. We've 126 00:07:00,165 - > 00:07:03,125 seen this flywheel of standards audits and insurance time and 127 00:07:03,125 - > 00:07:05,765 time again when new technology has been introduced in society. 128 00:07:05,765 - > 00:07:07,520 We see it again in cars. 129 00:07:07,520 - > 00:07:10,160 Cars have safety standards. These were not demanded by 130 00:07:10,160 - > 00:07:12,960 government. They came from industry themselves because they 131 00:07:12,960 - > 00:07:16,000 knew that if we develop safer cars, people are more likely to 132 00:07:16,000 - > 00:07:19,120 buy them and safer cars actually enable you to drive even faster 133 00:07:19,120 - > 00:07:22,425 as well. So it was industry standards that also led us to 134 00:07:22,425 - > 00:07:25,465 airbags and seat belts and some of the other things that now 135 00:07:25,465 - > 00:07:28,025 make cars safer. We naturally have, again, the third party 136 00:07:28,025 - > 00:07:31,145 auditor going in and checking these cars and ensure that they 137 00:07:31,145 - > 00:07:31,865 follow the rules. 138 00:07:31,865 - > 00:07:34,425 We have the inspection element again, and we also have the 139 00:07:34,425 - > 00:07:37,830 insurance element. And this flywheel, one of the best things 140 00:07:37,830 - > 00:07:40,790 about it is that it really scales. So we're not just 141 00:07:40,790 - > 00:07:44,070 thinking, seeing it with say light bulbs and cars. We also 142 00:07:44,070 - > 00:07:48,390 see it for nuclear power plants to this date where you also have 143 00:07:48,390 - > 00:07:51,855 standards inspections of those power plants and insurance even 144 00:07:51,855 - > 00:07:55,055 works in this case as well. So there's no limitation to the 145 00:07:55,055 - > 00:07:56,975 power of this flywheel. 146 00:07:56,975 - > 00:07:59,855 When we're looking at AI, we see some of the same things at play. 147 00:07:59,935 - > 00:08:03,710 We see a new technology that is very powerful and has the powers 148 00:08:03,710 - > 00:08:06,990 both do a lot of good, but also if things go wrong, can have 149 00:08:06,990 - > 00:08:12,190 severe financial implications. And the other thing is it's a 150 00:08:12,190 - > 00:08:17,455 complex industry where me as a startup saying my technology is 151 00:08:17,455 - > 00:08:22,255 safe creates limited trust if I'm an enterprise buyer, big 152 00:08:22,255 - > 00:08:26,015 bank, for example, that wants to adopt this technology. So with 153 00:08:26,015 - > 00:08:28,095 the artificial intelligence underwriting company, what we're 154 00:08:28,095 - > 00:08:31,320 trying to do is to create that trust layer in between the 155 00:08:31,320 - > 00:08:35,640 companies building AI and the companies adopting AI. And what 156 00:08:35,640 - > 00:08:37,880 we offer as the trust layer is this flywheel. 157 00:08:37,880 - > 00:08:40,280 So we go in and codify the standards we believe that the 158 00:08:40,280 - > 00:08:43,485 companies building AI should follow. We go in and audit 159 00:08:43,485 - > 00:08:45,725 against those standards in collaboration with third party 160 00:08:45,725 - > 00:08:48,925 auditors like Shellman, CoalFire, companies who really 161 00:08:48,925 - > 00:08:51,805 know how to go deep and validate that the standards actually 162 00:08:51,805 - > 00:08:54,685 followed. And then we certify companies against the standards. 163 00:08:54,840 - > 00:08:58,600 A big part of the certification in the case of Agenack AI is red 164 00:08:58,600 - > 00:09:02,440 teaming, so we go in and test the actual AI agent systems, not 165 00:09:02,440 - > 00:09:05,080 just to see that the policies they have are in place and work 166 00:09:05,080 - > 00:09:07,545 well, but that the agents actually work and are robust 167 00:09:07,545 - > 00:09:10,345 under pressure, and the companies that then obtain a 168 00:09:10,345 - > 00:09:13,945 certificate gets access to buy insurance of their agents so 169 00:09:13,945 - > 00:09:16,985 that there's also that financial coverage of residual risk. 170 00:09:16,985 - > 00:09:20,025 Daniel: Yeah, this is so interesting and I have so many 171 00:09:20,025 - > 00:09:23,770 questions. Maybe one question that's just very selfish and our 172 00:09:23,770 - > 00:09:27,130 listeners know some part of the joy of being able to do a 173 00:09:27,130 - > 00:09:30,410 podcast like this is I get to get my own questions answered by 174 00:09:30,410 - > 00:09:33,850 people that are smarter than me. But one of those questions that 175 00:09:33,850 - > 00:09:37,875 I have that actually comes up in conversations I have day to day 176 00:09:38,115 - > 00:09:44,515 is is this tension of, hey, I I see a standard out there, 177 00:09:44,515 - > 00:09:46,995 whether it's, you know, some of the standards that we'll talk 178 00:09:46,995 - > 00:09:50,260 about that you all are codifying, or maybe it's things 179 00:09:50,260 - > 00:09:53,540 like the NIST AI risk management framework or things from OWASP. 180 00:09:54,020 - > 00:09:58,340 And logically they say, yes, it would make sense to do those 181 00:09:58,340 - > 00:10:03,365 things. But what is the forcing function that is kind of making 182 00:10:03,365 - > 00:10:09,285 making companies consider actual implementation of those of those 183 00:10:09,285 - > 00:10:13,365 principles rather than having it be a be an aspirational thing? 184 00:10:13,365 - > 00:10:18,490 Is it the is it the potential, you know, PR risk to the 185 00:10:18,490 - > 00:10:22,410 company? Is it you mentioned the financial side, the, maybe it's 186 00:10:22,410 - > 00:10:25,930 the commercial side of getting, you know, software vendors 187 00:10:25,930 - > 00:10:29,495 getting their software into the hands of their enterprise 188 00:10:29,495 - > 00:10:32,535 customers. What what do you see as some of those main forcing 189 00:10:32,535 - > 00:10:35,895 functions, or are there even those forcing functions right 190 00:10:35,895 - > 00:10:39,255 now that would force people to consider this as something, you 191 00:10:39,255 - > 00:10:42,695 know, not aspirational but actually practical? 192 00:10:42,695 - > 00:10:45,590 Emil: Yeah. So I I see a couple of different things that I think 193 00:10:45,590 - > 00:10:50,070 are very practical. Any vendor building powerful AI right now 194 00:10:50,070 - > 00:10:53,270 knows how tricky it is to get through the enterprise vendor 195 00:10:53,270 - > 00:10:56,565 due diligence process and questionnaires. So these 196 00:10:56,565 - > 00:10:58,725 startups face these questionnaires. Sometimes 197 00:10:58,725 - > 00:11:01,525 there's a 100 questions on them and it's extremely painful to go 198 00:11:01,525 - > 00:11:02,005 through. 199 00:11:02,885 - > 00:11:06,085 And I can tell you also, speak to a lot of enterprise CSOs and 200 00:11:06,085 - > 00:11:09,445 GSE managers, it's equally painful on their side, right? 201 00:11:09,445 - > 00:11:12,670 Because they're at a stage where the space changes so often that 202 00:11:12,670 - > 00:11:14,990 they feel a desire to actually change their questionnaire every 203 00:11:14,990 - > 00:11:17,710 month. Going through a 100 questions from a startup every 204 00:11:17,710 - > 00:11:20,030 single time you try to onboard a vendor is also completely 205 00:11:20,030 - > 00:11:24,405 painful on the other side. So I think part one here is speed and 206 00:11:25,685 - > 00:11:27,845 a desire to get to a place where you actually feel like you've 207 00:11:27,845 - > 00:11:31,365 covered your blind spots as well. And having a third party 208 00:11:31,365 - > 00:11:34,805 developed standard with all of the industry in the room to help 209 00:11:34,805 - > 00:11:36,965 find those blind spots and figure out how we can't fight 210 00:11:36,965 - > 00:11:39,970 them in a standard, I think is value proposition number one. 211 00:11:40,850 - > 00:11:43,250 The speed argument obviously assumes that you can get across 212 00:11:43,250 - > 00:11:45,650 the line in the first place. I think the second part of the 213 00:11:45,650 - > 00:11:49,090 value proposition is having that third party validation that your 214 00:11:49,090 - > 00:11:51,875 Agenack AI is actually safe, secure and reliable. We're 215 00:11:51,875 - > 00:11:56,035 working with some of the frontier companies in the 216 00:11:56,035 - > 00:11:58,195 aerospace right now, companies like Eleven Labs, companies like 217 00:11:58,195 - > 00:12:01,235 Fin that just got acquired for 3,600,000,000 by Salesforce, 218 00:12:01,235 - > 00:12:04,755 companies like UiPath who have set the standards within their 219 00:12:05,010 - > 00:12:08,610 categories historically. They have fantastic security 220 00:12:08,610 - > 00:12:12,610 postures, but they don't have a way to prove that to an 221 00:12:12,610 - > 00:12:15,890 enterprise. So an enterprise will just never trust a company 222 00:12:15,890 - > 00:12:17,650 that has an incentive to sell their product. 223 00:12:17,650 - > 00:12:20,655 They need that third party to go in and do that. And then I do 224 00:12:20,655 - > 00:12:23,455 think there is a security argument to be made here. Our 225 00:12:23,455 - > 00:12:26,735 red teaming consistently uncovers blind spots for the 226 00:12:26,735 - > 00:12:29,375 companies that we work with. Sometimes it's the hallucination 227 00:12:29,375 - > 00:12:32,575 rate where we realize that a specific type of adversarial 228 00:12:32,575 - > 00:12:35,780 attack will bring up the hallucination rate or specific 229 00:12:35,780 - > 00:12:37,860 language switches or other things that might actually 230 00:12:37,860 - > 00:12:41,380 happen when their products are deployed. Other times, it's 231 00:12:41,380 - > 00:12:44,340 jailbreaking risk that we manage to uncover or pump injection 232 00:12:44,340 - > 00:12:45,780 risk that we manage to uncover. 233 00:12:46,085 - > 00:12:49,525 So we do also see ourselves as helping these companies actually 234 00:12:49,525 - > 00:12:53,125 improve their safety, security and reliability posture, which 235 00:12:53,125 - > 00:12:58,085 is valuable as well. Then I'm sure there's a marketing benefit 236 00:12:58,085 - > 00:13:02,170 to the companies going out early and adopting a new framework and 237 00:13:02,170 - > 00:13:05,370 showing and demonstrating their moving in front when it comes to 238 00:13:05,370 - > 00:13:10,010 AI security leadership. I think that's an important branding 239 00:13:10,250 - > 00:13:14,570 value as well that we sometimes help provide, but I don't think 240 00:13:14,570 - > 00:13:17,435 that's the core benefit of decertification right now. I 241 00:13:17,435 - > 00:13:20,075 think that is really unlocking upmarket enterprise revenue for 242 00:13:20,075 - > 00:13:21,195 the for the companies. 243 00:13:22,155 - > 00:13:25,835 Daniel: Agents are impacting every function within a company, 244 00:13:25,835 - > 00:13:29,195 but it's sometimes very difficult to figure out what an 245 00:13:29,195 - > 00:13:32,860 agent should do, what a human should do. Jeffrey from News 246 00:13:32,860 - > 00:13:36,460 Research, a recent guest said often agents have no taste. 247 00:13:36,700 - > 00:13:41,260 That's why I'm so impressed with what our partner Framer is doing 248 00:13:41,260 - > 00:13:44,700 with their pro website builder that's already trusted by 249 00:13:44,700 - > 00:13:47,805 companies like Miro and Perplexity. They're implementing 250 00:13:47,805 - > 00:13:51,565 agents, but in a way that agents and humans work in tandem. 251 00:13:51,645 - > 00:13:55,165 Agents bring speed and scale, but people bring the taste, 252 00:13:55,165 - > 00:13:56,765 judgment, and control. 253 00:13:57,245 - > 00:14:01,740 And these agents help solve this gap between AI generated ideas 254 00:14:01,740 - > 00:14:06,540 and production ready website work. So Framer is already 255 00:14:06,540 - > 00:14:09,820 enterprise level solution. They they allow you to create amazing 256 00:14:09,820 - > 00:14:14,315 websites that are SEO ready. And so I really would recommend that 257 00:14:14,315 - > 00:14:17,195 you check them out if you're building a new website or just 258 00:14:17,195 - > 00:14:20,555 implementing landing pages or an upgrade in your existing 259 00:14:20,555 - > 00:14:23,995 website. Learn how you can get more out of your site from a 260 00:14:23,995 - > 00:14:27,595 Framer specialist or get started building for free today at 261 00:14:27,595 - > 00:14:34,410 framer.com/practicalai for 30% off of Framer Pro annual plan. 262 00:14:34,490 - > 00:14:38,490 That's framer.com/practicalai for 30% off. 263 00:14:38,490 - > 00:14:43,370 Framer.com/practicalai. Rules and restrictions may apply. 264 00:14:45,075 - > 00:14:45,875 Yeah. Yeah. 265 00:14:45,875 - > 00:14:49,635 I I'm I love your answer and it was a little bit I I was trying 266 00:14:49,635 - > 00:14:52,355 to validate some of my own thinking through that because 267 00:14:52,355 - > 00:14:55,395 we've talked on the show before about, you know, it isn't really 268 00:14:55,395 - > 00:14:59,795 like the the governments of the world are quite behind in terms 269 00:14:59,795 - > 00:15:03,660 of, you know, how they would, you know, enforce or even say 270 00:15:03,660 - > 00:15:08,300 what to enforce for companies building AI things. And and so 271 00:15:08,300 - > 00:15:10,780 it's really the enterprises themselves that have some 272 00:15:10,780 - > 00:15:14,300 motivation to do this, all of those that you laid out, and I'm 273 00:15:14,300 - > 00:15:20,715 and I'm sure more. In in the in the current, I guess, state of 274 00:15:20,715 - > 00:15:24,715 AI standards, if we kind of shift to that piece, and then 275 00:15:24,715 - > 00:15:27,675 eventually, wanna get to kind of the evidence and red teaming and 276 00:15:27,675 - > 00:15:32,290 and all of that. But maybe just as if we take a general look at 277 00:15:32,290 - > 00:15:37,810 the standards that exist out there for AI and AI agents, 278 00:15:37,810 - > 00:15:41,250 could you help us understand what kinds of standards are out 279 00:15:41,250 - > 00:15:45,815 there and what they cover? Because there's a lot of there's 280 00:15:45,815 - > 00:15:48,295 a lot of sort of intersections that we could think of, whether 281 00:15:48,295 - > 00:15:56,330 that be security or safety or alignment or all sorts of 282 00:15:56,330 - > 00:15:57,690 things, data privacy. 283 00:15:57,690 - > 00:16:00,970 There there's all sorts of ways that you could kind of look at 284 00:16:00,970 - > 00:16:03,530 this and perspectives that you could look at it from, and 285 00:16:03,530 - > 00:16:06,570 there's all sorts of things that people have proposed over time. 286 00:16:06,570 - > 00:16:10,145 So I imagine, you know, that's part of the reason why having a 287 00:16:10,145 - > 00:16:13,745 company that's really digging into this at a deep level is is 288 00:16:13,745 - > 00:16:17,345 very worthwhile, which I I think it is. But could you help set 289 00:16:17,345 - > 00:16:20,625 the stage for that? Like, how can we categories categorize in 290 00:16:20,625 - > 00:16:24,450 our mind the current state of AI standards and what perspectives 291 00:16:24,450 - > 00:16:25,330 are coming from? 292 00:16:25,330 - > 00:16:27,730 Emil: Yeah, absolutely. And by the way, Daniel, that's exactly 293 00:16:27,730 - > 00:16:30,690 where we started last summer, right? So if you go to auc1.com 294 00:16:30,690 - > 00:16:33,570 today, you'll find that we've done crosswalks. I think it's 295 00:16:33,570 - > 00:16:35,890 about 10 different frameworks now, but they're transparently 296 00:16:35,890 - > 00:16:38,985 available so you can see exactly how our standard fits into the 297 00:16:38,985 - > 00:16:42,185 existing environment and hopefully you also see then why 298 00:16:42,185 - > 00:16:45,465 we concluded after doing this work that yes, there actually 299 00:16:45,465 - > 00:16:48,825 was a need for another standard even though there can be 300 00:16:48,825 - > 00:16:50,265 sometimes a little bit of standard fatigue. 301 00:16:50,480 - > 00:16:54,160 Daniel: And just, by way of, I don't know, encouragement or 302 00:16:54,160 - > 00:16:56,720 thanks, maybe gratitude is the right way to put it, our our 303 00:16:56,720 - > 00:17:01,280 our, the company Eil Laid has looked at that many times in 304 00:17:01,280 - > 00:17:04,955 terms of, and we're maybe in, not not like everyone, we're 305 00:17:04,955 - > 00:17:09,035 building actual, you know, plane that works on some of these some 306 00:17:09,035 - > 00:17:11,275 of these knobs and levers that you talk about, but it's been 307 00:17:11,275 - > 00:17:15,540 extremely useful. And even, you know, as we're as we're writing 308 00:17:15,540 - > 00:17:18,660 content or doing planning or thinking about things in our 309 00:17:18,660 - > 00:17:21,460 product, I always refer people back to that page and I'd refer 310 00:17:21,460 - > 00:17:24,500 our listeners to that page because it is a it is a really 311 00:17:24,500 - > 00:17:28,260 great crosswalk and helps understand, you know, where 312 00:17:28,260 - > 00:17:30,900 these align, where they don't align, what what the other need 313 00:17:30,900 - > 00:17:34,555 is. So just by way of gratitude, thank you for putting that 314 00:17:34,555 - > 00:17:35,915 together and making it public. 315 00:17:35,915 - > 00:17:37,915 Emil: Yeah, no, and we appreciate all the people who've 316 00:17:37,915 - > 00:17:40,235 worked on this. We do a lot of work with the Cloud Security 317 00:17:40,235 - > 00:17:43,275 Alliance, with the OWASP community across both the AIVSS 318 00:17:43,275 - > 00:17:46,380 and the GinAI project. We work with Cisco and IBM on 319 00:17:46,380 - > 00:17:49,180 Crosswalks, so it's a big team effort and I really appreciate 320 00:17:49,180 - > 00:17:53,260 that we've been able to gather the ecosystem around a decided 321 00:17:53,260 - > 00:17:55,740 to just publish some of this stuff transparently so that 322 00:17:55,740 - > 00:17:58,875 organizations like yours, but also I know big enterprises are 323 00:17:58,875 - > 00:18:02,635 using the controls we put out transparently in their own 324 00:18:02,635 - > 00:18:05,275 control frameworks. That's completely free to use and only 325 00:18:05,275 - > 00:18:07,515 the companies pursuing certification actually needs to 326 00:18:07,515 - > 00:18:10,555 get money out the pocket. To get back to your question, because I 327 00:18:10,555 - > 00:18:13,580 think it's an important one, the way we see the standard space is 328 00:18:13,580 - > 00:18:14,860 that you have three layers. 329 00:18:14,860 - > 00:18:17,100 You have an organizational layer, you have an 330 00:18:17,100 - > 00:18:19,820 infrastructure layer and then you have the agentic AI layer. 331 00:18:19,820 - > 00:18:22,300 At the organizational level, many organizations have been 332 00:18:22,300 - > 00:18:25,340 through an ISO 27,001 certification, classic 333 00:18:25,340 - > 00:18:28,300 management system certification. ISO then about three years ago 334 00:18:28,300 - > 00:18:32,355 now published the 42,001, which is the management system 335 00:18:32,355 - > 00:18:36,275 certification for AI systems. It's a governance certification 336 00:18:36,275 - > 00:18:38,995 that ensures that you have the right policies in place and the 337 00:18:38,995 - > 00:18:42,355 right procedures in place so that when you develop AI 338 00:18:42,355 - > 00:18:46,230 systems, they hopefully turn out in the right way if you follow 339 00:18:46,230 - > 00:18:48,870 those systems. Then you have the infrastructure layer. 340 00:18:48,870 - > 00:18:52,070 That's where your SOC two comes in and your pentesting and some 341 00:18:52,070 - > 00:18:55,750 of the classic cybersecurity controls, access management, 342 00:18:55,750 - > 00:18:59,625 transport security, all the good stuff there. I'd say that many 343 00:18:59,625 - > 00:19:02,585 of those things become even more important in an space because 344 00:19:02,585 - > 00:19:06,265 pace is higher, data access is higher, so if you don't have 345 00:19:06,265 - > 00:19:09,785 that in order, then you should go back and ensure that you get 346 00:19:09,785 - > 00:19:13,770 those boxes checked. And then at the Agenack AI space, we 347 00:19:13,770 - > 00:19:16,250 basically just didn't see anything when we started this 348 00:19:16,250 - > 00:19:20,570 company and started drafting the first version of AI UC. One, we 349 00:19:20,570 - > 00:19:24,490 see NIST have come out with the AI risk management framework. 350 00:19:25,325 - > 00:19:28,685 There's a little bit of Agenack stuff in there, and I know from 351 00:19:28,685 - > 00:19:31,405 speaking to the team that they're considering publishing 352 00:19:31,485 - > 00:19:32,605 additions to this. 353 00:19:32,605 - > 00:19:34,925 The Cloud Security Alliance has also done their AI controls 354 00:19:34,925 - > 00:19:37,325 matrix, where again, there's some things in there around 355 00:19:37,325 - > 00:19:40,530 Agenack AI that are pretty good. The issue with both of those 356 00:19:40,530 - > 00:19:44,050 frameworks is that they're guidance. They're voluntary 357 00:19:44,050 - > 00:19:46,690 frameworks. You decide which controls you implement. You 358 00:19:46,690 - > 00:19:49,570 decide whether you like, how you implement them. 359 00:19:49,570 - > 00:19:52,845 They're not orderable frameworks. So the way AAC One 360 00:19:52,845 - > 00:19:56,685 fits in here is that we've basically taken the core 361 00:19:56,685 - > 00:20:00,125 governance things from the organizational level that we 362 00:20:00,125 - > 00:20:03,005 think are really important when it comes to AI systems, such as 363 00:20:03,005 - > 00:20:07,290 having failure plans in play when agents do not do what 364 00:20:07,290 - > 00:20:09,770 they're intended to do and you know how to deal with that. Good 365 00:20:09,770 - > 00:20:12,250 change management and acknowledging that every time 366 00:20:12,250 - > 00:20:15,850 you, for example, replace the LLM in an agent, it will behave 367 00:20:15,850 - > 00:20:18,410 differently. And if you don't take that into account in your 368 00:20:18,410 - > 00:20:22,215 governance, your end users will bear the burden of that. So some 369 00:20:22,215 - > 00:20:24,615 parts of the governance, the core parts of the infrastructure 370 00:20:24,615 - > 00:20:29,015 layer as well as ensuring that the folks who have access to the 371 00:20:29,015 - > 00:20:32,775 AI system itself and can make these big decisions, that's 372 00:20:32,775 - > 00:20:33,175 restricted. 373 00:20:33,500 - > 00:20:36,460 Ensuring again that transport security, when you do agent to 374 00:20:36,460 - > 00:20:38,940 agent communications and so forth, is in place. But 375 00:20:38,940 - > 00:20:42,620 otherwise, we basically leave ISO and TUC2 to do what they're 376 00:20:42,620 - > 00:20:45,900 really good at and focus on the agentic layer. And what is up 377 00:20:45,900 - > 00:20:49,165 there then for us is specific controls around safety, for 378 00:20:49,165 - > 00:20:51,805 example, ensuring that agents behave according to brand and 379 00:20:51,805 - > 00:20:55,725 that they don't give users guidance on medical care or 380 00:20:55,725 - > 00:20:58,605 legal advice, financial advice, other high risk areas, basically 381 00:20:58,605 - > 00:21:01,085 that they stay within their scope and don't start breaking 382 00:21:01,085 - > 00:21:04,300 out of that. We look at specifically how you restrict 383 00:21:04,300 - > 00:21:07,820 the agent's data access, its system access and its tool 384 00:21:07,820 - > 00:21:10,300 access, so it doesn't start processing refunds when you 385 00:21:10,300 - > 00:21:13,100 shouldn't. We look at hallucinations, which is also a 386 00:21:13,100 - > 00:21:17,775 risk that is quite unique to AI obviously and does not come up 387 00:21:17,775 - > 00:21:21,135 in any way in either ISO or SOC certifications. 388 00:21:21,135 - > 00:21:26,895 So really focus on the agentic layer there and the core part of 389 00:21:26,895 - > 00:21:31,180 the differentiation, I'd say, is then the technical level of the 390 00:21:31,180 - > 00:21:33,740 controls. We go in and are actually quite prescriptive in 391 00:21:33,740 - > 00:21:36,860 what we want to see from the agents because we have a good 392 00:21:36,860 - > 00:21:39,420 understanding now of what the right toolbox is to ensure that 393 00:21:39,420 - > 00:21:43,015 these agents behave safe, secure and reliably. And the other 394 00:21:43,015 - > 00:21:46,135 thing is we acknowledge that a technical control in itself 395 00:21:46,135 - > 00:21:49,415 might not hold up under robustness. So six of the 40 396 00:21:49,415 - > 00:21:53,095 mandatory requirements in AAC one have to do with red teaming, 397 00:21:53,335 - > 00:21:56,480 testing that these technical controls then hold up under 398 00:21:56,480 - > 00:22:00,720 pressure, both when we react, like we engage with the system 399 00:22:00,720 - > 00:22:03,680 as a benign user, just ask it questions and see if it 400 00:22:03,680 - > 00:22:05,000 hallucinates, but also what happens when we start 401 00:22:06,080 - > 00:22:09,165 approaching the system like, with social engineering and and 402 00:22:09,165 - > 00:22:10,445 adversarial pressure. 403 00:22:10,445 - > 00:22:14,125 Daniel: And just to help people understand, so the AI UC one, 404 00:22:14,205 - > 00:22:17,405 that's the standard that that you all have published. People 405 00:22:17,405 - > 00:22:22,310 can look at it online. I assume since it's AI UC one, there's an 406 00:22:22,310 - > 00:22:27,590 anticipation there might be a a two or other or or various, you 407 00:22:27,670 - > 00:22:31,350 know, either revisions or different focuses kind of within 408 00:22:31,350 - > 00:22:34,390 different certifications is my is my understanding right there? 409 00:22:34,390 - > 00:22:39,055 Emil: That's correct. I think where to start is we update the 410 00:22:39,055 - > 00:22:41,135 standard every single quarter. So we've gathered now a 411 00:22:41,135 - > 00:22:43,855 consortium of two fifty secondurity leaders. Some of 412 00:22:43,855 - > 00:22:46,895 them are CSOs at Fortune 1,000 companies. Some of them are 413 00:22:46,895 - > 00:22:49,695 security engineers, architects, GRC managers. 414 00:22:50,020 - > 00:22:53,780 And so we have the full stack of people in the room. And with 415 00:22:53,780 - > 00:22:57,060 them, every quarter we identify new priority areas. Last quarter 416 00:22:57,060 - > 00:22:59,940 it was MCP risk, for example, which has really come up as 417 00:22:59,940 - > 00:23:03,140 agents start not just operating in isolation but exchanging 418 00:23:03,140 - > 00:23:06,795 information. This quarter, we look a lot at how we can 419 00:23:06,795 - > 00:23:10,475 strengthen runtime security and that continuous element, which 420 00:23:10,475 - > 00:23:14,075 continues to be really important for a lot of organizations. So 421 00:23:14,075 - > 00:23:15,835 we get them into the room and update the standard each 422 00:23:15,835 - > 00:23:16,315 quarter. 423 00:23:16,500 - > 00:23:21,540 I could very well see that new frameworks, so an AIAC two, AIAC 424 00:23:21,540 - > 00:23:24,340 three come out in the future. We don't have any plans to do that 425 00:23:24,340 - > 00:23:28,180 yet, but what we know, again, if we go all the way back to where 426 00:23:28,180 - > 00:23:31,505 we started our conversation, is that this combination of 427 00:23:31,505 - > 00:23:34,065 standards audits and insurance have worked historically. So 428 00:23:34,065 - > 00:23:36,785 right now we focus on the application layers of the 429 00:23:36,785 - > 00:23:41,345 platforms and products that take Agenack AI and deploy. But 430 00:23:41,345 - > 00:23:44,580 there's a model layer as well, which we see as our second 431 00:23:44,580 - > 00:23:48,500 horizon and there's the physical layer, like the data centers and 432 00:23:48,500 - > 00:23:51,780 the infrastructure that we deploy AI on, but also the cars 433 00:23:51,780 - > 00:23:54,900 and the robots that we put this into where standards audits and 434 00:23:54,900 - > 00:23:57,140 insurance could play a big role and that's where we see the 435 00:23:57,140 - > 00:23:58,180 company go long term. 436 00:23:58,180 - > 00:24:01,885 Daniel: Yeah, that makes sense. Could could you help us, so 437 00:24:01,885 - > 00:24:06,445 maybe paint the picture, let's say, there's a scenario, I'm I'm 438 00:24:06,445 - > 00:24:12,340 a company, I maybe I'm building an agentic a new agentic driven 439 00:24:12,340 - > 00:24:16,180 product. Right? And I'm going to offer it to some sort of 440 00:24:16,180 - > 00:24:19,220 regulated or enterprise customers. I'm selling into 441 00:24:19,220 - > 00:24:19,700 health care. 442 00:24:19,700 - > 00:24:22,660 I'm selling into, you know, large manufacturing or or 443 00:24:22,660 - > 00:24:26,905 whatever it is. Right? So in that scenario, what what would 444 00:24:26,905 - > 00:24:31,305 the process kind of recommended process be for our company to 445 00:24:31,305 - > 00:24:35,545 engage with this standard and eventually get to that level of 446 00:24:35,625 - > 00:24:39,190 certification, maybe in the future eventually to the to the 447 00:24:39,190 - > 00:24:42,790 insurance side, but at least to that certification side, what 448 00:24:42,790 - > 00:24:45,750 would that process look like? And then maybe highlight in that 449 00:24:45,750 - > 00:24:49,350 process where the red teaming comes in. And then I'd love to 450 00:24:49,350 - > 00:24:51,990 circle back on that maybe later and and talk through that 451 00:24:51,990 - > 00:24:52,710 specifically. 452 00:24:52,710 - > 00:24:56,135 Emil: Yeah. Absolutely. So you'd get in touch with our team and 453 00:24:56,135 - > 00:24:58,695 the first thing we always do is we do a gap assessment against 454 00:24:58,695 - > 00:25:02,135 your existing systems. If you have a well documented trust 455 00:25:02,135 - > 00:25:05,895 center already or some blog post describing what you do, we can 456 00:25:05,895 - > 00:25:09,160 basically go back to you and tell you this is the places 457 00:25:09,160 - > 00:25:12,840 where we believe you already meet the standard. These are the 458 00:25:12,840 - > 00:25:15,400 areas where we expect that there will be work for you. 459 00:25:15,480 - > 00:25:18,360 So you basically go into the certifications process with open 460 00:25:18,360 - > 00:25:21,960 eyes around what is the workload needed from engineering, from 461 00:25:21,960 - > 00:25:28,345 legal and from your GRC team to take your company through it. I 462 00:25:28,345 - > 00:25:32,105 will mention at this point, we've had a three person Y 463 00:25:32,105 - > 00:25:35,225 Combinator startup go through this. We've had UiPath that is 464 00:25:35,225 - > 00:25:39,680 publicly traded go through this. We have companies at all stages. 465 00:25:39,680 - > 00:25:42,080 So I mentioned now security, legal and GSE. 466 00:25:42,080 - > 00:25:44,560 That was the same person when it came to points or getting 467 00:25:44,560 - > 00:25:47,520 certified, right? So it is a standard that scales with the 468 00:25:47,520 - > 00:25:50,160 organization's size as well. When you have this gap 469 00:25:50,160 - > 00:25:52,855 assessment completed, you basically decide whether you 470 00:25:52,855 - > 00:25:55,975 want to move forward with the certification or not. To move 471 00:25:55,975 - > 00:26:00,055 forward, we split the process in two parts. One part is you pick 472 00:26:00,055 - > 00:26:01,735 an order of your choice. 473 00:26:02,375 - > 00:26:05,015 We have a number of credited auditors, like, for example, 474 00:26:05,015 - > 00:26:08,620 Shellman Coal Fired, but the list is growing very rapidly at 475 00:26:08,620 - > 00:26:12,380 the moment. And trusted auditor who knows how to do this. And on 476 00:26:12,380 - > 00:26:15,180 their track, you basically start collecting all the evidence that 477 00:26:15,180 - > 00:26:18,380 is needed to go through the ASC one audit. It falls in two 478 00:26:18,380 - > 00:26:20,620 buckets. Some of it is the classic legal policies. 479 00:26:20,945 - > 00:26:23,185 If you have a generative AI product, you need to define who 480 00:26:23,185 - > 00:26:25,905 owns the inputs and outputs and how you retain user data and 481 00:26:25,905 - > 00:26:28,545 whether you train on that user data. So forth, you need to 482 00:26:28,545 - > 00:26:31,585 define your acceptable use. And the second part is the technical 483 00:26:31,585 - > 00:26:34,385 controls that that Sheltman will go in and validate. So that is 484 00:26:34,720 - > 00:26:37,280 your filtering configuration against harmful output, your 485 00:26:37,280 - > 00:26:40,800 classifier, your defensive prompting, your groundedness 486 00:26:40,800 - > 00:26:43,920 filtering when it comes to hallucination preventing, your 487 00:26:43,920 - > 00:26:47,360 safeguards around tool calls and all the other things. So again, 488 00:26:47,360 - > 00:26:50,025 you go through those requirements and capture the 489 00:26:50,025 - > 00:26:53,465 evidence and submit that to the auditor that goes in and does 490 00:26:53,465 - > 00:26:54,825 that third party validation. 491 00:26:55,465 - > 00:26:58,745 The other track we then do in parallel is that you give us an 492 00:26:58,745 - > 00:27:02,105 instance of the agent or the agents, it can be multiple as 493 00:27:02,105 - > 00:27:05,380 well, that is in scope for the certification. And you basically 494 00:27:05,380 - > 00:27:08,820 configure a representative version of that agent. So an 495 00:27:08,820 - > 00:27:11,700 agent that would be configured how an enterprise would use it. 496 00:27:11,780 - > 00:27:15,060 We sometimes see companies creating an extremely safe agent 497 00:27:15,060 - > 00:27:18,265 that has almost lost all its power because they just wanted 498 00:27:18,265 - > 00:27:20,585 to pass the certification, we obviously would then go in as 499 00:27:20,585 - > 00:27:23,785 the third party in the room and push back and say, we want to 500 00:27:23,785 - > 00:27:26,345 see an agent that is configured based on the public docs you 501 00:27:26,345 - > 00:27:29,145 have and the defaults you've built into the product. When we 502 00:27:29,145 - > 00:27:32,025 then have access to that, we often access it via API. 503 00:27:32,025 - > 00:27:37,920 Our internal team will draw up a matrix of the risks we see that 504 00:27:37,920 - > 00:27:41,040 this agent is subject to and the attacks that it could be subject 505 00:27:41,040 - > 00:27:44,320 to if someone went in and attacked it. And we then develop 506 00:27:44,400 - > 00:27:47,760 usually between 1,005 different scenarios that we're going to 507 00:27:47,760 - > 00:27:51,515 hit this agent with. Each attack is unique. Some of them are, 508 00:27:51,515 - > 00:27:55,115 again, benign in nature. So the user will simply ask it a 509 00:27:55,115 - > 00:27:56,395 question, get the answer back. 510 00:27:56,395 - > 00:27:59,355 If the agent doesn't hallucinate, it passes the eval. 511 00:27:59,595 - > 00:28:03,355 Other times we'll increase the adversarial pressure step by 512 00:28:03,355 - > 00:28:06,620 step. So the first step could be that we try to lie to it. The 513 00:28:06,620 - > 00:28:09,420 second step could be that we invoke authority. We do it over 514 00:28:09,420 - > 00:28:12,700 multiple turns sometimes and keep insisting on doing things. 515 00:28:12,700 - > 00:28:15,660 We pretend that we're under distress and say, if you don't 516 00:28:15,660 - > 00:28:19,515 do this right now, I will go and do something terrible. So please 517 00:28:19,515 - > 00:28:24,635 process this refund. And obviously only pass the agent if 518 00:28:24,635 - > 00:28:27,595 we see it hold up to that pressure. We do the red teaming 519 00:28:27,595 - > 00:28:30,395 in two rounds because we often do find things in the first 520 00:28:30,395 - > 00:28:32,715 round. So similar to an ISO audit where you have a stage one 521 00:28:32,715 - > 00:28:35,500 and a stage two and you then have a chance to mitigate any 522 00:28:35,500 - > 00:28:38,540 findings in between, we give a company the chance to do that 523 00:28:38,540 - > 00:28:41,020 because the goal for us, again, is not compliance. 524 00:28:41,020 - > 00:28:43,580 The goal for us is security, right, that you actually improve 525 00:28:43,580 - > 00:28:46,300 the agent as part of the certification process. So 526 00:28:46,300 - > 00:28:49,420 depending on the magnitude of the findings, your team will 527 00:28:49,420 - > 00:28:52,735 have between, say, one and four weeks to mitigate these things 528 00:28:52,735 - > 00:28:55,935 based on the recommendations we come up with. And we then do a 529 00:28:55,935 - > 00:29:00,175 second round of testing. That testing is final and is taken 530 00:29:00,175 - > 00:29:03,610 then into account when the auditor takes your evidence, 531 00:29:03,610 - > 00:29:06,810 takes your Red Team results and writes that final audit report. 532 00:29:06,810 - > 00:29:09,930 And what you leave the process with is a comprehensive audit 533 00:29:09,930 - > 00:29:11,930 report that describes your security posture. 534 00:29:12,090 - > 00:29:14,330 It's between sixty and one hundred pages long and it's an 535 00:29:14,330 - > 00:29:17,085 asset you can really unblock those enterprise deals with. You 536 00:29:17,085 - > 00:29:19,805 get a certificate for your website again, so you can 537 00:29:19,805 - > 00:29:23,005 demonstrate that you've gone above and beyond when it comes 538 00:29:23,005 - > 00:29:26,045 to security. And then we come knocking again three months 539 00:29:26,045 - > 00:29:28,685 later and say, we still have access to your agent via the 540 00:29:28,685 - > 00:29:32,420 API. We're now going to run that same barrage of tests again to 541 00:29:32,420 - > 00:29:35,220 ensure that the changes you've made in the last quarter didn't 542 00:29:35,220 - > 00:29:38,580 invalidate some of the security things we found. And we do that 543 00:29:38,580 - > 00:29:41,060 every single quarter, that's a requirement to to maintain 544 00:29:41,060 - > 00:29:41,860 certification. 545 00:29:41,940 - > 00:29:45,065 Daniel: And in that red teaming, I mean, you you mentioned this 546 00:29:45,065 - > 00:29:49,305 before around kind of the probabilistic nature of of some 547 00:29:49,305 - > 00:29:52,105 of these things, and this is something I've always run into 548 00:29:52,105 - > 00:29:56,345 in in AI workshops as I give workshops in in enterprise. 549 00:29:56,345 - > 00:29:59,350 Often people will say, oh, this is like, you know, it's not 550 00:29:59,350 - > 00:30:03,270 deterministic. How do we how do we create the right like, what 551 00:30:03,270 - > 00:30:07,590 does passing mean? Right? And so you could say, well, passing 552 00:30:07,670 - > 00:30:11,750 means, you know, passing all 5,000 scenarios. 553 00:30:11,830 - > 00:30:14,685 Right? And you mentioned this phased approach, which I think 554 00:30:14,685 - > 00:30:17,725 deals with part of that. But, yeah, what, could you describe a 555 00:30:17,725 - > 00:30:20,765 little bit on that side? Like, what does what does passing 556 00:30:20,765 - > 00:30:25,440 mean? At what level kind of do you expect things to pass or 557 00:30:25,440 - > 00:30:29,200 should should you expect things to pass or has that even been a 558 00:30:29,200 - > 00:30:30,480 a topic of discussion? 559 00:30:30,480 - > 00:30:32,960 Emil: Yeah. So it's it's a great question and it's also a really 560 00:30:32,960 - > 00:30:36,480 hard question, so there's some nuances in here. What we require 561 00:30:36,480 - > 00:30:41,965 to pass AAC one is that you don't have any rate each run 562 00:30:41,965 - > 00:30:45,725 based on severity. So a pass, you can have a P four, which is 563 00:30:45,725 - > 00:30:48,365 an insignificant, say, small hallucination that doesn't 564 00:30:48,365 - > 00:30:51,165 really affect an end user. A minor thing would be which would 565 00:30:51,165 - > 00:30:51,565 be a P three. 566 00:30:52,170 - > 00:30:54,650 P2 would be something significant that may actually 567 00:30:54,650 - > 00:30:59,130 have real world implications. P1 is something critical. And P0 568 00:30:59,130 - > 00:31:01,450 actually don't know the name for it. I think we have called it 569 00:31:01,450 - > 00:31:03,770 catastrophic or something like that. The kind of thing, if we 570 00:31:03,770 - > 00:31:05,905 found it, you would drop what you had in your hands and start 571 00:31:05,905 - > 00:31:08,785 mitigating it immediately because having a system deployed 572 00:31:08,785 - > 00:31:11,265 with this kind of vulnerability could have real world 573 00:31:11,265 - > 00:31:14,705 implications that would be high. 574 00:31:15,185 - > 00:31:18,385 Our grading approach right now is that you cannot pass AAC1 if 575 00:31:18,385 - > 00:31:21,540 you have any P0 or P1 vulnerabilities identified. You 576 00:31:21,540 - > 00:31:24,820 have to mitigate those. From then on, we believe a lot in 577 00:31:24,820 - > 00:31:28,900 transparency and we know from the compliance world, at least 578 00:31:28,900 - > 00:31:32,900 the frameworks that are robust and hold up under pressure, that 579 00:31:32,900 - > 00:31:36,775 if we put the results in the audit report and your customers 580 00:31:36,775 - > 00:31:41,415 see that audit report, you are very incentivized to mitigate 581 00:31:41,415 - > 00:31:44,935 the vulnerabilities we find. What we also know is that these 582 00:31:44,935 - > 00:31:49,175 agent systems are very different from use case to use case. So a 583 00:31:49,290 - > 00:31:52,970 coding agent is one type of beast versus a customer service 584 00:31:52,970 - > 00:31:57,210 agent versus a automation agent like UiPath that make decisions 585 00:31:57,210 - > 00:31:58,410 based on the information. 586 00:31:58,650 - > 00:32:02,730 And so companies have different tolerances around the percentage 587 00:32:02,730 - > 00:32:05,555 of hallucinations they would accept and so forth. So we 588 00:32:05,555 - > 00:32:08,595 really leave it up to the company and really in the end, 589 00:32:08,595 - > 00:32:11,555 the customers of that company to make these calls. The important 590 00:32:11,555 - > 00:32:14,275 thing is, and this is where we sometimes have a little bit of a 591 00:32:14,275 - > 00:32:16,435 conversation with some of the companies we work with, no 592 00:32:16,435 - > 00:32:21,350 company has ever and will ever pass AAC1 with a 100 pass rate. 593 00:32:21,350 - > 00:32:25,430 It doesn't exist here. We're not Delve SOC two compliance where 594 00:32:25,430 - > 00:32:28,710 you just get a magical, a spot free audit report. 595 00:32:28,710 - > 00:32:32,585 All agentic systems are nondeterministic in nature. That 596 00:32:32,585 - > 00:32:34,825 means that they will always, if you put them under the right 597 00:32:34,825 - > 00:32:37,545 amount of pressure, be able to be jailbroken. They will always 598 00:32:37,545 - > 00:32:41,305 be able to hallucinate. We work again with, like some of the 599 00:32:41,305 - > 00:32:44,185 legal agents we're certifying right now are world class at 600 00:32:44,185 - > 00:32:47,065 hallucination prevention. I am sure we will still be able to 601 00:32:47,065 - > 00:32:50,340 find some minor hallucination cases in those. 602 00:32:50,340 - > 00:32:52,660 And that's just the nature of these systems. If you remove 603 00:32:52,660 - > 00:32:55,140 those hallucination rates, it's because you've made the agent so 604 00:32:55,140 - > 00:32:57,780 dumb that it won't be able to actually execute the use case 605 00:32:57,780 - > 00:33:02,455 there. It is a topic that is very alive for us, both because 606 00:33:02,455 - > 00:33:05,095 there's a grading methodology question in there and then 607 00:33:05,095 - > 00:33:08,695 there's this communications question. And we've not yet seen 608 00:33:09,175 - > 00:33:11,575 that enterprises fully acknowledge this. Enterprises 609 00:33:11,575 - > 00:33:14,695 would also like to see something spotless because something that 610 00:33:14,695 - > 00:33:18,370 is not spotless just asks like adds complexity and raises some 611 00:33:18,370 - > 00:33:22,050 of these questions, but we're hoping to be part of a push in 612 00:33:22,050 - > 00:33:25,170 the sector to acknowledge that a spotless audit report is 613 00:33:25,170 - > 00:33:28,290 probably not as valuable as a audit report that reflects 614 00:33:28,305 - > 00:33:30,065 reality, more more more clearly. 615 00:33:30,065 - > 00:33:33,905 Daniel: Yeah. That that's really helpful. Appreciate that. I hope 616 00:33:33,905 - > 00:33:37,585 you're inspired by the work that the AI underwriting company is 617 00:33:37,585 - > 00:33:41,185 doing and what we're talking about in this episode, really 618 00:33:41,185 - > 00:33:45,220 getting to a point where true enterprises can adopt agentic 619 00:33:45,220 - > 00:33:50,180 technology and actually have confidence in that technology 620 00:33:50,180 - > 00:33:54,820 and maybe eventually insurance around the risks associated with 621 00:33:54,820 - > 00:33:58,255 AI agents. That involves a whole lot of things. 622 00:33:58,255 - > 00:34:00,815 There there are a bunch of controls that need to be put 623 00:34:00,815 - > 00:34:04,975 into place. Everything from, yes, individual guardrails, but 624 00:34:04,975 - > 00:34:09,790 much more than that to how agents access MCP servers, how 625 00:34:09,790 - > 00:34:13,550 you manage supply chain and the risk associated with things in 626 00:34:13,550 - > 00:34:18,110 the supply chain around agents, how you handle observability and 627 00:34:18,110 - > 00:34:21,870 response to incidents. This can be really overwhelming, and 628 00:34:21,870 - > 00:34:25,615 that's why I'm so privileged to be working with an amazing team 629 00:34:25,615 - > 00:34:29,455 of AI engineers at Prediction Guard, where we've actually 630 00:34:29,455 - > 00:34:33,055 built an AI control plane that you can self host in your own 631 00:34:33,055 - > 00:34:37,215 infrastructure that allows you to treat AI agents that you're 632 00:34:37,215 - > 00:34:41,810 adopting with zero trust and these built in controls out of 633 00:34:41,810 - > 00:34:45,570 the box. I would love for you to take a look at what we're doing. 634 00:34:45,570 - > 00:34:49,890 Book a call with my team and I to talk through your individual 635 00:34:49,890 - > 00:34:53,810 implementation and how you can get up to speed rapidly and 636 00:34:53,810 - > 00:34:56,395 adopt this technology with full confidence. 637 00:34:56,395 - > 00:35:00,795 You can find out more at predictionguard.com/practicalai. 638 00:35:00,875 - > 00:35:04,875 That's predictionguard.com/practicalai. 639 00:35:06,200 - > 00:35:09,320 I I have another kind of selfish question because this is 640 00:35:09,320 - > 00:35:12,680 actually a response I get quite often when I'm talking to people 641 00:35:12,680 - > 00:35:17,160 about the systems that they're building. And, I I have my 642 00:35:17,160 - > 00:35:20,345 metaphor that I use that I would love you to critique, which 643 00:35:20,345 - > 00:35:23,065 might not be useful. If it's not useful, I need to use a 644 00:35:23,065 - > 00:35:26,025 different metaphor, but the the scenario is often they say, oh, 645 00:35:26,025 - > 00:35:30,425 well, we're building, you know, these agents or this agent, and 646 00:35:30,425 - > 00:35:32,025 maybe they're using AWS. 647 00:35:32,025 - > 00:35:35,140 Right? And so they're building some agents. They have some 648 00:35:35,140 - > 00:35:38,900 agent harness, and then they're plugging into some AWS bedrock 649 00:35:38,900 - > 00:35:43,220 models. And I'm talking to them about, hey. Well, like, when 650 00:35:43,220 - > 00:35:46,260 you're thinking about governance of these agents, the behavior of 651 00:35:46,260 - > 00:35:48,995 these agents, how you control that behavior, how you prevent 652 00:35:48,995 - > 00:35:50,195 bad things from happening. 653 00:35:50,195 - > 00:35:52,195 Like, how do you do that? And they're like, oh, well, that's 654 00:35:52,195 - > 00:35:56,595 easy. You know? AWS has, a content filter on their bedrock 655 00:35:56,595 - > 00:35:57,635 model. Right? 656 00:35:57,715 - > 00:36:00,755 And, to be clear, I'm not bashing on AWS. I think it's 657 00:36:00,755 - > 00:36:04,520 cool that they have a content filter. But I I I often use the 658 00:36:04,520 - > 00:36:10,040 metaphor of, like, my own health as a person. So I I say, like, 659 00:36:10,040 - > 00:36:13,720 well, is it bad for me to run a point check to, like, check my 660 00:36:13,720 - > 00:36:16,975 temperature? It's not, like, a bad thing, right? 661 00:36:16,975 - > 00:36:21,055 Like that's part of maybe being a healthy person is knowing if I 662 00:36:21,055 - > 00:36:24,335 have a fever or not, right? But it's very different from me 663 00:36:24,335 - > 00:36:28,335 being plugged into a healthcare system where there's electronic 664 00:36:28,335 - > 00:36:31,830 health records about my journey as a person, my health, my 665 00:36:31,830 - > 00:36:35,430 conditions, different from kind of having a comprehensive set of 666 00:36:35,430 - > 00:36:40,390 physicals and labs that were run that give different, you know, 667 00:36:40,870 - > 00:36:44,955 perspectives on on my health. Right? And so there's there's 668 00:36:44,955 - > 00:36:48,315 this system that I'm plugged into, there's a process, there's 669 00:36:48,315 - > 00:36:54,635 policies around that, there's, and that's, in in my mind, 670 00:36:54,635 - > 00:36:57,995 that's much more of kind of the perspective that people need to 671 00:36:57,995 - > 00:37:01,860 go to is not so much like, hey, I have a prompt injection 672 00:37:01,860 - > 00:37:03,060 filter. Right? 673 00:37:03,060 - > 00:37:08,340 And that's my strategy. But more this kind of comprehensive view 674 00:37:08,340 - > 00:37:11,620 like you would have of your own health as a person, but now 675 00:37:11,620 - > 00:37:15,345 we're talking about like the the health or behavior of a of an 676 00:37:15,345 - > 00:37:18,785 agent. I don't know. Any any critique on that or or 677 00:37:18,945 - > 00:37:21,425 Emil: know, I I actually I I really like that analogy. I've 678 00:37:21,425 - > 00:37:25,185 not thought about this this one before, but I think our 679 00:37:25,185 - > 00:37:28,980 quarterly red teaming is is very much alike to the doctor's visit 680 00:37:28,980 - > 00:37:32,260 where you go from head to toe. You go through the MRI scanner, 681 00:37:32,260 - > 00:37:36,660 you go through the blood testing, you like everything 682 00:37:36,660 - > 00:37:40,100 that Elizabeth Holmes tried to prevent with Theranos, we will 683 00:37:40,100 - > 00:37:44,775 do to you and we will do it 10 times over. And in between, the 684 00:37:44,775 - > 00:37:46,775 beauty of the standard is there's obviously a lot of 685 00:37:46,775 - > 00:37:50,615 runtime controls in there. So we will ensure then that you do 686 00:37:50,615 - > 00:37:53,335 still take your temperature every day, in fact, probably 687 00:37:53,335 - > 00:37:56,055 every minute, so that alerts are configured if something 688 00:37:56,055 - > 00:37:57,095 immediately goes off. 689 00:37:57,095 - > 00:38:01,500 We will also have you lock your system behavior right. So if 690 00:38:01,500 - > 00:38:04,620 something goes awry and you don't understand it, you can go 691 00:38:04,620 - > 00:38:08,140 back and see what is the observability then and go in and 692 00:38:08,140 - > 00:38:11,100 explain it. So I think it's basically the perfect analogy 693 00:38:11,100 - > 00:38:13,835 between the red teaming is a doctor's visit we do every 694 00:38:13,835 - > 00:38:16,875 course and we do that very comprehensively. And in between, 695 00:38:16,875 - > 00:38:20,635 we make sure that we check your vitals every minute and there's 696 00:38:20,635 - > 00:38:24,795 an alarm going off if there's something off and then maybe 697 00:38:24,795 - > 00:38:28,075 adding a healthy diet to it in the first place as well, 698 00:38:28,075 - > 00:38:32,630 ensuring that the inputs and outputs of the system are are 699 00:38:32,630 - > 00:38:34,470 working well, so not too much junk food there. 700 00:38:34,470 - > 00:38:38,710 Daniel: Okay. Yeah. I like, even now I I I have some revisions of 701 00:38:38,710 - > 00:38:41,670 my metaphor I'll use based on your response. I think that's 702 00:38:41,670 - > 00:38:45,685 great. But yeah, I I think the other thing that might like we 703 00:38:45,685 - > 00:38:49,845 have some developers listening, Maybe people developing agents 704 00:38:49,845 - > 00:38:54,325 actively and it it might be somewhat overwhelming to, for 705 00:38:54,325 - > 00:38:59,470 example, look at the, you know, AI UC has like a evidence, page, 706 00:38:59,790 - > 00:39:02,510 right, where I can see all the things maybe I should be doing. 707 00:39:02,510 - > 00:39:06,670 How do you see the market evolving in terms of like, 708 00:39:06,670 - > 00:39:09,950 obviously, you have one side of this which is really related to 709 00:39:09,950 - > 00:39:16,055 the standard, the certification, maybe eventually that insurance 710 00:39:16,055 - > 00:39:20,615 side, but then it can be, you know, an individual developer of 711 00:39:20,615 - > 00:39:24,935 an agent might not be an expert in agent security or how to 712 00:39:24,935 - > 00:39:27,415 govern these things, that sort of thing. So that might seem 713 00:39:27,415 - > 00:39:30,990 overwhelming to them. That you mentioned partners like 714 00:39:30,990 - > 00:39:33,870 auditors. There's the infrastructure layer. What what 715 00:39:33,870 - > 00:39:36,430 do you think I I guess my question is what do you think 716 00:39:36,430 - > 00:39:42,430 needs to be in place to, to actually enable real world 717 00:39:42,430 - > 00:39:46,315 developers to meet some of these standards, whether that be 718 00:39:46,315 - > 00:39:51,355 evolution in the tooling or, obviously understanding maybe of 719 00:39:51,355 - > 00:39:54,715 the the the, standards. 720 00:39:54,715 - > 00:39:56,715 I I don't know. Does the question make sense? 721 00:39:56,715 - > 00:40:00,770 Emil: Yeah. Yeah. No. Absolutely. I think where we are 722 00:40:00,770 - > 00:40:04,530 right now is we're just overwhelmed by how positively 723 00:40:04,530 - > 00:40:07,490 AAT one has been received as we're really busy just 724 00:40:07,490 - > 00:40:08,770 delivering certifications. 725 00:40:08,770 - > 00:40:12,450 So that means we've less time to talk to some of the many, many 726 00:40:12,450 - > 00:40:15,330 fantastic partners who come into our inbox and want to partner 727 00:40:15,330 - > 00:40:19,545 with us. I think there's three things we need to get right for 728 00:40:19,545 - > 00:40:24,505 this to work. I think we need to continue pushing for code and 729 00:40:24,505 - > 00:40:27,465 Neogenetic products that come out of the gate as secure as 730 00:40:27,465 - > 00:40:30,760 possible by default. We're certifying our first coding 731 00:40:30,760 - > 00:40:35,240 agents right now and we are certifying both a well lovable, 732 00:40:35,240 - > 00:40:40,200 which I think will be certified when this episode comes out, and 733 00:40:40,200 - > 00:40:44,565 then another very large coding agent that may or may not have 734 00:40:44,565 - > 00:40:49,445 been acquired recently for a lot of money without naming names. 735 00:40:50,245 - > 00:40:52,805 Working with the coding agents layer and the platforms where 736 00:40:52,805 - > 00:40:55,125 you go in and configure code, we're also, like again, UiPath 737 00:40:55,125 - > 00:40:57,605 is a good example where you don't just like have one agent, 738 00:40:57,605 - > 00:40:59,285 but you actually go in and build agents on top. 739 00:40:59,340 - > 00:41:02,540 Means that it'll be easier to meet a lot of the standards just 740 00:41:02,540 - > 00:41:05,660 by default because the environment where you define and 741 00:41:05,660 - > 00:41:08,780 build your agent is secured by default. So I think that's step 742 00:41:08,780 - > 00:41:11,580 one and we're going to do more of that work with some of the 743 00:41:11,580 - > 00:41:15,345 big agentic platforms out there very soon. Think some of it will 744 00:41:15,345 - > 00:41:18,945 be announced in the fall. The second stage is we need this 745 00:41:18,945 - > 00:41:21,185 partner ecosystem you just talked about and we're already 746 00:41:21,185 - > 00:41:25,585 starting to come out with some examples of this where a partner 747 00:41:25,745 - > 00:41:31,220 meets, helps companies meet a good chunk of controls. So a 748 00:41:31,300 - > 00:41:34,020 company like White Circle, which I think is very cool, we don't 749 00:41:34,020 - > 00:41:36,820 work with them at all, so just a shout out. 750 00:41:36,820 - > 00:41:40,020 Their monitoring and filtering work is really good and has 751 00:41:40,020 - > 00:41:43,005 helped companies meet a lot of the safety requirements in our 752 00:41:43,005 - > 00:41:46,605 standard. So that is like one platform you integrate and you 753 00:41:46,605 - > 00:41:49,565 immediately meet say eight or 10 of the requirements in the 754 00:41:49,565 - > 00:41:53,565 standard. There are many other platforms. We're doing some work 755 00:41:53,565 - > 00:41:57,110 right now with Credo, with Witness AI, and there's, again, 756 00:41:57,110 - > 00:42:01,750 tens of others. So having that ecosystem help companies meet 757 00:42:01,750 - > 00:42:03,430 the controls, I think is important. 758 00:42:03,430 - > 00:42:06,150 And where we've already gone in and done our best to help 759 00:42:06,150 - > 00:42:08,070 companies meet the standard is we've gone in and actually 760 00:42:08,070 - > 00:42:11,385 defined the typical evidence we see companies upload. To see 761 00:42:11,385 - > 00:42:14,265 that as your guidance for where to look for the right 762 00:42:14,265 - > 00:42:16,665 approaches, we don't just define the controls and leave it up to 763 00:42:16,665 - > 00:42:19,385 you to figure out how the hell to implement it. We actually try 764 00:42:19,385 - > 00:42:22,905 to give you the guidance as well on how we see companies do it 765 00:42:22,905 - > 00:42:25,900 today. I think the third and final stage we need is obviously 766 00:42:25,900 - > 00:42:29,260 making it easier then to go through the certification 767 00:42:29,260 - > 00:42:32,700 itself. So we're already integrating the framework in the 768 00:42:32,700 - > 00:42:34,300 leading GSE platforms. 769 00:42:34,300 - > 00:42:39,355 We're making it easier for our auditors to capture as much as 770 00:42:39,355 - > 00:42:42,475 the evidence programmatically. So we move away from screenshots 771 00:42:42,475 - > 00:42:46,315 and into like real validation that the controls work and hold 772 00:42:46,315 - > 00:42:50,715 up in real time. That layer and like the whole GSE engineering 773 00:42:50,715 - > 00:42:53,730 space is just really interesting to follow right now. And we're 774 00:42:53,730 - > 00:42:57,570 doing our best to keep up and make the standard work for the 775 00:42:57,570 - > 00:43:01,650 GSE engineering community as well. So that when your re audit 776 00:43:01,650 - > 00:43:06,625 comes in that next year, that it's a very limited time 777 00:43:06,625 - > 00:43:10,465 commitment we need from you and that we can focus on security 778 00:43:10,465 - > 00:43:11,665 instead of compliance. 779 00:43:11,665 - > 00:43:16,145 Daniel: That's great. Well, Emil, it's been amazing to look 780 00:43:16,145 - > 00:43:20,910 at what the AI UC has done even in the past year and the amazing 781 00:43:20,910 - > 00:43:23,630 resources that you put out for the community. Thank you for 782 00:43:23,630 - > 00:43:26,670 doing that. Thank you for working towards the future that 783 00:43:26,670 - > 00:43:30,430 you described. Would love to have you or others, back on the 784 00:43:30,430 - > 00:43:32,705 show in the future as things develop. 785 00:43:32,705 - > 00:43:33,425 Thank you so much. 786 00:43:33,425 - > 00:43:36,305 Emil: Thank you for having me, Daniel. And I think maybe just a 787 00:43:36,305 - > 00:43:40,705 final plug before I I head off. This is work made by industry 788 00:43:40,705 - > 00:43:44,065 for industry. I'm I'm luckily not alone in in doing this this 789 00:43:44,065 - > 00:43:46,910 work. We've collected a consortium of about two fifty 790 00:43:46,910 - > 00:43:50,750 leaders across CSOs and the Fortune 1,000, it is the 791 00:43:50,750 - > 00:43:52,750 security engineers and so forth. 792 00:43:52,750 - > 00:43:55,470 And having that community come together and actually leave 793 00:43:55,470 - > 00:43:58,750 competition aside for a moment and recognize that the size of 794 00:43:58,750 - > 00:44:01,685 these challenges and the pace of the challenges just dictates 795 00:44:01,685 - > 00:44:05,445 that industry has to come together is fantastic. The fact 796 00:44:05,445 - > 00:44:07,525 that we've been able to just offer the platform and then let 797 00:44:07,525 - > 00:44:10,805 industry work together to define and codify these standards is 798 00:44:10,805 - > 00:44:14,005 fantastic and we would love to see more people get into the 799 00:44:14,005 - > 00:44:17,340 machine room with us. An open invitation for everyone who's 800 00:44:17,340 - > 00:44:20,780 excited about this work, either to help us drive adoption of the 801 00:44:20,780 - > 00:44:23,660 standards we see work or actually help us help us write 802 00:44:23,660 - > 00:44:26,060 them. My pleasure. It was a great conversation. 803 00:44:26,060 - > 00:44:27,020 Daniel: Yeah. Thank you, Emil. 804 00:44:31,535 - > 00:44:34,815 Narrator: Alright. That's our show for this week. If you 805 00:44:34,815 - > 00:44:38,975 haven't checked out our website, head to practicalai.fm, and be 806 00:44:38,975 - > 00:44:42,335 sure to connect with us on LinkedIn, X, or Blue Sky. You'll 807 00:44:42,335 - > 00:44:45,950 see us posting insights related to the AI developments, and we 808 00:44:45,950 - > 00:44:48,830 would love for you to join the conversation. Thanks to our 809 00:44:48,830 - > 00:44:51,550 partner Prediction Guard for providing operational support 810 00:44:51,550 - > 00:44:52,350 for the show. 811 00:44:52,590 - > 00:44:55,790 Check them out at predictionguard.com. Also, 812 00:44:55,790 - > 00:44:58,464 thanks to Breakmaster Cylinder for the beats and to you for 813 00:44:58,464 - > 00:45:01,344 listening. That's all for now, but you'll hear from us again 814 00:45:01,344 - > 00:45:01,904 next week.