Bridging the gap in AI Governance

Speaker A: Hello, world.

Speaker B: This is Better Tech, a podcast where we chat with some of the most successful leaders about the latest industry developments. So join us as we explore the world reliance on tech.

Speaker C: Hello and welcome to, uh, Better Tech. I'm Jocelyn Houle, your host. I cover our data and AI beat here. Um, it's great to welcome Katharina Corner, who is the senior principal consultant at, ah, Trace three, uh, and a specialist in AI governance. Welcome. Katerina.

Speaker A: Hi. Thanks so much for having me. Really happy to be here today.

Speaker C: It's great to have you here. We haven't seen each other in a while, but we've actually connected a couple of times professionally. I've always been so impressed with your work and today we're going to be talking about bridging the gap in AI governance. That's a place that I've spent a lot of time focused as well. Um, but before we get there, let me just ask real quick, um, what is Trace three and what does a senior principal consultant in AI governance do there?

Speaker A: So thanks for the question. So Trace three is actually quite big. I'm the principal consultant in the AI Governance, AI Risk and AI Security team. We are about six people so far, uh, and growing. And Trace three was traditional, uh, it's a tech solution, um, consulting company. But we have scaled our um, consulting services, um, like a lot. We have uh, AI Consulting group with AI strategy consulting, AI, uh, solutioning consulting, AI governance consulting. So we are really, um, you know, can cover it all and 1500 people based in Irvine, California and covering all of the US with like, uh, great sales teams in a lot of, um,

Speaker C: local places, uh, in the different geographies. Yeah, you know, it's great to reconnect with you. The more that, you know, I focus on this area quite a bit. I come from the software development side and I always want to cure all the problems with software. But increasingly it's clear to me as I work with customers, it's such a complicated, intertwined problem that you need technical, behavioral change management, design. Um, there's so many components to adopt safely, adopting, uh, these AI solutions that um, I'm really interested in talking to you a little bit about what you're hearing from customers because they need consulting that is like multi, um, modal and not just, oh, let's do a software solution or oh, let's have a policy in place. And I, I know you agree with that because we've talked about that before. But, um, before we kind of get into that part, let's kind of, for our audience talk A little bit, um, the maturity gap in AI governance. Let's start with what is AI governance? And then like give us an idea of the AI governance landscape.

Speaker A: Um, yeah, happy to. So AI Governance. What is AI governance? That's really a very good question that is actually still discussed. Um, we see great um, research and overviews coming out on this question just recently I think by Stanford and others because there is still this is AI governance, is it all about regulations, AI related law or is it a um, nice to have or a good governance like ah, um aspirational um thing, um ethics, et cetera. It's both ends. But we should really distinguish here because we have hard law and then we have these good governance principles that also fall under AI governance, AI related laws uh, that we have to uh, adhere to. This needs to be implemented and operationalized by AI governance. On the other hand, AI governance is also inspired or um, covered by frameworks like NIST AI Risk Management Framework or ISO IEC 42001 for AI management systems. And there are other frameworks like uh, in Singapore uh, and other places they also published great advice but those are more oriented towards what does the organization define for itself, which principles does it want to follow in regards uh to AI. So those two things and the outcomes of AI governance are usually like you mentioned before, uh some policies, AI risk management policy, acceptable use policy for AI, um, AI intake policies or a set of very classical policies are um, um risks. What are the AI risks for my organization? So AI risk register and on the other hand we have uh training components so training staff on risks and opportunities and more um, not paper based but more procedural um efforts all with the goal of course to minimize the risks that AI um systems can bring uh, to the organization, to uh society, to um individuals while tapping into the opportunities that AI can provide. So this is in all the frameworks, in all the approaches, I would even say in all the laws. This is like this dual approach that AI governance um, should help with.

Speaker C: Yes, I love that you're talking about that. I think regular. Even if you work in any of these businesses, like I've worked a lot in finance until you really get into the governance component, you don't understand how many layers of risk management are required. It's really interesting. I forgot about the risk register. Of course you just mentioned that and uh, I've dealt with that in the past. Um, so you know everybody says they want to do a governance. Everybody has their, their intent in the right place. Everybody wants their stakeholders and shareholders to know that they're doing AI governance. What have you seen as kind of the big gap though from like what you say you want to do aspirationally and what people are really implementing today?

Speaker A: Um, it has already been coined a few years ago this term of um, AI governance gap, but we still are struggling with it. Of course AI is, there's no stop in um, new um, AI system capabilities coming out. So the gap is hard to close when the target is constantly moving. But I would say some of the biggest maturity gaps in AI governance that we're seeing is really very simple, straightforward, maybe similar to security and privacy in general. Um, how do we turn those high level principles into enforceable monitored practices? So the operationalization of AI governance, AI governance in this sense meaning all those principles that are um, part of AI governance, uh, privacy, security, transparency, explainability, accountability, robustness, safety, etc. So how do we really put them into practice then? Internal accountability structures? Because AI governance, uh, or AI actually is so cross functional, right? Everyone is involved, like the end user, like my colleague, like staff in general, then legal and IT and strategy, um, and product management. Like there's so many, many, many players, um, that this uh, question of who is accountable, who is the risk owner, for which cases do we need to escalate AI system, um, use it to some committee and when do we not have to do that? So that's pretty tricky as well. And then I think the third one maturity gap is really that you do not, even if you have an AI intake process already and you do a risk assessment at the beginning if necessary, if it crosses a specific threshold, uh, then this ongoing monitoring of the system is a challenge too because AI is now omnipresent. But the resources, staffing, uh, and tooling is usually not quite yet there. To really have this plan, do check, act and continuous monitoring that we would need to have in place.

Speaker C: And it fights you, right, because it's not deterministic. The outcomes are very hard to tie back to the original inputs. Unlike the old days of data warehouse where you, it might be painful but you could look back from say a credit decision all the way back to where we got that data from TransUnion or something like that, using a financial example. But in the world where AI is perhaps making a decision about who gets a product offering, who gets offered a certain credit line, um, by the time that thing's in production it's out of your hands. And so it's always been important to have governance of data and decisions and machine learning models certainly. I mean I'm speaking I know finance the best, but certainly we know it's super regulated. You have to have visibility. Um, there's a technical dropping off point where you um, don't always know with these AI models, um, you can't backtrace, uh, necessarily as conveniently as you could have, uh, perhaps in the warehouse world.

Speaker A: Um, I love that you're saying this actually because while, uh, I think it's not incorrect that plenty of people say AI is just software, it's also not completely accurate. Right. This is why NIST AI Risk Management Framework has this appendix saying uh, how does AI risk differ from traditional software risks? And it's uh, exactly due to those things that you mentioned, like probabilistic nature, new data coming in. So I always like, uh, I don't know, I feel a bit uncomfortable. This, you know, um, we have a conversation at the end, it's like, oh, you know, it's just software. Well, that's why we even have this new field. Exactly. Management.

Speaker C: I think people are soothing themselves a little bit. Right. I always hear people, it's just a prediction machine, it's just software. And I get that, uh, that's not untrue, but still a different situation. Um, and so, um, so I do see a lot of highly regulated industries. And so let me just backtrack here. And also for listeners who don't kind of know the tradition, governance and risk management world, uh, basically, uh, I'll just use data as an example. And AI is kind of a framework for. This is like all of us who work at the company are in the first line. We're all responsible for making sure customers get taken care of and we do ethical things. Second line, probably like in your division, that person is responsible to make sure the division sort of organizationally pushes down the right, um, and sort of unified governance tactics. And then like third line is like usually an standalone organization that at an enterprise level is setting the policy, understanding governance, writing the big risks up to the audit log in the sky that your, you know, your governmental auditor may look at and dealing with your auditors. Um, that's very loose, uh, uh, inadequate description. But AI uh is following the same general pattern, right, of having use cases approved at the organizational or second line level. And then these third line organizations are creating some sort of policy based on international national standards and pushing those down through the organization. Is that about kind of how it's working today or we'd like it to work?

Speaker A: Yeah, I would say that's accurate, yes.

Speaker C: Great. Um, so the interesting part here though is that you have people involved in this whole process, and it doesn't always, um, move the needle on risk management and on reducing the risk of AI, uh, implementations. And so I'm just curious, because you're working with customers and people doing this today. Do you have examples and you can anonymize it? Do you have examples of companies who are effectively navigating all this and closing the AI governance gap, uh, better than others?

Speaker A: I think we are really at this, uh, inflection point or turning point where companies are really scaling up and those we, uh, can we support usually already have something in place. So I think that's awesome. Uh, so if you as an organization have an Excel sheet, I mean, if you pick an Excel sheet, you'll probably not suffice. But, um, any kind of AI use, case inventory, or if you develop your own models, any kind of, okay, I have an overview, or I'm starting to have an overview of my models. That's usually what's happening already. But then it's really about, like, how do I, uh, set this up in a way that I can scale this? Like, what's my tech and tool support like? When does, which team come in? How does a risk score maybe get calculated? And how are thresholds defined? Like all those nitty gritty details that make a big difference when you want to scale because you create a bottleneck very quickly. If you do the same kind and type of risk assessment for all your AI use cases that come in or are requested by staff, you might want, uh, to say, hey, I only do AI risk assessment, um, piggybacking, kind of on the privacy impact assessment if there's personal information involved or if the AI system is contributing to a consequential decision. Maybe I'm like, um, falling under the Colorado AI act, or, uh, if some sensitive specific data or a specific context or a specific AI system or so. So I think this definition of thresholds is something that I have not yet seen, that it's so super widespread. So.

Speaker C: But it is useful. It has been successful. Yeah, I love that, um, as a good starting place. And I love that you even said Excel spreadsheet just to get started now. Um, because, um, let's assume I'm like the CEO of a multinational bank, and I come to you and I'm like, hey, Katerina, we got to put some AI governance in place. Uh, you know, tell me how, um, you know, based on your consulting work, where's the most common places in that huge laundry list of things you have to do? Where are the Common places where organizations really get stuck. So you're the CEO, like I'm running this thing, like what are the red flag areas in your mind of places where AI governance gets stuck?

Speaker A: So um, I think if you would have asked me this one or two years ago, I wouldn't have given the same answer. But by now as uh, tools and platforms for AI governance and AI security have evolved so much, I would, in this specific case, I mean of course I would say, hey, what do we have? What's the profile? What's the target state for us, what's our risk appetite, what's the risk tolerance? So that we have a kind of overall framework where we are and where we want to go. But in this case I would actually go for ah, tools, tooling, um, shadow AI detection tools. And I'm pretty excited about this actually because I mean there's so much AI use nowadays in organizations that is not tracked at all and we have no visibility into it. So with those new tools you can really get all the info like what's going on in your network. You can uh, integrate with Cscaler or with Palo Alto Networks, you can have your APIs tracked, you can have uh, a browser extension, what's happening there. You can have user behavior analytics and all, all of this, um, put together in kind of a neat dashboard where you see which user uses which tools, which are approved, which are not approved, who should access those and who shouldn't. So you can actually operationalize um, your AI policies, acceptable use policy. And I think what I find, um, additionally exciting in general, I love that, um, I like the thought and I love the thought and I wish it would be leveraged more that AI governance can be used for AI strategy, right? So it's not, not only about punishing people. Oh my God, it's terrible, right? But hey, what are they using it for? Are they. And then is it really for work? If it's for work and a lot of people are using this or we see then, then let's, let's embrace it, right?

Speaker C: There's bad actors and good actors, right? And they can both make mistakes. They can both do the wrong thing for the wrong right reasons. Um, and so I definitely agree with that. One of the smartest uh, leaders I talked to in AI about a year ago actually created a lab where you could, like a sandbox where you could just mess around and basically do shadow AI with some oversight. And I was like, that is the most genius thing ever. Because when people are hiding, that's the part you don't want you. Don't you want it to be brought to the fore? So I think using AI, uh, governance, I love that. And tools to just shine light on it and give people the ability to use AI safely. Um, I hope that more companies take that on because this is really too big. This isn't just like, oh, I'm, you know, checking my sports gambling site, uh, from work. This is too big. Ah. And too much, um, proprietary data is flowing out of the company potentially. Um, so, yeah, that's really interesting. And also being on the tech side, I love what you're saying because you just meet with the CEOs and they're like, we're gonna check every piece of data before it goes out. And then secretly you're just feeling like, oh, I don't know how we're gonna do that.

Speaker A: Absolutely. But. And, uh, the people component that you just mentioned is also, of course, super critical because I also see that often it's not 100% clear to people, to staff. What can I use? Where do I have to request usage? How fast will this, uh, be decided? Like, all those just clarity, clarity. When can I use what for?

Speaker C: Honestly, you bring up a great point. I've been doing this a little too long. And I will tell you, most people try to do the right thing, like, even more than I might try, I think. I look at how hard they're working to comply with the internal rules of the company. And so I do think 99% of the time, people will do it if they just know what they're supposed to do. Um, not to be too much of a Pollyanna about that, but I think people do want to be good actors for the most part. So let's talk. I don't know if I pushed on this enough in our first kind of discussion. Uh, the first chunk is like, what is that maturity gap in AI governance? And, like, who have you seen kind of working their way through it, know, with some success? And I love this answer. They're adding transparency now. They're tracking the use now. And they're not punishing people. Um, do you think? I just want to make sure. For our, you know, listeners, let's just briefly chat about the real world challenges, like, what are. Why do I need AI governance? Like, what could be the problem? I know it sort of seems obvious to you because you sit with this all day. Um, but, you know, a lot of people sort of see recent legislations kind of eased up. We're not as concerned as we once were. Um, you know, why you Know, I'm the, I'm the CEO, once again coming to you and saying, Katerina, like, where does this fit? How risky is it? Like, what are the risks? What are the risks of inadequate AI governance?

Speaker A: M. Yeah, I mean, it depends. It really depends on the organization, right? So, I mean, your question is very good and very high level. And nevertheless, I'm now thinking of something now very hands on with this threat modeling. So. Because how do I even understand what could go wrong, right? I can just take, uh, go to MIT AI Risk Repository, which is tracking over a thousand AI risks, AI related risks, and you're just like, okay, I don't know where to start. So we have to have means on various levels to assess what are the risks actually for us.

Speaker C: I do think that, uh, leaders should educate themselves on the type of risks. I think sometimes it's seen as just a policy risk or a compliance slap on the wrist, um, or paperwork problem. But, you know, there's like truly bad actors out there who can use prompt injection to poison your data. They can, uh, perhaps trick the AI into, uh, saying and doing things on behalf of the company that are not what you want at all financially or from an ethical or, um, branding standpoint. So, um, I love that you mentioned that MIT link, because I do think, um, there's a little bit of hand waviness about the risks of these downstream AI applications because they're fun to use, we're all using them in our regular lives. But when you explode that out to a corporate level, um, there are many risks. And so, um, you got to address those now. I'm going to kind of switch to the other side of the table. So you're like the vice president of a line of business, and the last thing you want to hear is, hey, I need to put resources behind a new AI governance program. So, um, what are some ways that, you know, you see as good ways to integrate governance and these product workflows, the business workflows? How can organizations integrate AI governance into the, into the business and the product workflows without slowing down the whole institution? What have you seen work? What's your opinion?

Speaker A: So I would say, um, when there's, when we have achieved awareness on AI risks, in particular, for example, we are an organization that really makes decisions that are impacting people like credit, um, approval, or do you get, uh, the job or do you get the apartment? And if you have issues here like that, first of all, you don't know why the AI system that you use to support those decisions even gets to this conclusion, that's a big problem. You need to have this transparency to the customer that you're using AI and the explainability to at least maybe via uh, counterfactual explanation. So if something changes then this would be a different outcome. So I can kind of determine how the AI system came to this conclusion, even if it's a black box system where we don't really know what is going on inside. So I think when we are there that via examples like this you don't want to be um, on the top of the headlines or news next week. Right. For any such thing. And of course there are many, many, many real world examples. The entire uh, um, Dutch, um, uh, government had to step back three years ago because the AI system four years ago because the AI system they were using for social uh, um, support, Social Security support was just completely discriminative. So we can via real world examples really get there I think to this awareness that this is a thing. And then what do we do? Well, I mean both of us, we come from privacy or at least part of our past work has been privacy. We have met on the Privacy Engineering board by the ipp, uh, which is all about implementing privacy into your um, products, services and infrastructure. So privacy by design or privacy by default, which was already piggy backing on security by design and now we're there that we just need AI governance or AI risk mitigation or responsibly, ethically trustworthy, if you wish to call it that. Um, mhm. By design or by default. And my opinion is that the AI product manager is like really at the center here and because he or she or them is the person who needs to bring all the stakeholders, all the subject matter experts together and bring them in at the right moment, meaning at the very beginning of the um, product development lifecycle or when I purchase an AI tool that I from the very beginning ask the right questions. Uh, procurement is. So we need this training like you said before for leadership but also for those roles that either develop products or purchase products. Mhm. To educate them or us on new risks that the AI brings.

Speaker C: Is that which is meant by shift left AI, yes, this is what I

Speaker A: would call a shift left. So we had this shift left in security meaning to the beginning of iteration phase of any product, uh, development. Not like bolting on security considerations at the end. Oh, we are almost tripping now security. Can you please give your okay or something? Of course this is not realistic and not sustainable. And the same we saw in privacy. There was also this with the gdpr, the general data Protection, um, regulation. In Europe there was this term privacy by design. I mean it was coined as you know as well by N. Kevukian long before who was in Canada. But so these concepts of really doing the right thing from the very beginning in a technical sense also. Because how can you achieve data minimization for example in privacy? Well, you could anonymize data or de. Identify data and nowadays you could consider differential privacy or synthetic data. So those principles need to be operationalized and brought to life. Right. The same with explainability, robustness, all those things. You have to have the right people, processes and tools in place to even um.

Speaker C: Yeah, no, I've been a product manager and I think that's really um, I believe it's almost old school when we used to have to do systems engineering early in product development that got separated away when AWS and cloud came along. And now I think a lot of people think the beginning of the product process is talking to customers, customers about what they want and that's fine. But now with AI, if you're building an AI product you still, and you're right to bring in cyber. If you're building an AI product you're in corporate, you really have to start a bit earlier with what are the non functional requirements perhaps or even the functional requirements related to keeping things private. Um, governing AI properly downstream for transparency. Um, because there, there kind of is a critical moment where you lose the opportunity to get that transparency.

Speaker A: Yeah, totally agree.

Speaker C: Yeah. Um, so anyway product managers are listening. Your job starts earlier. Um, so it is hard though I do think there's a tension that I've been stuck in YouTube probably between balancing the speed and responsibility. Do you, can you think of any companies who are doing a good job with that or any um, anonymized or by name? You've seen good examples of companies able to do that.

Speaker A: I mean when I was uh, thinking about this question ahead of time, I am sorry but really what came to mind was Microsoft. So it might not be like a niche example or something but they were early.

Speaker C: That's great.

Speaker A: Yeah. With so much guidance and such a professional approach to the entire realm of AI governance, uh, so much guidance and open source tools also that they uh, outsourced and that you can tap into just I think last week they came up with a uh, new AI risk assessment or security assessment like report. So they're constantly publishing something and it's

Speaker C: high quality, high quality.

Speaker A: And also for red teaming they outsourced um, their um, tools which is like One of the number one, I uh, would say mitigations that you need to uh, practice uh, for generative AI tools. So red teaming is really big. Um, not so much in practice yet. But a lot of security teams are actually coming to us and saying hey, um, AI risk might not be entirely in our sphere, but of course with security or we are maybe even the pen testing team and we want to know, we want to proactively understand what's our part. Is there a part that is our part? And then it's about like.

Speaker C: And do we get any budget?

Speaker A: Do we get budget? Yeah. And do I have to pass for bias and hallucinations as well? Because that's actually not really security. And then you know, it's like so hard.

Speaker C: It's early days. There are these, a lot of these conversations I think leaders, if you have any CEOs listening, like that would be the thing is leaders, I think it's very helpful. Don't turn this into a committee operation. Somebody has to at the end of the day kind of own this thing. It's either data or security or compliance and governance. Um, because you do have a lot of people who then executives get in a room and they're confused. Um, but uh, we can't say solve that problem on this discussion. But what I did want to talk. What is red teaming? I hear that all the time and I thought I knew what it meant. Now I feel like maybe I don't.

Speaker A: Yeah, it's basically ah, stress testing your um, in this case LLM M most often LLM M Specific um, applications for specific threat uh, scenarios. So red teaming is uh, the next step after threat modeling. So what could possibly go wrong? What would really be the, the highest risks that would make a most negative impact on our organization or uh, end users etc. And then test for it. So if, if uh. Okay, it would be really bad if the chatbot um, would as we had this example. Right. Some airline I think selling cheap tickets or Chevrolet or something.

Speaker C: Oh yeah, I just heard this. Yeah.

Speaker A: So. Okay, we don't want that. Okay, we don't want that. So now I'm, it's just very simply said sitting there and trying to get the chatbot to do that.

Speaker C: Okay. So it's just cracking break, trying to break things and figure out why. Okay, um, that's what I thought it was but I just wanted to double check. Um, and it's uh, it's got this sort of multi business perspective. Uh. Right. It's not just hey, we're trying to break it from the perspective of cyber, but we're going to try and break it from the perspective of branding and IP and privacy. Um, the other kind of term I wanted you to help us understand a little bit is like what is shadow AI exactly. And a company.

Speaker A: Mhm. Shadow AI is any AI that is not approved. So that is used in the organization with data, presumably with data that shouldn't be used. But I mean even if you just use the ChatGPT, even though only Copilot is approved, that's shadow AI. And of course especially when company data, IP data, sensitive data, personal information, some, I don't know, processes, workflows, whatever company specific are put into those tools, it's ah, an incredible risk. We don't even know they're in use, we don't know where the data goes. And if it's free tools then usually you cannot even turn this off that they use the data for training, et cetera and could potentially pop up, up um, somewhere else. So that's that. And this is why I'm so excited about those new tools, uh, like shadow AI detection tools.

Speaker C: Hm.

Speaker A: But also we have a big because you asked for ah, about like what are we seeing. And I find very interesting that many organizations are now getting really interested and we're having a couple of DSPM projects. Yes, let's talk about that.

Speaker C: What is DSPM in your mind? Let's uh, define it. It's kind of new terminology.

Speaker A: Yeah, it's new uh, terminology but it's not really new in terms of uh, what it is. Very interesting. So data security, uh, posture management is if I say it very simply is uh, scanning. Where is your data, what kind of data? Label the data, who is accessing the data. It's not really tracking flows in real time, but it's almost real time uh giving answers to those questions across your cloud environment on PREM or SaaS. And uh, uh, this tooling has actually evolved from privacy tech tools. So we see vendors in the data security posture management space and there are plenty of them now and big ones and really really really mature ones that uh, came from uh, privacy, ah, because they were starting with code, uh, scanning

Speaker C: for a pii, uh and they have to have a map of assets, right?

Speaker A: Yeah. And so where is PII in my organization? But it's not only about pii. Right. There is a lot of information and data that we want to know where it is, who uses it, who accesses it, are the access controls in place or not. So just this visibility, this basic visibility you can achieve with A DSPM tool. And yeah, I'm also pretty excited about this.

Speaker C: It is exciting, I think just to give, uh, for audience members too, a little context, which is, you know, cloud data just happened. It's hard to remember because AI is happening. But in the cloud data world there were tools for cloud for on prem, so it wasn't really clear how to figure out where your vulnerabilities were all at once. Anyway. Um, and so a lot of these DSPM tools, in my mind what they do is they give you this intersection of where you're vulnerable potentially and how much you care about that vulnerability. Because if you have a server facing the Internet and there's nothing on it, I don't care as much if it's faced. It has a huge security problem and it's tons of customer data. Right. And so to me, that's been the missing piece of remediation. Uh, how do I know what to fix first? Uh-huh.

Speaker A: Yeah, it's really, uh, informing my security of posture strategy. Right?

Speaker C: I think so. And it sounds funny when you say it out loud, like all good ideas. You're like, oh, doesn't that exist already? But it didn't, it really didn't. Uh, and so it's an exciting time. And I think owasp, um, has like a framework around this that a lot of companies are using. Can you talk a little bit about the OWASP top 10?

Speaker A: So, yeah, that's uh, I would say, um, not directly related uh, to dspm. But of course any DSPM solution can address some of those, uh, top ten vulnerabilities. So OWASP has maybe already two years ago and now we have version two published. Ah, top ten for large language models vulnerabilities. I mean, I think it still yet has to reach widespread use. Um, it's very well known. I am not 100% convinced that organizations, uh, are really utilizing it yet because AI security is such a tricky and huge field. And AI security, I mean security teams in general, they are like very specifically trained for classic security, um, challenges and usually I would say in many cases also already understaffed.

Speaker B: Yes.

Speaker A: Oh my God. Okay. AI security. Hey, security team. What about. And they're like, oh my God, I'm not trained in AI. I don't even know how AI works. I don't know. I mean, that's.

Speaker C: No, it's true, it's true. It's good to think about.

Speaker A: Yeah, we all had to learn. And not everyone has time to learn this because they already have something else to do. And Then it's like, where does this sit? And then maybe you have, um, enterprise risk management efforts and security says yeah, yeah, I'm covering AI risk without knowing what are AI risk. What is AI specific security risk. Right. So sorry, I was deviating a little bit from your question.

Speaker C: That's okay. I like it.

Speaker A: But I really want to praise OWASP because it has not only published and is updating constantly Those, um, top 10 for LLMs M but also they have published so much recently. So Genti, uh, red Teaming guide together with Cloud Security Alliance Red. Yeah, mitigation. So they have.

Speaker C: I'm going to give you, I'm going to give you a shout out because I will just say like I like OWASP a lot and you also mentioned NIST earlier, which I also like a lot. I think it's a little overwhelming right now because so much guidance is coming out and I, I love what you post on LinkedIn and I like that we're covering NIST and OWASP here because they're really good if you don't know what to do. It's a great. Both I think offer great starting points, uh, in framing up the problem statement, figuring out what at least are the like buckets of activities that need to occur. Um, so I would say, uh, definitely people should follow you on LinkedIn because you always have good recommendations about this and I agree with you 100%. Uh, I know OAS pretty well and I think it' very good and it's easy to read guys. Just, it's easy to get through guys. So it's like 10 things.

Speaker A: Yeah. Okay. If it wasn't this, this, if it was this report only, but all of us can also not help itself use more and more because there's just more

Speaker C: and more things to talk about. There's so much to talk about. Yes, that's true. M. Um, so anything I missed that I, I should talk about strictly, I want to talk a switch from. To like Ethics Washington. Right. Uh, like we have ethics washing and then we have execution. And I want to sort of switch the conversation from just straight up AI governance at companies to like, what is the ethics challenge here? Um, but before I do that, is there anything else you wanted to make sure that you talked about in the AI gap? Okay. Um, yeah. So the ethic. What do you think, like the principles and checklists are? Well, let me back up a little bit. What is responsible AI in your mind and what are the principles and checklists that you have to think about as a leader? If you want to do responsible AI.

Speaker A: So responsible AI I would say means the same thing as when we're saying trustworthy AI and trustworthy AI. So there's this is, I would say a global movement in a nutshell. And there are many, many layers of responsible and trustworthy that we can refer to. So the international treaties, the OECD has already established its AI principles. I'm um, not 100% sure. Test me. I will fail. Four years ago maybe or five. And they have defined alignment with human values, um, data quality, et cetera. Then we have um, uh, the G20, uh, picking up on the same principles and then it was just uh, everywhere. So many self regulatory initiatives. This was actually before explicit AI regulation came into the picture. So Meta or Microsoft or Google or I don't know, Salesforce. And so, so, so, so so many companies have those self regulatory responsive AI principles, um, that are then also reflected in existing or new laws. And what are those responsible AI principles? Also NIST AI Risk Management Framework lists them explicitly. So they're mostly overlapping in those various resources. And it comes down to security, uh, safety, like also you know, LLM Chatbot should not give you know, harmful advice, um, uh, privacy, accuracy, robustness. So it should not be brought off guard. Yeah, accuracy transformation.

Speaker C: I think it's interesting. One of the themes I've been thinking about a lot lately is how um, there's like AI newness and then there's AI ifying the oldness. And so we've got old processes. I think a lot of times, myself included we're thinking about AI governance as a straight through process. Some data is going to get trained and it's going to go in the system and then we're going to have a model and, and yeah, we've governed that sort of in the past and it has some differences. But you know, when you think about this ethics challenge and doing things like hey, we want it to maintain, um, you know, work for the good of humans, make uh, sure it's not being sneaky and lying. These are very new elements that you to try to govern. Uh, that is a new like true ethics challenge. And what are you seeing that companies maybe like even companies who build these foundation models, like do you see a foundation model company that like really doing this or is it all like kind of ethics washing for the, you know, anthropics and open AIs of the world?

Speaker A: I wish, I wish I had more insight into what is happening behind the curtain because I also do not know now that I spend like a lot of Time researching this what I would like to. So it's not. But I don't know what OpenAI uh is using. Right. Is this a uh, data aggregation? And so under like California for example, under California privacy law, aggregated data is actually also out of scope or do they use I don't know differential privacy for something. I don't know if they're doing ethics washing or not. And what I wanted to stress also is the difference between responsible AI and trustworthy AI. So those principles that I mentioned twice already and ethics. So some people say like ethical AI is the same as responsible and trustworthy AI but I think it goes beyond. Right. Ethics.

Speaker C: Oh I like that. Let's slow down and really cover that. I think that's really interesting. Um, so what's the, the what's the top of the pyramid in your mind?

Speaker A: The pyramid? Not for me. I'm. I mean I studied law so you know.

Speaker C: Oh yeah, no pyramid. But let's, let's like focus on these three terms that you're mentioning and slow down for a second because I think that's so interesting and I. We kind of mix and match.

Speaker A: Yeah. So ethics I would say is related to questions that are not, not covered by hard law and also not really falling under these principles which like I said or argue can also be tracked back laws in general. Ethics. Should we replace unit ABC with um, AI agents and just fire those 40 people? Um, um. Is it okay to use AI in this specific process to assist with our decision making? Mhm. It's legal. But should we do it? So it's more about this should we and not are we allowed to.

Speaker C: Okay, I like that.

Speaker A: Yeah, that's it.

Speaker C: And then you said trustworthy and responsible were the other two.

Speaker A: Trustworthy is the term that is used by NIST AI Risk Management Framework. And I would say my point of view is identical with the term responsible AI and all those self regulatory like Google. Okay, I have my 5 or 7 or what not trustworthy slash responsible AI uh principles. Their privacy. They're decent that so the you know, trust evoking but probably must ah have anyways.

Speaker C: Right, right, right.

Speaker A: Um so those two terms I would say are the same and regard to self regulatory initiatives, these international um initiatives, the initiatives by standardization bodies because they always, always refer to a very very similar sounding set of principles. And ethics is all those things that people are concerned about. I would say that go beyond those things.

Speaker C: I'm glad you're talking about that. I feel like a lot of companies are just like give me the List of things to do and I'll do it and you know I come from the data world and like nothing's going to fit perfectly in every bucket. You do have to think for yourself about what are the ethics behind this. What do I generally want to be true? Even if you're a business, I don't think there's any one template uh, so that's really great, I'm glad you're um, defining that. And then the other thing I've seen you posting about, we're kind of coming up to the end and we got to get to our rapid fire questions but um, you know privacy enhancing technologies are evolving. Uh and there's some really interesting ones out there. Um, how. What are privacy enhancing technologies? These pets. You'll see that uh, and um, what are you seeing people adopt and use out there?

Speaker A: So privacy enhancing technologies since a couple of years is um, very embraced especially by uh, governments also and standardization bodies. Ah so by everyone who really wants to do the right thing is a term for maybe 6, 7, 8, 9, 10 what not uh, technologies that do protect personal information while processing data or doing data analytics. So we also have the term privacy preserving machine learning if those privacy enhancing technologies are applied in AI and machine learning. Some examples are synthetic data meaning I um, reproduce data uh, that mimics the original data set without like you know referring to real people. Or we have secure enclaves also called trusted execution environments. That's basically ah, we have this on our phones that's actually some hardware components where the code is um, processed inside this enclave and it's much harder for intruders to see the code.

Speaker C: Mhm.

Speaker A: Um, or we have differential privacy which is kind of the golden standard of data anonymization these days. While still hard to implement. NIST has just published the final version or some months ago of its guidance on different privacy which is randomizing data so that when you process and analyze the data you cannot tell anymore with mathematical proof if I was part of the outcome, if that I have a black cat was part of aha. Blonde women sometimes have black cats. Um and their homomorphic encryption is such ah, a fascinating thing because it's uh, computation on encrypted data. Super practical. It must be very familiar to you because it's more or less for.

Speaker C: I worked with a uh, homomorphic encryption company when I uh in the past. Yeah yeah, yeah. And uh, it's got some serious big brain math people behind this. But it was actually it's been around uh, and that people are optimizing it for um, for uh, corporate use very effectively. Um, so I'm glad you're talking about that because it's a, you know, it feels like an intractable problem. But there are some really interesting, very new technologies addressing this and being adopted. It's not, it's not just in the research lab, but they're really being used uh, effectively. So that's always kind of uplifting, um, big companies though.

Speaker A: Yeah.

Speaker C: Yes, yes. So, um, so excited to get a chance to catch up with you. I love uh, all the stuff that you post on LinkedIn. I do recommend people follow you if they can. Um, and um, you know, we love to do these kind of last um, rapid fire questions if you're ready. Uh, I'll run through, uh, maybe not every single one, but we'll do a few of them. Um, what one quality that you think separates good leaders from great leaders?

Speaker A: I personally think it's ah, safety, safe. Feeling safe in the team so that people can be themselves and say what they think and it's not like perceived as a uh, you know, critique or like you have to just follow the rules. You're not allowed to think. It's only when you feel safe and you can share, then you know, it benefits the entire team and organization. So I think that's super important.

Speaker C: Yeah, I love that you're saying that. That's a different answer than we've gotten in the past, but it's so true. Um, what is one? I'm actually curious about this because you produce so much work that I see, uh, what is a habit or routine that you think keeps you like at your best professionally?

Speaker A: Yeah, that was a, that was a tough question. I just, I think I hold myself accountable to post and to, on LinkedIn, for example. I really do that. And I do that since four or five years. Posting two to three times per week. Well, recently not so much, but. And trying to share the content that I'm learning about benefits me and it obviously benefits others as well because I got so much great feedback and so I just tried to um.

Speaker C: Yeah, yeah, it's really good. Check uh, it out guys, because she really, you really do give kind of your thoughts on it, which I think a lot of people are just like, here's a link. And I'm like, well I could read that but it's very helpful to get your opinions. Um, is there a book or movie that you would recommend, uh, or a periodical, something that you think uh, we should all be checking out because it's

Speaker A: AI related and I did like, it is um, Companion. Ah. So not only have I watched Companion, which is a movie that, uh, a couple of people have AI Like LLM in a robot form.

Speaker C: Oh, great.

Speaker A: I love it as companions. And. And it's. So that's actually. It's a good movie. But on my plane yesterday from Europe to. To back to California, I watch. I like Bollywood movies, too. And I watched a Bollywood movie which had the same topic, and he also didn't realize it was a robot, et cetera. So. Yeah, and there's a book that I read, Machines, um, Like me by Ian McEvin, which also has this, uh, same topic. And he. He wrote that, like, five years ago.

Speaker B: So.

Speaker A: And it's so spot.

Speaker C: Is that interesting?

Speaker A: So good.

Speaker C: All right, that's good. That's a good summer read. Good, good. Uh, examples for going into the fourth of July weekend. It's not gonna. You don't have to learn anything new. You can just enjoy a book. That's really good. Um, what is, um, if you had to summarize your leadership philosophy into, like, a tagline or a motto, like, what's one thing, um, that you would tell your team? Um,

Speaker A: I have led with passion in the past, I think, and I love when, you know, we can share the passion and we have a common goal. I mean, it sounds too obvious to. To be interesting, but really, this common goal, and then everyone can speak up and share. So this feeling safe is. Is really important for me.

Speaker C: Also, you're mirroring that back, right? That's the best thing you've experienced, and that's what you want to mirror back. I think that's, um. That' amazing. And, you know, it actually is kind of rare, um, for leaders to do that and to be in a team like that. So if you guys are listening and you're on a team like that, appreciate it. It's something special. Um, Katerina, what a nice opportunity to catch up with you. Uh, you're just working in such an interesting, evolving area right now, and, um, it's been great for our listeners. And, uh, I'm going to continue to read the stuff that you write, uh, and see maybe we can catch, uh, up again on the show and find out, uh, what more you've done, because I do think AI is going to be so next year.

Speaker A: Thank you so much for having me. It was a pleasure. It was great to be here. Um, good luck, and all the best to our listeners.

Speaker C: Thank you.

Speaker B: We look forward to bringing the latest industry news in our next episode. In the meantime, check our other episodes@techcell.com podcast and be sure to subscribe to our YouTube channel so that you never miss the an episode.