Why AI Governance Keeps Failing - and the Framework That Fixes It with Mark Khater

Speaker A: Awkward time to ask this, but. Hey, did you download the trail map?

Speaker B: Yeah. No, I don't need to.

Speaker A: I don't understand. You're trusting your signal out here.

Speaker B: I'm trusting T Mobile. They have the best network. And if we end up in bumtots nowhere, well, we've got T Satellite for backup.

Speaker C: Whoa.

Speaker A: I don't trust my carrier that much.

Speaker B: We'll just use your phone as a flashlight.

Speaker D: With America's best network and T Satellite, we're keeping you connected in places you never thought possible. And if you switch today, you get free phones for zero down and only 25 bucks a month per line for four. Find out more@t mobile.com or visit your local store.

Speaker A: Best Mobile network Based on analysis by Ooklev Speed Test intelligence data to age 2025 with 24 monthly bill credits and 4 eligible port ins on essentials for well qualified customers with autopay plus taxes, fees and $35 connection charge per line credits and in balance 2 if you pay off earlier, cancel contact US Finance Agreement example $299.99 Moto Edge 5G required T Satellite available with compatible device in most outdoor areas in the US where you can see the sky included with experience beyond or $10 a month. However, news monthly cancel anytime visit t

Speaker E: mobile.com it's not about how much AI. Uh, it's not about going out and buying all the smart gadgets in the world and putting them in your organization and then trying to figure out what is the impact of that later on in dealing with it. No, it's about exercising the empathy. It's about exercising, uh, the judgment is about getting the right data. Uh, it's about really making the very simple decision of uh, where am I going to deploy AI and for what reason? It's a technology pull by the organization rather than a technology push by the manufacturers of that technology.

Speaker F: Hello ladies and gentlemen and welcome to our Risk Management show podcast where we explore critical trends and strategies shaping risk, security and leadership. Today I'm your host Boris agranovic, founder and CEO at UH Global Risk Community and our guest today is Dr. Mark Khatter. He's a Cambridge academic AI pioneer, fintech CEO and one of the rare voices who has been building AI systems continuously since 1994. Mark heads the center for Strategy and Performance at Cambridge uk, runs AQM M Squared. Sorry, AQM Squared, an AI driven investment platform and has co founded hedge funds with more than 1 billion in assets under management. So what? Uh, M. Mark is also fluent in Arabic, Hebrew And English, which is kind of makes it uh, rare quality in our age of uh, global conflict. So Mark, welcome to our Risk Management show podcast today.

Speaker E: Thank you for having me, Boris. Pleasure to be here.

Speaker F: Absolutely, it's my pleasure. So today I believe we'll have a thoughtful conversation on uh, a topic why AI governance, uh, keeps failing and the framework that fixes it. But before that, Mark, could you tell us a short story about yourself, your career path and what brought you to where you are right now and what you guys at uh, AQM M squared up to this day.

Speaker E: So, um, I started my, My career as an engineer, uh, many, many years ago, Boris. And uh, I, I was actually a medical engineer, uh, and that drew me to looking at uh, bio signals and, and how best to analyze them to help doctors, uh, do a better job of their diagnostics. And uh, I got into this in 94 when I joined the uh, school of Engineering. And uh, back then artificial intelligence was, was uh, around.

Speaker F: Uh.

Speaker E: But uh, people uh, were speaking more about it from a theoretical point of view because the computers and, and the software to implement wasn't there yet. So uh, my, My first project was an attempt to. It was very ambitious attempt to get computers to help diagnose images of cancers which um, at the uh, because the computers weren't fast enough and the images weren't clear enough and there were many, many other problems. But I learned uh, quite a lot about applying artificial intelligence to bio signals. And uh, that was the start of my, my very first AI project. Um, fast forward now 30 years later. And um, and my. My Last sort of 12, 15 years have been focused on applying uh, AI to signals, uh, coming out of the stock market. And that's really what AQM Square focuses on. It looks at those signals coming out of the st, tries to pick stocks that we think are going to do very well, invest in them and hopefully uh, deliver uh, superior returns to our, uh, to our clients.

Speaker F: Fantastic. Okay, let's uh, jump to our topic. So you uh, with regards to AI governance, uh, you arguing that it's kind of strategic problem rather than a compliance problem. Why do so many organizations still approach governance as a checklist exercise? And what are uh, they missing in the leadership level?

Speaker E: I think, Boris, if you look at risk and you sort of want to simplify it, you know, you probably say risk is the likelihood of an event multiplied by the impact of the event. And I think the fear of missing out that a lot of CEOs are facing now when it comes to AI means that they're trying to actually handle the impact of events, that part of the risk, as opposed to the likelihood of the event. So it's all about, let's get AI in there, um, when the problem happens, let's deal with the impact. So a lot of the time, and I think this is what's causing that governance, what's increasing that risk, uh, when deploying AI is we're not trying to think carefully and clearly about the likelihood of the risk events, but rather control the impact of that risk when it happens. And, and this is where a lot of the danger comes, uh, from. And I think a big part of it is that fear of missing out.

Speaker F: Your framework named Edge, which calls empathy Data Governance Execution, which proposes embedding governance directly into AI systems from the beginning. Can you walk us through how this framework works in practice and why bolt on governance almost always fails?

Speaker E: The idea of what we've called the Edge framework really is that we want to, to say before you embark on these AI projects, you need to empathize, look at the data, examine, uh, governance and plan execution very, very carefully. Um, a big part of leadership, as you know, is about understanding people, their incentives, their behaviors, and to a great extent trying to predict those unintended consequences of introducing new technology into the firm. And that's what we call empathy, uh, when it comes to data, a big part of the effectiveness, the qual, the cost of the artificial intelligence is about the quality of the data, the integrity of the data, the ownership of the data, the security, and whether that data is actually just fit for purpose or not. And governance is all about accountability. Now AI is incredibly good at generating a decision set, but it's in no way, uh, uh, responsible or accountable for what a leader decides or an organization decides to do with that decision matrix. So governance is all around that accountability controls, oversight, the transparency, and also escalation mechanisms that need to be built into the firm. When AI does the wrong thing, then last and in no way least is that execution, that translation of the intentions, the plans around AI into operational reality, the processes, the culture and the KPIs that we, uh, implement into the organizations to make sure that AI is having the intended, um, impact. So effectively, by looking at that sort of what we call the Edge framework, we embed that aspect of governance very early on, uh, into, uh, AI as it sort of starts to integrate within our organizations.

Speaker G: And now we have a quick message to share. If you are listening to this podcast, it probably means you have a keen interest in risk and compliance. You're not alone as globalriskcommunity.com has already more than 100,000 active members. Together we share knowledge, resources and the latest events. On top of that, Global Risk Community is a great and easy way to network and broaden your opportunities. Visit globalriskcommunity.com and sign up as a member. The link is in the description. Thank you for listening. Now back to the episode.

Speaker F: So, because you have been building AI systems since 1994, which was no buzzword then, but before today's generative AI way, looking back across three decades, which risks are uh, genuinely, genuinely new, and which current uh, fears do you believe are being misunderstood or exaggerated?

Speaker E: I think up until uh, now, uh, a lot of the worry around AI and so there is some of that worry today was around the technical failure. So a lot of the time engine, uh, because AI was more of an engineering, a technology problem really. We sat around the rooms and we talked about technical failure, the failure of servers, the failure of the computer system, the failure of the data, failure of the models, the algorithms, etc. I think what's what this, this generation, what this um, uh, period of this era is going to face is more a focus on organizational failure. I think that's the risk that's going to start coming. There's the organizational failure. We're going to look at infrastructure failure because to a great extent now AI is no longer a technology problem, it's an infrastructure problem. It's impacting our electrical grids, it's impacting our water systems, it's impacting the um, the environment. It's going to have wider implications on, on various policies within our um, societies, whether it's the educational policies, health policies, uh, and defense policies. So uh, and maybe this is to a great extent why a lot of people are speaking about AI as the next industrial revolution. I always say, I don't know if that's the. It might be, it might not be. But what is incredibly clear is AI is becoming an infrastructural problem and not a technology problem. And that's going to come with its own matrix, uh, of risks. Um, when you look at AI as well today, there is uh, a lot of problems around um, the impact of poor decisions that are being built on AI. Um, AI is incredibly good at creating decision matrices that look quite good on the, you know, uh, on the onset. When you look at whatever AI produces to you, your first impression usually is, oh wow, that's great. But then again, the implementation of those decisions is not as easy as AI makes them seem. And they have implications on, on organizations. Um, I think today's greatest risks often emerge from successful AI adoption rather than actually failing failed AI adoption. And that's another thing that, you know, 30 years ago wasn't there. And every major AI wave has exposed weaknesses in leadership, culture and decision making within organizations. And organizations are going to have to sort of adapt their operating models and try and reduce, as we said, the likelihood of those risks and the impact of future disruptions of implementing those AI models. So these are all risks and that are sort of emerging, emerging now that we're not there, uh, on that 30 year horizon when AI was just a budding technology.

Speaker F: So one of your most uh, quoted ideas is machines think fast, humans think deep. How does this apply to modern organizations deploying AI in areas like uh, compliance, fraud detection and financial decision making?

Speaker E: M. I remember I, I said, I, I came up with this quote on stage about a year or so ago and because somebody asked me, what do you think is the biggest difference between the way humans think and machines think? Inverted commas think because I don't think machines think. And I said, well if, if I had to sort of differentiate, I'd say machines think fast and humans think deep. And, and if you, if you examine history, Boris, and, and philosophy, thinking fast was never a trait of women. Uh, fast thinking, uh, produces a lot of information, a lot of ideas, but was never a trait of wisdom. Um, and so it's important to note that this is a, ah, quite, quite a substantial differentiation in the way humans think and uh, machines think. Um, now the AI, because of its ability to go through vast amounts of information very quickly and create relationships, reduces the cost of generating answers. But what it doesn't do is it doesn't guarantee better decisions, decisions. And the likelihood of error increases when, when, when humans stop questioning those AI recommendations, when they stop applying deep thinking. So lots and lots of information, lots and lots of patterns and relationships, uh, drawn by AI. Lots and lots of outcomes, uh, very fast thinking by AI which requires that deep thinking by human beings that questions those recommendations. And the impact of the error grows when the AI decisions are scaled across an organization. So use it and you use it and other people use it within the organization. And we all come into the same room carrying those errors to probably very similar errors. They don't end up canceling one another because we are no longer turning up to meetings with uncorrelated decisions. To a great extent our uh, decisions have been correlated by the machines and the models operating in the background. So again, another incredibly Dangerous um, risk of this fast thinking without deep contemplation from humans. And I think the goal isn't really here for humans to work against machine. It's not a human versus machine situation, but it's more of a human judgment amplified by machine capability. And I think that's where we should be uh, moving or targeting as sort of deep thinking societies.

Speaker F: Perfect. So Mark, I would like to ask you your personal opinion of uh, what is the major misconception in this, your field of uh, building AI driven investment platforms that you kind of uh, strongly disagree with.

Speaker E: I think a lot of people uh, today believe that there are platforms that uh, invest using AI. There's actually very few platforms that apply AI correctly to the investment field. Um, to a great extent there are a lot of large language models that are being deployed to look uh, at investment decision making. Um, and large language models don't do a lot of what investment professionals do. Um, in the investment world markets reward better decisions, not simply more information. And large language models turn up with a lot of information and they produce a lot of options uh, for um, traders to work with. Now as I said, that's a lot of information but not necessarily better decisions. Um, you're going to notice that most investment mistakes originate from what we call human cognitive biases rather than data shortages. It's sort of a uh, trader or an investor being married to a particular investment. That's not necessarily uh, a data shortage. This is more of a cognitive bias. And to a great extent AI at times actually reinforces those cognitive biases and makes the human make a lot more mistakes. So again, incredibly important to notice that um, AI also can reduce the likelihood of certain errors by sort of systematically processing the information, which is wonderful. But then again it can also eliminate a lot of necessary information that you might want to look at before you deploy um, down investments. And so to a great extent I'm still a very strong believer that human oversight remains incredibly essential uh, because the impact sort of the investment mistakes uh, can have. And a lot of the time we're so eager to come in and disrupt a field. You know, let's disrupt uh, the world of investment with AI. Uh, I don't think that's very smart because the world of investment has been built with safeguards, with governance, with risk control, with compliance, uh, with financial uh, authorities regulating it in particular ways to safeguard an investor's money. And so sometimes coming in and disrupting it with a technology is not really uh, uh, the smartest way to safeguard uh, the uh, investors money. So I think these are all things that the investment world is grappling with at the moment and uh, trying to sort of insert AI correctly. Um, and I think it is actually one of the few fields that currently is pushing back and thinking carefully about how to deploy AI.

Speaker F: No. Interesting. So, because you have advised governments across uh, Europe and the Gulf on AI policy and strategy. Right. So how do you see the data sovereignty and geopolitical AI competition reshaping risk management over the next few uh, years?

Speaker E: I mean Boris, I think data, uh, not a lot of people would disagree about this, but data is becoming a strategic asset. It's comparable to capital, to energy, to infrastructure. And so ownership of data, ownership of your data, um, as a government is a geopolitical uh, risk that everyone is looking at and considering, uh, very, very carefully. And you're seeing that nations are increasingly viewing data control as a matter of economic and uh, national security. So to a great extent when I'm, when I'm speaking with a lot of those, those governments, it is around, you know, that uh, um, um, sort of um, sensitivity of data and how we can best use it for economic and uh, national security, um, also sort of current geopolitical events have shown us that uh, uh, both the likelihood and the impact of AI related disruption can really, really have uh, a negative um, um impact on the um. We've seen what happens when data sites get bombed. We've seen what happens when choke points for trade, which are also choke points for data, get controlled in ways that we don't want them to be controlled. And all of this is going to have uh, uh, an impact on the way you process um, uh, a lot of the necessities uh, of daily life within your country. And more and more I think organizations are beginning to understand that where the data resides, who controls it and how it moves is essential to their competitive advantage. And so, so future uh, resilience really depends on balancing uh, the very sophisticated equation of innovation and sovereignty. How am I going to keep up with global innovation? How am m. I going to maintain sovereignty of data and sovereignty of control over those core capabilities that are essential for my economic development and also my national security? And I think those are the questions that most of these governments are asking and where I sit with them in closed rooms to have those discussions.

Speaker F: So your research at Cambridge highlights uh, what you call the competence coordination gap, where organization may have the expertise but fail operationally because teams cannot align effectively. So why is this gap becoming even more dangerous in the AI driven organizations? And what can leaders do to close it. And also what, uh, kind of piece of advice, um, practical advice you would give to, uh, CEOs looking to improve their organizational, um, resilience against these complex, uh, operational risks.

Speaker E: Competence really examines whether we have the ability, uh, to do a, ah, particular thing. So it answers the question, can we do this? And coordination, uh, asks a question, how can we do this together? So it's about the alignment of the leaders or, uh, the various role players within the organization. Um, and you don't just have to have the. It's not enough to have the capability in order to get something done. You need to be able to align everybody to get it done. And I think that is really the coordination and the competence and how they fit together. What AI is doing is actually, uh, doing something that I think is quite ironic really, because it's aligning people very quickly. People get coordinated very quickly, but the question is, are they aligning on the right thing? Now we, we spoke about it, you know, uh, very quickly and we said you, you will walk into the meeting. I will walk into the meeting. Uh, both of us probably denying that we've ever used AI because it's taboo, because it's still perceived as a shortcut. And we've both used the exact same models. Uh, we probably have both had very similar education because we're sitting in the same organization that has an HR process that guarantees that you and I are professionals that have received particular, uh, uh, pedigree of education and degree of experience. And we're both sitting in the meeting having asked the exact same model, the same questions. And suddenly your thought process, your decisions and mine are silently correlated because we didn't exercise any independent thinking. And very quickly everyone in the meeting is now agreeing or coordinating or aligning, uh, on the wrong page. And probably you've seen this happen during the Iran war. All the information is coming from the same sources. It's being analyzed in the same way. A quick decision is being made, quick alignment. But unfortunately we might have all been aligned on the wrong page. And our analysis might not have been great and the outcomes might not be going where we want them to be. And I think that's one of the things AI is accelerating and it's accelerating coordination, group alignment, group thinking, uh, without us noticing this, we what. That, that is what's happening. It's also doing other great things, uh, that, you know, it's helping us all get, get the, all of the information we need very quickly. But it's also meaning that we're inheriting all the bias in that information in our decision making. And these are all things that are now emerging in boardrooms that, um, some, some leaders can, can observe, others don't. And so I think the most important thing really moving forward when it comes to this is diversity. Uh, diversity is no longer a luxury. It's not something there because, um, you know, it's a nice to have, but actually having diverse thinkers in the room, devil's advocates, people who question, who think deep, is becoming absolutely more and more necessary for organizations than ever before. Um, you know, and I think it's very simple, simple. Um, if you, if you as a leader surround yourself with yes men, it's incredibly dangerous. And if you as a leader surround yourself with yes, uh, models, models that agree with you all the time, it's also incredibly dangerous. And that's really what we're doing. We're just substituting yes men with yes models. And that can be catastrophic.

Speaker F: Absolutely. Wow. This is kind of a very, very powerful, uh, powerful, uh, thinking. Yeah. Uh, maybe we have to create a. Models for, uh, devil's advocates. So it's good idea.

Speaker E: It's not really a bad idea, Boris, to be honest. I mean. But, but again, the reality is that's why we have humans, because humans have been throughout history incredibly capable of, uh, observing the, the, the, the opposite opinion. You know, humans are wonderful at playing the role of devil's advocates, argumentative, independent thinkers, innovative. That's what makes us human, empathetic. You know, they'll ask, they'll ask the questions like, is this the right thing to do? Is this ethical right? Would I like this being done to me? You know, these are all incredibly important questions. That empathy aspect that we as humans bring into the room. And to a great extent, you know, this younger generation that is coming out, they're, they're, they're incredibly smart, budding, they're ethical, uh, they're environmentally conscious, uh, they're diverse, uh, and they will no longer work with organizations, institutions and governments that they perceive as unethical. And so we're going to have to accommodate all those things if we actually want to be in the good books and do business with this new generation.

Speaker F: Fantastic. So maybe if we summarize our interview for someone who is listening and would like to walk away with one or two major takeaways, what would it be?

Speaker E: I'd say most organizations currently, they believe that AI governance is about controlling the technology. I don't think it's only that. I think it's about reducing the likelihood of the poor decisions and also limiting the impact uh, of mistakes that are going to occur because we're going to take in AI. Uh, I think organizations that are going to succeed moving forward are not going to be uh, um, the ones with the most AI. No, they're going to be the ones who exercise the best judgment as to where and how to use AI. Um, and I think that's really uh, the biggest takeaway from here. It's, it's not about how much AI. It's not about going out and buying all the smart gadgets in the world and putting them in your organization and then trying to figure out what is the impact of that later on and dealing with it. No, it's about exercising the empathy. It's about exercising uh, the uh, judgment is about getting the right data. It's about really making the very simple decision of where am I going to deploy AI and for what reason. It's a technology pull by the organization rather than the technology push by the manufacturers of that technology.

Speaker F: Mhm. All right, fantastic Mark, that were all my questions. Perhaps if I forgot to ask you something that is important to you, uh, to convey to our audience, please go ahead.

Speaker E: Um, I think you've covered quite a lot of ground there Boris, to be honest. And all I'll say is if uh, if I can be of any help to anyone in your audience, uh, please feel free to, to reach out to me. I on my email, uh, mark twoam.ac.uk or uh, in my LinkedIn which I'm sure you'll, you'll include somewhere uh, in the podcast.

Speaker F: Absolutely. So thank you Mark again. Thank you and I wish you a great success and your many uh, companies. Uh, is uh, one of this uh, AQM Squared.

Speaker E: Thank you so much for having me Boris. It's been an absolute pleasure.

Speaker C: If you like this episode, please give it five stars on your favorite podcast app. It will help us in spreading the word. Don't forget to subscribe to receive your notifications of future episodes straight to your phone. If you like to be connected with your peers, risk managers and compliance executives from all over the world world, make sure to go to our main site at www.globalriskcommunity.com and click on the sign up button to join in. There are some incredible conversations happening inside the community. If you work in a fast growing company operating in the risk management space and your job is to acquire new customers, generate third leadership and awareness about your products and services, consider to become our partner here is the global risk community is looking to work with a limited number of innovative risk management companies interested in a new partnership model. What do I mean by that? We will put you in front of our engaged community of more than 100,000 subscribers. By using our multichannel approach such as website, email, events, online business, social communities, video and podcast, podcast, to name a few, you generate leads, awareness and advocacy.

Speaker F: Everybody wins.

Speaker C: Interested? Send your request to infoobalriskconsult.com last but not least, if you or someone you know will be an incredible guest on our show, email us@infolobalriskconsult.com and let us know. See you in the next episode.