Can we trust something we don't fully understand? With Simon McDougall and Avtar Benning

Speaker A: Trust is both delicate and hard earned. It can also be the difference between experimentation and scale. As organizations race to adopt AI, the challenge isn't just about capability, it's about confidence. Because the companies that benefit most from AI may not be the ones with the most advanced technology, but those create enough trust for people to use it at scale. Today we're asking the big question, can we trust something we don't fully understand?

Speaker B: AI has become distributed and widely used very quickly. We haven't had time to build all of these components up and we're going to have to kind of build some of them as we go along.

Speaker C: I would argue trust right now is probably more important than more intelligent models.

Speaker B: The good organizations aren't just doing this stuff for the sake of it.

Speaker C: If you get the trust angle right, it really becomes a, um, transformation enabler.

Speaker A: Hello and welcome to the Green Room by Deloitte. I'm Steph Dubbs and I'm joined today by my co host and another new face of the team, Andrea Boxall. How are you feeling, Andrea?

Speaker D: Uh, I am absolutely thrilled. You could even say I'm Deloited. Oh, gosh, sorry. Let's get that in.

Speaker A: Good start to the episode. So for those of you who are watching the episode on YouTube or on Spotify, you may actually notice that our green Room has turned a shade of pink this week. But don't worry, the screen isn't broken. We'll actually be back at our normal studio soon. So as we continue our series looking into the impact of AI could have on businesses, individuals and society, there is one topic which has come up in every episode. Trust. For the last couple of years, it's felt like capability has been the main question asked about AI. But as we move into a new phase of adoption, the question of trust and confidence in the systems becomes even more important. Trust can be seen as a break on innovation, but is it actually what allows AI to scale past experience, experimentation and into transformation for organizations? Trust is central to any AI adoption and it's a non negotiable that needs to be built in from the start, not added, uh, later. So as the pace of change shows no sign of slowing down, how can we keep up with technology and build confidence at scale? Today we're answering the big question, can we trust what we don't fully understand?

Speaker D: Today we're joined by Simon McDougall, who's the chief strategist for privacy and AI at ZoomInfo, and Avatar Benning, who's a director in our, uh, Deloitte's Trustworthy AI offering. It is a pleasure to have you on the sofa.

Speaker B: It's a pleasure to be here.

Speaker D: So I wanted to begin with a tension point we often hear. So it's this idea that because individuals are adopting AI at pace that we haven't seen before, it's putting pressure on businesses to move quicker. So is there any truth in that thought and kind of what are the differences in what individuals and organizations have to think about when it comes to AI adoption?

Speaker B: I think that's very true. I think that, uh, one of the key stats that I've picked up in the last year is that it took ChatGPT just a couple of months to get to 100 million users. Uh, when it took uh, TikTok well over half a year to get there, it took Instagram well over two years to get there. The adoption of this technology by individuals is faster than anything we've had before. And that I think is where often you see a bit of a trust gap because people are getting hold of this technology and using it, uh, and then m worrying about trust later. And that's a big challenge for businesses because businesses have different interests and different concerns and different obligations and uh, so they're going to have to try to keep up with that.

Speaker D: Um, yeah, totally. I know that. I think once you understand what IT can do as well, you kind of find yourself coming back to it and asking it certain questions. And that kind of trust, as you say, doesn't tend to be thought about. How are the questions that companies need to ask about AI changing as it develops and as we get more used to it being part of our lives?

Speaker B: In terms of the questions that companies are asking, a lot of those are well worn. So before I was at Zoom Info, I was a regulator. I was Deputy Commissioner, uh, and the Information Commissioner's office. And we were doing work on AI going back to 2018, 2019. And then it wasn't called AI is machine learning and it was synonymous with machine learning. And this whole wave of AI now wasn't really envisaged, but if you look at the guidance we produced at the time, a lot of the risks were the same. We discussed explainability a lot, we discussed transparency a lot, uh, and uh, how AI could be used to make decisions about people hiring decisions, uh, decisions on credit or immigration. So those things were already, those are well worn risks. And then you have other risks on top of that, not least just how widely spread this technology is now. But also there are other unique risks that weren't there before. I mean Deepfakes was not something we delved very heavily into, for instance. But I think one of the challenges for organizations is saying, well, what are the actual risks represented by this new technology? How do they affect me and what is the risk of harm? The risk of harm to our consumers, to, to society, to our staff, to our customers, if they're corporate customers. And how do we manage those risks of harm?

Speaker A: Definitely, and I think that's the thing because you often see companies obviously sort of promoting the power of an AI tool, don't you? And sort of all the capabilities that it's able to do. But actually that trust and transparency point is something that isn't often widely promoted or it hasn't necessarily been until now, I suppose. Avatar, let's come to you first. Is AI advancing faster than public confidence in it? Is. Is that why?

Speaker C: Yeah, absolutely. Um, actually I would argue trust right now, where we are in time is probably more important than more intelligent models. I think trust is becoming the real barrier from going, going from simple POCs to full scale enterprise wide adoption. And I see this with clients a lot. You know, they, there's no shortage of, uh, example of going to a client, seeing a very long list of great and wonderful exciting things they would like to do, but then sort of prioritizing that and then sort of figuring out which ones are the where are you going to get the most roi? Where are they more safe and reliable? I think all of these questions come a bit later and then there's a bit of a bottleneck going from that to full enterprise sort of rollout.

Speaker B: If you look at how innovation works and you're going all the way from uh, innovation being an idea that some people have, whether it's in a lab, if it's in a garage, in a university, all the way through to actually being fully scaled and actually distributed. You need trust to get from one to the other. So very often the people who are building the cool things underestimate how hard it's going to be to get everybody else to adopt those cool things.

Speaker A: I know this sounds like a really fundamental question, but why is that trust actually so critical?

Speaker B: Well, I think that's very context specific because when we're saying trust, the issue with trust is sometimes you say it, everyone just nods. And so yes, trust is very important and it is important. But are you talking about trust among a consumer group? Are you talking about trust among a group of citizens? Uh, are your customers a bunch of corporates? Is it B2B? Is it societal trust? There's all These different things flowing through and uh, the importance of trust I think is very context specific there. In the end, what does unify all of those cases is that people or corporates or whatever aren't going to fully, uh, engage with your solution if they don't trust it. They might play with it, they might experiment with it, but they're not going to actually invest and use it on a day to day basis. And that becomes more and more critical the more sensitive the usage.

Speaker A: I suppose. Avatar, in your world as well, there are times where trust actually matters more than it does, um, than other decisions. Let's say there are times where the trust actually matters even more than the AI system does in some ways.

Speaker C: Yeah, absolutely. I think there are certain scenarios and I like to think of it in the classifications, in the way the EUA act classifies in high, medium, low risk categories. And when you're in the sort of high risk bucket where there is repercussions or material repercussions of it going wrong, I think trust outweighs say, accuracy by some level of percentage points. And I think I've worked across a range of different risk categories and actually a lot of them are falling into the high risk categories. And so often the conversation is around, um, we're excited about this but we can't get this wrong. Um, and so indeed very much so. I think trust is quite right at the top of everyone's list.

Speaker D: And Avatar, I'm interested to know, what do you think is the kind of one thing that's holding businesses back from having trust in AI systems and capabilities?

Speaker C: I think the challenge is with these AI models, uh, some of these closed black box models, there's a sort of misconception there that you need to. Well, firstly, you can't understand each and every component of those systems. Um, but you don't need to understand each and every individual component of these models to gain trust. You need to be able to understand, um, certain components of these models, but enough kind of confidence around reliability, accuracy. So there are a lot of things you could do around the peripherals of these models to gain trust. It's the classic kind of aeroplane analogy. You don't understand how a jet engine works, but you're confident enough to get on an airplane. And that's because there's a lot of testing that's been performed, there's a lot of controls and risks around it that are well understood. So I think it sort of comes back down to not necessarily needing to know all the details, but enough confidence that we will perform reliably. That gives you trust.

Speaker B: And I think one of the key things there is that this is not a new conversation. We have had trust conversations forever and ever. One of my favorite stories around this is going back to 1891. I think the U.S. president's name was Benjamin Harris from memory. So fingers crossed on that one. They electrified the White House. They put lights into the White House. Very exciting at the time. Revolutionary. Uh, the President and the first lady refused to touch the light switches because they were worried they're going to get electrocuted to the point where there was somebody employed at the White House. One of their jobs was to turn the switches on and off on behalf of the President. And now obviously we have all, probably a number of times today, use electric switches and turn them on and off, and we don't worry about that now. Why is that? That is not because we've all become electricians and we haven't tested these particular circuits very well. But there are layers of assurance and testing and common use and life experience. All these things come together to build trust. So we use it, uh, unthinkingly. And if somebody does get electrocuted by a light switch, that's something you tell your friends about and it becomes a newspaper story and everyone wonders what happened and there's an investigation into it. So there's all these components and we'll talk about bits and pieces of, I'm sure, as we go through that, come together again. Going back to my first points, one of the challenges we have here is that, uh, compared to other technologies, AI has become distributed and widely used very quickly. This post, chatgpt AI in particular. So we haven't had time to build all of these components up, and we're going to have to kind of build some of them as we go along. But that's. We need all of those components before we really get to a stage of saying we are going to trust AI.

Speaker C: And the interesting thing is when you then go and speak to clients about this problem, it's exactly what you said, that some of these risks are well understood, but many of them aren't. So there's a big educational challenge. And then they're quite often faced with the dilemma that they don't necessarily always have the resources or even the technology to be able to perform the testing or assurance or whatever's required to, to give them trust. So there is, you know, the technology's moving faster than they can keep up. Um, and yeah, it's a very challenging time, I think.

Speaker B: And one of the other challenges there, I think, for a lot of businesses, is that on the other side of the coin, there is real commercial pressure to innovate in AI. And this isn't really a full trust point, but it's the other side of it. It's what's pushing these businesses. The boards are challenging CEOs, uh, CEOs are challenging the business, saying, well, our peers are doing X, Y and Z. And I've read this thing over here, and we should be adopting AI here. How do we go about that quickly? So there is a pressure to move fast. And I think a lot of, uh, risk management professionals, whether it's, uh, chief risk officers or general counsel, people in corporates, are caught between a business that's clamoring for rapid deployment of these solutions and a low trust environment over here. And they've got to kind of balance those two things out.

Speaker D: There's a real tension in those two, isn't there?

Speaker B: Real tension?

Speaker D: Yeah, there's a real tension.

Speaker A: Bring us on very well to the point around, like, what actually creates trust itself. So, you know, we've got the example of the light switch, which I'll be thinking about next time I turn a light on. Thank you for that. And as you say, there's the example around planes, you know, the fact that we trust planes, you know, despite the fact that very few of us probably know how they actually work. So what actually makes people trust a system? Does it come from just, does it have to be a positive experience that drives that? Simon, I'll start with you.

Speaker B: So the good news is that there's a whole, uh, world of research and academia about this. And if you want to go down a rabbit hole, then you can go down a trust rabbit hole again. The trust conversation itself is not new. AI is the new thing, and it's raising trust questions. And, uh, one of the traditional models talks about gaining trust through, uh, three different values, uh, you have to display, and that's ability, uh, benevolence and integrity. And so when they say ability, it's, do you believe that the thing you're going to trust the organization or the person actually has the ability to do what they say they're going to do? Are they competent in this benevolence? Is, are their interests, uh, the same as mine? So it's, uh, an emotional thing. Are our values aligned or are they always going to be a bit sneaky about things? Integrity is, are they actually going to follow through? Do their principles, Will they deliver on something that they say they're going to do. But evidence is being well meaning. Integrity is saying we will always deliver. And to have trust in something, you have to build up a confidence in those three values, which are all slightly different. So that's one way to think about it. But there's a lot of other different ways of going around it. In the end, if there's just one strap line, people need to just believe that you're going to do what you say you're going to do. I think that is kind of the baseline for these things.

Speaker C: Yeah, I'd agree. I think, um, sort of almost rolling that up into there needs to be some kind of evidence. Right. Like, I come at this from a sort of a bit more of a technical testing angle. And my conversations very quickly always boil down to, uh, have you done this test? And what is the outcome of this? And that's great. But exactly to your point, there's a so much bigger kind of trustworthy AI framework out there which spans across many different angles. And I think all of that is what forms your evidence base. Back to my earlier point, it is difficult to explain each kind of, um, component of the neural network models underneath. And that just won't happen. If AI providers did more in the transparency side of things, I think that will help the pace of this space. But they also have a motive to get these models out there very quickly. And so it's on us to make sure we come in and assure it.

Speaker B: I think that supply chain point is super important. Uh, it's got better along the way, I think in that immediate post, ChatGPT 3.5 Rush, there was this sense, I think, among the hyperscalers, the model providers that, well, just take it or leave it, uh, and then over time model cards have got better and explanations of how the models work have got better, but they're still by no means perfect. And very often what you see corporates doing, corporates and governments and all organizations is having to buy solutions that they have a partial understanding of. And going back to that point around ability, benevolence, integrity. If you're buying solutions you don't fully understand, and then you're talking to your customer base and they're asking for detailed explanations, you're caught in the middle. And I think the worst thing you can do then is actually bluff your way through this. You've got to find a way to square that off and be quite upfront about what you know and what you don't know.

Speaker D: And I think we've talked on previous podcasts about how our ability to Think critically as humans is so important when it comes to AI and being able to ask those questions of what it produces and how we use it. And I know there'll be people out there that would love to know, that would want to see behind and see how it works and understand and have those three things covers. But then there's also people like me who like to use it and see what happens and kind of maybe have a positive experience of it. So then we return to it and we kind of see good results and it provides us with what we need. Is there a link between that kind of positive experience and our ability to trust it?

Speaker B: I think, again, context is everything here. And if I go back again to my days as a regulator, we did work with Manchester University in 2018, 2019 on explainability and trust, and we were running, uh, these large workshops called Citizens Juries, with lots of people who are representative of the uk, so not specialists, people who are just representative of the UK with the same level of understanding. And what we found was that people were willing to trust AI in certain contexts much more than others. So, for instance, if it was a diagnostic tool which was approved and used by the nhs, where there's already a huge level of trust, and it was being used for their benefit, people would say, fine, we're going to go with that because we can see the benefit and we trust the nhs. If it was a hiring decision by a faceless corporation, they were less willing. So a lot of it, uh, I think actually people are much smarter in this area than we give them credit for. We assume people are just blithely going in and trying things just for fun. I think people are making smart decisions. If it's a really cool toy and you're not sharing much personal data and it's just on your phone, it's doing some cool things, well, then, great. If it makes your dog look like a cat on a fine. But if it's going in and it's telling you, it's giving you a guess on your medical diagnosis with some symptoms, then you're going to go, okay, well, who's telling me this and why they're telling me this and why are they directing me towards this particular supplier for a solution? So I think people make lots of these little decisions almost intuitively, and I think normally they're pretty smart about it. So I think, yeah, I think if something gives you a bit of joy and it's fairly low risk, you're going to go and do it.

Speaker C: I think there's another interesting angle to this as well, which I've encountered a couple of times, this kind of over reliance on trust almost. There was a very specific example where um, a client had developed an AI tool in their financial crime space, was doing really well, exceptionally well, to the point where its accuracy was far greater than what was previously done by humans. Um, and over time they trusted it so much that eventually there was a bit of, um, people left the team and skill set retention was an issue. Fast forward a couple of years later when it started deteriorating in performance, they didn't have the skills around that team to fix the problem. Um, and it was all a bit of a mess. And, and so I think there's also a bit of a danger on what happens when you trust it too much and you become over reliant on it. So it is threading that needle a bit carefully as well.

Speaker A: Definitely. I suppose you're thinking about applications of AI within firms themselves. So coming back to that workforce point in particular, how important is it to have almost like an audit trail of understanding, particularly if you're talking about something like an agentic workforce, how AI is actually being worked, where it's been used, why it's being used, et cetera. How important is it to have an understanding built in of what the agentic workforce is actually up to and how you understand it?

Speaker C: Yeah, I mean, I think the agentic problem is, uh, sort of extrapolates the whole problem by factor of 10 or more. You know, we're sort of moving into this task orientated system to now a goal orientated system where you have multiple agents all autonomously having the power to make decisions, read, write, access tools. Um, and so in that world I've experienced many firms expressing nervousness about letting these agents go loose in their systems without the necessary risks and controls. Um, but I think part of the answer to that problem is as you say, the kind of auditability, the kind of traceability of these systems. I think what you really need is a kind of real time monitoring system in place that enables you to pull out the telemetry of all of the interactions in real time and having in place the right guardrails, risks and controls. That way you're able to at least monitor uh, these actions. And to the extent possible, I think safety by design should always be first principles as well. I think when you're setting up an agentic system there may be certain agents that um, you restrict access to or you make them more deterministic by the way you construct them. So I think there are definitely ways to make it make agentic systems less sort of adventurous almost and a bit more deterministic. Uh, and so I think, you know. But to your original question, yeah, I think um, that the kind of traceability, auditability of it is super important. Otherwise you don't really know what's going on.

Speaker B: I think one of the challenges there is that these new technologies, whether talking about AI and talking about LLMs or agentic or we're moving to world models, whatever they do, and they will fail in different ways the way humans fail. One of the challenges I've seen again and again is organizations trying to apply risk models which are based on deterministic technology and humans to AI. And it's going to fail in different ways. And so if you're not actually kind of thinking through how things could go wrong and just using old world controls, then you're going to get caught out along the way. So you've got to be thinking through how this works. And avtal to your point, I think with agentic, where you could have a sequence of decisions being made without any human supervision and an outcome in the real world, I think we're going to see sadly some um, real interesting but maybe messy examples of failure which will seem very strange to us because we're just used to kind of like the old world of failures. These are going to be different control failures.

Speaker D: Looking for a way to make a difference while you shop well with scope, you can. Scope is one of the national charities Deloitte supports as a part of our social impact program. Visiting any of their retail stores or online shop helps to support scope's work to create an equal future for disabled people. Every pre loved item purchased or donated funds vital services and helps keep millions of items out of landfills. It's good for people and it's good for the planet. Just visit scope.org.uk to find out more.

Speaker A: If we maybe move on to talk about how trust itself is actually built. Um, leadership is obviously such an important thing here. So who actually should own trust in an organisation, you know, is this board level. Who needs to take ownership?

Speaker B: I can tell you who. It shouldn't be by themselves and it shouldn't be the Chief Technology Officer by themselves. It shouldn't be the General Counsel by themselves. It shouldn't even be the Chief Risk Officer by themselves. What we're talking about here is general purpose technologies, uh, and we're talking about often, as you said, rapid adoption of those technologies, uh, across an enterprise. So it has to be uh, a cross functional play, I think what's more important than whether it sits with any particular individual is having clear allocation of responsibilities across the board. Personally I am more in favor of trying to actually allocate responsibility across existing roles than creating brand new AI risk individuals. Because in the end this is just another technology or another set of technologies. It's very exciting, it's fast moving fine. But the risks to consumers are going to be the same kind of risks you've always had. You know, if it's a bank it's going to be unfair credit decisions. For instance, if it's a social media firm it might be about protecting children. Um, it's just another iteration of technology. So I'd be challenging all the usual risk management functions to say in this new world how are you managing these new risks?

Speaker C: AI is quite pervasive in an organization. It touches a lot of different areas of a firm. Um, it's not just constrained to uh, one area. So uh, that friction is proving to be challenging to get uh, an aligned ownership. But I completely agree, I think there's some shared responsibilities there across all the different areas.

Speaker D: I'd love to hear from both of your experiences about what you've been seeing in organisations that are getting it right. Yeah, if you've got any examples.

Speaker B: So I'll start with a point that I wanted to cover and this comes in here I think which is around use of standards and frameworks in those kind of areas. I think my heart slightly sinks even though I'm a regulatory nerd I should love these things. My heart slightly sinks sometimes when we get on talking around these areas. But that is kind of we're moving to a more mature uh, world where use of standards and frameworks is starting to become appropriate for AI. The big international standard is ISO 42001. Uh, that seems to be being adopted by a lot of organizations now and it's come on leaps and bounds in adoption in the last couple of years. There's also the NIST AI Risk Management Framework which touches uh, on some of these points we already spoke about. Building trust actually discusses trust in it. So that's also helpful uh, and starting to look at this and there's many other things out there. I'd say those are the two main uh, frameworks out there. So I wouldn't say, I wouldn't recommend one or the other. In particular I think what I've seen among the better managed firms is, is they are looking at those frameworks and either taking the best of both, they are quite different ways of doing it, or Going down one road or going somewhere else. But they uh, are now saying, okay, how do we actually have something which encompasses a level of maturity around how we're using AI in the organization?

Speaker C: I think the firms that are doing it well, definitely some common characteristics there. Safety by design, not thinking about trust at the end, but at the offset whilst they're initiating and producing their use cases. If you get the trust angle right, it really becomes a uh, kind of transformation enabler rather than a thing that slows you down. If you've got good governance frameworks, you've got good testing, you've got good risk and controls. Actually it helps you go quicker because you move through the process quicker. If you, you don't have that, you just end up sort of a bit of a bottleneck at the end. But I think again, sort of to your question, I think also taking a risk based approach has been, I think a really, um, good way of putting in proportionate risks and controls because you don't want to be spending the same length of time testing a PowerPoint AI tool versus something that has medical implications. I think having a risk based approach uh, enables you to allocate your time more effectively and I think that. And then sort of good governance. You've mentioned it a couple of times, but I think it's a no brainer. If you've got good governance frameworks in place, risks, policies, roles and responsibilities with authority to make decisions. I think again that will help you move M through the process quicker. And so some of the firms that I've seen doing exceptionally well and have use cases in production, I think they've ticked most of those boxes.

Speaker B: I think one of the key things is joining up the good practice, which is obviously a good thing to do. It's often the ethical thing to do. We all like doing these things. Tying this up with the commercial realities of what you're doing. Zoom Infra is a B2B company. We focus on enterprise customers. They have high expectations around what we do and we use things like certification to actually articulate that we do this well. But the good organizations aren't just doing this stuff for the sake of it worthy though that is they can show how having good governance in place and good controls and maybe one of these frameworks is building trust with their customer base and enabling them to sell more stuff, whatever the stuff is.

Speaker D: So we've talked about kind of ways of building trust and we've mentioned assurance. So I wondered if you could just talk a little bit. Maybe either of you take this one about what is AI assurance?

Speaker B: I'll go first because I think it's actually a really interesting question for the whole of the uk because AI assurance is something that the UK government wants to be a leader in and for good reason. We have a very vibrant market in the UK around AI assurance. If there's anyone wants to go down the rabbit hole. There was a great UK government report on this a couple of years ago, sizing the market and looking at how it all works. I think the key thing to say is that AI assurance is not one technique. AI assurance is the whole sweep of uh, stuff we can use to get confidence and ultimately to build trust in AI. So it covers technical things such as model audits, it covers standards and certifications, it covers looking at the overall process and the life cycle. And with all these areas you're looking to kind of build up this patchwork of areas of assurance, in which case to finally kind of build some trust.

Speaker C: I think that's a great summary and I think just building on that, the UK are indeed doing a lot in this space and you've got DCIT and NPL and they're all working towards some common understanding of the definition AI assurance, but also to some extent some protocols guidelines around this. But I think it's probably one of the key answers of building trust, which is if you are able to and assurance in the sort of traditional sense. Many may interpreters assuring to some form of a standard. I think we all know that there are some standards out there, nist, ISO, so but it's not uncommon to go to a client and they ask. I'm not really sure how to test for bias or hallucinations. And I think that has resulted in a bit of a sort of wild old west scenario where no one's quite sure what to do. But AI assurance is definitely uh, a big piece of that jigsaw, which is sort of making sure you've got a good governance framework in place, but also good testing protocols in place. Whether that's extends from accuracy, bias, fairness, explainability, you know, we've got a good sort of trustworthy framework which we align our principles to. But I think if you work your way around the principles of what makes trustworthy AI and assure it towards that, I think you then build an ecosystem that is trustworthy. So I think AI assurance there is super important to building uh, that trust in an organization.

Speaker A: I suppose it's how do you measure that public confidence piece? As you say, there are obviously standards and let's take another example from another Industry, you know, food hygiene, for example, There are standardized inspections which you know, have gradings which are clearly displayed, which we can understand even if we're not specialists. How do you measure public confidence though when it comes to AI? That's trickier, isn't it?

Speaker B: We've got a long way to go. You've got things like food safety, safety and airlines as these poster childs elsewhere for how this should all work in terms of trust. And when you have that kind of environment, it's taken decades to get there. And whenever anything goes wrong, food poisoning or heaven forbid, uh, a kind of plane accident, then that is headline news and often takes of a lot long time to unravel. And they dig deep to find those root causes and they improve. We are, you know, the maturity of what we have over here is nothing near that. I hate to say it, and I very much hope that it's not organizations that any of us are working with. But there'll be many bumps in the road before you get to that level of maturity and trust with AI and technology without being doomstra about it. But we should be realistic. Right now, trust in AI, at least among the general public, but I'd say among some corporates as well, is plummeting. You know, as we record this. A few weeks ago, Eric Schmidt was giving a commencement speech in the US and was booed when he mentioned AI. If we walked out of the studio and asked people about AI in the street, they'd say I'm worried about what's doing to my kids and I'm worried about whether they're my kids or indeed I will have a job in a few years time. They're not going to say I'm really excited about AI. So, uh, we've got some headwinds to work with before we get to this kind of high trust environment. We will get there because in the end it's the only way any product or any technology succeeds. So we'll get there through some bumps in the road, but in the end we'll get to a mature, high trust environment. We have regulators and standards and certifications and people just do it without asking questions around it. But we're nowhere near there right now.

Speaker A: So actually as we start to come to a close, let's maybe sort of round off by starting to think about maybe some takeaway, uh, advice for some of our listeners. So aft, let's come to you first. What is the first step that firms actually need to take when they're ensuring that they're embedding trust in AI, in order to be able to scale with

Speaker C: confidence, there are many things, but one of the first things firms should try and do is get an AI inventory in place that also classifies the use cases in risk categories. So if you've got a sense of where AI is being used across the entire organization and you have a sense of high, medium, low, or whatever the categorization is, you're then able to start putting in the appropriate set of proportional controls, risks and the subsequent actions. But I think that's one of the first steps. And you would be surprised at how few firms out there still don't have a sense of where AI is being used in their firm. I think only once you've got that, then you can start thinking about the next steps.

Speaker B: Yeah. And that I think is a very fast evolving discipline. On the one hand you have shadow AI, which we haven't touched on too much, but I'm sure is well known to the listeners. Going back to the very first point you made, Andrea, uh, are the staff just using their phones because it's a better solution than what's the in the firm then? There's quite a good ecosystem now of discovery tools to shed info when AI is being used in an organization. But also I've heard of corporates getting some quick wins just by looking at corporate credit card spend. You'd be amazed how often, especially if, uh, the center is being a bit slow, businesses will just go and use their corporate credit cards to get a subscription to a new model and suddenly you've got a whole new service you didn't know about. So running through these different ways of discovering, uh, where the tech is being used is helpful.

Speaker D: So I think I'm really struck by, um, I think the two takeaways for me are around the benevolence point because, and the bumps in the road not to kind of center on the negative thing. But I think from what you said, we are needing to build trust as organizations in AI but we also have to live with the fact that it won't always be be right and there will be bumps and we have to live with that tension, but we have to maybe trust in the kind of benevolence factor that actually our intentions and what we want to use it for align with. You know, we're all on the same page with that. Uh, um, I think I'm really struck by what you said there.

Speaker B: Honesty and transparency here is really key. I completely agree and I think very often, and this may be me m being optimistic, optimistic. But if you are honest and transparent around what you're doing. If you're clear about what the risks are and you're clear you're striving to manage them, then whether it's consumers or whether it's other businesses that you're working with, they'll accept a few bumps in the road within reason if they know that you are striving to manage it. Conversely, if you sit there and you bluff your way through, then the first time you get caught out, you've lost all trust.

Speaker A: So let's round everything up then and come back to the big question. Um, can we trust what we don't fully understand? Simon, let's go first.

Speaker B: You can if you trust everything else around it. So in terms of what's building this trust, you know, we just spoke again about, you know, the ability, uh, benevolence integrity points. Is it whether you trust the institution that is using the AI? Is it trusting their track record? Are you trusting the fact they've explained things to you and they're transparent? Is it because everyone else is using it and trust some of the people who are using it? You can do all these things, which means you don't need to understand the model in the same way you don't understand the dreamliner you're flying on or the entire supply chain for the burgers you're buying in the supermarket. But you need to trust something if you're going to trust the thing like it.

Speaker C: ABTA yeah, I'd agree with that. I mean, if you can trust another human, you can trust an AI system. I think if you've got the right governance in place, if you can put the right risk and controls in place, solid testing, enough humans involved where there needs to be, you've got an ecosystem where you're building an evidence base that enables you to understand the system more. And if you can understand the system more, you can then make informed decisions on how to use it safely. And sometimes the answer may be it's not appropriate in this scenario, but in other cases it may be. And I think if you do that, you can take a risk based approach to her.

Speaker A: Well, both, thank you so much for joining us. In the Green room that is pink today, you can trust that it is normally green. But thank you so much for joining us.

Speaker B: Thank you.

Speaker C: Thank you.

Speaker A: Thanks for listening to this episode of the Green Room by Deloitte. We release a new episode every other Tuesday with another big question, so don't forget to hit follow or subscribe to this podcast wherever you're listening or watching. And make sure your notifications are on so that way you'll be alerted whenever a new episode drops. This podcast is produced by our very own Pod Squad. Original music m by Ali Barrett.

Speaker C: Sa.