Auditing in the Age of AI: Risks, Rewards, and Practical Steps

Hello, I'm Richard Chambers, Senior Advisor of Risk and Audit for Optro, and welcome to another in my continuing podcast series, Speaking of Risk and Audit. I'm joined today by a colleague from KPMG, he's a partner in advisory services, Charles King. Now, for those of you who don't know Charles, I've asked him if maybe you could just share a little bit about his background as we get into our topics for today. Yeah, sure.

It's good to be here. Thanks for having me. I started my career at KPMG as an internal auditor. I was really interested in business process improvement, and that's what drew me to audit.

But I pretty quickly realized after a few days of lugging around three-ring binders and numbering pages with red pencils that I thought there's just got to be a better way to do this. And so really early in my career, I started down the path of learning data analytics and did that for several years as an auditor and eventually went back to school. I got a master's from NYU in data analytics and went off into industry to start a data analytics practice. Did that for about five years.

Got totally out of audit. I was leading revenue management and optimization for a big food service company. But one day the siren song of the audit profession called me back and I rejoined KPMG and have really had a focus ever since I got back on trying to embed technology into the way we do audits. You're what we call in the service industry kind of a boomerang.

That's right. You were there, you left, you came back. That's great. Well, I admire the work you do, and you were very fortuitous in your choice of getting some expertise in data analytics when you did.

Let me ask you, what do you think are the most significant pressures and challenges that internal audits are currently facing, whether it be data analytics or beyond that? What strikes you as one of those challenges that might be keeping chief audit executives awake at night? I think there are a lot of challenges that are keeping chief audit executives up at night right now. The risk landscape is getting more and more complex in almost every aspect, whether it's geopolitical risk.

Obviously, the advent of AI and agentic AI and all of the technology investments that are being made by enterprises today is driving a lot of new technology and bringing with it both opportunities and risks. Fewer and fewer people are studying the majors that traditionally lead to an internal audit. Career so recruiting and retaining talent is difficult and generally as all of these headwinds in terms of risk and complexity get bigger and bigger we're not seeing internal audit budgets or internal audit resources expand commensurately so they are really struggling to meet the needs of the organization with fairly limited resources so there's a.

Chief audit executors are battling on all fronts right now. It's a tough job. Absolutely. And I agree with you.

If anything, I think the profession's going through a very stagnant phase in terms of resources. And in some ways, they're declining in terms of real purchasing power just because they're not keeping up budgets, they're not keeping up with inflation. That's right. So Charles, beyond compliance, where's internal audit adding the most demonstrable strategic value to organizations today?

Well, I think audit's always at its best when it's helping leadership make decisions in the face of uncertainty. It's not about checking boxes and compliance. It's about being an advisor. I think on our best days, we do that well.

But as an AI person, I'm probably a little bit biased. But I think as we look at the AI programs that almost every enterprise is rolling out these days, sometimes the risk and the cost of developing AI agents in the first line or even the second line can be pretty high stakes and internal audit teams in many cases are stepping up and saying we will be the laboratory where we can build some of these solutions it's relatively low cost it's relatively low risk versus some of the other key areas and it's an area where I think internal audit doesn't have to work quite as hard to earn a seat at the table.

And by being first out of the gate or one of the first out of the gate, they learn a lot about the technology that serves them well as they go through auditing other business processes or AI governance functions that are in the business of rolling AI out to the entire enterprise. So let me ask you, so what is one major trend in the audit and risk community that you believe is being overlooked or underestimated by most organizations. And you mentioned AI. I don't know to what extent you think that could be one of those things.

Our research from Optro has sort of reflected that maybe internal auditors are a little bit timid with that. But what would be a major area or trend that perhaps internal audit needs to be a bit more aggressive and deal with? Yeah. So you might think I'm going to say AI, but I think one of the areas that I'm not hearing enough people talk about is post-quantum cryptography.

We've been five years away from quantum computing for 30 years. I totally recognize that. But I think some of the advances that have been made in the last few years makes the likelihood of quantum computers at scale much more likely in the next few years. And that is really going to upend our approach to cybersecurity and privacy.

And I'm not seeing that on enough audit plans, and I'm not hearing enough people talk about it. And I think that that is really going to catch us off guard when it happens. We're one major breakthrough away from a quantum computer that can break all of our encryption. And I think regardless of whether you're in banking or healthcare or technology or any number of other sectors, making sure that we're ready when that happens and not scrambling to catch up is really, really important and under-discussed.

So what'd you call it? Post-quantum cryptography. Post-quantum cryptography. One of the trends that you're seeing now is a lot of organizations are taking a harvest now, decrypt later approach.

And by organizations, I mean primarily nation states. They're just collecting all of the internet traffic. And they understand that when they're able to decrypt it, they will. And so all the documents, the contracts, the banking information, they'll be able to decrypt all at once.

And they may or may not announce that that's happened. And the ability to crack the encryption also means they're not just intercepting data, they're also able to impersonate actors online and send emails or log into accounts. And to the extent that we are not hardening our organizations against that reality, I think we are really missing the boat on something important. Certainly something for our listeners to pay attention to.

In your role as the internal audit and controls innovation leader, in your firm. How do you see the conversation around AI evolving, specifically the shift from a theoretical concept to the structured AI governance programs? What's your observation been? Yeah.

Well, it's funny. I've been talking to internal audit leaders since ChatGPT came out over three years ago now. And for the first 18 months, I felt like every conversation was almost exactly the same conversation. People asking fairly introductory questions about what the technology can do and how they should be thinking about it, both from a risk standpoint and from the standpoint of how they can use it in their departments.

What has changed, I think, over the last 12 to 18 months is I've really started to see a separation between the teams that are leaning into AI and building agents and moving forward with their program versus the teams that either haven't done much or have maybe rolled out a tool like Copilot, but then left it at that. What we learn as we start to build AI solutions in organizations is that it's really a momentum game, building your first agent is tricky. You've got to figure out how to navigate your organization's governance.

You've got to understand what your particular tool set is able to do. You've got to think about what you want agents to do. And there are all kinds of personalities and people that are asking questions along the way about how this agent's going to work and what data it's going to touch and how you're going to protect the data and all this. But once you get to your second or third or fourth agent, you start to get some muscle memory in terms of how you build agents.

And what that means is that momentum starts to really build up and the leaders start to lead even more. And the people that haven't started down that journey. Or left behind. And so what I really encourage everyone to do is, is just dive in.

Even if your first couple agents are not miraculous revolutionary agents, if they're just small, little incremental things, the, the lessons that you learn from building these initial AI solutions will serve you well when you do get to the big transformational ones later. Yeah. When we, uh, We did some research recently, and what came back from the population that we surveyed, chief audit executives, about 25% of them said they were actively using AI. About 50% said they're piloting or experimenting with it.

And then about 25% said, you know what, we're not really going to do anything for at least a year. Does that sound about right? Yeah, I think that's about right. Right.

And I think that probably reflects, in many cases, the enterprise posture towards AI. Generally speaking, the enterprise is not making investments or not making investments because of what the chief audit executive wants or doesn't want. They are leaning in to AI and hopefully the internal audit team is a fast follower and using that enterprise investment, the enterprise strategy to drive what they're doing in audit a little bit better. But I think to the extent that you are part of an organization where the organization makes AI a priority and your team isn't, I think that's a pretty bad place to be.

Well, you sort of touched on it a second ago when you were talking about agendic AI, but from a. From a practical perspective, what are some of the most immediate and impactful cases, use cases for AI within internal audit processes itself? I mean, what would you say to a listener who's sort of on the fence? They know they want to try something.

You know, what would your advice be? Yeah. Well, generally, when people first get an AI tool, they will start to do what I call ad hoc prompting. They'll just ask questions and get answers.

And over time, with some practice, you and your team members will start to find that you're really getting good results for certain types of prompts and maybe getting less desirable or less efficient results for other types of prompts. And that will start to coalesce into a kind of prompt library. And you can have prompts that are engineered and tested a few times that you can publish to your team. Usually where I see teams start is in the planning phase of the audit, when you're trying to synthesize a lot of documentation, whether that's policies and procedures, prior audit work papers and findings, regulations.

Other industry content, maybe from the Institute of Internal Auditors or your co-source provider, wherever you kind of get that material. The ability to synthesize that all together, think about what should the scope of my audit be, what questions should I be asking, what controls should I expect to see, that's usually a really good early starting point. We also see teams have a lot of luck in the reporting phase of the audit when they're trying to write findings or review and analyze management action plans.

That's another good starting point. When you start to build agentic AI, then you can really start to have an impact in field work with things like testing controls or doing data analytics. And as you get more and more advanced along your journey, you can start to become more sophisticated about how you expose even just simple prompts to your team members so that there's a little bit less of the copy and paste and moving back and forth between tools. That's usually one of the areas that people get into once they've been down the road a little bit.

So Charles, what skills and competencies do internal auditors and consultants need to be cultivating to remain relevant in a world that's increasingly being augmented by AI and sophisticated technology? Yeah. Well, one of the conversations that is ongoing with my team is that all risk now is technology risk. There is not a business process.

There is not a department. There is not a transaction that is not touched by technology. And. Many organizations still have this distinction between a business process auditor, a generalist, and an IT auditor.

I think that distinction is becoming less and less relevant. So if you are one of the generalists, I think you really need to lean into better understanding technology. AI is a great place to start. AI is an opportunity and a risk, like I said, everywhere in the organization.

But AI relies on data, data quality, data governance. That becomes more and more important. AI raises new questions about cybersecurity that we all need to be more comfortable with. And I think we're heading into a world where we might actually see some of these traditional metrics like control counts go up.

As we get into a world where there's less management review, less approval, and more configuration of systems. And so understanding the capabilities that systems have, what can and can't be configured, I think is going to be really, really important for all auditors. But I, you know, I'm a tech guy. I love technology.

It interests me. I continue to believe that for every auditor, regardless of your industry, your function, your years of experience the most important skill is curiosity yeah and i tell everyone to cultivate a sense of curiosity and inquisitiveness that will serve you well regardless of what happens with technology regardless of what happens in how your department is shaped uh in the years to come that is that is the fundamental skill set along along with relationship building and a few other things, but I think curiosity is the killer app of internal auditing.

Yeah, no, I absolutely agree with you. So, so Charles, you're, you're leading innovation in KPMG. Um, what, what is your philosophy on balancing the traditional audit rigor. Um, with the need for continuous technology innovation?

Yeah. Well, you know, I don't think that those two things are really in tension with each other. I believe that as we develop more and more sophisticated technology, it actually allows us to be more rigorous if we're careful about the work that we do. You think about something like data analytics, the ability to look at a full population, really extract bigger trends out of data, or even do simple things like picking better samples, that adds rigor to our work.

And I think now that most of us have AI tools on our desktop, that is giving us even more opportunities to... Dig deeper and ask different questions and get different perspectives that we might not think about because of our own internal biases that an AI conversation might clue us into. So between all of these technologies, we have to never take off our skeptical auditor hat. And we need to always treat AI.

I always tell people, treat AI like a precocious intern. They are precocious enough that you want to give them some big, complicated task, but they're still an intern and you want to rely on your judgment, your experience, your competency to review what they're doing. And I think generally we have a pretty good culture of that internal audit. It's kind of built into our DNA.

Skepticism. Right. That's right. And we have this one up review, you know, the senior does the work, the manager reviews the work.

I think that's kind of baked in. And that's how I think about. AI, you know, there's this concept of work slop. I don't know if you've heard that term before, but the concept of work slop is just to sort of mindlessly accept the answer from the AI.

And I think as we embed AI, not just into our day-to-day work, but into the way we collaborate with others, there's a norm building that's happening right now. Just like when email came out. We agreed with each other that we weren't going to send 20 emails to the same person in an hour and we weren't going to write in all caps and all of these things that nobody puts in a policy, but we all know there's just good professional conduct, right? I think the same thing is happening right now with AI where these norms are starting to develop.

And one of the things that I think we can do as leaders is to harden those norms and make sure that we're talking about what is and is not acceptable behavior when it comes to AI or any other technology. You know, people sometimes over-rely and we need to make sure that they are, doing what we want them to do as auditors, which is use all the technology to their advantage. But first and foremost, bring your judgment and your experience as an auditor to the bear. Great conversation, some real insights.

I think you're hitting right on the head some of the issues and challenges, particularly in relying on and becoming comfortable with the use of AI. Charles, thank you again for joining us today. It's been a fascinating conversation. And I want to thank you for joining us on another episode of Speaking of Risk and Audit.

For Optro, I'm Richard Chambers. Thank you.