Governing AI at Speed: Policies, Processes & People

Hello, I'm Richard Chambers, and welcome back to another episode in my podcast series, Speaking of Risk and Audit. Today, I have with me a very special guest, the General Manager of AI Governance at Optro, Guru Sathopathy. Most recently, Guru was the founder and CEO of FairNow, which was in the headlines recently when it was acquired by then Audit Board, Now, of course, Optro. Guru, welcome to the program.

Richard, thanks for having me. A pleasure to be here with you today. For those who are watching and listening, tell us a little bit about yourself. I know you're well known in the field of AI governance, but maybe some of our listeners and viewers wouldn't know of your background.

So I would just ask you to share a little bit about it. Thank you, Richard. I'm happy to. So yeah, I've been in the AI and the technology space for pretty much my whole career, 20, 25 years across my time in academia, the corporate world, as well as the startup world.

Obviously, before I joined Optro, I was a founder and CEO of FairNow, and I've spent time kind of not only focused on governance, but overall AI, how AI is transforming the business world, building products and building technologies in the AI space. But then over time, I've also moved into the governance of these technologies, which I think we're going to obviously talk about over the next few minutes, but I've become really passionate about this topic. Well, it's just great to have you with us.

You know, Guru, for our audience, I'm sure if they're like me, they hear the term AI governance with greater and greater frequency. I mean, first, you know, we were inundated with AI more broadly. And then in the last year, we're hearing a lot about AI governance in a way that internal auditors and risk managers really need to understand. And maybe you could help us by explaining what's sort of the core components of AI governance.

If someone asked you to kind of define it and talk about what its core components are, what would you say? Yeah, I tend to use a framework called the three Ps, policies, processes, and people. I think you can break it down into those things in a kind of a simple way to kind of think about the overall things. Otherwise, AI governance can be quite complex and confusing to people.

So let's start with policies. Policies are how an organization, Richard, thinks about how it intends to use AI, right? What are the legal constraints that it has? What are the ethical constraints?

You know, hey, we're not going to use AI in these particular use cases for XYZ reasons. We're not going to use data to be trained, to be used for training for these types of AI models, right? So those kinds of guardrails and policies that an organization develops are its first order of business, so to speak. Next, how does it actually implement and execute those policies, right?

That's where the process, the second P comes in. How are you monitoring your AI technologies and tools? How are you testing them? What steps do you take before you go ahead and deploy a system?

How are you working with vendors to ascertain that their technology is well-governed? How are you thinking about the continuous monitoring and documentation and approvals that need to go into a place before a system gets used, right? So those processes are, in a sense, how you execute your policies, so to speak. And then the third P is, at the end of the day, you absolutely need humans overseeing and having final arbitration on what happens.

And that could be everything from remediating issues that come up to being held accountable when things go wrong. And so having, again, those clearly defined roles and responsibilities of where humans need to be involved in the governance process. So again, I think those three things together define and constitute a really good governance program. Of course, the devil is in the details, and hopefully we'll talk about that some more.

I think that's an overarching framework that I think most companies are starting to work with today. I think for our listeners and viewers, those certainly align with how we undertake our mission on a regular basis. I mean, the whole issue of controls, I sort of think about the policies or the controls. So I like that 3P model, and I think it will resonate with those who are out there watching us today.

Okay. So tell me a little bit about what do you think are the primary factors from regulatory pressure to technology maturity that have made AI governance as critical and as urgent of a priority for organizations as it's become? Yeah, you know what's interesting is I think there's been a shift in what's driving that priority of AI governance. I think initially it was a bit of the regulatory pressure.

There were a bunch of laws that came out in 23, 24. the EU AI Act made a big splash. I think it was in 24 when it got first passed and said, hey, enforcement's coming in a year to two years. And of course, people often remember GDPR and were like, hey, this is going to be part two of that.

We need to get prepared. And so I think the initial impetus was regulatory pressure. I have to say, in my view, that has shifted, Richard. And the speed at which this technology is moving is.

And the business risks are what I'm hearing are driving more of that need for AI governance. And so let me explain that, right? So AI has been around for quite some time, and especially machine learning. Methodologies have been around in many sectors for decades.

But very quickly in 2022, we pivoted to the release of large language models and these chatbots and this generative AI. And then we've moved on from there. And now we have AI agents that are starting to work and deploy and perform tasks. And so the rapidity of that, and agents have many additional layers of risk that need to be governed.

So that progress of this technology, and then when we think about the business risk, right? First, the performance of these systems, are they performing as intended? Are they actually driving more efficient outcomes? Are they giving you better results?

Are they biased? Are they reliable? Are they hallucinating, right? Like there's a whole dimensionality of performance that we take for granted, but they need to be constantly tested and monitored to ensure that's actually happening.

So that's one dimension. The second dimension is around security. And I know cybersecurity has been around for some time, but here's where it gets complicated. Cybersecurity has been traditionally focused on data breaches, right?

Making sure sensitive data, private data is not being released, being breached. AI agents take that to a whole nother level. There was a recent example with McKinsey where a third-party vendor was able to breach their model endpoints, their agentic endpoints, and not only have read access to data, Richard, but they had write access. What does write access mean?

It means you can delete data. You can override data. You can even rewrite prompt prompts for those internal McKinsey agents. And through those prompts, as you know, you can reimagine how they think and how they process information and what kind of outputs they give you.

So the risk surface area has increased dramatically. And then the third component of business risk is just transparency. Do we know how these systems work? If someone came and said, hey, explain that decision that your agent made, can we do that?

Are you able to do that as an organization? Do you have the auditability, transparency, and explainability that needs to go along with it? So I would say it's how quickly the technology is evolving and the surface area of the risks, business risks that are growing significantly that's driving the need for AI governance, even more than the regulatory pressure at this point. Yeah, that's some great insight.

You know, Guru, the term agentic AI, AI agents is getting used a lot. I just read an article just very recently by someone who was arguing that that that there are a lot of that that term is being misused a lot, that that there are people who are claiming that that they're delivering agentic AI or they're using agentic AI when in fact it's really not. I mean, they're using AI, but it's not agentic AI. How would you distinguish the difference?

I mean, if I'm going to say I'm using agentic AI, what does that mean to you? Yeah, that's not surprising. I agree with that article, first of all. I don't know which article you're referring to, but the broad description that you laid there, I agree with it.

And not surprising, right? I think this is par for the course, throughout the course of technology, right? I mean, even going back to like, what is AI? Yeah.

I think for over the last 10, 15 years, you would have companies claiming they were using AI or had AI and you dug deeper and it wasn't really AI. It might have been some simple analytics or whatnot, right? So on and so forth. So I think you're seeing a repeat of that from a marketing perspective in the use of agentic AI.

There's always a tendency, I think, I'm sorry, there's always a tendency, I think, to latch on to a buzzword and then suddenly you're using it in everything. And so, yeah, I think you make a good distinction there. I'm sorry, I didn't mean to interrupt. No, no, I completely agree with you.

I completely agree with you. And that's always the clash between kind of the engineering folks and the marketing folks, right? Like, I mean, how much can we push the terminology and the nomenclature of how we describe things? But so the way I think about agentic AI, the real evolution and innovation of agentic AI, as opposed to generative AI, right?

And that's what I'm contrasting with, is the ability to write and perform tasks, right? So as opposed to just generating content, these systems can actually interact with tools in your ecosystem. They can do things. They can perform tasks.

They can execute things on your behalf. So it's moving from kind of. Writing and responding and so on and so summarizing, which is what the generative AI tools to actually performing tasks. And I think that that distinction is quite noteworthy, because when systems can perform tasks, that means they have access to a lot of your data that they need to be able to use to perform tasks.

And they can read and write and access those tools. This could be the tools like your financial systems and records, your human capital systems and records, your audit systems and records, they have access to these tools now as humans would and can do a lot. And so you can see how both the potential is so much higher in terms of what these systems can do, but also the risks are so much higher. But that's how I make that differentiation.

Can they actually perform tasks on your behalf? Great distinction. Thank you very much. You know, as a leader in the risk and compliance technology space, how is Optro addressing the need for integrated AI governance within its own platform and for its customers from your perspective?

Yeah, so I think I'm projecting out a little bit into the future, Richard. And as I think we move forward, governance is going to require both a technological component as well as a human component and what I call kind of the human over the loop, so to speak. I want to give you an analogy that I think hopefully your audience will find useful and helpful. When we drive, right, there's so many technological components in the vehicle that are monitoring our seatbelt usage, our speed, our gas tank, or our battery charge, our blind spots, our lane maneuvering, all of these things, right?

That's that technological governance layer. And then when something's off, if you're tired, if the air pressure and your tire is too low, it notifies you and it's up to the human to kind of go, you know, intervene, so to speak. Right. I imagine something similar with AI governance.

You're going to need continuous monitoring. Right. With AI systems, just as you do as you're driving. And humans can't do continuous monitoring.

That's just not what we do. We can't do continuous monitoring. And so you need a technological layer to do the monitoring, to do the orchestration and then kind of inform the humans. So when I think about that technological layer, there's an orchestration layer and a monitoring layer.

And so today, Optro is a leader, Richard, in the orchestration layer of AI governance, the place where you can implement your policies, your controls, your compliance, your governance workflow, all of these things, right? And where we're moving, our vision is to move into the continuous monitoring layer in the future. And so combining that orchestration layer with the monitoring layer is how we at Optro believe that we can be a leader in the AI governance space. I know that's something that a lot of people are looking at and looking for when they're looking for technology solutions is how can it help them better implement and better execute AI governance.

So I think you gave some great insight there. AI governance is inherently cross-functional. So I would just ask you, What is the most effective way to foster collaboration, to get your audit, risk, legal, data science, and IT teams working together? 100 percent.

This is something I talk about all the time. And I'm actually going to take your point, which is I completely agree with and expand it even further, Richard. Absolutely. These organizations are all going to be involved in AI governance.

And I think that's a meaningful difference in terms of how compliance and governance has happened in the past, where you kind of isolate it to the audit function or the GRC function, et cetera. Everyone in the organization is going to be involved. In fact, there's a great quote from Jack Clark that I like to use, a co-founder of Anthropic, who says, the work of every company is going to increasingly shift towards monitoring and overseeing all the different AI systems and agents running around in your organization, right?

As these AI systems and agents start to do more and more of the day-to-day work, the role of the organization and the humans, Richard, is actually going to be one of governance. So governance is going to be everyone's job, not just audit or risk or compliance, right? And so I'm kind of putting your point, your question on steroids a bit, right? It's like, how does everyone get involved in this and how do we make sure that's the case?

And that's where I think, again, having that orchestration layer that I was talking about, that kind of orchestrates that governance process across all your stakeholders, right? Having your data scientists be able to demonstrate what they're doing from a testing standpoint, from a documentation standpoint, from an explainability standpoint, and having your risk folks being able to see that documentation and review it and be able to approve it. And then having different folks having different views and lenses on this overall governance process, that orchestration, you need the help of technology to do that.

Otherwise, it becomes incredibly manual, incredibly cumbersome, incredibly high friction. And so, again, this is an area where technology and companies like Optro, I think, are going to really help that coordination and orchestration across all the different actors in your organization. Great insights. Guru, I have really enjoyed the conversation.

You're so knowledgeable and you've got such a deep background in this space. I hope we can invite you back again, because I know that as this continues to evolve, as the opportunities continue to evolve and the risks, our viewers and listeners are going to hear more from folks with your kind of insight and background. So Richard, pleasure being with you. Thank you for having me.

And look, I'll offer, this space is moving so rapidly. And so more than happy to come on and share more as these things just move so quickly over the next few months and years to come. Thank you for joining us today. And to our viewers and listeners, thank you for joining us as well.

For Optro, I'm Richard Chambers.