Understanding the Drivers of AI Compliance

Welcome back to the Risk and Cybersecurity Podcast. I'm Lauren Ross, a senior manager in Cherry Bekaert Cybersecurity Practice. Today I am joined once again by Steve Ursillo, a partner in our cybersecurity group, and Morgan Haag, a senior manager at Meditology Services, which is a top-ranked provider of information risk management, cybersecurity, privacy, and regulatory compliance consulting services exclusively for healthcare organizations. And so today, our second part of our 3-part series, we're discussing the drivers of AI compliance and how to navigate them. So, Steven, Morgan, welcome back. One of the first drivers that comes to mind is regulatory pressure. So Morgan, I want to start with you. What impact do emerging regulations like the EU AI Act or US executive order have on global AI deployment strategies? Yeah, it's really interesting. I think, you know, honestly, there's been such a divide between what we kind of see on, on the domestic front in terms of regulation, how kind of heavy-handed is probably the best word for it, right? But kind of how involved we've seen the movement versus in the EU as an example. But even then, you know, organizations have to be prepared for regulatory shift. I think the kind of gauntlet's been thrown down in AI that countries and unions like European Union kind of let organizations figure it out on their own, candidly. So they had the EU AI Act. There were a lot of requirements there, but enforcement for that has continued to get pushed back. There's some speculation that that's due to some of the influence from, from large American companies like Meta, et cetera. But it seems like in the short term, you know, if you're just doing business in the American market, it doesn't seem like there's going to be any really significant regulatory action specific to AI. And that's probably going to be the same story, at least for a couple of years here, just because there's a big incentive, even on a geopolitical side, to have American companies at the forefront of that conversation. And in the EU, even, I think due to some of that influence too, The AI Act has a lot of detail around development of models and prohibited use cases and things like that. The security language, candidly, is fairly light. There are some general obligations you have to meet, but outside of that, it's mostly around kind of what are the use cases for the model, how are you sourcing your data, and things like that. But again, the enforcement for that has been knocked back a couple of times. And so we'll see when it does go into effect if it stays there. Either way, just like with any evolving regulation, I think we saw with what we kind of colloquially dub HIPAA 2.0, you know, the notice of proposed rulemaking on the updates to the HIPAA security rule, organizations, it's best to start preparing for that, right? So if you operate in European Union at this point, hopefully you've already gotten yourself fairly well aligned with the AI Act. You're aware of things that you might have to change when that does go into full effect from an enforcement perspective. In the same vein, in the U.S., executive orders don't quite hold the same weight. But it's always better to take a look at the signals, adjust as needed, because there's significant investment in the AI space. And so if you go the wrong way and your model gets flagged or you get a significant fine, that's not good for anybody. So yeah, and you know, you mentioned being prepared and there's been a lot of changes. So there's a great deal of uncertainty in that space. But Steve, you know, how would you recommend companies prepare for this regulatory uncertainty and what kind of things can they start doing now? I think many organizations are, you know, already have a program. Hopefully they have a program in effect to look at both contractual and regulatory obligations to the organization. If not, you know, it's certainly something that is much broader than AI. But to your point and to Morgan's points about the evolution and fast-tracking of some of these regulatory requirements, and as it emerges in with different groups getting involved and obviously different both domestic and international influence, it's really important to have a sound structure that allows you to be, you know, adaptive and agile. So I think having, excuse me, having that AI governance model structured as such so that it's, you know, it's clear folks are accountable, there's executive key management and leadership oversight, that there's, you know, a cross-functional AI and risk and ethics committee that may be established in order to help govern and drive some of the decisions as far as what's happening there. Folks that can stay on top of the different regulatory requirements. So whether that's in-house folks that are really monitoring current legislation, or if you have, you know, other folks that are, whether it's lobbyists or other folks that are on top of, you know, some of the more emerging things that might come to fruition that you might need a longer runway on. I think that's important. And then of course, you know, anytime you empower kind of a committee or a group, you have to make sure that they properly have the tools tools in order to, you know, really interject if there's high risk, right? Pausing any type of deployment, escalating any type of risk, and adapting quickly to manage that risk, I think, are a real key there. So, you know, identifying what regulatory requirements are coming down the pipe, assigning kind of that ownership, governance, and foresight to make sure that folks are doing what they need to do. And then you're going to go back to typical risk management protocol, right? So identifying the risks— our last podcast talked a lot about different frameworks and ways that you can identify risks across your, your AI stack, right? Ways in which you're doing your risk assessments to make sure that you've defined obviously the likelihood, the impact, and any regulatory relevance there as it pertains to being assessing on that risk and then bringing in the proper control mitigation and of course the reporting there. You know, we talk about part of this process and some of the challenges organizations have is really understanding once again that register of all the different types of AI models that they have, having that inventory, that includes the different types of data sources and any type of lawful bias or any type of ownership is gonna be important to all of that. And having that asset register, you can't properly deal with risk mitigation regulatory effects if you don't know the systems and the data that you have, where it's processed, and any sovereignty issues that you may be up against there. So having those measures as part of what you need to assess becomes really key. And then of course, when you're talking about the governance of AI, you're talking about kind of these different components of ongoing monitoring for that governance structure and whether or not you have any triggers to do any type of deeper reassessments against it. So as you're— are you revisiting these new models, new data, or new regulations, or potential impact of incidents, and what's the definition of an incident from that perspective as it pertains to these changes, right? So regulatory obligations are becoming stricter and stricter on time for notification, and do you have the right capabilities built into your information security program and your cyber response program to pivot based on the use of AI and how that's being used and how it's being propagated throughout the organization. So again, you know, this, this kind of culminates just a more modular compliance approach, which we talked about with different frameworks, having the proper risk management measures and having the right governance and oversight to stay on top of the regulatory implications and how that impacts the organization on an ongoing basis. And beyond regulatory pressures, another component to certainly consider is enterprise and vendor expectations. So Steve, you know, what are the top compliance criteria enterprises are looking for when they're evaluating AI vendors? So, you know, you're seeing a lot more targeted questions that are coming in now. I would say 18 months ago there were questions, are you using AI type of thing? And they were pretty straightforward. You're starting to see a much more granular look based on the levels of risk and the expectations. With all these frameworks and regulatory expectations coming out, you know, folks are properly dealing with it from the third-party risk management front, right? So they're going in, they're looking at the frameworks, they're looking at the risk mitigation, they're coming back and they're saying, hey, you know, I'm not hosting this. This is an AI SaaS product or it's something that's outside out of my control. So how do I know whether or not the vendors are doing all of the things that are prescribed in 4200— I mean, 42001 or the risk management framework for AI? So what's happening is you're getting a much more prescriptive runbook on the more mature organizations to understand where the data is coming from, how their data is going to be used, if it's going to be used in training, what service providers, what are the boundaries and the flows of data. What type of safeguards are they putting in, you know, on the front end through the processing and on the back end? You know, are they protecting against prompt injection? Do they have kind of— and when you think about, you know, prompts, you're thinking about the new zero trust, right? You can't— you got to be able to filter and make sure that the prompts that are going in are governed and they're within the guardrails of expectation to avoid any direct or indirect prompt injection, as an example. So are they doing those types of things to make sure that they're safeguarding their And again, depending on how autonomous that system is, you, the more autonomy you have, the more you have to really rely upon a level of least privilege, right? Because there's going to be different aspects of that autonomy that needs to be broken out, monitored, and safeguarded, and transparent for folks, and understanding how that affects your business and the nature of the transactions you do becomes really important. So I think that's another part of, of what's expected. Going through that. You know, there are certifications that folks look at. You know, you look at ISO certifications, whether it's ISO 27001 for security, 42001, you know, for AI governance, SOC 2 properly designed to take into account, you know, the AI systems and the risks that are predicated there, both from a, a workflow perspective, a tactical risk management perspective, and a governance perspective becomes really important. Making sure that there are strong proofs of responsible AI. So getting some transparency on whether it's artifacts or things that show they're testing for bias and fairness, what they're doing to make sure there's no— or they're minimizing drift. AI red teaming, you're starting to see in more mature organizations more AI-targeted red teaming where there are products in the market that will not only perpetrate certain types of red team attacks against the AI stack, but it'll also evaluate that against the organization's policies and procedures. And you can also take the, the same types of attack vectors and build that into almost a continuous monitoring perspective where you can have the traffic go through these particular guardrails and it'll detect certain types of attacks and any type of sophistication on top of those attacks where it minimizes, again, doesn't eliminate, but minimizes the risk against some adversarial attacks or prompt injection, things like that. These are all questions that you'd want to formulate and ask. Does the enterprise have any type of model cards where they're giving a better understanding of exactly what what the model does? Is there clear lines of the data, the utilization of the data, and how once again it's impacted? And then what's the ongoing maintenance? Are they just— is this just a one-time thing that they do an audit annually, or have they built in kind of that continuous governance approach that we would expect? So I think these are all key considerations, and there's always a balance there in making sure you're getting a certain level of depth and transparency. Don't just take a report and say, oh, they have a they get a certification for ISO, they got a, you know, a SOC report, check a box and move on. You really need to understand the risk to your organization, understand the concepts of the shared responsibilities to mitigate that risk, and really hold these vendors accountable for what they need to do to keep up their end of the bargain. And Morgan, you know, what are you seeing? How are procurement teams evolving to assess this AI risk as part of their vendor onboarding? Yeah, I think I'll last on really the last thing that Steve said there. It's, I would say, the responsible procurement teams or kind of the cross-functional teams, if security owns this or if an internal GRC team owns like third-party risk or supply chain risk as an example, really are starting to educate themselves on, okay, well, what, like, how does AI work, air quotes, right? What does a model mean? Like, where does it come from? Because if you're not familiar with some of the core components, You know, the layman might not understand that a huge majority, right, of Air quotes AI capabilities are really built on a very small selection of like frontier labs capabilities. So everybody's using Claude or OpenAI, whoever it is, right, to do some sort of summarization or analysis on a data set. Very few organizations are actually developing internal models purely based off of their own kind of logic and analysis. And so If you don't know that, as an example, if you're trying to understand how is this system secured, who's responsible for the data, etc., you're just not going to ask the right questions, candidly, or you're going to get a response and it's not going to make sense to you. But maybe you just say, okay, great, sounds good, we're good to go, green light, which can obviously introduce a significant amount of risk. Just the nature of security nowadays is that third parties are arguably the largest vector there is. And so, you know, AI serves to complicate that quite a bit. I think beyond that, we've seen kind of a varied approach. So I think for a really, really small team, Steve touched on this a little bit too, but just updating your questionnaires, that that's how you manage your third-party risk. Just adding in a few specific areas around the AI model might be 6 to 7 questions above and beyond what you've kind of asked in the past to at least evaluate, hey, are they addressing these things? Are they thinking about these questions? Sometimes that's kind of as good as it gets. I think beyond that, another piece is adding some validation into the mix. So not just asking your questions or, you know, asking your vendors, hey, look, how are you addressing this or that? But making sure that you do have or enforce some need for a certification or an attestation, something along the lines of, hey, these people take AI security seriously and buy as a byproduct of that, our security and our data seriously. That's kind of become table stakes. I think generally there needs to be some sort of validation that people are doing something, whether that's directly or via like a managed service in the third-party space. I think beyond that too, like, ultimately the best way to really protect yourself as an organization is something contractual. So that's going to be updating your BAAs, updating any agreements you have, or even like SLAs to specifically codify what the requirements are around that, that AI use. Some organizations, especially once ChatGPT kind of came onto the scene, that's really what started this, this whole avalanche, was, or rather, would outright put prohibitions like, hey, you cannot use my data in an AI model. Now the tone has shifted there because a lot of organizations want to leverage AI-enabled service providers, but putting specific considerations just like you would around your data subjects, privacy, et cetera, is an absolute need. Organizations that haven't quite got to that point yet, definitely sit down with legal, sit down with your compliance department and sort that out because we haven't had a huge, you know, event yet, but we certainly will where a model is compromised and the data just comes all over the place. And then again, if that's your third-party provider, you're still on the hook for that. Outside of that, I think what a lot of organizations have started to do is move into firstly kind of a little bit of a cross-collaborative environment. So it's not just now that your GRC team can kind of check these things off during procurement. There's not just one stage where security is going to give you the okay in your ServiceNow ticket or whatever it is, right, depending on how your procurement process works. If you've got data leaders, if you've got a data science function or anything that's in that realm, tapping on their shoulders to get them to kind of look over what you're seeing from your vendor is really, really critically important. And then above and beyond that, you know, not just kind of diving headfirst into some of these solutions. Give yourself a POC period where you can test these things out, see how the organization actually operates. Are they taking care of your data? Are there any kind of concerns you have beyond that? That's kind of where the approach has shifted. Just again because of the quantity of data that these organizations might be ingesting from you. The stakes are a little bit higher, and so organizations have started to get a little bit more touchy in terms of how they share their data. I think that's a good thing, but those are a couple ways that they— that kind of manifests. And that's a great point, Morgan, and you both have given some examples on the strategies being deployed by organizations to address these AI-related risks. So Steve, I'm curious, what do you think are some of the most overlooked risks in AI systems today? You know, like bias, privacy, does anything else come to mind? Well, so I think Morgan and I could probably do an hour just on, on some of those components, right? But I mean, we mentioned a lot of different elements of risk just in passing, but, you know, I think overall when you if you think about breaking risk out into a few different areas, it kind of helps you maybe drive some of the focal points. So we've talked about regulatory risks, you know, the auditability of what you have, whether it's explainable, if it's transparent and you can have accountability with, you know, the actions, information, and the processing that's taken place. You know, obviously there's elements of data protection that's going to be impacted by different organizations. So obviously PHI for healthcare, you know, any type of card processing for PCI, CUI or FCI for CMMC or the DIB and defense supply chain. So, you know, all of these different elements of the data flow and just making sure that those risks are not presented in the way in which your AI is being handled and you don't know the flow of that data. If there's elements of decision-making that cast any type of bias or discrimination, you know, you could be putting yourself into a bad spot there with applicable laws and regs. So those are things to consider. And then, you know, there's also obviously the contractual requirements that you may have with various customers and just making sure that, you know, you're mitigating the appropriate risk around that. So those are some regulatory ones just to say. Some of the technical ones we talked about really, you know, constitute some of the cyberattacks against these AI models that can provide for data leakage. There's different types of adversarial attacks where, you know, folks are trying to, or bad actors are trying to get away with different types of prompt injections and model theft and things like that that they may be trying to circumvent, replicate, and/or skew a model to behave in a way in which it wasn't originally designed. And there's some really tricky ways to do that. There's a few sessions at Black Hat this year that kind of went through some pretty elaborate use cases on indirect prompt injections that were pretty interesting there. And like I said, the more that those proof of concepts and use cases become apparent, the more you're gonna see more and more attacks like that, you know, really get orchestrated in the, in the threat landscape. I think the use of, you know, bringing in different types of plugins or add-ins to the AI models and interoperability with Agentic AI, I think, is going to allow for, you know, certain types of dependencies and problems there along the cycle. So when you think about AI supply chain, just thinking about making sure that you're, you're safeguarding and thinking about the threats across not only your model, but anything that you're bringing in from a third-party perspective or any interoperability with any other existing systems out there through API calls or whatever, in a, whatever way that that's architected, I think is important. There's a number of operational risks that, you know, we've talked about that could potentially have an impact on decision-making, you know, key elements of process integrity, anything like that, that I think become really apparent. But there are some interesting ones. I mean, obviously we've all seen elaborate phishing attacks, and this is getting away a little bit more of the generative AI, but just, you know, from the standpoint of thinking about the attackers are using AI to attack like we are using it to defend, right? So you're starting to see more sophisticated attacks using AI, whether it's through phishing, vishing, or deepfakes or anything like that. That's another one to consider. Obviously that, you know, I think kind of traverses down a little bit further than the AI stack, but employee awareness and other types of safeguards and mitigating controls to make sure that assets aren't improperly reallocated or stolen or theft or anything like that. There is a— there could be a huge automation bias where, you know, folks start to rely on the AI more than they should, right? Unless it's properly designed, you— I mean, you still always have to have these safeguards. It's part of what's expected for AI monitoring. So I think that, you know, there's a— there's a huge tendency to— because AI is doing things at a rapid speed and seems like it's, you know, really on top of what it needs to do, validating and knowing the level of trust that you need to put into that and making sure that you validate that. I think that's a huge risk. Shadow AI is another one where organizations have designated systems that they've vetted. You know, think about all this work that Morgan and I are talking about here on risk mitigation, and then somebody just, you know, goes on and subscribes to a cloud AI and starts using that to do their work. I mean, shadow AI is a real thing and it's a big deal, and they can open up a lot of exposure there. So having the right systems that can detect inappropriate use of AI behind the scenes. And of course, you know, we did talk about model drift before. Just, you know, if again, if you're not monitoring the integrity and the output of these models, inherently the information being generated can continuously get inaccurate over time and feed bad behavior to the model. And you have this drift that occurs. And if that's not being properly safeguarded, you're going to be very much overreliant on a model and you're going to have things going way off the rails. So Those are some of the, I'd say, the bigger ones. But like I said, Morgan and I could probably talk for another half hour just on that. Yeah. I love, Steve, I do gotta say, I love the last one. It might have been the last one, but the one around kind of people being overly dependent upon the systems. That's a big one. I think if you've got kids, you know, it's, that's always my concern of my daughter. And, but as defenders, as people that are in the organization, I think it is something that's really important to kind of consider and, and love that call out. I mean, you think about the social aspects of it as well, but it just, you know, again, it goes a little bit further than just what you'd have in, in potentially just in the enterprise. Yeah, the responsibility for some of these sites that are providing services, social services for kids and things like that, or just any type of generative AI that could be used inappropriately or skewing a particular behavior. Absolutely, to Yeah, and Steve, you know, you laid out several risks there. I know there's a lot more that come to mind, but, you know, let's say there is an AI failure. You know, Morgan, how can compliance frameworks help mitigate reputational damage that could be associated with AI failures? Yeah, I think it's interesting because it's similar in some notion to what we've talked about before, where some of the security kind of components are the same. It doesn't matter whether it's a traditional system or kind of an air quotes AI system, but from a from a reputational harm perspective, it is a little bit different. And so there's just being where we're at, right, with the last 3 or so, 3 or 4 years-ish, there's been such a big avalanche of AI adoption and implementation. It is a couple of major things if you are leveraging an existing compliance framework. I think first things first, due diligence is a major, major player for you. It's a major contributor to ensuring that you do have that compliance framework that you're leaning on, whether that's, historically been the CSF or HITRUST or ISO, whatever it is. I think making sure that internally people are committed to some common language of security control, privacy control, and at this point, right, kind of AI control goes a really, really long way. Making sure that you have that framework and you've also got a document, documented paper trail of kind of leveraging that goes a really, really long way, not only kind of internally from a cultural perspective or even from a business perspective if you've got investors But from a regulatory perspective, if the OCRC sees that due diligence— using them as one example of regulatory agency— it's a significantly different conversation than if you hadn't thought about it, you just went out and bought something, or you just went out and built something, and then something bad happens. So I think due diligence kind of is part and parcel with a compliance framework, but it's really, really critical and a big benefit for organizations in this space. I think another piece there And there's maybe two ways to look at this, but, you know, some degree of signaling and transparency for your clients, for your investors, and even for the people at your organization or your patients if you're in the healthcare space as an example, goes a really, really long way. I'll use Anthropic as an example. They are probably the most transparent or appear to be, that's what I'll say, the most transparent from a security and commitment to privacy perspective. Of the, the frontier labs that are out there. So, you know, OpenAI is, is the biggest of the brands. They've gotten a little bit of flack as an example, and so their perception is a little bit different than what Anthropic has in the market. So if you can kind of signal your commitment to security, privacy, etc., and, and kind of speak out into the market, hey, look, we're committed to security and privacy by aligning with XYZ framework, whether that's ISO or HITRUST, whatever it is, that goes a really, really long way in making sure that, A, you know, you've demonstrated your commitment to transparency for your clients and for the market and also regulators, but also, B, it does a lot for brand perception. If something bad does happen, it's not, oh, well, of course, you know, they weren't taking care of what they needed to. It becomes more of a, well, hey, look, you know, there's only so much they can control. And it's kind of interesting how it can shape human perception and goes a really, really long way. And I think resilience to a reputational, uh, impact— whereas if you'd never told anybody you cared about security or privacy, all you were worried about was shipping new features, and then something bad happens, people are likely to be a lot less forgiving. And so that's really the difference between kind of being able to survive that and closing up shop, which can happen in the result of a really significant breach. And so that's another big thing. I think in the same vein though, You know, leveraging a compliance framework that is tied into some sort of an attestation or certification can go a long way as well. So that's where we want to lean on ISO or HITRUST as an example. Those are frameworks that people can come in and basically check your work on, right? So it's easier for you to use those in negotiations. It's easier for you to use them as assurance as well. If something bad does happen, you know, hey, look, HITRUST brands themselves pretty well around how many of the organizations percentage-wise have been breached, right? It's a pretty low number. There's a number of reasons for that, but it definitely won't hurt you if you're an organization that does have something happen where reputational harm is a consideration if you've got one of those rubber stamps, so to speak. And so those are some of the major things to think of. Some of it's a little bit more kind of softer business considerations, but it goes a long way when you're thinking about reputational harm. Awesome. Thank you so much, Morgan. Well, thank you, Morgan. Thank you, Steve, for joining us again today. And thank you, everyone listening, for tuning in to the Risk and Cybersecurity Podcast. Don't forget to subscribe and join us next time as we close out this series with the differences on managing internal versus vendor AI. Thank you.