Balancing AI Governance and Innovation with Erica Werneman Root of EWR Consulting

There is a lot of hesitancy in industry because they're not necessarily sure about what kind of regulatory requirements attached. Is it just through any kind of AI system? Where do you draw the line? How do you figure out what matters? We might find ourselves in the future in a position where we perhaps scale back some of the ideas around what we can use AI for. We know automation is a real risk, so there's some studies done around pilots and even an experienced pilot with their own life on the line here year was a lot less likely to override an AI recommendation than a manual checklist. Welcome to AI Government and the Future, a podcast by Corner Alliance. We explore the intersection of artificial intelligence, government and the future. We work with government to create results. We ignite your agency's mission by helping you to design and implement high impact and innovative federal programs in AI, broadband, cybersecurity, public safety, and more. Being a government ally is at the core of all we do. Welcome back to AI Government in the Future. I'm your host Max Romanick and today I'm excited to welcome Erika Wernham. En route to the show, Erika is the founder of EWR Consulting, specializing in AI governance, data protection and compliance. With extensive experience in technology law, she helps organizations navigate the complexities of AI integration while ensuring ethical and regulatory adherence. Today we'll discuss the intersection of AI governance, workforce readiness and global standards, exploring how AI can be deployed responsibly while safeguarding societal values. Welcome to the show, Erika. Thank you. It's a pleasure to be here. Definitely. As a means of getting us started today, I'd love to hear a little bit about your journey. You've really built quite an impressive career in technology and laws, particularly focusing on AI governance and data protection. Can you share with us what really got your interest initially sparked in AI and its governance? Yes, of course. So I actually started out in litigation as a litigator. That's the team I qualified into long time ago and I ended up working with a lot of technology companies and I thought it was fascinating. It was a really interesting space to be in. So in terms of AI, I've probably been in that space since 2015, I want to say 2016 maybe when I started properly working with the coins of technology. But we didn't actually call it AI then. We very much referred to the actual use case. It was facial recognition or it was robotic. It wasn't like an overarching AI, if you will. And I having done disputes and focusing on technology, I sort of pivoted into the data Protection space in the run up and subsequent to gdpr. So my involvement was primarily from a data perspective, not so much product liability or any other, but I found out, I thought it was fascinating. It's really interesting to see the technology, what it could do. And then I ended up being invited onto a government panel. We were looking at AI ethics, overlaps with laws, regulation. This was probably 2018, 2019 and really I've been involved in it ever since, but I would say it's only in the last year and a half, two years that I've been doing it exclusively. So it's always primarily been sort of ancillary to my data protection work. Yeah, that's really interesting. You know, sort of my own background's in similar areas. I've been tangentially involved in AI since like the 2000 and tens as well. And to your point, it was not called that back then, but now we understand like, oh, a deep learning algorithm. Yep, that's A.I. okay. Yeah, yeah, yeah. We've had, we've had all of it. It was ethical AI, responsible AI, trustworthy AI. I forget what it was called back in 2014, but we've had all of them, haven't we? Yep. It's sort of like the old cybersecurity days. It did not start as cybersecurity and certainly not cyber. Cyber meant something else back at the beginnings of the aol. So that's awesome. You know, I'm very interested in your sort of personal view on the relationship between technology and its ethics and regulation, especially when it comes to things like AI. Where do you land on this? I mean, there's quite a spectrum available for how to categorize the ethics and whatnot when it comes to new cutting edge technologies like this. Yeah. So the question was the relationship between technology, ethics and regulation. I don't get like the technology for me is just the. I wouldn't say a constant is always changing, but it is one of those it is what it is kind of things versus ethics and regulation, where you can get into a lot of trouble. I think so with the overlap with the true. I often find myself talking, in particular when I'm not talking to other legal professionals. We're talking about a concept and we think we have a common understanding, but actually we're talking about slightly different nuances and I think a slightly tricky part with ethics and regulation because there's so many concepts that span the two but might mean different things. So take transparency or explainability. Right. They mean slightly different things. So in my view, and I don't want to get the ethicists coming after me here, but I think to a large extent, where there was a legal vacuum, there was a lot of ethical frameworks that were helpful for companies. I think we moved now into more of a regulatory and legal space, which means that, I mean, one way of looking at law is as society's minimum standard of ethical behavior. So if you're not hitting the legal requirements, even if you think it's ethical, it's not going to be enough. So I think you need to understand your baseline from a legal compliance and then beyond that you can then start looking at ethical considerations. So I think law and regulation has come in and sort of superseded some of the earlier ethical conversations. I definitely still think there's a place for ethics, but I think it's for more the ethical questions rather than legal, if you will. So in the organizations that I work with. So let me give you a couple of examples of what I mean by that. If you're wondering about my accent, I'm Swedish, by the way. I've had people had a full on conversation with me before and they're like, where are you from? I'm Swedish, I've lived in the us, the uk, France, the Middle east, all over the place basically. Sure. So a hybrid accent, hybrid. It's completely international. So in terms of the ethics, right, there are different kinds of ethical questions. I think there's ethical conduct questions and I see this pop up a lot in quite sophisticated companies. Around what kind of AI capabilities are we prepared to make available to what kind of a purchaser is, in what kind of a jurisdiction, how could it be misused? These are ethical questions. As long as there's no sanctions in place, the law doesn't tell you what to do. So I think that's a straight up ethical question. You then have a lot of leeway, I guess, and the law will set a standard, but then it's up to the company to actually translate that into practical decisions. So what kind of things? Looking at an AI system or an AI model, what's the objective? Is that an ethical objective? What kind of priorities are we looking at? Are we not looking at in that context? I think those are as long as you comply with the law, I think those are ethical questions as well. I do find myself quite often having conversations with companies that have an ethical framework in place or ethical risk assessment and by that they think they have covered off the legal requirements and that is quite often not the case. Right. So I think it's important that you don't confuse the two. Have your legal and your compliance program your baseline and then actually figure out which ones are the ethical questions and what's your ethical position on those. And it can vary from company to company versus the law that applies to everybody. So I'd also love to throw one more sort of buzzword into there, and that would be governance. Governance within that. So let me pose something to you and see if you agree. So I love that, like the ethics sort of forms the basis of how you want to govern your operations in terms of what you want to use it for. And all of those questions, if there's general agreement, then sometimes that gets turned into a regulatory framework where we've converted some of those ethical guidelines into a legal standard for minimum ethical compliance. Of how any of this stuff is going to go, I would say then governance has a lot to do with how you stitch together multiple systems with different regulatory or ethical standards into a unified whole. And so the governance sort of helps you meet the regulatory requirement as informed by the ethics. Yes, I actually agree with that. That's a very boring topic. Well, it's good because we have a lot to talk about and making sure that we have agreement on what those terms mean right at the front end is going to be super helpful to the rest of our conversation. I'm a practitioner, so for me governance is also fundamentally very practical. So it's the actual structure for making all of that happen. Who's involved in the decision making, your authority delegate, like who actually gets to make those kinds of decision, how you structure it, how your ethics committee fits into your risk. The practicalities of this, I also see it as governance. That's the only thing really I would add to that. Otherwise. Yeah, completely agree. Yeah, using a fun legal word. I'll stipulate to that. Yeah, that's great. Okay, so if we then sort of have that, that sort of three way mechanism of ethics, regulation and governance in place, where do you feel like humanity's best interests are served in terms of aligning the introduction of a new technology, like some sort of AI system, into preserving what we determine to be societal values? Like how do we use this sort of three part mechanism to try to help protect and serve the public interest? Great. I mean, there are some very, quite a lot of very opinionated people in these topics. I'm not an activist, I should caveat, I'm a lawyer. So I think there's a lot of concern, some very justified concerns. I think the way you practically manage those is by containing the different projects and the issues that you're looking, looking at. I completely agree that there's so many different issues though, right. It almost becomes unmanageable because the legal feeds into the economic, which feeds into the societal, which feels, you know, there's a whole circle of different considerations. I didn't actually start out in law, I started out in economics. So I was very much a big fan of the whole systems thinking approach from a long time ago. And I've also applied that to my sort of legal reasoning. I think there's so many different variables here. It's very easy to get caught up in societal risk when what you should really be probably worrying about is practical risks. Right? So at the moment, in terms of the technology, there are people far smarter than me talking about existential risks and what it means for job displacements and all of that. But I think at a corporate level, you can manage some of those really material practical risks by having a good understanding of what you're actually talking about, having a good understanding of, of the technology, how it can be used, what are the risks inherent in that and what are the risks at an enterprise level and at a customer or a subject level. And I think it really does depend on who you are and that whole economic value chain. Like if you are a PR agency, your campaigns for a toothpaste brand is not going to be that material. Like, don't worry about it too much in terms of using AI. But if you're all of a sudden engaging on political campaigns, it does cross over into some societal aspects. So I think. Right. Sizing or spectrum of concerns is probably a good starting point. I think we should all become more informed. I think we should all become better at spotting AI when we are receiving it, engaging with it and all of that. So there's a whole civic part of it as well. But they do, I take your point. They do sort of tie into the governance point is, is pertinent, but sometimes people worry too much about is that they're not actually focusing on the practical risk inherent in what they're doing. And I understand you do a lot of government work. So my personal view is that the risks in the public sector are a lot more material than they are in the private sector because you don't usually have a very good feedback loop or feedback mechanism. And it's very easy to incorporate skewed incentive models and not actually get any feedback on it. So I think in the public sector, having a really good understanding of your governance is all the more important in the private Sector people will just leave you. They'll start writing really bad press about you, you'll start getting a lot of claims versus in the public sector, people don't have an option so much in terms of who they deal with. And there's a lot of bureaucracy involved in that as well. It's a lot more difficult for individuals to be heard. So, yeah, I'm interested in your view on that, actually, since you're very much involved in that space. Yeah. So there's a couple of threads I'd love to pull on there for sure. I think that you're really onto something with. You have to determine this based on your use case. And so in the public sector, like an easy compare and contrast, we have guests on the show all the time from Department of Defense, and they're using AI for target acquisition, for kill, no kill decisions for all manner of different things that are of the highest consequence that you can possibly imagine an AI system getting engaged in. And then we also have people that we hear from that are using AI for workflow, process engineering so that they can respond to inquiry faster. It's like, okay, well, so if we mess that up, we lose a day in providing information to a constituent who's asking or to one of our clients who's asking. And then on the other side, if we get it wrong and we apply these things incorrectly, they're life and death decisions. And so I definitely agree with you. I think that you've got a very salient point around what's your use case. And we need to determine all of this ethical, regulatory and governance structures based on the actual use case. Applying the highest level to a life and death decision from a military perspective to your basic office clerical work seems really inappropriate. And the same is true in reverse. So, like, that's a big deal. And then I think it also begs at. There probably shouldn't be a single regulatory mechanism for all of this sort of stuff because that sort of invites the baby splitting where it's like, well, maybe somewhere in between the two is the answer. I don't think so. Like, if you lessen the standard on the military side with a life and death decision and then heighten the standard on the clerical side, you've just figured out a way to make an outcome that's bad for everybody. And even within an organization, you can have all these different use cases, right? So figuring out the optimum cadence for ordering new pencils using some kind of algorithmic feature, you know, that's very different from the kind of decisions you're talking about. So within any given organization you're going to have a huge increasingly so in the future you're going to have a huge range of different kinds of risk profiles for what you're doing. And I think understanding the ones that matter versus the ones that don't necessarily, I think that's, you know, we talk about it, but actually there is a lot of hesitancy in industry because they're not necessarily sure about what kind of regulatory requirements attached. Is it just through any kind of AI system? What is it? Where do you draw the line? How do you figure out what matters? Yes, absolutely. And I think a very nuanced understanding of this too also sort of begs the question at okay, well what part in the chain of this system would we even want to apply the regs at? There are plenty of systems where the actual AI algorithm is pretty uninteresting, but the data backing it that is trained on that's very interesting. So like think about, I don't know, social media where we've all sort of participated in a semi not consented to grand social experiment over the last decade plus and that has largely existed in a completely unregulated environment and what that data is being used for, who owns it, how they're being trained on. There's a few things to say about privacy and a few like, you know, have you collected information from a underage person? Like okay, yeah, there's some of that sort of stuff. But the general mass we've now used as all this information to train a large language model to perform question mark function that's largely been in an unregulated space. Is that a good idea? Is that a bad idea? Like that's a very open question to me at this point. It is, but it's also the tricky position that Europe and lots of other countries find themselves because if we're introducing regulations now is essentially just creating a barrier, right? A barrier to entry for competitors. And once you're in a position of having done it, lots of these companies are very pro regulation because it is a very effective barrier to market entry. So you have to think about it from the multiple different sources. On the flip side, you then have the rights holders, you know, the IP holders. A lot of this data isn't just your status updates, is actual creative works. It is works as subject to intellectual property rights. So there's so many different aspects to that. I see. I didn't have the answer to that. So we're seeing how it's unfolding in real time, right? With lawsuits, with the regulatory guidance and all of that. I think for the vast majority of companies it's more about wait and see what happens rather than if you're not building the LLMs yourself and you're primarily looking at it from a flow down due diligence procurement perspective, you're probably just going to wait and see how that plays out. Yeah, and I can't fault them for that. I mean, we're looking for predictability and it doesn't exist yet, so. Fine. Exactly. But it's probably going to trigger some interesting markets. Right. Of vetted and legal databases for all kinds of different use cases. So it's not just the wild west and then there was nothing. It's probably going to be an evolution towards more structured and you can get a lot of benefits in that as well. If you, rather than relying on web scraping, if you're actually curating the content, you can hopefully manage some of the risk we're seeing around bias, inaccuracies, et cetera as well to a better degree. So I think it's all a great evolution. Absolutely. And I think that another interesting thread there on bias and inaccuracies and a lot of that is sort of coming from the data that you're training on and trying to make sure that you're constructing these algorithms to be free of this very, very difficult challenge. We see it from a multiple group of sort of factors. Like a good example in law enforcement data, there is inherent bias in the law enforcement data since it's been collected. It's been part of how that function has operated, at least in the US I can't speak to the international community, but if you tried to train an LLM on historic law enforcement data and you didn't give it any parameters, you would adopt all of the bias from how that's happened in this country that's just been existing. It's a very non controversial position. It's there. And so if you're trying to build an LLM for a law enforcement function using that training data in the past, you have to be very sensitive to how you try to scrub for those sorts of issues within the data set itself. And then you have on the other end of the spectrum functions that either the data is new or they've been sensitive to it since its data collection. And so as those data sets were built, some of the bias or some of the potential issues there had been sort of scrubbed and sanitized from the get go. And so you have fairly clean data sets that you can use to Train on. But generally speaking, do you think that there's a role for standard setting bodies or something in the governance and regulatory mechanism to try to help control for bias in those sort of intended or unintended ethical dangers? Yeah, for sure. So I had a look at. There's a really good, it's based in the uk, it's called the AI Standards Hub. It's the out of the Alan Turing Institute, I believe. And they're tracking all of these different things. The last time I checked, there's something like 300 different standards or frameworks and over a thousand different identified risks. There's a lot, there's a lot that's happening on the standards development side. There's a lot that's happening in academia. And I think the problem for, and I, you know, caveating my answer heavily here, I'm on the practitioner side, I'm on the implementation, building the framework, trying to get businesses to do these things right. So there's a lot of information but not a lot of curation in terms of what's relevant, what applies, when should we worry about things and bias, you know, we have as lawyers, certainly my mind you might be better at this, but when I hear bias I immediately jump to discrimination and all of these things. But there's different kinds of bias depending on who you talk to. Statistical bias, like all there's different computational, so there's different ways of looking at these issues as well and they mean different things. So I had one interesting conversation was in the context of a medical provider that were looking at implementing some kind of a system to basically prioritize for transplants. And they kept getting the recommendations, kept being women and they went back and had. It wasn't implemented, by the way. This was just like rough sketching here. So don't worry, you're not at risk of not getting a transplant. But it was very, very basic. So women on average live longer. And it was set to look for how long could you utilize the donated organs inside women more often than not because of the average life length. And then also within that same issue it was publicized so you can have a look at it. But it was also around incentives in that case. So it prioritized cases depending on how long you could actually make use of an organization. So immediately you have the length of your life, women in favor. But the other incident, it was, you know, if you're actually potentially going to need a transplant, it's much better to leave sooner rather than later. So you're now incentivizing, trashing your organs to get on this list to be a better candidate. Right. So with all of these different things, there's the biases you have to look out for, but then there's the incentives that you're putting into the model. Like recruitment is another brilliant example. I love those. So I was reading out the rampant use of AI for CV screening in the us. They don't really do it as much in Europe. It's considered quite high risk. But you now have companies that will come in and make your CV better for a. For an algorithm. It'll push in in whites, it's not detectable to the human eye. It'll put in Harvard mba, like all of these buzzwords, litter your CV with it, but you can't see it. So I think humans is sometimes the unpredictable variable with these things that we fail to account for when we're deciding and there's some kind of indicate we, you know, we plan them for a purpose and we consider the risk for the purpose and all of that. And then it can still go haywire because people find ways, right? They like to game systems. That's another thing I think you have to really account for. So the bias is the main one. And then what kind of incentives are you actually creating with this structure? And it's. Sorry, I've gone off on a tangent here. So coming back to your original question around standards and how important they are, I think they can be really helpful if you make good selections. So if you know what you're looking for. But most of the very sophisticated companies, they have their own way of doing it and they contribute towards the standard setting, but they're not necessarily following all of it themselves. They have their own ways of doing it. So I think the standards, just like when you're contracting with an entity, you'll have certain standards that are required, like ISO 27001 from cybersecurity. It gives you the assurance that you don't have to go and check all of their infosec documentation. Like, have they got. All of. It's just a tick box. Right. I think we'll start seeing a lot of standards used in that kind of contractual setting, just as an assurance mechanism, because it gives you comfort, at least of a baseline of understanding and management of some of those issues. And they do cover to a certain extent, you know, issues around bias detection, all that sort of. So standards I think will be helpful. I think the legal industry still catching up on how to utilize them effectively, because there's all kinds of different requirements you can put In a contract, if you know what to look out for, like third party assurance. All of these different things. Yeah. And definitely not all standards are created equal as well. You know, like you sort of run into that classic joke of like, oh, there's 13 competing standards for this thing and someone says, oh, we really need to find one unified standard that fixes all of this sort of stuff and gets everybody aligned. Basically there are 14 competing standards. They just add another one to the pile. Where we've seen the standards development in our government work come to fruition, where it's actually pretty helpful is when you have sort of public private contributors to that. And so you have some sort of public entity sitting on that standards board, you have a consortium of private entities sitting on that standard board. And then when you really have your way about it, it sometimes that then includes international, both public and private sector so that you're able to come up with this sort of partnership. Every one of those entities has a voice at the table. And the negotiation that gets forced by having all those people in the room tends to create a standard that is going to be generally more acceptable than others. So like 3 gpp for wireless stuff is a good example of that, where it has a really broad contributing group of people to it because everybody has to use those wireless standards in order to have cellular networks in their countries, whether it's a publicly owned network or a privately held network. And then there's not much utility if people can't interoperate in roam between networks. Otherwise like, oh, well, here's a cell phone that works only in your country or only in your state or province or something like that, incentives are going to suggest that company's not going to last very long. But when they have the international cooperation, like yeah, the US phone will roam to a European network which can roam to an Asian network, which can roam to an Australian network. And so you have one device with enough general agreement that you can make its basic functions work. And then out at the periphery there might be some functions that are sort of country specific and that's fine. But as long as the basic utility of the device can be generally enjoyed by anybody who shows up to the network with them, it you've probably got a pretty functional standard. Yeah, I agree with that. And that I think you can only ever be dealing with a rather limited number of participants and that's just that. So that works because your number of telcos is pretty limited. It's not like every, every single city will have its own telcos. So by taking that analogy, I think that can work on the provider side of things if you're looking at standardizing some of the aspects by which companies are developing and releasing. But then if you look on the deployer side, just before Christmas I did 50 different risk assessments for CoPilot, just the one product that we were looking to divide in one of my client entities. So on the deployer side, the risks, I think the actual use cases are so materially will so materially impact the risk profile of what you're doing. But we're talking about a different kind of risks here. We're talking about your enterprise and your, to a certain extent, your legal compliance, reputational, end user risk, not the risk inherent in how you build the system necessarily. So in that whole life cycle question I think is really interesting and depending on. So I don't work with people or companies that are developing the actual LLMs. I do work with fine tuners and people that are building machine learning and all that. So the developer side of it I think is very different from your deployer. So, and I think this is again a common misconception around the EU AI act, for instance, 90% of companies probably will never be caught by it, even if they're using AI is a product liability regime and it's very focused on the higher risk scenarios and your models. If you're on the deployer side and you're not in those categories, it's not going to materially impact. It's not like gdpr. I keep hearing that from Americans. It's the GDPR of AI. It's not, it's quite contained in its scope to certain risk profiles and certain entities. So for the majority of companies your risks are still going to be your data protection, your actual operational, your accuracy, like very practical aspects. But the standardization and in that side of things, I think the standardizations are a lot trickier because there's such a huge spectrum. So you almost have to customize something for each client that you're working with because they overlap between them. Different industries, different level of sophistications, it's just a completely different story. And if you did a standard like take one standardized approach, you'd probably over eggs for most of them and not capture enough of the risk for some of them. So I think knowing how to write, understanding and having access to the standards I think is really helpful. And then knowing how to write right sizes for an organization is key. Yeah. So when the EU act comes up on our show, the most common way that it is presented is representing one End of the spectrum in terms of regulatory approach where you can try to create regulation proactively before you start to see harms emerge with the hope that you'll keep them contained and they won't emerge. Or you can take a more laissez faire approach where you innovate to the sky's the limit. And then when you start to see harms emerging due to the innovation that's coming around, then you go after them one at a time when it's like, oh, okay, so we didn't intend for miners to use that, so we better curtail it for miners. Oh, we didn't mean for that to percolate over into that sector. It does harm over there, so we better control it for that sector. And you sort of walk it back. And so it all becomes a say the EU proactive prescriptive approach is innovation stifling, and the laissez faire approach is very innovation fostering, but you run the risk of getting caught with some harms. That's a very quick and dirty summary of the vast majority of the debate that has happened on our show around a laissez faire approach versus eu. Where do you land? Is that a fair assessment? And do you have additional factors you would like to contribute to that discourse? Yeah, I'm outing myself completely here. I'm what they call a champagne socialist, so I'm Swedish. I'll definitely rave about the benefits of the welfare state, but preferably from a tax haven like Dubai, I'm hedging my bets on these things. Right. I think there's a societal cost to overregulating, just like there's a harm space cost to underregulating. If we right now, like, if we scale it back a little bit. So what is the regulation? So it is. Is an extension of the welfare state in its attempt to limit and protect society from harms before they occur. I think that's a very sensible approach when risks are known. So I see no issue with regulating for pharmaceuticals or. Sorry. When risks are known or are irreversible or just completely unacceptable. I think that's when you regulate. So if the risks are known and you know how to manage them, why not. What is the risk? Why would you want people to potentially fulfill that then if it's irreversible and unacceptable? I think that's also a good case for regulating. I think beyond that, and there is definitely quite a bit of bureaucracy in Europe that I don't agree with, because there's a cost to regulation. You're Putting a cost on compliance and that if it doesn't have the benefits that you intend for it to have, that's within wasted costs. And it does stifle innovation. I don't think people can really, you know, there's the whole argument around it creates certainty. It does. If you have the time and resources to read the law, which is what happens when you're a well resourced, established company. If you are a smaller company, that is simply not the case. And I work with a lot of startups, so I'm very sympathetic to we shouldn't over regulate, but I do think there's a place for regulation when the harms are just unacceptable. And when you draw that line, that becomes a huge. Coming back to the whole ethics point, right. Where do you draw the line? And as you have wealthier societies, you have people slightly older, wealthier, you become more risk adverse. And there is certainly a tendency in civil society to view risks as unacceptable if you don't understand them. But let's take driving as an example. People get killed every day across Europe in road accidents. But we're not banning cars, we're not regulating towards, you know, you can only drive at 30 miles per hour, which would be the prevent or death sort of spectrum of it. We say that's the risk you take if you're on the roads. Right. There's a spectrum. You go to Germany, you have autobahn, there's no limit, like, you know, jesus, take the wheels. It's the whole thing. But there is that whole spectrum. And where you find that balance, I think, is quite tricky for societies. You then have vested interests, you know, you have academia shouting at the top of their lungs around every conceivable kind of risks, real and imagined. You then have activists, you know, no one's going to say we don't have activists in Europe, we certainly do. And you do have, you know, your political spectrum goes back and forth. So where do I land? Was that the question? Where do I land? I mean, you know, like I said, we have lots of people who've come on and given the polar positions, I'm always more interested in some of the nuance. Like every time you want to apply that and say, oh, laissez faire is going to be great for innovation. Sure. Like if you don't have anybody telling you what you can and can't do and it's greenfield space, yes, of course that's great for innovation. You can kind of go crazy with it. And then if you have have an instance where it's Very prescriptive. And here's the narrow swim lane that you're allowed to exist in. Yeah, it's hard to innovate outside of that if it's ipso facto illegal to do so. I also am highly suspicious anytime things are presented in completely black and white terms like that. And so I'd love to hear where you land in some of the gray space. In some of the gray space. So I think if it's private sector, I think you should have a greater risk tolerance because people can choose if they want to participate in that or not. So if you are developing some kind of product that's high risk, if you don't agree with it and it's private sector, and obviously there's some nuances to this. Right. But on the whole, I think if it's private sector, your risk tolerance should be higher because as individuals we have options and we can choose not to participate, we can choose to Boyco companies, we can choose to just go with a competitor. And you're also creating incentives there for actually developing, developing better products because to attract customers, to attract employees, et cetera. Obviously there's a bit of nuance around that as well, but in the public sector, I think the risks are greater and I think they're less understood. There's a tendency sometimes to fix political problems, to fix efficiency problems in ways that lead to harm, but without any good options or recourse for individuals. So I'm more suspect of that kind of. I know you're massively in the public sector, so really interesting views there, but there are. If we're going to go down the slightly more permissive and experimental sort of way, how are you going to manage that safely and in accordance with your responsibilities as a policy institution, as a government? And there's also, you know, I'm quite skeptical of some of the use cases personally. So if you look at, let's take CV sorting as an example. So I was reading through the terms and conditions for a company, won't name them, and they had all kinds of, of good content around the bias mitigation, which I thought was really good. And they had really good content around data protection, the governance posture, but there's nothing in there around their accuracy. So, you know, does it work? Does it do the thing you say it can do. And when I looked at it, actually nothing in terms of warranties around it being suitable for or fit for purpose or able to actually produce good results. And then they struck me to sort of what? Well, okay, well, is it actually predictable? You know and I'm still not sure you can actually predict if someone will be a good employee or not. Same with decisions around loan application et. You can automate some of the decision flows but actually making a decision, you know, how predictable is that kind of thing. So I think we might find ourselves in the future in a position where we've perhaps scaled back some of the, the ideas around what we can use AI for. It can certainly help automate processes and sort of block out obvious nos. But in terms of using it for justice. So looking at, in the justice system, looking at it for what am I trying to get here? I'm sure there's a common thing in here, it's non verifiable future activities I think, I mean someone clout will come up with a much better word. But what I'm trying to get at is say you use AI to predict the likelihood of a disease. You can then test for the disease and you can say if it was accurate or not. Say you say AI for facial recognition, you know, massive privacy concern, but not really that much of accuracy concern because you will then test it. You'll know is it right or is it wrong versus like the kinds of decisions like admission to higher education, like loan application, is there a right or wrong answer? I don't know. I mean there must be something that people are basing it on. And as you probably know that it's working with some of these companies. But it seems to me that that's so much more subjective that the accuracy point and whether or not it's suitable should really be a major factor. And if you then applying that to public sector. So if a bank is doing that, I can still opt to go to another bank. It might be a bit more cumbersome and I might have to change banks, but I can still do that. Versus if I'm dealing with the public sector, what are my options? Right? I mean it seems like there's a common thread running through all of those points around AI literacy though. Like yes, if you have capacity to consent to whatever's going on, that says very little about whether or not you actually understand. Like you can say yes, I've reached the age of majority, I have the ability to say yes and no in a contractual, legally binding basis. But it, there is no requirement in capacity to consent. But also says and I understand what AI systems are, how they're being used and how this might affect me going forward. That's not part of the calculus. And then on the other side of that too, if you're more on the executive side, you're saying, oh, I want to consider AI for performing some function in my enterprise. You can do that, make a buy decision and have no idea how any of these things work, how it was trained, how it operates. And so you might be seemingly like a very sophisticated, savvy participant, but you really have no business in this marketplace whatsoever. Obviously, AI literacy is going to be a big deal going forward, and I think it's going to be one of the bigger separators in terms of people who can understand what AI is, how to use it, how to get the benefit of it, and what some of its known pitfalls and dangers are. And then there's going to be the people that are not understanding why, whoa. Last time I applied for that loan, I got it, and this time I didn't. And I don't understand the difference. That seems like it's going to be a very big trend running through. And so I don't dispute or have a different opinion of it, but I just am noticing just a sort of a common thread of like, everything you're saying is a lot easier to agree with if you also assume a level of general AI literacy. Very true. No, I agree with that. And I think that's a fair observation. Right. Everything becomes more easier with information. And if I didn't caveat it before, I probably should have that your ability to access is. These services matter. Your ability to understand or engage with them and the kind of options you have from a practical, individual perspective, they also massively matter. So I was talking quite broad brush approach. But no, I agree. And I think AI literacy, I mean, there are. Let's take some examples. Right. So the loan example that we talked about, I think that's a really good material example where, you know, having an understanding. But then you could also take an example of AI trading, the stock market. Would you understand that even if it was capable of explaining all of the parameters? I don't know that I would. But I wouldn't want myself to be precluded from using that if I deem myself to be. Or the law says I'm sophisticated enough to accept that risk. Right. Gives you consent or risk acceptance. I think we're more in the risk acceptance. I don't understand what my financial advice is like. Explain to me sometimes not because I'm a poor judge of market conditions, I like investing. But some of the variables that he's talking about doesn't necessarily click with me in terms of that's not how I would exercise it or See it. And if you're then looking at a whole marketplace scenario and all of the different factors that an AI system could take into account, I don't even know that you could explain it. So you probably just have to make an assessment in terms of broad strokes. Do I understand roughly? And am I prepared to accept the risk if it is to make a profit? You know, I think that's an individual call. Obviously there would be regulatory considerations, not misrepresentation, like all of these different considerations. I think that's then quite different from the kind of harms that might arise from. What's another good example, human oversight. So in a lot of AI systems that I see, actually if you look at the fine print, they will say this is a random recommendation. You are ultimately responsible. We're providing something that's intended to assist you, but you are responsible for the oversight and the ultimate decision. Again, whole different range of use cases where that could be material or immaterial, ranging from spelling suggestions to emergency dispatchers. There's that whole spectrum. But if you don't have AI literacy, as in that mitigation, that safeguard, that human in the loop is completely ineffective because they're never going to intervene and they wouldn't know what to do if they did. Right. That's where it's really, really material. So I think it's tempting to sort of try to cover off everything within the AI literacy. But again, paring it back to what is relevant to what you're trying to achieve, what your business is doing, what's the context, what do I need to do this safely? I think that's good AI literacy and it's manageable. And there's so many interesting projects on the horizon. I'm quite active in the space myself, looking at not necessarily the technical aspects, just that kind of thing that can go wrong. Like the human in the loop. It's a very good safeguard if people know enough that they can actually do the thing they're supposed to do. And there's coming out of research, you know, we know automation bias is a risk. So there were some studies done around pilots and even an experienced pilot with their own life on the line here was a lot less likely to override an AI recommendation than a manual checklist. So the automation bias is a thing that should be part of AI literacy. Like people are understanding that bias, but it's not enough just to understand that. You then have to create some kind of incentive system to incentivize the good behavior, the actual checking, the actual verifying. So, yeah, it's certainly keeping me busy thinking about these things. It's all so interconnected. You know, we started our discussion today trying to get some agreement around what some of these terms mean. Governance, the ethics, the regulatory spectrum. And great. I mean, we got there. But the AI literacy required to just be able to follow the baseline conversation and then make informed decisions around how to use these sort of systems is really intense. I mean, it's a terrible litmus test, but you can sort of tell the type of environment you're in based on what concerns are being brought to your attention. I know I'm dealing with someone who has a limited understanding when they're worried about Terminator Skynet sentient algorithm taking over and exterminating humankind. It's like that's probably not where you should be focusing your attention. Like that's maybe more tangential versus a very sophisticated person who's come in and saying, talk to me about your data sanitization practices, how you've trained this algorithm, what your inputs are, and where do you keep a human in the loop so that things don't run amok like a much more sophisticated set of concerns there? I guess the point I'm arriving at is a final question for you, and it's one I like to pose to everybody who comes on the show. You know, we have as our listener base a lot of policymakers, a lot of governmental folks, and a lot of private sector development companies who actually are very much engaged in this. In your perfect world, if you were able to give all of these folks advice that they would then go take and it would better the AI environment for all, what is your advice to them? What's important? What should they be paying attention to from a policy perspective? I would like people in those kinds of positions to think very carefully about the systems, the system approach, these things, the incentive models. This is way back when I was doing economics, but I recall my professor had a really good example for us undergraduates, right? So he came in and he said, this is the scenario. We have predatory loan companies engaging in these practices. You're a policymaker, what would you do? And we all got up on our high horses and we all bandaged, like banished immediately. But the whole point of that story was a real story, and that's exactly what had happened. And what happened then was more murders, more theft, more, the American word for it, assault. Assault. Like all of these horrible things were happening because actually people were using these letters, not for weeks or months getting these 800% interest. They were using it for a day to bridge a gap. So that really resonated and it's happened. If you look at regulations with unintended consequences, they're all over the place. There's a really good example to start in the UK here, there's some kind of endangered bird that had been around for 40 odd years and an environmentalist in and basically said, we're going to regulate for this and if that bird is on your land, you need government approval to sell your land because we don't want it going into the hands of industrialists, etc. Etc. And every farmer went out and shot all the birds because they're not going to have, they're not going to compromise the value of their property over a bird, like with the best of intentions, right? So I think, and I think this is something they don't necessarily teach in lots of scenarios, maybe I just happen to have a really insightful professor, but you create systems with regulations and they have intended and unintended consequences. So the most talked about will be innovation, stifling competition, et cetera. But there'll be all kinds of other ones as well. So thinking about how permanent what you're putting in place, ideally you would take a slightly private sector approach and test something out and see how it actually pans out before you mandate it. So I think if I had to give advice to policymakers, I would think very carefully about intended unintended consequences. I'd think about resilience in the ecosystem. It's not necessarily a good idea to have everybody doing the same thing because it creates a lot of vulnerabilities in the system as well. I would think about how can we do the most amount of good with a decent amount of harm? So what are the things, things we really, really, really care about? Who do we really care about doing this? And can we basically let other people figure it out as they go? Is there a recourse? Is there some kind of justice? If you did suffer harm, could you actually claim for it? Could you get compensated? This is not a very straightforward. This is just a list of different considerations. It wasn't like a very succinct. It's not a straightforward topic and I have direct knowledge of more than one person at the end of one of these shows with a pen and paper writing down a list saying, okay, check this out, read this, think about this, this. And that's exactly the help we're trying to provide people. So does not need to be a direct pithy response. I appreciate the more thorough, like, hey, here's your checklist, people? Yeah, it's a checklist, not a sound bite. Sorry. That's great. Listen, Erica, thank you so much for sharing your wisdom with us today. This has been a really great conversation, and to everyone else in the listening public, we'll see you next time. Thank you. Thank you. AI Government and the Future is brought to you by Corner Alliance. To find out more about Corner alliance and how we work with government to create results, visit our website@corneralliance.com and then make sure to search for AI government future in Apple Podcasts, Spotify and Google Podcasts, or anywhere else podcasts are found. And click subscribe so you don't, you don't miss any future episodes. On behalf of the team here at Corner alliance, thanks for listening.