Navigating AI's Ethical Nightmare Challenges (Reid Blackman)

Today we'll talk about how you can address AI's ethical Nightmare Challenges, and who better to talk about it than someone who's actually just written a new book about that. Reid Blackman. Hey, re, thank you so much for joining. Thanks for having me.

Thank you. You've been on the show at the end of 2022, which seems like a lifetime ago. Definitely in AI terms. But maybe for those of our listeners and viewers in, in the audience who are not familiar with you, maybe can you share a little bit about yourself, who you are and what you do?

Yeah, sure. So I'm the founder and CEO of a company called Virtue. Back in 2022, I came on because I had just published a book called Ethical Machines with Harvard Business Review. I, for the past six plus years, have been advising organizations mostly in the Fortune 500 on how to avoid what I would call now the ethical nightmares of AI or the ethical reputational regulatory and legal risks of AI, that sort of thing.

So mostly advising those companies. Do a lot of keynote speaking. Wrote this new book. Oh.

Oh. The other thing is that by way of background, I'm not a technologist by background. My PhD's in philosophy. I was a philosophy professor for 10 years specializing in ethics.

So my background is philosophy, not data science, computer science, et cetera. Awesome. Wonderful. And if I think of the term having a spiky point of view, quite honestly, you are one of the first people that come to mind.

So really great appreciate you bringing this topic to people's minds and forefront with a little bit of an edge. So I'm excited for our conversation today. Yeah. Sharp edge.

Awesome. Yeah. Hey, that's what you need these days to, to stand out. So folks, if you're in the audience, if you're just joining, drop a comment in the chat where you're joining us from.

I'm always curious to see how global our audience is, and if you want to find out how you can prepare your teams for the next generation of AI readiness, consider picking up a copy of my book called The HUMAN Agentic AI Edge that's on Amazon and everywhere else you can get your books online. Paperback ebook audiobook. Now with that out of the way, re should we play a little game to kick things off? Let's do it.

We got the button. Okay. So here we go. Exactly.

There's the button. When I press the button you'll see the wheel spinning and when they stop turning I would like for you to answer with the first thing that comes to mind and why, in your own words, and to make it look more interesting. You only have 60 seconds. Okay.

Are you ready? Alright, let's do it. No worries. Okay, here we go.

If AI were a sports car, where would it be? 60 seconds on the clock. Oh, man. It would be it would be a Lamborghini without breaks.

Ha. Why that? Because we know how to build these things. We don't how to stitch these things together, but we don't really know how to control 'em very well a car that can go extremely fast, but we don't necessarily have the requisite safety equipment, like a, like brakes on a car.

I like that. It's very easy if you go down the quarter mile just straight. But we know life in business has a lot of curves and turns. And turns.

Yeah. So we need to know how we can break and steer. Which, helps steer the conversation. Obviously a lot has happened since you were on, on the show last time in November 22, and it feels like the industry keeps riding the AI hype cycle like a rollercoaster up and down and super fast.

And. It's no longer just AI everywhere. But now we're talking about agent AI everywhere. Again, like my book too.

Yeah. But what ethical challenges do you see that introduces now for organizations and even for society as a whole? Yeah. Okay.

So there's obviously a lot to say here. Reminds me, is gonna depend upon what you think about it. How you think about AI agents, what they are. So I'll give a very quick articulation of how I think about them.

Two simple conditions. Number one, it's usually, it's an LLM, typically, at least now an LLM in the center that you interact with. And number one, the first condition is that it's connected to other tools. So it could be the internet, it could be other, say, narrow AI or other generative AI.

It could be connected to enterprise software databases, so on and so forth. So it's gonna be connected to those things. The second condition for being an agent is that instead of you saying, Hey, listen, I want you to go to narrow, database number three, and put it through narrow eye number five, and then also consult with gender value number seven. And instead of you giving it step by step, you just say, Hey, here's what I want to do.

And it quote unquote figures out by itself autonomously, so to speak how this should go. And then it. Goes and does the thing. So you don't give it step by step instructions, you give it the goal.

So that's how I think about an agent. It's an LLM technical budget tools and it has some degree of autonomy and how it pursues the user specified goal. Okay. Lots of places for things to go wrong.

So first of all, we already have all the stuff that is, that the sets, narrow AI and general AI. So you still have things like. Biased outputs, privacy violations, including IP violations, copyright, viol violations, you still have hallucinations. All that stuff still applies.

You also have some other things because we're getting really complex and messy here because we're connected to all these different things. So I'll highlight three things that, that, where things potentially go sideways. New nightmares. Number one, you have what you can call cascading failures or a snowball effect.

We're talking about big systems here. So this thing does something wrong, and then it passes on the information to the next thing and that, so that amplifies the error and then it goes to the next thing. It amplifies the error. So error.

So with all these systems interconnected, and if one of them gets the wrong answer goes sideways, ethically, or otherwise, in some way, it's going to echo throughout the system until you get really big bad stuff. So you can think about things like I like to think about this sort of, I don't dunno. I actually don't know how the, if there's a, if there's a stock market crash, like the flash crash, you could have a sort of, if you like, ethical flash crash where Something goes wrong over here and then it just resounds throughout the system.

Yeah. So that's one source of error is just called cascading failures. A second kind of problem is systemic or emergent risks. So the way that I like to think about this is, suppose you've got a bunch of basketball players and they're good, they're each good basketball players.

No problems with any of them. They're not making errors, let's say, but there's something about the way they play. This one is the, this one plays a really fast game. This one plays a really slow game.

This one is all about passing. This one is not, and for whatever reason, even though they're individually good players, they don't play well together, they don't make for a good team. And you can have the same thing with. AI where all the tools to which it's connected.

They're all on the sort of ethical reputational, brand legal, up and up, but something about how your agent stitches them together, it doesn't make for a good team. And so you wind up with these emergent risks. And then the last thing that I'll say is this autonomy bit. So you, it's that autonomy is not a risk unto itself.

It's that it exacerbates all the other problems. Because when we go through, when we, when AI creates biased outputs, privacy violations, hallucinates there are cascading failures. There are systemic or emergent risks. It does it at tremendous speed and scale.

And if the AI is autonomous and you say, go do the thing, and you go walk away and you've come back and has been doing the things 'cause it's autonomous and it turns out that. One or more of those errors, ethical reputational that have occurred, you're in trouble. So autonomy pours gasoline on the other ethical reputational fires. It also sounds like if you have increasing levels of autonomy, there are fewer humans in the loop or aware of what is actually happening or being able to step in, right?

So to your point multiplying or exponentially growing the risk of failure in that system, not just additive, but. Like multiple Exponential. Yeah. It's, it becomes exponential and the human in the loop strategy breaks down.

So human in the loop as probably your audience knows, is meant a human loop is meant to stand between, if you like, the AI outputs and the outcome because the a, the human takes the outputs, reviews it let's say it's a very simple, I broken bone identifier for radiologists, right? So the radiologist is the human in the loop. She says, yeah that's legit. That really is broken.

Or actually the AI's mistaken here and then prescribes whatever she prescribes and that's the outcome. But when you have, it already begins to difficult with generative AI, but with a agentic AI, let alone multi-agent AI, when you have agents talking to each other. Human loop really begins to break down because imagine this sort of mountain of data and decision points made by the AI and then you go to, Sam in accounting and you say, Hey, Sam, validate that. Is that cool?

Sam can't do that. Sam can't handle that amount of data. And those, that quantitative decision that the speed at which they were delivered and the speed at which Sam is supposed to validate, it's not possible. So one thing that we see with.

Age agent AI and multi-agent AI is that the human, the loop strategy for risk mitigation really begins to just completely fall apart. So what's the answer then? Oh, there's lots of things to say here. We still need some kind of, certainly some kind of oversight, some kind of validation.

People use other acronyms now. It have for a while, like human over the loop human, in charge of the loop, human, behind the loop, ahead of the loop. There's all sorts of crazy places. There's the short of it though is that I don't think that there's one generic answer of here's what you do, then it's for this particular generative AI solution, this particular agent solution, this particular multi-gen solution, it's how do we identify how things can go sideways?

What can we do now to decrease the likelihood of those things going sideways? And what can we do now to monitor the thing once it's in the wild, so that if it does start to go sideways, we can intervene in the right kind of way. And the answer to who should be doing what, when, how? It's must vary on a per use basis, a per use case basis.

It can't be decided ahead of time because the context in which you could use an agentic system. So what do you do then? Yeah, so I don't think there is the thing to do. It's not with all these different kinds of agents, with all the different contexts of deployment, there's the way that you avoid the risks that a human in the loop might play in, say, narrow AI.

It the issue is that there's lots of different ways that things can go sideways. There's lots of different things that we can do to decrease the likelihood of it, and there's lots of different things that we can do to monitor the AI. And so to me the question is not what's the thing, what's the silver bullet? If it's not a human in the loop.

Rather it's for this particular use case, for this particular context of deployment, what's the set of things that we can do in order to decrease the likelihood of really bad outcomes? And in some cases that might be a data. I feel like a data scientific solution. It might be someone.

Down the downstream of the data scientist, like a marketing person or a person in HR using the tool. It might be some combination, usually will be some combination of that and more. And so YI just think that's when it comes to human, the loop, it is not a panacea. It's not the thing that's always gonna fix your problem by any means.

It's just going to, in many cases, heap a level of responsibility on someone that it's not fair to give them. And then things will break down. So it sounds like definitely take a deliberate approach. Look at the risk of what is, what is that task, what is the risk in involve anyways as it is.

And then when you layer a genetic AI on, on top of it how does that risk change? And is it even feasible? To automate it. Is it feasible to have a human in the loop and make decisions more?

Yeah. Yeah, that's exactly right. And even more generally, I think the key thing in the era of Agen AI is what does it look like to do that kind of problem solving at scale? I think that's the question that we have to wrestle with.

Because I don't think it's a matter of, and I'm sure we'll talk about this some more, but it's not just a matter of, oh, just do the things or do the thing. Yeah, it's okay. In each particular agent that you're developing, let's solve the problem of. How do, what are the various things that we can do to decrease the likelihood of various kinds of nightmares occurring?

So I've been obviously following your content for a number of years. And I really love the perspective that you bring to that topic of not just responsible AI but really the ethical piece of it. And I'm sure you purposefully call it. Ethics as opposed to this broader term of responsible AI.

But companies have been standing up ethics committees. They've been standing up responsible AI programs. But from what I see and keep hearing from you, it seems that something's broken. What are you seeing?

What's really changed when it comes to AI ethics and governance and responsible AI? Okay, so there's tons to say here. Actually there's a whole chapter of my book where I tear apart what I call the standard approach to responsible AI or AI ethics. And yeah, we could talk about the difference of AI ethics versus responsible AI, but it's all in the same kind of mix.

You can also call it AI governance trustworthy AI, if that sort of thing. Anyway, so there's this standard approach that I think is totally broken, and there's a couple of things going on here. The standard approach to responsible AI is very top down and very policy driven, so it roughly looks like this. You get some executives together, you create something like an AI ethics statement, you know where you talk about you com.

You're committed to privacy and fairness and transparency and accountability and that sort of thing that ultimately gets those values get. Two things happen to those values. Number one, they get translated into procedures. So what does it mean to be, committed to fairness?

We'll check for, we'll check for bias in our models throughout the AI life cycle. What does it mean to be transparent? If we should use an AI in a customer interaction, we'll tell them, we'll disclose to them that an AI was used. So you, you translate it all to com to the procedure procedures.

And the other thing is that the values get some way or other embedded in your enterprise risk policy. Now one of the big problems with this, because it requires an enterprise wide policy, that's the approach. And if you talk to one with Fortune 500 companies, this is gonna take forever, at least a year. A year is the fastest that you're going to get an enterprise wide, say AI risk policy passed, and once it's passed.

Great. You can celebrate for half a second and now you have to implement, 'cause no one cares about your policy. It's not like your employees wake up with a thirst to read the new policy. And so now you've gotta implement that policy and that's going to take.

Years, at least six months for a department. And if you're talking about organizations that spanned hundreds of markets and dozens of countries, that's a long implementation roadmap. Meanwhile the technology AI is leaping ahead. So you pass your AI policy on error AI.

Great. Oh no. Gender value just came out. You finally passed your policy on gender AI.

Oh my god. Now it's AI agents. Yep. You.

You considered AI agents, but not multi-agent AI. Oh, another, what do we do now? And especially 'cause the technology keeps changing, but your policy, say bans in one instance, one year. They might not wanna ban the next year because the technology has changed such that now safer and we don't need the ban anymore.

For instance, if monitoring agents there's techniques and software that doesn't exist next year that doesn't exist now. Exist now, it's gonna be way safer to deploy them. Next year as opposed to now. And your policy can't really speak to that and accept an extremely general generic way.

So you take forever to pass this policy. Then you're trying to implement a policy, but the policy is stale and dated and dead. And so you implement the policy but the policy's outta date. So get the C-suite back in and the board back in and update the policy.

This is just a chaotic nightmare where you're never actually getting too robust. Nightmare avoidance. And so that's one big problem with the timeline. The other, that's one reason why it's crumbling.

The other reason that it's crumbling is that you take all these things you translate all the values to procedures and given the complexity of ag agentic systems, especially, half of those procedures don't apply anymore. Half of them do, and there's another, if you like half, there's another 10 50 whatever procedures that you do need to go through with the new, with this identity system that are not mentioned the policy. That never works. And the other thing is that you never know if you're successful with this procedure, policy, procedure based first approach because what does success look like?

You, did you achieve fair outcomes? I don't know. We did the things. We did the procedures right, but did the procedures make it a fair outcome or a privacy respecting outcome or a transparency, respecting outcome?

I don't know. We did the things you told us that the value is just our procedures. We did the procedures and so that's it. And so it's not outcome oriented.

You don't know what success looks like. It's too slow to implement. It's just the whole thing's a mess. And it's always hard to put a price tag on something that hasn't happened.

Risk avoidance Yeah. Is a lot harder to, to estimate than if something materializes. So it sounds like there's a lot of a lot of talk, a lot of red tape, a lot of, back and forth between all the different departments and stakeholders and players to go through. Then you finally roll it out and by the time the world has moved on makes things very always in, in this iterative mode of catch up.

And hey, here's something new and we need to think about this and roll it out. Yeah. Now you talk about this very purposefully in, in, in terms of nightmares and maybe if you hold up your book, we can all get a look at the Oh yeah. Cover of it.

So that's what it looks like, right? The Ethical Nightmare Challenge. It's been out for a couple weeks now. So I'm super excited to, to get some time with you to talk about this, but why specifically?

Yeah. Why specifically do you talk about this as nightmares? There's a lot of reasons. I have a chapter called Why I like Nightmares, and you should too.

It, so when you talk about values, like I just said they're very abstract. They're, it's, if I ask you to tell me what heaven looks like. We're not very good at ex explaining what heaven looks like. If I ask you what are some hell hellscape scenarios, you can do those.

We're very experienced talking about nightmare scenarios. We have, all of history, we have, the news, we have books and film and TV and et cetera, et cetera. So we have a deep familiarity with nightmares. This means that we can make them concrete and specific, vivid, they're also emotionally engaging.

So they're motivating. They carry a level of urgency that values doesn't, if you don't live up to your values, or not fully, you say we're doing the best we can, it's hard. But if you run to a nightmare that's you don't say, oh you can run into nightmare sometimes. There's a whole, oh my God, no.

We really gotta avoid nightmares. So I like nightmares because they are concrete specifications with outcomes. Specifically outcomes that you want to avoid. You can build strategies and tactics around avoiding those nightmares in a way that you don't really build strategies around abstract values like fairness.

You can, they're very shareable. I can tell you with say a nightmare of mine and you're gonna get it in a way that you might not get it. If I just say I'm for fairness. Okay.

But the KK is, there're for fairness too, but they are different conceptions. So when you say you're for fairness, what do you mean? Nightmares are much more relatable in that way. They're also, the relevant to organizations, you could specify what's organizationally organization relevant, nightmare.

So I'm not asking for what's your particular, your personal nightmare, but rather for the organization for which you work, what are some ethical and reputational and legal nightmares that your organization can run into? That's pretty grabble. And the last thing I'll say, and then I'll shut off, is that everyone can speak the language of nightmares. So maybe you're a data scientist, maybe you're in hr, maybe you're in marketing.

Maybe your own product. You know what it, if I say so, tell me some, kinds of nightmare scenarios that could result if this, that goes sideways, everyone gets that. And if everybody gets that, now the data scientist can talk to Mark and HR fruitfully in a way that they couldn't necessarily earlier if they said we're for fairness. And then you can see, data scientists like what are.

What is that? What do you mean by that? But I think the language of nightmares is, it's a touchstone that everyone can relate to and so they can have conversations with each other, which is the kind of thing that enables collaboration. But hey, at a at a time when there's so much talk about opportunities and.

Ideally bright future on one end on. Yeah. On the other the super dystopian, everything is going down downhill and will erase humankind. Yeah.

Doesn't the term nightmare, ethical, nightmare, specifically, fuel this angst, this fear of something going wrong. Yeah. Isn't it leaving out that, that opportunity of what the technology can actually do. Couple things.

So yeah, I'm not a pessimist. I'm not I don't think the sky is falling. I think it can fall if we don't do things the right way. So this is me saying, let's face the problems head on.

I don't think we get to avoid the nightmares and we get to the pro the promise end of all this opportunity. If we just close our eyes to the nightmares. No, nothing bad here. Nothing to see here.

Let's just think about the opportunity. That's a really bad way. That's, let's wait for things to go really bad. The other thing is that two things, one I'm focused on the book is focused on making sure that the nightmares that you specify are organizationally relevant, which means that you can actually do something about it.

So if you're worried about, a GI, artificial general intelligence, taking over an ex, extinguishing humanity look, you, look, you work on insurance. There's nothing your insurance company's gonna do about it. There's nothing that you know your CPG company's gonna do about it. There's nothing your healthcare company's gonna do about it.

It's just, if that's gonna happen, it's beyond the purview of the organization's nightmare. That's like humanities nightmares as opposed to something that's the organization's nightmares. That's one thing to say. Another thing to say is that I think people are already anxious and fearful of not just the complete doomsday scenarios, but lots of other sort of if you like, middle ranged, bad things happening, like discriminating against people at scale.

Creating reports for clients that contain lots of hallucinated material. Things going sideways in some massive way that you didn't see coming at all and didn't have the tools with which to see it. And I think people are anxious about that stuff. And the answer is not to, let's just not focus on the, in the anxieties producing stuff.

Let's just focus on the good stuff. It's silly. I think we justifiably quell anxiety by facing the problem head on and then problem solving, let's solve it. Let's, we can solve this problem.

We can't, we can reduce the likelihood of catastrophe here. I'll say one last thing. Suppose someone said, Hey listen, we're gonna engage in developing nuclear technology. But listen, we can blow up humanity and some were to say, oh, don't focus on that.

Focus on all the opportunity that nuclear has to work. It's are you outta your mind? You need safety, right? You need you, that's insane.

That's obviously save you. So you need safety. You need to, and you need to attack the problem head on so that you can take advantage of the opportunity. So this is not a.

Eliminate all risk by not pursuing AI. That's not what I'm suggesting at all. I think pursue all the cool things, make sure you don't blow up things, whether it's people, your brand or both. Buy, then in the book I specify a method for doing such a thing, et cetera, et cetera, but yeah.

So that's my short answer to why focus on the nightmares. I think it's, I think it's compelling in the right kind of way. I can absolutely get behind that. Personally I see myself as a techno pragmatist.

Yes, there's lots of opportunity, but there are risks in the truth is somewhere in, in the middle. And we need to find a way to make the technology practical without exposing everybody else to, unreasonable amounts of risks or, yeah. Unmitigated bill. Yeah, in that sense.

Now Reid, we're getting close to the end of the show and I was wondering if you can summarize the key three takeaways for our audience today. We covered a lot of ground about ethical nightmares. Yeah. What are the three things that stick?

I'd say one of those things is that the AI risk landscape is getting insanely complex, diabolically complex, as I call it in the book. The move from narrow to generative to agent AI are big leaps in the AI risk landscape. And when you get to multi-gen AI, we're really talking about. In the in the book, I call it an AI risk cluster fuck.

Yeah. And that one big takeaway is the AI risk landscape is getting crazy takeaway. Two, the standard approach for dealing with AI risk, responsible AI, AI ethics approach, et cetera. The kind that I've developed over the past six plus years, the kind that the big four develop over the, have developed.

I've advised three of the four big four, so I know exactly what they're doing. Any consultant who's been doing the same kind of approach I don't think we're necessarily me and my colleagues and other or, and competitors deserve blame, just didn't know what was coming. But it's broken. It's not just the right way of doing it anymore.

And anyone who's trying to sell you that sort of standard way of doing responsible AI, I think they're setting you up for failure. The third and last thing, which is more, less the takeaway 'cause we didn't talk about it too much, but it's something like. This is what I try to articulate in the book is whatever the solution looks like, and I have my own, the Ethical Nightmare Challenge is my solution. But whatever the solution looks like, it has to be something that could be rapidly implemented and scaled.

If you can't rapidly engage in ethical, reputational, legal nightmare avoidance, then either you're not going to innovate at all because you're gonna be too scared or you're going to innovate totally recklessly. And so the Ethical Nightmare Challenge is my solution for rapid scalable. Governance respons to AI, AI ethics, AI go risk management. But even if you don't use mine, you've gotta find something that is, can move at the pace at which AI evolved and is implemented and scaled.

To me, that's really the big piece to the last one, right? It's moving incredibly fast where blazing through all these different step, scale innovations and it's not just a matter of keeping up, but really also keeping pace with it. So really, yeah. Thank you so much for sharing your expertise with us.

It's always a pleasure talking to you and learning what you're seeing around ethical AI, responsible AI. Folks, if you're in the audience, make sure to check out Reid's book, the Ethical Nightmare Challenge that's out now. And yeah, thank you so much for joining re really appreciate it. Yeah, it was my pleasure.

Thanks for having me.