Episode 111: Building Your Defences Against AI Misinformation

Dr Genevieve Hayes: Hello, and welcome to Value-Driven Data Science, where data professionals become strategic experts. I'm your host, Dr. Genevieve Hayes, and today I'm joined by Derek Gibson. Derek is a decision scientist, analytics educator, and has recently wrapped up his long career in financial services at Wells Fargo.

He serves on the Wake Forest University MS Business Analytics Advisory Board. He is also a co-author of Data Duped: How to Avoid Being Hoodwinked by Misinformation, and author of the upcoming Data, AI, and the Noise: Searching for Truth in Information and Algorithms. In this episode, we'll explore strategies you can deploy for identifying unreliable AI outputs and building the defenses necessary to keep AI-generated misinformation from reaching your stakeholders. Derek, welcome to the show.

Derek Gibson: Thank you very much. It's really great to be here Dr Genevieve Hayes: Recently, I've been approached on several occasions by consulting clients asking me to review statistical methodologies developed with AI assistance. In all cases, my clients have come from non-technical backgrounds, and what they've managed to produce is impressive. Detailed statistical specifications that would otherwise have required years of study to develop.

Their work is living proof of just how far AI has come. However, on reviewing the work, the one thing I invariably notice is that while on the surface what the AI has produced seems polished and professional, if you dig deeper, subtle methodological issues emerge, issues that are invisible without specialist expertise. Now, in these situations, this is exactly why I've been brought in. But conducting these reviews has made me wonder about the situations where no one in the room has the expertise to validate AI outputs, including situations where I'm potentially the one producing AI outputs in areas that are beyond my own expertise.

Depending on the circumstances, the results could range from embarrassing to career-limiting or even catastrophic. AI has the potential to amplify human capabilities, but it also makes us vulnerable in ways we've never had to face before. Now, Derek, you've spent over 20 years in business analytics and decision science, and now you write about defending against AI misinformation. Given your experience, there are undoubtedly dozens of other topics you're equally qualified to write about.

What led to your decision to focus on helping people avoid being deceived by AI? Derek Gibson: Yeah. So it's not just being deceived by AI, but I think deceived by data and information or misinformation. The work that Dr.

Jeff Cam and I did on our first book, "Data Dupe," was really coming out of the classroom. The skill sets that we were teaching young professionals to make better decisions really led to my wanting to further how people make decisions and really having the skill set to make adequate decisions based on the information they had. And the challenge with AI is that - I'm sure you've used it, lots of people listening to this use it every day, and they probably notice that AI is confidently correct.

AI is, if you pull back the covers, it is not a truth tool. It is a tool that is looking at what is the best answer to the question that has been prompted to, resolve. And It's literally looking at the probability of the next word or the next phrase in the context of what's being asked of it. But if it doesn't know the answer to a question, it'll fill in those gaps.

And I think the risk right now with AI is that not only does it do that in a manner that is seemingly trustworthy part of that is not only the fact that it'll fill in the gaps, but it's also because of the interface we have with AI is very different than what we've experienced before working with computers. And that interface, that very personal back-and-forth conversation feels human-like. And there's part of our sort of ancient being that when we have these interactions with other people, we trust them.

You're giving up your trust right away. So because it's doing that, we have to be a little more aware that maybe the answer isn't exactly right. And then further, what makes this a bigger risk is that AI is doing this at a scale never seen before. So misinformation can move much faster than it's ever done before, and AI can in fact create it.

We no longer have to rely on humans to do that. So I think those are some of the initial challenges. The skills that we wanna focus on whether it's my research, my writing, what we're teaching in the classroom, is how do you get people to be a little bit skeptical without being too alarmist, but also, how do you teach people to trace back to the source of the information they've just been given? So those are the techniques and the frameworks that we're focusing on in our work.

Dr Genevieve Hayes: Yeah, it's very interesting the source of AI's information is just Whatever the dominant opinion is on the internet. You could hypothetically have a situation where... let's say we go back to Galileo's time. Everyone believed the world was flat.

Doesn't mean it's true, but if the internet had of existed back then, every internet site would say the world is flat. Derek Gibson: That's exactly right. So yes it's the dominant opinion. I like how you phrased that.

Researchers have shown that AI is relatively good on things like science and math, although there's been a few people who've pointed out online that sometimes it doesn't always do the right math. But you know, on things that are, I'll say, concrete and well-known facts, it does really well. Where it strays is the things that are often debated or subject to human interpretation. So if you're asking AI about a fiction novel or your favorite movie or poetry, it might come up with different answers.

Or if there are topics even in history that had a long discourse on the internet or whatever the training data source is, a discourse about what actually happened and why, you could have answers from AI, even the same AI in a different chat, that are different than the first time you asked it, if that makes sense. So the inconsistency, which others might call the hallucinations, are still gonna be out there, and it tends to be in areas that are less known, the ambiguous areas.

And some other researchers have even trained AI to create false answers by creating content that it then feeds into the AI. So you get this loop of information. So I guess my main point there is AI is not a truth tool. It is just reflective of the information it has access to.

Dr Genevieve Hayes: So when ChatGPT first launched, its output was riddled with errors and hallucinations, and that initially tripped up a lot of people. Since then, I've noticed a marked decline in the number of AI hallucinations I've observed. But you also have people becoming a lot more aware of the potential for AI to get things wrong. Given this, where do you now see people getting caught out by AI misinformation?

Derek Gibson: Yeah, I think it goes back to the confidence that people have in some of the responses And probably also the wow factor of people when they first start using ChatGPT or whatever tool. The wow factor is they get a really good answer, and it immediately builds credibility as they're using the tool, and then they move on to the next question. And because that credibility's been built up with the first few responses, it continues, so we as people, never lower our guard and do a reset on how much do we trust the answers we're getting out of this machine.

It seems to go up over time because we've had good answers. And then we'll find ourselves believing an answer down that path that isn't accurate. So the overconfidence blends into a lot of places. So it could be, people taking shortcuts to do , their homework.

And notably in legal cases there's been a number of them, they seem to make the press quite well, that, people are filing documents with courts, and they'd have references to cases that don't exist. It becomes so prevalent, there's actually a researcher who is creating a database of all the times that this has happened including some cases in California where they've had some significant fines for filing information with false references. I think that's where it comes to people having to learn new skills to work with AI, that first, you can't just take the first answer you get from it.

It needs to be an iterative process. And you also need to treat AI like it's this eager new assistant you have at work. They're really smart, they're really capable, but they often rush through answering the question because they wanna please you. And that's how AI is.

So when you get the answers to those legal questions or you're trying to summarize a brief, you have to ask those follow-up questions that say explicitly, "Show me the references. Provide me the links." And then here's the hard part. Even though we wanna use AI as a time-saving tool, we have to take additional time to go and look up and cross-reference the information it gave you.

We still have that role to do. And in a way you are saving time on one task, but you're creating a new task for yourself, which is the verification task Dr Genevieve Hayes: I think the key point is that whole relationship between you and AI. What you said there is you have to treat AI as being your junior assistant. So you are effectively positioning yourself as being the knowledgeable person above that assistant.

And I think historically, that's how we've always positioned ourselves with regard to the pre-AI internet. We might look something up, but you can verify whether a site looks reliable or not. You might trust encyclopediabritannica.com, but you're not going to trust some cheesy-looking HTML site that looks like something developed around the year 2000 So We were always in control when we were using the internet.

But because The chatbots are designed so that they do sound so confident. We have, in many ways, positioned ourselves as being the student or the underling of the AI. We trust what that AI says, just like you would trust a teacher or an experienced manager, and you can't do that because the AI is just a really confident-sounding junior at the end of the day. Derek Gibson: Yeah, that's absolutely right.

So it goes back to this new skill set, the verification skill that we need to build. And I love the comparison you just made. We used to look at a Google search, and if you went past the first page, you'd have a page and page of what it references And we know that algorithm at the time was stack ranking on those that were most useful to people who had previously asked a similar question. So that builds our confidence that Google's gonna give you something reasonable to look at, and it gives you the evidence, the link to that source.

With AI, it doesn't give you that at all, again, unless you explicitly ask for it. It'll give you a response, but you don't know where among its training data it has come up with that information. And if it is information that's verifiable, you can then, of course, ask that follow-up question and say, "Give me the link," but then you're doing that extra work. So I do think that what's changing is how we get to information.

And this is the premise of a lot of my more recent writing. It is the noise that we are having to filter through on our own, and that skill set of going past the noise, going past the initial answer, going to a source is how we can get there, but it requires us to do things just a little bit differently than we used to. Dr Genevieve Hayes: Are there any non-obvious red flags that signal an AI output shouldn't be trusted? Derek Gibson: Non-obvious Dr Genevieve Hayes: So there are some that are pretty obvious I never trust any bibliographical references that AI comes out with after seeing so many fictional journal articles created by AI.

Derek Gibson: Yeah. So back to what we were saying earlier, you have to have some domain knowledge about a topic to verify it on your own. And if you don't, I think that's when individuals can become vulnerable. And then if you put that in the framing of a topic that might not be well known is definitely something AI will give you an answer to, but maybe that's the red flag you're talking about that, how could it possibly know the answer to something so ambiguous?

Those are the sort of things I'd be looking for. Dr Genevieve Hayes: It's like - this isn't a reference to generative AI, but at my gym they have the fancy scales. Those sets of scales that tell you not just your weight, but, muscle mass and bone density and all sorts of things. And I was thinking about that the other day, and I was thinking: How can this possibly tell you all of these things?

I can understand how it can tell you things like weight, but how can it possibly tell me, basal metabolic rate based on standing on this for five seconds? And, I did some searching on the internet and I realized, no, it calculates about two things, and then everything else is just feeding numbers into various formulae. And so I think it's that step of about thinking, okay, given the information that's going in, how can this AI come up with this conclusion? What would be required to conclude it?

A set of scales can't possibly determine my basal metabolic rate. And similarly, AI couldn't determine that either because it wouldn't have the information going in. Derek Gibson: Yes. There's a lot of inference there.

Inference used in a different way from AI, but it's inferring something. My favorite one of those, by the way, is the body battery. I don't know how it comes up with this, but it seems to have a little icon that says when you're full or not full based on your sleep and your activity and all that sort of thing. And it's a great example 'cause I kinda wonder how did they come to that?

And again, back to being a data skeptic , there's lots of things where we need to be a little more objective about how did it come to that conclusion. Why do they know this thing? You remind me of some of the other things we see that are, like maybe headlines where someone has a very specific percentage of something that they found, and I think what's the denominator? How did they come to that conclusion?

What's not only the source, but what's their methodology? So it's interesting that we see those things that again, AI's giving us confidently, but we have to still use those old school tools of how could that possibly be known? 'Cause it's gonna wanna do that. Dr Genevieve Hayes: Yeah.

The other thing that's a big red flag for me is if AI comes up with a polished summary that lacks technical depth, it's almost certain that once you start looking into the specifics of it, there's going to be something major missing. That's what I've found from my review work. It can come up with all this statistical gloss, and then it's but it doesn't explain how you would actually get that data in the first place. Derek Gibson: Great.

Dr Genevieve Hayes: sample in order to test this hypothesis. It was just saying, "Go collect the test data." But you can't prove a hypothesis without a control group. Derek Gibson: That's such an excellent point.

So one of the things that I'll offer up that hopefully is useful not only to you but maybe everybody listening, is when we put a prompt into AI, we should not only ask it like, "What's the answer to this question?" Or, "Do the statistics on this," but we should ask it to explain itself step by step. And then that's how you, again, as the manager or the person overseeing this virtual worker can inspect what it's gone through, and if it can't explain those steps, then you know that there's a problem.

If it can explain those steps, then your domain knowledge around the topic should be helpful. It doesn't mean that everybody needs to be smarter than the internet to be able to work with AI and validate it, but you need to think of it as, like my experience as a manager. You can supervise people who know things that you don't, but what's important is how you manage them and how you ask those questions about how did you come to this conclusion? What was your source of data?

Were there any problems with the data before you ran your statistical analysis? Once you did, how did the data support the conclusion that you're providing today? These kind of basic questions don't change with AI, but actually become more like vital in managing the outputs we get from AI Dr Genevieve Hayes: reminds me of writing exams for my students when I was teaching. Every single question I'd write would always have at the end of it, "Show all working," or, "Justify your solution."

So what you're saying is basically treat AI like you would if you were an examiner writing an exam for students. Put in that final sentence at the end. Derek Gibson: You're right, show your work. And again for some topics that are undisputable science those are less risky.

I think the riskier ones is when you're asking AI to make an interpretation of ambiguous data in the first place, or data that comes from conflicting sources, or data that might have been collected in a way , that isn't necessarily clean. There's messy data out there. So if you threw multiple data sources into AI and then asked it to look at the trend of how your company is performing, it could come up with an answer, it absolutely will, but you have to ask it what steps it went through to come to that answer, just like you would, a regular employee or a member of your analytics team.

Did they cleanse the data? Did they find anomalies in the data? Was there anything in the data that surprised them before they came to that conclusion? What about the data might cause that conclusion to be different?

What could go wrong? And again I think what's so important is that AI is here. It's not gonna go away. But we have to adapt to how we work with Dr Genevieve Hayes: Okay.

And so this is what you were referring to before as the validation step. Is that right? Derek Gibson: That's right. Absolutely.

Dr Genevieve Hayes: The way I work with AI is I tend to do things one step at a time so that I can validate things along the way because I don't want AI to get to step 12 and have a mistake back at step two. I'd rather realize there's a mistake at step two when it happens and then correct it. But, what you're saying is it's creating pro-productivity gains, but there's also a cost of those gains. Derek Gibson: Yes, there can be.

But let me give you another example where I'm - I don't wanna be negative on AI. I really am optimistic about what AI is gonna do for us. But right now, probably the best use case for AI in a organization is as a coding assistant. And I think it is a multiplier.

Not only does it just save time, but one of the things in my experience with it is that if you ask it to write a bit of code it doesn't just give you the ultimate answer. It's often very iterative. It'll show you- Like the first step it wants to do, and then ask you to go test that. And it'll tell you why that worked or why it didn't, and then it'll go on to the next step.

It's very much like a coaching tool and it also by way of the steps it goes through, helps you understand as the human what it's doing and why. And that's important because that means that the next time you ask AI to do code for you, you'll be that much more knowledgeable about how the code should look, so that now as the supervisor of the output, you have more knowledge. So it becomes like this really good reinforcement loop that it starts out as the coder, it helps teach you how good code should look.

On the next task, you now have a better ability to inspect that code to make sure that it's accurate or going in the right direction. In most cases, it's accurate, it's gonna work. The questions you have to ask is why did it use this method of doing this task? Dr Genevieve Hayes: That's a good use case.

Yeah, I've found with things that I've never done before, getting AI to coach me through them has really helped me to learn how to do that. So at the moment, I'm doing some work around building AI agents. I don't have a clue what I'm doing. So it occurred to me, why don't I ask AI to help me to do it?

And so I've been having a lot of conversations with AI and saying, "Okay, this is what I wanna do. Where do I even begin?" And it's coaching me through doing that development work. Derek Gibson: That's excellent.

And hopefully it knew enough about you to know your background, knew your current capability. Maybe it's from its memory, it looked at other projects you've done that were similar. So it didn't start out at just the base level. It stepped up to match your current learning level.

Is that right? Dr Genevieve Hayes: It started by asking me a bunch of questions. So for example do I wanna do it via code or via some sort of GUI-type int-interface? And, what's my goal with these agents?

And we had a lot of conversations and, AI started by asking me, what's my vision for everything that I wanna build? And so by doing that, Claude was able to say, "Okay, you've basically got three main components for this that you need to build out. Perhaps start by building out-" This component, then the next component, and so on. And so Claude's helping me to figure out a strategy for doing this, whereas when I started it's like I don't even know where to begin because I've never done this before.

Derek Gibson: And I think having, that honest conversation with AI when you start is a good approach because you don't want just the answer. You want to know how it got there. So starting with what are your goals, what are you trying to solve, what's your output? That's excellent.

If you just say, "Give me the output," you'll get something, but you won't know if that's really what you want, and then you spend a lot of time, I think maybe more time going this is not quite what I needed." Like that communication skill is something that we also need to revisit as we're working with AI in this conversational way. Dr Genevieve Hayes: Yeah, I find the more time I spend talking to AI, the better the outcome I get. So I actually try and slow things down rather than just saying, "Here's a mega prompt about what I wanna do.

Go do it while I go off and make myself a cup of coffee or something." Derek Gibson: Yeah, you'll get much better results. Dr Genevieve Hayes: And I've actually saw a post about it on LinkedIn the other day where they were saying that MIT research showed that people who argued with AI and pushed back on AI results tended to get more creative outputs than people who just blindly accepted those without putting up a fight. Derek Gibson: I'm not surprised by that research.

I think I've seen something similar to that, that it'll in a sense work harder to give you a better answer is what it's doing. Dr Genevieve Hayes: Yeah and that's what I've seen 'cause often the first answer's rubbish, and I'll say no. That's not at all what I want. No.

This is what I want. Can you do it again with these conditions?" But I couldn't come up with those conditions until I saw the first bad answer, and then was able to say, "No, this is wrong because of X, Y, and Z." Derek Gibson: And it probably very eagerly said, "Thank you.

I'll consider that in my next response," right? It takes feedback quite well. Dr Genevieve Hayes: Yes, it's wonderful. In that sense, it's a lot easier than working with human coworkers because AI loves getting feedback, whereas no human likes being criticized So what's the single most important thing a data professional can do right now to strengthen their AI defenses?

Derek Gibson: I think the single most important thing is to not wait on an institution or organization or a tool to help you build the skills you need to verify information. Right now, the current state of AI is it is moving at an unprecedented pace. There are different players. So the players being the AI development organizations, the companies using those tools governments and regulators all have a stake in the future of AI.

But at the moment, they're not coordinated, and some of them are even moving in different directions, largely driven by economics. So if we're waiting for that moment that AI is going to give us truthful and only truthful and helpful information, then that's not gonna happen until those parties coordinate. So it's up to us to build our own data defense, to build our own skills, to look at how we're using the tool so that we can verify information and not get distracted by the misinformation or the disinformation and move forward.

So that's the most important part. Dr Genevieve Hayes: For listeners who wanna get in contact with you, Derek, what can they do? Derek Gibson: They can reach out. They can find me on LinkedIn.

They can also find me on my site, derekwgibson.com. Dr Genevieve Hayes: And that's it for today's episode of Value-Driven Data Science. But if you want more from Derek, next week you can catch our Value Boost episode, where we explore how data professionals can help their stakeholders avoid being duped by misleading data beyond AI outputs.

And if you found today's episode useful and think others could benefit, please leave us a rating and review on your favorite podcast platform. That way, we'll be able to reach more data scientists just like you. Thanks for joining us today, Derek. Derek Gibson: Thank you Dr Genevieve Hayes: And for those in the audience, thanks for listening.

I'm Dr. Genevieve Hayes, and this has been Value-Driven Data Science.