AI Hallucinations in Court The Sullivan & Cromwell Lesson

Speaker A: Hi and welcome back to another episode of the Law Practice Podcast. In today's special episode, we are talking about a high profile cautionary tale of how AI hallucinations impacted an AmLaw50 firm. In this episode, our special guest discussed why these costly mistakes keep happening, even to seasoned legal professionals. And they share some tips on what you can do to help protect your law firm from these types of issues that could drastically impact your reputation. So stay tuned for today's special episode. Welcome to the Law Practice Today podcast where you will hear updates on hot topics, tips and ideas that can help you run a more successful law practice. Now, this show is brought to you by the Law Practice Division of the American Bar Association. So that means the views and the opinions shared on this podcast do not represent the ABA or the Law Practice Division, but they are the views of each individual participant and any products or services discussed do not represent any kind of endorsement from the ABA or the Law Practice Division. Now let's get back to the show.

Speaker B: Welcome to the Fireside Chat. I'm, um, here in sunny San Diego with my friend Jennifer Ellis and my friend Michael Eisenberg at the, uh, Law Practice Division spring meeting. Some big news involving an MLA 50 firm, Sullivan and Cromwell Mobile law firm, and they filed a brief in court with some citations to cases that didn't exist or some citations that were just plain inaccurate. The case didn't stand for the proposition which they claim, and they got caught and had to apologize to the lawyers on the other side and throw themselves on the mercy of the court. And what's going to happen to them, uh, remains to be seen. But I wonder, as many of us have wondered over the past couple of years, why does this keep happening? What's the, um. By now everybody knows that this is a flaw in how these large language models work. And yet people keep doing it. Jennifer, why?

Speaker C: Hi. I find it fascinating that how quickly this all happened because normally when a new technology is introduced, it takes a little while before someone gets in trouble. But, uh, as you guys know, and you out there probably know, generative AI was introduced in 2022 with ChatGPT. And very quickly after we had the MATA case in which someone got very publicly in trouble for dabbling and using AI to dabble. So when you dabble, certainly you are more likely to make a mistake. Of course, in this case we don't have dabbling. We have some affirm that had a very clear AI use policy. And if you read the articles about it, they make it clear that the policy was ignored. So when policies are created but ignored, that leads to problems. And I think every firm that is using AI should have an AI use policy. But why does it happen? It could be laziness, it could be confusion. It could be a failure to understand the limitations of the tools.

Speaker B: Time pressure.

Speaker C: I think time pressure is a big one. I know one case happened because a woman had just lost her husband, my sympathies to her force. And she was rushing to get the work done. We all have so many pressures on us. I think it's very easy to. With the confidence. AI is so confident sounding right. It gives wrong information very confidently.

Speaker B: I think like a lot of lawyers

Speaker C: I know, like a lot of lawyers, a lot of politicians, and we believe, uh, confident language. So I think it's a mixture of all of those things that leads to these ending up in court filings and getting people in trouble.

Speaker B: Yeah. Michael, you're a student of all this. I know and have written about it and I have a podcast about it. What's your view? Why does this happen?

Speaker D: I find something that Jennifer said interesting, that it's slow for attorneys to get in trouble or using this type of technology. Yet I've always noticed that attorneys, uh, typically are slow to use this type of technology. Not to pick on one product or another, but how many attorneys are still using WordPerfect because of their old programming days that they simply are so comfortable with? They don't want to shift from that to something else that may be more efficient in getting the work done today. It's interesting how everyone just starts using ChatGPT and its other AI cousins. Uh, they seem to be under the impression that because it's there, it's infallible. And yet by doing that, you're violating, you're violating simple ABA rules like 1.1 and comment 8. You need to be competent in what it is that you're using. You don't have to know how to program and understand the nuances of ChatGPT to the granular, but you still have to understand its basic limitations, which includes hallucinations. But what I find amazing is that regardless if you're using ChatGPT or having a clerk write something for you, one of the final steps should always be Shepardization of your of your citations. Because quite frankly, if the citations aren't any good or if it's the wrong premise, then you have no business submitting briefs. In my practice in the past, when I'm dealing with the other party, certain parties that I'm consistently up against, I've noticed that they had a tendency not to always use the right citation. And that was before ChatGPT and AI. And that being said, I would always check their citations to make sure that they're actually accurate. But before I submit a brief, one of my final steps is to double check all my chapterization citations and make sure that they are accurate. Because not only do we have, uh, a competency issue, we have to make sure that we're doing candor with the court and that we're supervising whoever we're using. When I say whoever, it's either a clerk under 5.1 or a computer or an AI under 5.3. So there are a lot of rules that are already baked into the system. And why we're not following following them, we're not. Why we're not doing our due diligence is beyond me. But as to why I think people don't really understand the technology and its limitations.

Speaker B: Yeah. Now, I want to, uh, push back a little bit on what you said, Michael, because, uh, part of the problem here is what you're talking about, the obligations you're talking about really upends our work processes for years. Let's take the judiciary. Historically, a judge would go to their court, would do the research, judge would take it and read it, and probably not go read each and every citation to make sure it's accurate. Now, that may be the obligation of the judge. Local counsel gets a brief from national counsel, two hours to file it into court. It's got 50 citations. What are you supposed to do? Go check and read every single citation. So I don't think our workflows and work processes have gotten to the point to recognize that if you're going to use these tools, you may have that obligation to do all that. And we have to work through that a little bit.

Speaker D: But I disagree with you just a little bit, because when you're using programs like Lexis and Westlaw to check your citations, I don't think in most cases you have to read the entire case to understand whether or not it is actually supporting the premise that's being proffered for it. If I look at a citation for a page specific citation, usually I can tell, okay, this seems to be flowing with either their argument or this makes no sense. And quite frankly, from that I should be able to glean through the article itself. That doesn't take more than a few minutes, especially with technology today and even 10 years ago.

Speaker B: It's an interesting point. I've heard people, and I want to get your thoughts, too, Jennifer. But I've heard people say that more senior lawyers that are used to using these tools, when they get the result back and read it, they can say, that doesn't sound right. Something off with that. It's the more junior lawyers that don't have that sort of ability and experience level to read something and see that it doesn't look quite right. That, I think, is a problem. Jennifer, we ignored you for a minute or two, but, uh, that's okay.

Speaker C: I find it interesting how people are violating 1.1. And you have to be competent with whatever technology you're using, which doesn't mean you have to be an expert, as you said. It just means you need to be trained in it, you need to play with it, experience it. Because I think it is harder to understand the hallucinations if you are not really working with the AI. So I always recommend people start with something simple and not dangerous, whether that be a, uh, PowerPoint presentation of recipes based on what you have in your cabinet. And you just get used to the tools. And as you do that, you are, uh, become better at spotting hallucinations over time. Every day that I used AI, something happens with the AI. Some kind of mistake, some kind of fabrication. It's constant. And I think a lot of lawyers don't understand how deep of a problem this is and how it is a feature not above.

Speaker B: Yeah, uh, it is an engineering risk that's built in. And there is a school of thought that says that you can at least start to predict when a hallucination might occur, and that occurs when you ask it. Something novel, something that if you ask it, what's the holding in Marbury vs Madison? It'll probably do okay because there's so much information, but something that doesn't exist. Uh, a case for a proposition that's a little harder for it to do, and it's at that point where the risk goes up.

Speaker D: But I think regardless of either situation, whether it's Margaret versus Madison or something brand new, you always have to do your due diligence review, check those citations, then submit your brief.

Speaker B: Huh. It's funny, though, when you start talking about that, you could very quickly end up spending more time checking the citations than if you never use AI uh, to begin with. So go ahead, Jennifer.

Speaker C: Well, I was going to say, I'm sure all of us. I've been a lawyer for 27 years now, so when I've done litigation, it's been pretty common for me to find that opposing counsel has cited cases. But then the proposition, they tell Me is completely incorrect. And we can assume either. We can look at it two ways. They either didn't read the case and maybe got a summary from the headnotes. And the headnotes are not always right.

Speaker B: The Westmore head notes. Right.

Speaker C: I was certainly warned not to go by the headnotes in legal methods all those years ago. Or they're intentionally being dishonest with me and thinking maybe she won't catch it. Which I will, because I read my cases. I, uh, read the opposing counsel's cases. I think what we're seeing is the reality of lawyers who historically don't read cases because it is easy to catch a, uh, fabricated citation. All you have to do is go to Google Scholar, go to your non AI Lexis, your non AI Westlaw, your non AI. And I say non AI very intentionally. Should not be checking AI with AI.

Speaker D: Right.

Speaker C: Should go to a non AI tool. But it's a common phenomenon. I think what we're seeing is that more lawyers than we realize just don't read the cases. And as a result, if you have a completely fabricated citation, obviously you're not reading the cases because it doesn't exist. Right. So it's. You could go to just tell if you're using perplexity, for example, tell it. Give me a link to the Google Scholar citation for the case, and assuming it's a reported case, you'll find it. And then you don't have to read the whole case. You can skim it, make sure it stands for the proposition for which you're citing it.

Speaker B: Yeah, and it's, it is interesting, though, because all of us as lawyers have taken holdings of cases and push them as far as we can push without going over the line. And sometimes when you're checking these cases to see if the citation proposition, the case is cited for the right proposition. How close to that line do you get? Could make all the difference. Michael, Jennifer, thank you. It's been a fascinating discussion. We can sit here all day, but I guess we better stop, quit while we're ahead.

Speaker A: So thanks, thanks for tuning in to the Law Practice Today podcast, where we bring you conversations and tips on how to run a more effective law practice. Now, if you want to hear more, be sure to stop by the American Bar association website to see more updates on, uh, new episodes as they come out and other awesome resources that are offered by the American Bar Association. Thanks for tuning in.