From Shepard's to ChatGPT: Same Lesson, New Tool - Verify Everything

Speaker A: Hello, welcome to today's episode of the Legal Tech Podcast. I'm Dan Sigle and I will be your moderator today. The Legal Tech Podcast is sponsored by the law offices of Daniel J. Sigel, LLC and Integrated Technology Services, llc, both based in Havertown, Pennsylvania. Integrated Technology Services provides assistance with workflow software management services, software control, and training to law firms and small businesses throughout the country. The law Offices of Daniel J. Siegel provides a variety of services to both attorneys and to the public in Pennsylvania. Today I'm going to talk about Shepherds to chatgpt Same lesson, New tool Verify everything I want to start today with with a lyric that's more than 60 years old but yet somehow seems perfectly suited to today's legal profession. Peter, Paul, and Mary famously asked, when will they ever learn? And every time I needed read another judicial opinion sanctioning a lawyer for filing a brief filed with AI generative generated mistakes. That lyric immediately pops into my head, when will they ever learn? The interesting thing is that these cases really aren't about artificial intelligence at all. They're about lawyers. They could be about other people, but they're about lawyers. And they're always about shortcuts. They're about trusting technology instead of verifying its work. And more importantly, they're about forgetting lessons that every generation of lawyers has had to learn. The technology changes. The lessons do not I've been practicing law for more than four decades. I've watched legal research evolve from something that took hours in a law library to something that can now be done in seconds on a laptop. When I started practicing and in law school, which was even more arduous, legal research wasn't glamorous. If you wanted to know what whether a case was still good law, you didn't click a button. You shepherdized. Actually, shepherdized. That meant walking over to a row of books, finding the exact volume of Shepherd's citations on the shelf, and checking every citation. Then you would check the pocket part, which was a slightly smaller version than the hardbound version you just had. And then you check the supplementary pamphlet, and finally you check another supplement. And sometimes it felt as if the supplements were multiplying faster than rabbits. Every lawyer remembers that sinking feeling. You'd spend 20 minutes checking a citation and still wonder whether you'd somehow missed something. The process was slow, it was tedious, and it demanded attention to detail. Some lawyers loved it, most didn't, and some decided there had to be an easier way. So they cut corners. Maybe they didn't look through every supplement. Maybe they stopped one Update short. Maybe they assumed that if they hadn't found Anything adverse after 15 minutes, there probably wasn't anything there. Sometimes they got away with it, sometimes they didn't. And when they didn't, judges would call them out on it, as would opposing counsel. The technology has changed dramatically since then. Human nature really hasn't. When computerized legal research became commonplace, it transformed our profession. Suddenly we could search millions of cases in seconds. We could key cite, we could shepherdize electronically. Research that once required half a day now took minutes. It was revolutionary. But here's what didn't change. The lawyer still had to read the case. The lawyer still had to understand the holding, what the case stood for. The lawyer still had to, uh, exercise his or her independent judgment. Technology made research faster. It never really replaced the lawyer. Then came November 2022. Chat GPT arrived on the scene, and within weeks, it seemed as though everyone was talking about it. Lawyers, judges, law professors, law students, clients, everyone. For the first time, you could ask a computer to draft an outline for a brief. It could write a motion. It could summarize deposition testimony. It could even explain, explain a complicated statute, might even suggest arguments. It certainly could write email. It could generate a first draft of almost anything. It felt magical. And it wasn't just in the law. It was in every profession. In colleges, uh, academia, you name it. It felt magical. In fact, though, I remember hearing lawyers say exactly that. This is magic. In many ways, it was. But magic has always been dangerous. There's a quotation that's often attributed to either Daniel Wallace or Terry Brooks. I've seen both names attached to it. And perhaps that uncertainty actually proves the point. Today the quotation says, magic is dangerous. It can be both a, ah, blessing and a curse. It takes stress to make the magic serve you and not the other way around. Whether Wallace actually wrote it or Brooks actually wrote it isn't terribly important. For a podcast. If I accidentally attribute it to the wrong author during a presentation, someone may correct me. After all, life goes on. But imagine doing the same brief, the same thing in a brief that you're submitting to a court. Imagine citing a quotation to a judicial opinion where those words never appeared, or citing a case that never existed. That's not an innocent mistake anymore. That's a serious problem, particularly an ethical problem, and a problem of image and your reputation. By now, almost everyone has heard about the famous New York sanctions case. From 2023, a lawyer relied on ChatGPT to perform legal research. The brief cited cases that simply didn't exist. When the lawyer questioned chatgpt, it assured him that the cases were authentic. Instead of checking independently, he accepted the answer. Then when the judge questioned him, he doubled down. Eventually, however, reality caught up with him. Uh, the cases were fictional, and the sanctions became international news. Law schools discussed the case. CLE programs discussed the case. Bar associations discussed the case. Judges discussed the case. Everyone, it seemed, discussed the case. Surely that would be enough. Surely lawyers would learn. No, apparently not. Every few months, another opinion appears. Now it seems like every few days, another opinion appears. A different lawyer, a different jurisdiction, same story. Artificial intelligence generated inaccurate information. The lawyer failed to verify it. The court noticed. Sometimes opposing counsel noticed first. Sometimes the judge noticed. Sometimes nobody noticed until much later. But eventually, the mistake surfaced, and the explanation always sounded the same. I trusted the software. Think about that phrase. I trusted the software. Would you ever say that about a client? I trusted everything my client told me. Of course not. Clients make mistakes. Sometimes unintentionally, sometimes intentionally. That's why we verify. Would you blindly trust opposing counsel's factual recitation of the facts or of the law? Again, no. You really wouldn't. Would you file a brief if you're a lawyer that's drafted by your first year associate without reading it? Hopefully not. Would you sign an expert report without reviewing it? Certainly not. Then why would anyone treat AI differently? One misconception deserves particular attention. Many people assume these problems only occur when lawyers use free public AI tools. That's simply incorrect. One of the most important recent decisions came from the United States Court, Court of Appeals for the 6th Circuit in the United States vs. Farris, decided in April of 2026. The attorney involved wasn't using some random chatbot. He was using Westlaw Co Counsel. That's professional software designed specifically for lawyers. Lawyers pay lots of money to use the, uh, Wexlaw and other products like it. The attorney admitted that inaccurate quotations generated by AI somehow found their way into his Appell brief. The quotations did not exist. The cases didn't say what the brief claim. They said the holdings were inaccurate. To his credit, the lawyer accepted responsibility. He admitted exactly what happened. He failed to adequately review the AI generated draft. Notice what the court focused on, though. Not that he used A.I. uh, not that he experimented with new technology. The problem was that he stopped doing what lawyers have always been required to do. Review their work, verify the citations, and read the authorities. That's an incredibly important distinction. Generative artificial intelligence is not unethical. Using ChatGPT isn't unethical. Using Westlaw Co Counsel Isn't unethical using Lexis AI isn't unethical. In fact, I think lawyers who refuse to lose, to use these tools and to learn these tools may eventually place themselves at a competitive disadvantage. These systems can dramatically improve productivity. They can summarize records. They can identify issues. They can generate timelines. They can compare documents. They can organize evidence. They can draft correspondence. They certainly can suggest research paths. They're remarkable. I use AI regularly. My office uses AI regularly. But we also verify everything, especially anything that ends up in court. This isn't just about citations. It's about professional judgment and eventually, reputation. Good lawyers don't simply collect information. They evaluate it. They ask questions. They notice inconsistencies. They recognize nuance. Artificial intelligence doesn't replace those skills. If anything, it makes them even more valuable. Because AI produces work so quickly and often so confidently that it becomes even easier to assume it's correct. But confidence isn't accuracy. Fluency isn't truth. And an answer that sounds convincing can still be completely wrong. Lawyers have to remember that the ethical rules haven't changed. Rule of Professional Conduct 1.1. That's in every state's Rules of Ethical. Of ethical obligations. Still requires confidence. Huh? Excuse me? Competence. And Rule of Professional Conduct 3.3 still requires candor to the tribunal. Rule 5.1 still requires supervision. Rule 5.3 requires lawyers to supervise non lawyer assistance. Increasingly, ethics committees have recognized that AI functions very much like another assistant in the office. An extraordinary capable assistant, but still an assistant. You wouldn't allow, I hope, a paralegal to file papers without an attorney or reviewing them. You wouldn't allow a summer law clerk to submit a brief directly to the court. AI deserves exactly the same supervision. No more, no less. So what should lawyers actually do? The answer is surprisingly simple. First, verify every citation. Not most citations. Every citation. Second, read every case you rely upon. Don't quote headnotes. Don't quote AI summaries. Read the opinion. Third, compare quotations with the original source. One missing word can change legal meaning. And I have found again and again and again that quotations from AI are routinely inaccurate. Fourth, establish office procedures. Don't leave verification to chance. Build checklists. Require second reviews. Develop habits that make accuracy automatic. Finally, train. Yes, train everyone in the office. And I know that's a hard subject for a lot of lawyers and law firms to understand. They don't do enough training. Many don't do any. Well, not just AI but a whole wide range of software and. And other tools that they use. They should be Training everyone. Lawyers, associates, paralegals, law clerks. Everyone should understand that AI is an assistant, not an authority. Looking back, it's fascinating how little has really changed. Forty years ago, lawyers skipped Shepards. Today, some lawyers still skip Shepherds, but certainly skip verification of AI output. That's today's Shepherds. Yesterday, shortcuts involved books. Today, shortcuts involve software. The underlying mistake is identical. Trying to save 10 minutes here and there and everywhere else. But you're risking your reputation, you're risking sanctions, you're risking disciplinary, uh, action. And more importantly, you're risking credibility with the court. And credibility most likely is and should be the most valuable asset any lawyer possesses. You can lose a case. You can recover from that, lose your credibility. And every future argument becomes harder to make. So where do we stand? Well, the question when will we ever learn? When will they ever learn? Remains important. I hope the answer is now, because generative artificial intelligence is not going away. If anything, it will become more sophisticated, more capable, and more deeply integrated into everyday law legal practice. That's exciting. It should be exciting. But every technological advance comes with responsibility. The lesson our, uh, profession learned with Shepherds, the lesson we learned when we had online research, and the lesson we're Learning now with ChatGPT, is exactly the same. Technology can make lawyers faster. It can't make them more responsible. Only lawyers can do that. So embrace the technology, learn it, and experiment with it. This has been the Legal Tech Podcast sponsored by the law offices of Daniel J. Siegel, LLC and Integrated Technology Services. We hope you've enjoyed this podcast. If you have, please share it, please subscribe, and please let all your friends and lawyers know about it. I look forward to future podcasts, and thank you again for listening.