06/19/26: Anthropic withdraws Fable 5, AI might already be sexist, and more

Host: Welcome everyone to the Juneteenth edition of Legal. Legal Talk. Legal Tech week. Legal Talk. What. What is the name of this show? Sometimes I never know. Uh, we were just talking about something else which got my tongue tangled. Uh, if you came here looking for the US uh in Australia game, sorry you've tuned into the wrong channel. Uh, but maybe if things get boring here maybe we can give you a few highlights as as this goes along. Uh, but this is the show where we talk about the top stories in legal tech and innovation for the past week. And uh, we have a special guest panelist joining us today. Uh so let's go around and start with introductions and uh Doug you've been on the show before but uh, why don't you kick it off

Doug Austin: Austin as it's for Ediscovery today. They lead a uh blog about ediscovery and related topic topics. Um I'm been uh in the industry 30 plus years. Oh already a ah already a shot on goal. I'm and if I'm distracted in doing this or doing you know, you'll know why because I was over here. Um, uh anyway been uh in the industry 30 plus years. I'm also right now uh serving as interim direct interim editor uh of the Pinhawk Law Technology uh Daily Digest. Um so uh, keeping that uh, um. Keeping the seat warm for um uh for for Jeff uh until he gets back and uh. Um. Hopefully uh uh you know he's hopefully he's doing well. You know I hope he gets well asap. Uh and uh, that's uh, that's what I'm here to do is uh you know, talk. Talk about the topics.

Host: Sounds good. Yeah. Jeff uh, has been also been a guest on this show and uh, uh uh you know so tragically had a sort of a very simple accident that had uh bad uh physical consequences for him. It's taking. Taking him a bit of time to uh recover and uh, we are all hoping uh for his very speedy recovery and he gets back to full fledged uh, full time job at his main job uh which is at a law firm and uh, and uh newsletter editing duties as well. So good for you for filling in um and a somewhat usual uh guest uh coming over from the uk. Caroline Health.

Caroline Health: Happy Juneteenth. Good of you to be working all of you. Uh, yes, in the uk. This is how much I love you lot that I'm here after two days of legal tech talk feeling amazing and not tired at all.

Host: Probably the only fact that you. Only the fact that you've been at a conference for two days that you're actually home tonight and able to join us. So we have that to think about.

Caroline Health: That is also true.

Host: All right, and, uh, Steve.

Steve Empry: Hey, Steve Empry here. I write the blog techfall Crossroads, and I write for above the Law, and it's good to see everybody on this holiday weekend.

Host: And last but not least, Joe.

Joe Patrice: Joe Patrice from, uh, above the Law. And I'm on the Thinking Like a Lawyer podcast. And I am, um. Uh, you know, we did have the day off, but, uh, since I don't view participating this show as work as much as my free time to hang out with folks, so I was happy to join, even though it's a holiday.

Host: See, I was thinking the same thing. I mean, a whole bunch of the regular panelists said, hey, sorry, you know, it's. It's a holiday for us, so we can't join. Join the show. Like, is that a thing? Like, I don't think of it that way at all. It's just like, I just like hanging out on Friday afternoons. And, uh, it's not like whether I'm at work or not. Of course, I work for myself, so it's not quite. Not quite the same. I don't get holidays, actually. But, uh, still, it just never feels like. I don't do this because it feels like work, that's for sure. So I'm offended that all of our other panels consider hanging out with us to be work. I don't know.

Joe Patrice: Did. Did you even take a button?

Steve Empry: Maybe we should.

Joe Patrice: I was just gonna say, did you take Bob Ambrosi day off?

Host: I did take Bob Ambrosi day. I did earlier this week, have a birthday, and I took Bob Ambrosi day off, uh, and more because I wanted to do it like, just a digital detox and just not. I did not wanna. I didn't wanna open my computer once all day. Did have to open my phone a few times, like, just to figure out where I was going when I was driving around and things like that. But, uh, yeah, it was really nice to not open my computer for a day.

Caroline Health: I saw some of our panelists at, uh, Legal Tech Talk. Horace. I don't know if Horace is on with this. I haven't got the. That open, but it was really nice. That's like, you know, it was really sweet to see what a community, you know, the impact of the community. And a lot of people are like, oh, hey, you know, I'm a big sort of. They join the panel all the time. It was really nice.

Host: Yeah, this is what, the second Legal Tech Talk. The second time they did this, or was this the first one? I lost her.

Caroline Health: Yeah. Amazing. Okay, I'll shut up about it until it's my turn. But I just wanted to say, because I saw Horace and obviously Horace.

Doug Austin: Um.

Host: Well, tell us about it. You can. You can do that now. Why don't you tell us about it?

Caroline Health: I haven't really. Shut up about it. Do you want to talk about Legal Tech talk, Caroline, or you didn't. Didn't sort of show anything. Yes. So thanks. So, yeah, so I, I, um. I've been there for the last couple of days, um, and it was great. So five and a half thousand people. This is the third year, so they've grown again and they're saying that they're going to grow further next year to about 7,000. I think they've just announced in the last couple of days they're launching in the usa, so they're launching. They're having a Miami conference in December. Um, so that came out this week. Um, and so who did a lot. Michelle did a lot of work on Legal Tech Talk. Yeah, it's massive, Michelle. So, um, it's interesting. So a lot of people absolutely love it. It's vibrant. I hosted one of the stages, um, and so was kind of confined to one room for a lot of yesterday, which I really enjoyed. But obviously, you know, you're in one place hosting. And then I hosted a panel at the end of the day, which, um, was talking about partly taking pilots and making them stick with people like, Hell Desantis from Bird and Bird, who you'll know. And, um, the CIO of Quatchicas and someone from Paul Weiss and, um, someone from. From a Croatian firm. It was really good content, I think. Um, and in terms of just the general busyness, like, so there's still this sort of sense of who's. Who's there, who's the audience. It's really busy, you know. Um, but who, Who. Who is there? Is it really beneficial? There's different feedback on that I get from different people. Uh, but one of the funny things which m. Blew my mind, to be frank with you, is Lagora built a pub, so it's near the O2. It's hosted at the Intercontinental. And I went to the Red lion, um, on. At the invite of. Well, actually, I think I gate crashed. A gate crash. But, um. I know.

Host: No, I'm shocked.

Caroline Health: Um, and, um, it's. It's been built. It was built. It looks like a real pub. It's called the Red lion. But the Red lion sign has got Ligora on it. Inside it looks like an English pub, but all the mirrors have got Ligora on it. The landlady is an actress. She's like a good old Eastern. All right, darling.

Host: Coming in.

Caroline Health: And then pint glasses all had Lagora on them. Uh, and there were photos. All the photos around the pub, like you get in a normal pub were law firms. Um, I'm not kidding you. The attention to detail was just. And it was just, I mean, two things. One, amazing, great marketing, and two, I mean, the money. You know, we being frank, we talk about this a lot. Right. I don't know what that cost, but that's a lot of money. So obviously that's something else that people were talking about in this continuing vein of how much is everyone spending on this stuff? But it was super fun.

Host: And you get to hang around with Jude Law.

Caroline Health: I didn't, uh, that's the only reason I went. I was like, hey, Jude,

Host: good singing. Uh, I mean, that is, it is kind of obscene. I mean, these companies just have so much money that they don't know what to do with it. I mean, it's cute and it's funny, but like, I don't know, I don't know what just feels upset.

Steve Empry: What's the vendor that brought the. The chef in the past couple of conferences? Professional chef somewhere. Not, not quite as to that degree, but

Joe Patrice: I believe Case Point had a chef.

Steve Empry: Yeah.

Joe Patrice: Right.

Steve Empry: Case. Yeah, it's Case Point. Yeah, yeah. Uh, it's pretty good.

Host: I was at some conference where Kelly, Kelly Twigger brought in her brother who's a professional chef. But that was more for, like to cook dinner that was for a legitimate purpose.

Caroline Health: Yeah, it was all a bit, it was pretty overwhelming. Um, and the costs, you know, respectively. Bradley, like Bradley, I haven't written anything up yet, so I'll do that on Monday because that's the other thing. Like, I've been done nothing but go to conferences. And I haven't actually done anything of my day job. I think I've forgotten what I do for a living that actually produce content.

Joe Patrice: Look, you know, I, I to take this in a different direction, but, you know, we're all writers here. Uh, I, you know, I resist AI uh, and the idea that AI can create good content because it spits out drafts and then I end up having to take an hour to fix them, which is probably better than the two hours it would have taken me to start from scratch. But whatever. Uh, but it can't really do things on its own. But I really do think I need to start working out a way in which it can do conference coverage for me. Because I, I have notes that I need to find a way where it's not going to be good, but it just is passable so that I could get an article out and I could be like, look, this is, this is best I could do tonight. I will have more thoughts later.

Caroline Health: You're so right. Because otherwise nothing happens and it just sits and then ages later like, oh, I didn't really do anything with that because it's just like so much. And then the next load of news came in and then it just. Then it's too late.

Joe Patrice: I had a non legal tech conference I was at two weeks ago and I got my first write up of it out yesterday. Just. That's how it, how it goes.

Caroline Health: Yeah, I think we can see. I'd start. I use AI sometimes for process stuff like that, you know. But the trouble with that, and this makes me think about the data question that we all talk about with law firms. So sometimes the notes are very understandable to me. But it would take, you know, obviously you have to get it in shape where actually uh, anyone could make any sense of it anyway. And then I sort of thought to myself, God, if I'm going to go through the exercise of getting it in good shape so that I can do something with AI, I might as well just pretty much get it in good shape and then write it. It feels like it would, you know, just what would be the quickest? Like I don't know.

Host: Well, you can, I mean I have gotten to the point of using my Fireflies whenever I have an interview. I'm using Fireflies to record the interview and get a transcript that way. Uh, because I've also. I don't know if this is like an age thing or the fact that I just don't write as much, but I just might. My handwritten notes have just gotten to be atrocious and even my type notes are just not as good as I used to be able to do. Uh, and maybe I've just gotten reliant on it, but uh, pretty much if I'm sitting down doing a one to one interview with somebody, I'm going to ask, I'll ask if it's okay. But they all say yes these days. And uh, turn on my Fireflies right on my iPhone and get a transcript. So that saves a lot of time right there.

Caroline Health: Because when you're in an in Person one rather than online. So you just have it on your phone.

Host: I just lay my phone down on the table and.

Doug Austin: Yeah, um, yeah, I do the same. I've been. I've just kind of at this point said, you know what? I'm going to record these things. I'm going to take some notes, but I'm going to throw it into notebook. Lm see what it generates. And usually if I'm gonna interview somebody, I'm going to say, hey, what. You know, here. Here it is. What do you think about it? They may give me a few edits and then I try to roll with it. And it's. It's actually pretty good. It does a pretty good job.

Caroline Health: So I'll start using that, I think.

Doug Austin: Yeah, yeah, it's pretty amazing.

Host: And.

Doug Austin: And what? And now I'm doing, like, anytime I do a webinar, uh, we'll have like a prep call. Record that. Prep call, Throw it. Throw it in and say, crank out some slides. It's. It's amazing at cranking up slides. Really.

Steve Empry: I use really good. I will usually take notes and use the AI tool, but then I will go back and check my notes against the. To make sure. And most of the time they're. They're pretty close. Um, and, you know, some of the. Some of the. There are. Because I'll miss a word or two or things like that. But.

Host: Or.

Steve Empry: Or I look at my notes and can't read the handwriting. That kind of helpful to have it.

Doug Austin: Yeah, that's a, That's a common problem for me is that it's just kind of like, what did I say here?

Joe Patrice: I've started. I've started using granola where you can do both. Where it's recording it and you're writing your own notes and then it takes the audio and it takes where and the times that you wrote your notes and like, thinks about all those and integrates them into a. It. It's pretty slick.

Caroline Health: Interesting.

Host: Um, it's comparing their transcript against your handwritten notes or something.

Joe Patrice: So, yeah, so like, it's. It's hearing the transcript and it's going like, oh, Speaker A, Speaker B. As all that's going. But I'll be writing things like, oh, at that point, you know, Stasha said this or that, whatever. And it will, when it puts out its final version of it, be like, all right, Speaker B was clearly Stasha because that's what Joe wrote when. But yet, like that sort of effect. It's really nice.

Caroline Health: Uh, Steve, I was thinking about when I Borrowed your notes when I managed to lose mine. And then you went, I've got half written notes, but you can.

Steve Empry: Nobody could read them.

Caroline Health: You're in the same briefing. And I somehow managed to delete a chunk of notes, which was extra clever. I haven't done that since. Anyway. And then I said, steve, and I have your notes. He's like,

Steve Empry: I was relying upon you.

Host: I've gotten to the point where if like, I'm on a Zoom demo or something and for some reason my Fireflies doesn't show up, like, I'll panic. Like, oh shit, I'm going to have to actually take notes now. And how do I do that again? Because I love it on a demo. I love it because you're so demo. I'm so distracted in the sense that I'm trying to pay attention to what they're saying and what they're read what's on the screen and also be taking notes. So when I can just eliminate that taking the notes part of it, it's. It's just great.

Joe Patrice: Yeah, sure.

Doug Austin: I, I used a recording app, uh, uh, Ilticon, a couple years ago, where I didn't realize it at the time, but if I got a call during the recording, it not only stopped the recording, but it lost the recording. And so I found out later that like three of my interviews were gone. And I was like, I had just had to apologize to folks and say, I am so sorry. This is what happened. So now it's like, uh, if I use it, I. I, uh, I put my phone in airplane mode so I can't get a call, I can't get interrupted. Uh, and that way it works.

Steve Empry: So I'll have to say that I've found though, if I don't take notes, when I take notes, if something's really important and will form m the central theme of something I'm going to write, it'll be in my notes. And if it's not in my notes, then, you know, it's like, it's not really that important right about. So it's, you know, I still scribble a lot of notes, but it changes

Doug Austin: the way I write notes because now I'm not trying to get every little thing. I'm trying to get certain key things, get them down and, you know, and try to listen and, and try to know that, okay, it's catching some of the details I'm gonna miss, but I'm gonna focus when I hear something really important. I'll get that.

Steve Empry: So, yeah, I'll put in brackets this is the theme.

Doug Austin: Right.

Steve Empry: But this is the point.

Doug Austin: Yeah, uh, yeah, absolutely.

Host: I've had just the opposite happen where I'll. I'll be thinking about. During an interview. I'll be thinking about, these are the key points. These are the important points. And then I'll look at the transcript and I'll say, wow, I totally missed that. Or something. There'll be something in the transcript that I just didn't pick up on while they were. While they were talking. It just kind of flew by me. But anyway, all of this happens to be a perfect segue to the story Doug wanted to talk about this week, which has to do with AI note takers of all things.

Doug Austin: Right. And, I mean, I think I saw somebody had their AI note taker on this. I've done web webinars.

Host: We mostly just get AI note takers attending now. We don't get actual humans coming anymore.

Doug Austin: There's note takers on just about every web meeting we have these days. So I've been looking for a case that hadn't involved an AI note taker. And so this case was one where basically it was a criminal defendant, uh, who was trying to, uh, uh, who basically, uh, had, uh, a motion to suppress evidence, including the interview, uh, you know, that he did when he was a suspect with the detectives. And it came down to, you know, they provided to the judge, um, certain things and also provided the, um, AI generated transcript of the interview. Um, and the AI generated transcript at some point said, um, do I need to have an attorney here? But the judge also listened to the actual recording. And the recording said, uh, I'm sorry, I had it backwards. The, um, the recordings, uh, the transcript said, but I have to have an attorney here. And the, the actual recording said, do I need to have an attorney here? So it just was a little bit off, but just enough to change the tone of it. And the court, you know, basically the judge, um, uh, ultimately said, you know, your motion was denied, and so forth. So that's the first case I've seen where an AI transcript mistake came into play. And fortunately, the court didn't get fooled by it. But I can imagine just how many cases could eventually, uh, be turned by a transcript missing something that's actually said. So to me that this is going to be one of the growing sources of ESI in discovery. And it's going to be really, um. I know there's always, you know, transcript errors that can happen when humans do it too. But there's so many more transcripts now that I think it's going to be a real challenge to make sure that it's saying what you think it's saying. So, um, I thought it was an interesting case.

Steve Empry: Judge Schaegles tells the story about his wife who went to see her medical provider and the medical provider used a similar tool and it got it all wrong. I mean, she got the transcript and went wrong. I don't have that. But that's crazy, you know, so it's a real across the board danger for these things.

Caroline Health: Sounds like they're using a summary or were they using the transcript? You know, because it sounds like almost like, you know, I'd be sort of. The transcripts are usually pretty good. Like, I wonder if it was like the summary where it kind of had a bit of license rather than the actual verbatim transcript. Does that make sense?

Doug Austin: It did the. I don't think the, I don't recall that the, uh, ruling made it specific, but it, it, it didn't say summary. It said AI Generated transcript. So I think it was the entire transcript. And honestly, I mean, when I've done meetings and I've gone and ahead and said, give me the transcript, and I look through it at times I'm like, okay, that's not what we said. So, I mean, I catch little things all the time. And this was a little, this was a little thing. That's a big thing in terms of just exactly how the wording is.

Caroline Health: So you know what, you're right, actually, because especially if someone's got an accent or you know, that it's better with some people's accents than others. Uh, selling with a vendor who. I looked at the transcript and according to the transcript, we were talking for the entire time about dolphins. I don't really remember what the word dolphin I must have worked out at the time. But, um.

Joe Patrice: But yeah, were you in that, Were you in the Lagora Pub when you were doing this? Maybe you were talking about dolphins. I don't know how long you were in there.

Doug Austin: Who would want to talk about dolphins?

Caroline Health: I think with the accents, you know. And you're right, actually. I mean, that is a m. Quite a small point. That's a big point. So.

Doug Austin: Right. And yeah. Ah, to me, as many of these note take, you know, meetings are being transcribed and what have you, there's bound to be more cases like this. I'm sure there can be, uh, several cases too. Uh, so I just kind of had f. Found that one. I was like, well, that's interesting. Now, this is the first one I've seen Where, you know, it was an actual mistake and fortunately was caught. But it's, I, I pretty sure, pretty soon there will be a case where it won't be caught.

Joe Patrice: See, I'm, I, I took it very much the other way, I think. I mean I, it that the tech wise I took that way. Uh, but stepping back, I thought this is an indication of exactly how broken our, uh, our criminal procedure has gotten. That clearly, clearly he was trying to find out if he needed a lawyer. Uh, he was doing what the spirit of the Miranda decision said. Said. Uh, but the courts have gotten so, so, uh, uh, um, they have such animosity towards the idea of people having lawyers, uh, that they, it reminded me of. Does anybody remember, like it's like seven, eight years ago there was a big case where a defendant asked, he was like, hey, you know, if this is what's going on, I need, I need my lawyer dog. But because we're not going to talk about this is what the person said. And the courts were like, he didn't invoke a need for a lawyer because there's no such thing as a lawyer dog. Uh, and it's clear that's not what he was saying, but that's what this reminded me of. It was one of those situations where it was under the law as we have it, a really critical semantic mistake that changes things. But it also was almost like a gut check of, uh, maybe that shouldn't be a semantic change mistake. Uh, that changes things. Maybe the AI is, uh, because it's not steeped in the nonsense of our legal system. Uh, it's the common sense answer, baby. Like the guy, do I need a lawyer here? That's a person saying, I think I need a lawyer. But like I'm, yeah, I don't know.

Doug Austin: Right, yeah.

Host: Or they're saying I need a comma here. Yeah, right. I think there's a little comma between lawyer and dog.

Doug Austin: Right.

Caroline Health: About having a recording. I mean, you've got to have the recording. I feel like this one is one of these areas where if you've got the recording, which presumably they will, they should be stupid to delete it. Then it's one of those things. It's going to be the small points where you can then go back to that part of the recording and check like the judge did. I feel like, you know, that's hopefully going to be something that's not too hard to resolve.

Host: This is where all the court reporters in the audience are jumping up and down saying, see, this is why you can't get rid of us. This is why you still need reliable court reporters. But I'm wondering whether anybody's actually ever done a study of the relative accuracy of a human court reporter versus an AI Transcription tool. Um, I've never heard of such a thing, but it might be interesting to do.

Doug Austin: I bet there's been something, um. Because, I mean, you know, obviously humans make mistakes, too, but, uh, to me, I think the difference is there's just so. The AI Transcripts are just so voluminous now. They're everywhere. Um, that I think that's one of the challenges. Oh, we just. Just, uh, kept. Kept m. Them from scoring, so, uh, still got an eye on the game. Um, but, yeah, I mean, that's. I think it's, you know, if you're an organization and you're having all these meetings that are being transcribed and then you've got, you know, a case, you got to figure out if any of that's potentially responsive, possibly. So that's going to be. It's going to be, uh. It's just going to be a major challenge.

Steve Empry: Yeah.

Host: I mean, if you're not going to give us your full attention here, I think maybe you ought to just step

Doug Austin: away, you know, it's all right. I'll only look every, you know, 60 seconds.

Steve Empry: But, you know, to your point, Bob, whereas before, yeah, I mean, there would be mistakes in transcripts, but most of the time, you know, the lawyer would read the transcript and. And gloss over the irrelevant points and the relevant points. If there was a mistake, it would be caught. And so. But now a lot of those deposition summaries and reviews are being done by AI and it's might not be caught.

Joe Patrice: Yeah.

Host: Yeah. Is it still a thing that lawyers read over the transcripts and make corrections on them? They used to do that.

Steve Empry: Yeah, I used to. I don't know.

Caroline Health: Now it's to do it right.

Steve Empry: You would do those errata, uh, sheets, and you would have them signed and send them back and forth and argue about them and.

Host: Yeah, well, in some cases, uh, AI isn't necessarily making a mistake so much as showing its sexist leanings. And, Joe, you've got an interesting story on that this week.

Joe Patrice: Yeah. Um, and at this point, Caroline, I'm really glad you were able to join, because if this show was just the four of us, as I explain how sexism works, it wouldn't have been as

Host: good as we're all thumping our chest and drinking beer.

Joe Patrice: Right. So this one came up just, uh, by happenstance, there was a viral, uh, Social media post earlier in the week of a woman who said that she wanted to update her resume. And like, my assumption is a lot of people, because a lot of people hate updating the resumes. She turned to AI and just put it into, she used Google Gemini, put her existing resume in and said, hey, add some stuff about this other stuff that's happened since then and what it spit, it spit out a resume. And then she had the idea, what if I changed my name to a male name and told it to do the same thing and it spit out a much better resume to the extent that little things like tasks that were, that were described, that she had assisted on, he had collaborated or led, uh, stuff like that, like the AI just guessed, uh, more important roles, more engagement all around with the male named resume, even though it's the exact same material, underlying material. Uh, so I saw this and obviously we talked about AI bias before and all the work that goes into that, I just put all that aside. I was like, oh, I wonder if this would work in how legal would handle this. So I created a fake, uh, life experience of a big law associate which I put together by using an AI to amalgamate eight different real associates who I picked largely at random. Well, not at random to the extent that I limited it to big law offices in New York corporate practices, but like grab one from Clery, grab one from Caveat, like all that and put them all together. Four men, four women put their experiences together, said based on this, create a uh, fictional version of these, uh, of these folks. Then I used Google Gemini not to pick on them, but since it was from the original experiment and I created a male version and female version of that to see what differences there were. Uh, and the good news is it wasn't nearly as bad as the original poster's situation was. I uh, think I venture to guess and this is just pure hypothesis. I think legal resumes are, and the nature of legal work is so homogenized that you know, represented client, client name in this deal, blah, blah, blah. That language is so ubiquitous in our resumes that maybe the LLM, you know, it's always predicting the next most likely word. And I think we, we may tend to use words that are common more than other professions do. So on that level, uh, it was better that said it found a way, uh, like just a few places, uh, where the guy that was described as highly motivated and with extensive experience just thrown into the discussion, her resume didn't have any of that, uh, all of her pro bono work was Just left off as unimportant. His was. Was put in, explained why he had taken a role in it. Uh, one of the ones that I thought was interesting that just from my perspective as somebody who used to interview folks, uh, in the education part, both of them were a semifinalist in Moot Court, uh, by put, uh, as one of their things. And for him, it was listed as one of his honors that he was semifinalist nationally at blah blah blah. And for her it was listed as an actor activity, Moot Court, blah, blah, blah. Uh, and as somebody who interviewed folks, I, you know, you look at that resume trying to find something to talk about, and you go to those sorts of areas and your eye goes to that honors section where there's like a, ooh, semi finalist, blah, blah. That's something that generates conversations in ways that, oh, you are a member of the Moot Court team. Probably didn't, uh, you know, like, I don't know, uh, I thought there were little subtle areas where there were mistakes. And to be fair, there were a few places where she got the better of it. She, uh, was described as the primary author on one thing, whereas he was just listed as being on the team. But by and large, yeah, it's still sexist. It's not as bad as the viral post, but it wasn't great.

Host: I don't know if, um, you read Joe's story and Joe pointed to, uh, the thread on X, the woman who did the original comparison. I thought what was really fascinating is she then kind of called Gemini to task, like pointing out the bias and asking it to explain itself. Um, and she's got all these screen captures of the way it responded. And it said things like, my training sees a woman's volunteer work and categorizes it as community service. It sees a man's and calls it board leadership. It says, I have a waiting problem, where I tend to use words like passionate, dedicated and helpful for women, and forensic, architectural and decisive for men. And it goes, I just got a ton of screenshots. It's like, just shocking.

Joe Patrice: Yeah, it was wet.

Caroline Health: I've done studies because I've done some sort of female leadership stuff and helping, you know, sort of around this and the language. It's. What it's doing is it's surfacing the existing bias. Right. So I, uh, A meeting years ago from the head of diversity at one of the big, really big international firms, and she showed us a word cloud, and she didn't tell us what it was. And she was like, just try and think you know, is there anything that springs to mind what this might be? And there was a word cloud that was like perfect and to neat and you know, and then there was like brave, Bold and you can. And it was, it was the workload of all the advertising that women and men were subjected to. And obviously this is a, ah. These are all these subliminal things which are continuously perpetuated but we aren't necessarily aware of them. But obviously the AI presumably is picking up on them and you know, no small measure. And I think, I think, um, didn't they have like. They had. There's been a few instances of people having to ditch AI recruitment tools because they're inherently biased. Like, um, I was trying to find it, but I think it was. There was Amazon or something that was you trying to use AI to select people, but it was selecting men. Like, I don't know, I'm laughing. It's not funny. But you know,

Host: that may explain the story I did this week on why all the new, all the new CIO CEOs and legal tech. Geez, they're all men. Maybe they're all the legal tech companies are using AI hiring tools. I don't know.

Steve Empry: Oh, wait, did you write that story, Bob, or did somebody else write this?

Host: Somebody else did a better version of it. That was actually the funny. I think I pointed out that, um, Jesse from Platform is the one. What we're talking about is a story of mine. A couple of stories about it get ripped off by some pseudo publication called the London Independent. What did I say it was? London Observer. I forget.

Caroline Health: London Insider.

Host: London Insider. But the person who pointed out to me is Jesse from Platform VR, who said somebody pointed it out to her, saying, here's this really insightful article about this trend of these new CEOs in legal tech. And uh, Jesse said she was reading and saying, I swear I just read that somewhere, uh, and then remembered my article.

Caroline Health: You should tell us, uh, tell us about the story. Is that the one that you were going to talk about this week?

Host: That is the one I was going to talk about. I was going to last week, but we didn't do a show last week. Um, where was I? I was somewhere last week. Oh, I know. I was at the AAA conference. Speaking of God, there's so many conferences now. Um, but no, but. So, yeah, I thought it was interesting that, that there was this sort of coincidence of, uh, four legal tech companies announcing new chief executive officers in the span of sort of in the span of a week. Although one of them was Actually announced a couple weeks earlier, but wasn't really made, wasn't really publicized much. Uh, and then also another one of Clio naming a new president at the company. Uh, and so there was the. The uh, 8:00am um, the, uh, the company, uh, uh, that our usual panelist, Nikki, uh, works for and. But. Which by the way, my son works for. Uh, but, you know, Drew Armstrong, uh, left the company as CEO and they brought in a new. A new person. Uh, Jeff Hughes had been a board member, uh, uh, for a number of years as CEO. There uh, were Casepoint, ah, announced a new CEO there, uh, uh, after having been acquired by Thoma Bravo, uh, what, almost two years ago, 18 months ago, uh, and kind of going with some interim people in that role, uh, for a couple of years. As a matter of fact, wasn't, uh, it. The, uh, former Alm, uh, CEO was acting CEO at casepoint for a while.

Doug Austin: Uh, right.

Host: And then there was, uh, Steno, uh, uh, promoted its coo, who they had hired a couple years ago into the CEO role. And the. The co founder of the company, Greg Hogg, who had been the CEO, uh, moves out of that role and just takes a board seat. Uh, and then, uh, there was Integrity announced a new CEO, Krishna Nacha there. And so I guess in part I was just kind of conjecturing on the idea, is there something going on here, uh, that's kind of causing this, uh. Oh, and the Clio, uh, was the uh, Ronnie Gurion, who's already been a coo, has been promoted to be also president and coo, but, uh, Jack Newton remains CEO. There a lot of Alphabet soup and all of that. Um, but, you know, I think maybe. Maybe there's sort of something going on here in the idea that, uh. I think a lot of these companies, um, suggest that, um, we're kind of in a new phase of uh. We're kind of going from the startup focus, uh, product focus, early sales focus, uh, to much, uh, more sophisticated companies, uh, that are, uh, trying to drive this sort of AI transition, uh, for their organizations and for their customers. And uh, in a number of cases, I think that's calling for new leadership. Um, so I think that's. That's one side of it. The other side of it though, is just what I was just alluding to, which is, uh, you know, regrettably, out of all these five appointments we're talking about, not a single one of them was a woman. Uh, and uh, you know, Drew Armstrong left. Left 8:00am I mean, Drew had been a really, you know, Dynamic and invisible. Uh, leader of a legal tech company. Um, somebody I really, you know, I thought was very thoughtful and enjoyed talking to her. My podcast, a couple of different times. Uh, and, you know, sorry to see her go. She did. She did go through a bout of breast cancer and had to kind of slow down for a bit, but. But from everything I've. I've heard she never really stopped working during that time. And supposedly she's now in full remission and recovered. And, uh. Uh, so maybe she just wanted to go smell the roses a little bit. I don't know. But, um. Interesting to see this.

Doug Austin: Yeah, there's been another one since your article. LVM Global announced a new CEO too. Um, and another man. So. Yeah.

Host: When was that? Just this week. I didn't see that.

Doug Austin: Uh, yeah, yesterday or the day before. I think it was, uh, Jamie Berry, um, a new CEO at LDM Global. So. So, yeah, uh, this can. It continues. One of the things you pointed out, private equity being a major driver. Uh, yeah, I definitely, definitely. See, I experienced that, you know, when I was a Cloud nine. So, uh, definitely, I think, um, that's. That's. That's happening more and more as these companies have the private equity, uh, investment, and they'll bring in somebody new when they feel like there's the need, so. Not surprised.

Caroline Health: There's a real tech bro thing, isn't there? You know, not just legal, but unfortunately it's very much the wider tech industry.

Doug Austin: Yeah, I, I absolutely think there is. Yeah.

Steve Empry: No doubt, no question, no question about that.

Caroline Health: Yeah. And that's the vc, you know, that is feeding in from the VC world. I'd suggest, um, I actually wrote. It's funny because I hadn't. I hadn't really thought about it, Bob. I just. I meant to post you on first, but I post mine because I only posted about three. Uh, and you're right. I've got the picture of the three guys all next to each other. I mean, it. It isn't unusual to see.

Host: I just stole your article.

Caroline Health: I. Yeah, I. You are.

Steve Empry: Bob just ran your article through AI.

Caroline Health: It looked like I hadn't copied it. I was just trying to read those and then make it look new. Um, but, I mean, yeah, it's very unfortunate.

Host: Yeah.

Caroline Health: And you're right. Imagine you have two stories about, like, about not enough women and not. Not have a woman on the panel. But, um, I mean, I don't know what we do about it. Like. So I think, as we know, you know. So I was thinking about re bringing back my the say gap, which is teaching women to speak up in public. And I'm going to reinvigorate that and do it perhaps as a not, not for profit or something. But the issue is time, of course. Um, but um, you know, there's obviously such a pushback, right? There's been. One of the reasons we stopped doing that was because there was a big pushback from firms on DE&I, which we've covered before.

Doug Austin: Right.

Caroline Health: So post all the political situation that we know about, um, we, there was a big pushback. People didn't want to put any money into dni. It's actually they're being sued for discrimination if they have any kind of program to help women. So I actually think that it's just going to get worse for a while until we have some kind of change in climate.

Host: Because the fear, I mean we, you know, I've alluded to Kristen Sunday. She did it a couple of times, I think back, you know, a decade ago or something. We should look back. Numbers of women and people of color in leadership and founding roles at legal tech companies and the numbers were pretty dismal. And now we've come a long way, baby, as the cigarette ad used to say. Uh, and we have. I mean there is certainly far greater diversity, uh, among the founders, ranks among the C suite ranks across the legal tech industry. But uh, we are certainly living in a climate in which diversity is being, you know, not just de emphasized but denigrated in some corners. Right. And so I don't know what, you know, how, how does that play out? It's, it's a, it's a scary thought.

Doug Austin: Many of those, what percentage of those are like. Some of the um, leaders that are women or minorities are in companies where they're self started. They're the ones who started the company and basically uh, uh, became. Were the leader from the get go as opposed to a company hiring or appointing a woman or a minority as leader. I, I have a feeling that the numbers are pretty a lot lower in those situations than they are with, you know, the people who've started their own companies that happen to be women, minorities and made a success out of them. Because I think there's plenty of examples of those. I'm not sure there's as many examples of people getting uh, appointed to those positions.

Steve Empry: Yeah, yeah, I think you're right about that. Uh, absolutely.

Host: All right, well, what, what would be a week of this show? I'm sorry, you want to say something else, Steve? No, I was just going to queue it up for Your story. So I was going to say what would be a week of the story without at least some story about lawyers hallucinating and their briefs there.

Steve Empry: Yeah, this is um, this is a really interesting uh, case not only because uh, of the hallucinations by every lawyer in the case or failure to verify citations by every lawyer case. But pardon my language, but it's a really great example of why you don't with a federal district court judge. So, so what happened here is uh, this was in front of Judge Acock down in Mississippi, who's a very good

Host: judge unless it's in their chambers. I think Joe covered that story previously.

Doug Austin: Right.

Steve Empry: Oh, I'll never recover from that one.

Joe Patrice: I was preparing to make a joke about that.

Doug Austin: So.

Joe Patrice: Yeah, good job. Okay, good job.

Steve Empry: So the two two party case, uh, plaintiff had an out of state lawyer admitted pro hawk and local counsel. Defendant had an out of state lawyer and local counsel out uh, of state lawyer admitted pro hoc and local counsel. The out of state lawyers prepared the pleadings with fictitious citations. The local council didn't read them, didn't read the pleadings, didn't check the cases and the judge figured it out, which is never good. Uh, and so he used the. She used rule 11, inherent power of the court and the local rules to sort of punish. The judge disqualified, all of the lawyers revoke. Ah, the prohawks find them. Um, but the interesting thing was what the out of state lawyers said in response to the first one said, um, well, I only used, I've only used AI for about six months and I really didn't know that it would hallucinate and make up fictitious cases. And Judge Acock said, I'm going to call BS on that one. Sorry, didn't work. Uh, and one of the reasons it didn't work is in the process. This lawyer admitted that she had filed other pleadings in the same case with uh, fictitious cases in them that were not verified. Kind of, kind of strike two. And then the judge on her own did some investigation and discovered that the same lawyer had filed pleadings after the fact that she was discovered to have filed fictitious cases. In this case, she had gone out and filed unverified fictitious cases and pleadings in other cases in other jurisdictions. Uh, and uh, you know, that sort of stud said yeah, uh, sorry and, and then remarked that, you know, it's also kind of bad that you didn't. You let your local council take the fall for all of this and didn't really own up to it. You made your local council went up to It. Then the other out of state lawyer, she said, well, you know, uh, I'm a partner in my firm and we had a, a policy that required all of our citations to be verified. Uh, but I didn't follow it. Judge said, that's not good. And then she said, well, you know, I tool an AI ah, tool that, that um, would look up cases in Texas and Oklahoma, but not in Mississippi. And the judge said, say what? Said what did you do now? And she goes no, no, no, no, no, that's not what I meant. I, I meant that the tool would look up regional cases that included the region in which Mississippi, uh, was, uh, was involved. This judge was like, really? I mean that's, that's what you're doing, what you're saying. And then, um, then she said well, um, I'll tell you what, the proposition I cited the case for is a valid proposition. It's just not. The case didn't stand for that. Sort of like no harm, no foul. Uh, and so, you know, the judge said, I'm not buying that one either. And then to top it all off, the judge called all the lawyers in for a show cause hearing. And this particular out of state lawyer called up and said, I have a scheduling conflict to have a hearing in another matter at the same time that you've set this one. Now, um, Judge Acock by this point was probably pretty suspicious. So she, she stuck to check the docket in the other case and discovered that the lawyer did not have a scheduling conflict. So um, you put all that together and it's a pretty bleak picture. Picture. I mean it's kind of, I want to say it's humorous in a way, except for the fact that I can't believe that these lawyers would do this. Um, the local council for both parties on the other side, I mean they, they came in, they owned up to it and they said, yeah, we, we violated Rule 11 and we didn't read the pleadings. We, we don't use AI but we should have read them. And they got off, you know, um, a, ah, lot easier. They weren't found to be in bad faith. They were, but they were disqualified from the case. And so it's kind of a, kind of an interesting situation. And the judge remarked on this and by the way, the, the opinion is a very well reasoned and well documented opinion. It's got the law of most, most of these cases that have been decided in it. So we really want a source for all the, all of the law that's been decided. Judge Acock's, opinion is, is pretty, uh, pretty comprehensive and, but the judge did remark that, you know, she realized that this could put the, the clients at a disadvantage because all of their lawyers are disqualified. But on the other hand, recognize that, you know, we just can't let this kind of stuff go on and there's going to be penalties and this is, this is one of them. I just thought it was a, it, it kind of stood out for me because it was not just your typical sort of. Okay, yeah, you, you didn't read the pleadings, you didn't read this, you didn't read the sides. Okay, here's your fun. Good luck. But I mean there was just a lot more to this and it really sort of painted a very dismal picture of the lawyers and the ends to which they would try to go when they should have just said sorry, I mean we messed up. You know, it's, it's. Wait, it happens.

Caroline Health: Um, honesty though, doesn't it, it's that core.

Steve Empry: Yeah, it's not even.

Caroline Health: It's a tech story, but that is just basic honesty. And like they, they, she. Like they lied to a child.

Steve Empry: Yeah, yeah, yeah. I mean, and I was, you know, she, she, she found the uh, find the one out of state lawyer, I think three $500 and the other one, two $500. And I'm thinking a lot of judges I'd been before, it would have been a lot worse than that. I mean I'm, uh, probably the tongue lashing probably was much worse than that that they got. But I mean they, they were. And, and she did disqualify them from um, appearing in any other cases in her district. Um, I think for two years.

Doug Austin: Right, exactly. Yeah, yeah, yeah, that. I mean, yeah, the uh, you know, I was a little surprised that the two um, uh, the two local lawyers were actually were taken off the case too since their, their main thing was failing to review and catch um, the hallucinations and um, and owning up to the fact that they should have caught him. Um, but so I was still, I was surprised that still they were kicked off the case as well.

Steve Empry: So obviously, yeah, the judge used, the judge used a local rule which most many local rules have the same thing. If you're, you know, ah, in state counsel for somebody, that lawyer has been pro hocked, you are responsible for their conduct. Uh, and that was the sort of the tool that she used that you know. Yeah, it's a little harsh, but you're responsible and you know it does bring into question, you know, some of the Discussions we've had before where you are, where local council are put in kind of a difficult spot sometimes where they get these pleadings of time sensitive, they've got to be filed within two hours. You don't know who did them, you don't know who did the research. Uh, it's got, in this case was not like this but in some cases it's got an AmLaw50 or AML100 firm is lead counsel on it and you wonder whether to question it or whether you can get paid to question it. And so yeah, there's a lot of, a lot of interesting issues that float out of this. But um, man, this is pretty bad facts.

Caroline Health: They're starting to make an example of people, aren't they? Basically because you know, we're far enough down the track and they're starting, we're starting to see the consequences in a lot of cases become a lot more severe. As you know, they, they've got zero tolerance for it now and it's a case of, you know, you should know by now and actually only by doing this, we're talking about it, uh, everybody's talking about it I guess that the embarrassment and the, and the knuckle wrap should have been enough for most people, but clearly hasn't been. Uh, but, but like you said, you know, we, we wrote a report this week talking about, there's been quite a few cases where like Vincent Masons referred themselves to the SRA because they made screw ups not only in the, in the court filing but then in the apology to the judge. They included, you'll have seen, you may have discussed it when I wasn't here, but so they included an apology, AI inaccuracies. And we've done some sort of studies recently about how people are becoming already quite, you know, just the human brain is becoming quite lazy and there are sort of changes in the way that we're interrogating information and our brains are switching off and we wrote this report, Neil Cameron, my colleague, about whether we're going to have to start to have AI instead of just talking about human in the loop. We're going to have to have AI in the loop to check the AI. Right, right. And actually whether that should be as a course of, as a course in terms of the process that you use to avoid this type of stuff, the dishonesty stuff aside, like actually whether you have to start using AI to check

Steve Empry: the AI like you raise an interesting point. Uh, Caroline, I was thinking you said it would embarrass these lawyers, you know When I was practicing, I ran across a few lawyers, like, would pull these kinds of stunts. And like, they weren't embarrassed. They just, like, maybe I'll get away with it next time. What the hell? I mean, it is kind of. Well, I don't think we've seen.

Doug Austin: We've seen cases for months where the. I mean, the sanctions have been really severe. And so I have a little trivia thing for you because I. Every week in my, uh, uh, kitchen sink, I do a count. Obviously, you all know about Damon Damien Charlatan's AI Hallucinations case database. Okay, so I reported it. I did my kitchen sink earlier today. 1624 cases, uh, um, in. In the database. Now, I started doing it a little over a year ago, June 13, 2025. Anybody want to guess how many cases were in it at that point?

Joe Patrice: 800 then.

Doug Austin: Any other guesses?

Host: 300.

Caroline Health: Josh. Josh? Joshua Lennon says 400. I'll go with.

Doug Austin: All right, Stephen, want to hazard a guess?

Steve Empry: I'll go with. I'll go with Josh Lennon's 400.

Doug Austin: All right. Well, if this were the prices, right, you'd all be, uh, over because we only had 105 cases a year ago. Now we have 16. Yeah, it's kind of one of those deals where it's, uh. I don't know if it's. They're reporting a lot more or the lawyers are getting used to using AI a lot more. I will say that, uh, the 1624. So still a majority of them are pro se parties doing it. So, um, uh, so, you know, there. It's not all lawyers, but it's still a decent percentage, uh, are lawyers. And by the way, I was wrong earlier on the oops note with 2 USA 2O. The offsides wasn't called, so it is USA 2. You know, for anyone who's wondering, it's

Steve Empry: one of the fall fallouts of, of, uh, AI and legal research, I think is, you know, you have to think, um, if I have to spend, uh. Like if I'm a senior partner and I have to spend so many hours reading all these cases that the associate is cited because the associate used AI, it almost becomes to the point where the AI is not saving you any time at all. In fact, it's costing you time, I mean, because it's going to be terribly expensive. But there's no real solution because you don't really know if the associate or the law clerk for a judge, for example, used AI or not. So even if you say don't use it because it's not. Because I can't read every single case. I don't have time. You're still probably going to have to. So it's a kind of. A. Kind of a weird situation.

Host: Yeah.

Joe Patrice: So this. This one scared me more because, uh, I remember a while back, we talked. I think we had a story where we talked about the situation where hallucinations made it up to an appeal because the trial level judge just accepted. Accepted the hallucinations, uh, without checking them. Uh, and at the appeal, you know that the other side, the adversarial system, did its job and pointed out this isn't right, and it got fixed. When both sides of the adversarial system are screwing up, then you're relying on the judge to do homework. One, uh, of the fallouts of the sex in chambers story we talked about a few weeks ago was what they didn't punish her for, by the way, they noted, but didn't punish her for, and I thought was probably more of an egregious, uh, violation of her duties. She only worked on the criminal docket and otherwise told her clerks just do everything on the civil. I don't really care. Uh, so she was abdicating all of that. According to the report, the idea that judges aren't caring what's going on and the adversarial system is. Is both screwing up. That's when, like, the whole thing breaks down. That's the. The process problem, I guess, that Caroline's talking about, too.

Host: Yeah.

Steve Empry: Yeah. And it's. You know, it's. In this situation. It was a federal district court judge, and I don't know this for a fact, but I would. I think there's a fair chance that the clerk. The judge's law court probably figured it out, but when you get down to the state court level, there's not that resources around to do that. And that's. Yeah, it makes it really bad.

Host: All right, well, we have to, uh, end our little pop. Pop up pub here on, uh, on Zoom for the day. And, uh, lucky, I've got my.

Caroline Health: I've got my AI Generated to do list.

Host: Bob, you've got your to do list right.

Caroline Health: Right up and publish a written talk about legal tech talk and the Lagora Lion Red lion pub by Monday.

Steve Empry: I hope they have. I hope they do something like that at the. You know, our. What's our next big conference? Get on that.

Joe Patrice: At Ilta.

Host: Double A, Double L, maybe That's not a big one. Okay.

Caroline Health: Yeah. And, uh, my stuff.

Host: Doug, I'm gonna have to get you back because I wanted to get your, uh, thoughts on the relativity acquisition of Gavel, but we're kind of out of time today. I thought that was an interesting one. A lot of people have asked me about that. It's kind of.

Doug Austin: I'll say. I'll just say this interesting direction. Uh, I'm not entirely surprised. They're looking for new directions to go, so, uh, yeah, it'll be interesting to see what happens. We could get into that a lot more, but, uh, I love if I

Caroline Health: can join you, Doug. So next week. I saw Phil Saunders and Dawna Moyni this week because it was Railfest in London, and I interviewed, um, them both. Um, and so, you know, perhaps can hopefully maybe join you and chip in on that one as well.

Joe Patrice: Emergency podcast.

Host: Emergency podcast. Or we can just do it next week. Yeah.

Doug Austin: All right.

Host: All right, well, thanks, everybody. Have a great weekend. All Good to see you.

Doug Austin: Bye, everybody. Thanks for having me.

Caroline Health: Thank you, guys.