*Special Episode* Featuring Caroline Hill, Editor-in-Chief of Legal IT Insider, on Charting Change in Legal Ep 57

[Music] [Laughs] Welcome to Charting Change in Legal! I'm Ari Kaplan, I'm an analyst who covers the legal industry, and I'm here with my friend. Caroline Hill, hello, nice to be here. I'm the editor-in-chief of Legal IT Insider.

I'm great to see you, Ari. How are you? It's great to see you. I'm well excited to have this conversation.

We find ourselves now on the sort of cusp of spring. I feel like we should be deep into spring, but you're here in the northeast, it's still cold. And I don't know what it's like in the UK, but on the northeast, the east and the east side of the United States is still kind of chilly. But great to see you.

I like this, I like this the start of our podcast. We can have a little weather update. And in the UK, we have that. [Laughter] That's right, for a while, it's moving.

That looks like on the video. I have to say, I am really excited about where many of the conversations are going on the legal tech front. I feel like we are now in a moment where people, and I'm lucky to get to speak to people all over the world, lots in the US, but people have a really strong foundational knowledge of lots of things, not just artificial intelligence, but just lots of areas where they are using their tools, where they are evolving their processes. I think that they are getting a lot of benefit from all the different events that we are all lucky to attend.

And the conversations have elevated in terms of their substance. I feel like the, a lot of the stuff that we do is a continuation of the same. And an evolution, as you said, people doing things are hopefully better. But what's really fascinating over the past, a week, two weeks, a bit longer, has been, I feel like there's a bit of a seed change in this AI, in the AI first law firm space.

I don't know if you've been tracking that, but whereas a lot of the stuff I wrote about as you completely correctly identified as conversations, we've been having for a long time, and are they doing things differently? Are they doing better? But there's been quite a lot of focus in the last two weeks on, there's a new AI native law firm index, which I wrote about, which has been created by the co-founder of Loop or Matt Pollins, talking about what, you know, 40 firms are on there now, so it had a chat with Matt about, you know, how do you define it an AI first firm?

And there's obviously a lot of focus on things like being free from the Bill of Allow, which you and I have talked about for donkeys years, but finally I'm like, oh, this feels kind of fun and new. You know, they're free from a lot of the constraints we've talked for years about, can people ditch the Bill of Allow, and there's a lot of firms now that are just being formed without the constraints. And then very recently, there's a new company called Manifest OS, who's just raised a 60 million series A to help create these AI native law firms.

And they, you know, you and I, we've been doing this a long time, you don't want to overplay, and there's a lot of natural and quite right cynicism in the industry about what it is and what it is. But I think this is, it's kind of exciting, you know, to see these, these new types of firms growing, and there's a lot of money behind them. It's very frothy, like you and I have already said, and there's risks in that, but it's exciting. I was lucky to interview Sanjay Kamlani, who you may know is the co-founder of PennGa3 with David Perla, and he started a firm called Fairplay, and the title of my podcast episode is Understanding the Emergence of AI First Law Firms, which I posted in mid February.

So you're absolutely right. I think there's like a lot of interest. There's a lot of enthusiasm for this kind of approach, and not surprising to me that someone as experienced and sort of transformative in terms of a figure in legal, like Sanjay. Like Sanjay would be involved because he was involved at the early iteration of changing legal with PennGa3.

So you get a lot of visionaries who are doing this. It's very cool. Very cool. And actually, you know, one of the things I've been trying to remind myself of, you mentioning PennGa, you know, it's really important to remember that innovation has been happening for a long time, and you've had the elevates and the axioms.

And you know, not every single thing is new, and also what exactly is AI native? And does it massively different, different to those types of companies? And you know, that we've had before that are basically doing things in a tech first way without the constraints that bringing together legal services and technology, without the constraints of the bill of allower. And actually, you quite rightly point out, you know, there's been some amazing companies.

PennGa3 was absolutely so ahead of its time, elevate, axiom. There are other stellar legal that has just come together with its parent company, which provides legal services. Does it matter that it's AI first? I think, you know, I'm not sure.

So sure about that. Actually, I mean, it is different, but this sort of concept of these, this whole swath of new firms coming to the fall is definitely exciting. You're also seeing, I mean, I feel like you're also seeing much better case studies. So we've had lots of conversations about use cases, but I was at Patrick D.

Domenico's legal research conference yesterday through his legal tech connect series. And there were several speakers who talked about rating different systems, comparing different applications and trying to gauge their response and incorporating feedback and then iterating on the deployment of certain tools. It was just great to hear and see how firms are measuring. You know, I mentioned that I had done this report on ROI of legal AI with LaGora.

And, you know, the objective was to just to kind of spark a conversation. The beginning, not the end, not to conclude, but to serve as a catalyst for us all to have a conversation. And it was great to see yesterday that there were firms who are thinking about this in a very methodical way. And, you know, and of course, it always comes back to the people and knowledge management and innovation and library science who are driving that conversation.

They're the sort of both the ambassador for technology to legal, but also the kind of the technicians and they kind of speak many languages at once, which is really benefiting their firms. Yeah, it's been great to see that role coming more and more to the fore. I'm still there's been some more AI chief, you know, chief AI officer appointments. I'm still curious as to how that fits in with knowledge and, you know, hoping that they work collaboratively.

And, you know, because I think that there's the knowledge officer has is the natural AI officer as well to certainly send. I think what's been so interesting on the tech front as well as this big tech. There's been a lot of emphasis on big tech now in legal tech. So anthropics been really busy.

And, you know, they've rolled out this word, you've now got native functionality with them words. You can red line with them word using anthropic Microsoft, doing similar stuff with code pilot. And apparently there was a Claude, a Claude webinar, which I missed, but I got the link. I need to watch it.

And I was talking about how they're working with legal and apparently they had over 20,000 people join, which I was on that webinar. I was on that webinar. I did not see a number of how many people were on. I've heard that number of 20,000 repeated many times.

And we're definitely a lot of people. And I know that because people would put emojis or responses or reactions to a comment or a question. And there were at a minimum hundreds of those in ways I had not seen before in a program. I don't know the exact number, but I have no reason to dispute.

And so, you know, no purpose of disputing it, but I will say that I am very, very lucky next week on Tuesday to be the MC of an event that Lexus nexus is producing. It's an AI symposium. And the keynote is the general counsel of anthropic. And I have the privilege of interviewing him in a fireside chat.

And so I'm looking forward to learning from him directly because he's obviously at the center of this conversation. And he is driving lots of these discussions. It's just, it's, it's, it's very exciting. It's exciting.

And it's, you know, I've said before we've had big tech. Microsoft has been a feature of our lives. SharePoint has been a feature of our lives. This big tech, the legal tech has been a feature of our lives for many, many, many years.

But this, this again, you know, this is kind of exciting. So it's never really worked as well. It's never been part of a lawyer's workflow to the same extent. There's been lots of reasons why you would choose something which is dedicated to the legal sector.

But, and you know, these big players, so I can't wait to hear from you how the conversation goes. You know, legal is such a small vertical for them. But I feel like, you know, people are really looking at this now and saying, you know, in terms of the level of the functionality, all of the, the billions that get they get to invest in the technology that powers it. There's, there's just, there's a real seat.

And fresh wheels as well. They're working directly with Google. I spoke to their head of innovation, girl Perez about the work they're doing with Google and it gets again. You know, fresh wheels is looking at big tech, not just, they've also just signed a deal with anthropic to roll out Claude across the firm and to, so they're using Google's AI Gemini that using Claude.

They're putting that in the hands of their lawyers are sort of allowing them to within their own lab within the interest of the government secure way. Allowing them to put it in their hands and start to create some, some workflows and, you know, see what comes out of it. And it's fascinating. There is a certain responsibility now for each of us and particularly those of us who are most directly affected by the emergence of these tools.

And I specifically thinking about junior professionals or professionals that, you know, may have some of their work performed by these tools. And I think that the responsibility is on each of us. I've been talking about this for many years that the responsibility for upscaling the responsibility for business development, the responsibility for networking is our own. And you can't really outsource that or rely on somebody else to give you that education.

There are so many tools available. And I think that once you do that, you kind of get engaged by the adventure of it. Once you start using one of these tools and in a safe contained way without disclosing any information, obviously. But once you start doing, I think I think there's excitement because whenever I hear people talk about the results when I'm at some sort of public event or a breakfast, I hosted a breakfast last weekend, Dallas.

And whenever people start talking about it, they become very animated. It's a very enthusiastic discussion. Even if it goes wrong, you know, it's not unlike, you know, trying a recipe out. And you're like, oh, but I added too much paprika.

And it turned out to be X or Y. And so, but I think that there's this adventure of it and the discussion of it and the sort of the sharing in the mistakes and in the adventure and in the idea that we didn't anticipate this being a result or we didn't anticipate this being appealing. And we sort of leaned into it or we made an adjustment. And only through that, that sort of tactical and and experimental approach will really elevate.

But I still think that it's on us. Take that step avoiding or delaying is probably not your best tactic. I was debating whether I can say this on the sort of hitting on all of the points that you just made. There's a vibathon coming up.

I don't know if I can say like because they were trying to keep it to themselves. Maybe I should just be a little bit fuzzy. I'm not sure if it's out there, but. So everyone just Google vibathon and I have a feeling you will find whatever my co-host is secretly but not secretly talking about.

No, not to self do not tell Caroline any secrets. Okay, wait, wait, I'm taking that. This is not vibathon. It's something called.

People doing hackathons. Obviously the big thing. And now there's a vibathon of some description of some somewhere at some point and bringing together bringing together minds and on topics like what's the feature of training. I think for me that's super exciting to not just I mean, I suppose hackathons by their nature try to come up with solutions.

I just am not 100% clear on the difference between a hackathon and a vibathon to a certain extent. But anyway, it sounds cool and it's new. It's exciting that people are coming together and collaboratively coming up with trying to come up with solutions to some of these newer problems like how are we going to replicate training given the whole layer of work that's currently done to constitute training of some description. Not that I'm convinced that that's the best type of training at all.

But coming together in this collaborative way to come up with some solutions. Also just using different tools as a sparring partner or as an advisor of some kind or a mirror. And just to say, gosh, I have this list of I have a to do list that's not getting done. How should I start?

And I think people would be surprised. I mean, I'm sort of telling telling you this and telling it to myself at the exact same moment because I'm just thinking, gosh, I have these things. I just don't get checked off my list. How should I get started?

Where should I take the first step? And I always think when I'm fine myself. Be entering maybe wasting time. I can't get started on something.

I either get up and go somewhere, take a walk. So will start a timer because I keep a track of my time, but I will sometimes start a timer sometimes even make it tick like Pomodoro just so that it forces me back into this moment of, okay, I may not get 30 minutes of work done. But if I sit down and I get five, I can build five and do another five later, but I won't just keep wasting my day. So that's a really important component.

And I work for myself. I don't have the accountability of a bill of lower, but I do have the accountability of needing to eat. And so I have to earn money to eat and pay bills. And so I don't know if that's, you know, I don't get paid unless I do the work and get the work.

And I think that, you know, that's, that's important for all of us. But if you use these tools to say, hey, I'm sort of meandering, where am I? And then, what should I focus? Obviously not for client confidential information or anything like that, unless it's an approved tool within your organization, et cetera.

But I do think there's something there like a, like a, you know, people describe it as a second set of eyes or another opinion. And, you know, you take it, take that opinion for what it's worth. There is people are starting to realize the value beyond producing a sentence or an email. And, you know, that's, that's, that's going to be important.

It'll be interesting to see. And once we go through this first series of beginning of the year events where people are talking about this, what the conversation becomes toward the second half of the year, like Delta, for example. And where that conversation has, has moved, how it's developed, how hopefully it's expanded and become more sophisticated. Yeah, I can't wait.

I can't wait. I think, I think this, yeah, I totally agree. I found myself using it for continuously expanding way, you know, like, and actually what's really interesting about that is this whole partnering with AI concept. You know, that, that, we are moving, going to be moving into more personalized agenda, agente, perhaps, but definitely more personalized AI.

And I think this whole idea of you started to view it as a collaborative partner. Almost, that's how I feel a little bit. I use different tools. So it's not like I'm wedded to one, but, but this whole, I now, there's a massive difference to this time last year and the way that I'm using it.

And I kind of bounced ideas even, you know, or, or say, can you just check. Are there any, I mean, obviously it doesn't know substitute for reading things, but like, is there any typos in here or? You shouldn't have it completely do your assignment and go play that, or go to lunch, but that's not what you should do. That's what is no one, don't take me in, you just go do my work.

I'm off for lunch, have it done by the time I get by. Yeah, and people, I think people are also recognizing limitations, notwithstanding that we still see hallucinations and we still see those submitted. I have no opinion there. I just think there are tools.

You know, we're bringing up. Yeah, there are tools. I don't mention any firm, but there are tools out there that allow you to. Prevent that, and so it's just interesting to me that we're, we're still struggle there, but that said, I think that, you know, that hopefully is happening less often and people are understanding the limitations and where they need to add.

Personal responsibility. I think that's the interesting thing and not getting you to come up, but just for the being mature journalism, the Sullivan and Kroma recently in the press because they set, they fired emotion for. With hallucination material and and the response was interesting because it was very, it was a mature response. It was, it was a, why were they, what were they using?

Why were they not citations? How did they not manage to verify, you know, trust, but verify that approach? It was a, I felt like it was a much more mature response, you know, to perhaps some of the previous hysteria. We all know now what you need and how to prevent these sorts of things and and it was like people pouring over their processes and they did and you know, so I think I felt that the response was quite interesting and quite telling in its maturity.

Yeah, it demonstrated leadership with an acknowledgement that there was a mistake and I meant the response in the industry rather than necessarily there is what's just in terms of how we respond things when they go wrong out knowledge. Now, the evolution that's happened in the industry where everyone is just a maturity of, okay, well, you should be looking at citations, you know, there's just this sort of like common understanding. I feel that it's just sometimes these things happen and you know how we were all like blindsided maybe two years ago and now everyone's like, okay, did you have this?

Did you have this? You know, we've all become, we've all become much better experts. Well, I actually also think we all live in the same glass house. So that's that that's really, you know, a challenge.

I think we all can kind of appreciate each other's position in this new era. And the other thing that I think is really interesting is I've been having lots of conversations on the client side of, you know, what is the reaction to leveraging AI? Are you having conversations? And I still think there's a wonderful, wonderful opportunity.

You know, we have clock coming up the corporate legal operations consortium, global institute in a couple of weeks. And I think there's going to be a wonderful opportunity to have an exchange between the law firms and their clients. And what they would like the remainder of 2026 to look like and how that relationship can be enhanced, not how can we take work away? How can we suffer nothing like that?

But much more, how do we elevate the work that we do? And how do we now take some of the time that we've saved or the efficiencies that we gained to do what we would do best, which is help guide you to strengthen your business opportunities. And I think that once that conversation starts to happen, this will flow much more quickly. The resistance will just sort of fade away.

And people will get very excited, not unlike the adventure you feel when you start using it. And once these conversations start to happen, my sense is that there will be an enthusiasm and an proactive approach to disclosing how it's being used and where else we can find it. And we just sort of start spotting, you know, coins to grab in the air like a video game, because that seems to be what will happen. And once there's a better understanding of how it can be used and where it can be used in an actual setting, you know, and in a sort of a live game setting, I think that people will, people will then, you know, the enthusiasm will translate into actionable results.

I still feel, have you been playing video games? Absolutely. I still feel it's quite adversarial, the in house. I'm, I'm fascinating right now.

I've, you and I, we've reflected how old we are. And, you know, I remember in 2008, the shift of power from law firms to in house legal team. So previously in house legal teams were reviewed as sort of slightly sleepy. That's where you went.

If you wanted a lower sort of a quieter pace of life. And then after the financial crash, suddenly the power really flowed into the hands of in house legal. And it was never the same again, you know, and I feel like we're one of those moments still where the power is more flowing into in house legal hands. And I, and I, I do think that slightly less adversarial approach would be super beneficial, you know, like exploring how things can be done together because there's also sort of pros.

And I've spoken to some amazing law firm leaders who, you know, they're very acknowledging of the fact that they need to perhaps work where the client is we need to mature as an industry and we need to perhaps, you know, follow the examples of the big four, for example, and being very system agnostic and working where the client lives and all of these kind of great things that we've maybe not done enough of. And there's maybe going to be some changes, but then private practice have got this advantage of this huge deep expertise from working with loads of different corporate clients, you know, which you can't get when you're in house.

I just feel like that's such an interesting area. I can't wait to see what comes out of clock on those sorts of conversations. You know, the other thing that I think it's important when I was at this event yesterday, the legal tech connect event, someone mentioned how this emergence of AI and legal reminded them of the growth of the internet and how an even back to putting a computer on all the desks of everyone at a firm and how it could be misused. And it's going to do all the work for everyone and we started this conversation out about the native or AI first law firms and I mentioned fair play with Sanjay Comlani and when Sanjay and David launched panchia three was sort of seen, you know, some people may have thought, you know, an LSP is going to take work from a law firm and the clients will rely directly on them.

And what we've seen is that relationship evolved and the law firm still sits in a very central position because they provide the judgment and the guidance on which the companies rely. We're just getting to that point faster. It's like, I already know path A or path B. My question to you is, which do you recommend I take?

And I think that that's going to be a really interesting way of looking at this. You know, I just just released the research from my the general council report that I do every year with with FTI and relativity. We were just at a webinar talking about this and these issues are really top of mind and you know, in house teams as you say have gone from, you know, 20, 30% usage of generative AI last year or two years ago to 80 something percent in the past year. And that number is only going to rise to 100.

And once we sort of figure out the where the lanes are again, I think that everyone will sort of figure out where their strengths are and law firms and their clients will remain kind of connected because there's a very important sort of relationship that can't really be substituted by. We've seen historically by pretty much any other option, however, the work that's kind of reallocated and the value proposition seem to adjust. Yeah, a little bit like us or no substitute. There is no such a big out there.

It's an conversation yet another 50 something episodes. Another wonderful discussion. I'm honored to be doing this. So thank you so much.

Thanks. All right. Thanks everyone.