The Strategic Evolution of Legal Ops with Olivier Plummer of Coinbase

Lawyers are actually solving a bunch of problems and you have to be creative for that. So to be able to DIY your own agents or your own workflows or your own solutions, come up with data models that you would have never been able to do before and on the fly is a form of creativity on its own. So I think, like, this is unlocking a lot of value in legal teams. I'm Hal Marcus. And I'm Mae Maeung Wudiwe. And this is Meeting of the Minds, the legal AI podcast from evisort. We interview lawyers, professors and legal operations pioneers that are pushing the envelope leaders using technology to drive great business outcomes and shape the future of the profession. Howdy. How. Hey, Mamie. We just had a really fun conversation with someone I'd never met before. I'm so glad you brought him to the podcast. Olivier Plummer. I want to get his title right here. He's the program Manager for legal technology and Operational Excellence at Coinbase. Yeah, I met Olivier at Concero events at Clock. We're both part of the black legal operations community. I've always been so impressed with how deep his skill set goes. It's not just traditional CLM or E billing tools. He's deep on contract intelligence, analytics, agents, all kinds of advanced technologies and how they're applicable to the legal department. Yeah. And working at Coinbase, you know, fast high tech crypto finance company, I think it puts extra pressure on to make sure that they're being really creative and efficient in their use of technology. That said, a lot of acronyms in this episode, so let me call out a few of them just so people are comfortable with them when they go by one that comes up a lot. APIs, people are pretty familiar with that. Application programming interface lets you connect computers and programs and share data. Elm, Enterprise Legal Management, which is a practice management strategy for corporate legal departments and integrating with insurance, government, et cetera. But mcp, also one that people may be less familiar with, Model Context Protocol. This is an open source standard that was introduced by anthropic in 2024 and it enables LLMs to access local files, databases, and APIs for that matter. So we're full circle on the acronyms. There's awesome. Thank you for setting the groundwork and let's hit it. Olivier Plummer, welcome to Meeting in the Minds. Hey, everybody. Super excited for this podcast. Thanks for joining us. So, hey, could you walk everyone through a little bit of your background and how you got to the role that you're in? Yeah, so I'm Olivier I currently am a program manager at Coinbase. I lead up our contracting side of the house and it's been a journey to get here. An interesting evolution of legal ops space. I was at a marketing tech company before this and did a little bit of entertainment work. And then I've been at Coinbase for about four years. I came on to implement our CLM and I've been here for seeing the whole evolution from like a very immature CLM to fully fledged integrating to the business. We now have our contract analytics tools. So EvalStore has been a great new addition and I've really watched like our full tech stack evolve and now AI come to the picture. So really my role now is to manage our legal tech side. We work on a lot of different projects and we have a lot of cool things going on at Coinbase. Awesome. I'm excited to have you on because in a world where so many folks are focused on clm, you've built so many skills, ES and clm, but also around contract intelligence, contract analytics, using AI to track information and agreements. What would you say for folks who might be maybe just coming up to speed on traditional CLMs and they're curious about, hey, how can I use AI and agents to help contracting be more efficient? I would say start simple and find ways that you could actually have value. I feel like there's a lot of pressure in the legal space to throw AI at everything and make it stick, but I feel like impact driving your use cases is really important. And if you're new to the industry, keep your data model clean. I've seen a lot of people who have to clean up a really messy data model. A period somewhere can mess up exactly what you're doing. So coming up with conventions, just thinking, like, how does this connect into everything? I don't feel like in this space we had to worry about systems connecting and data being transferred to everything having unique identifiers, and now everything is interconnectivity. I feel like it's the flip that's switched in the past year or two. So start simple, focus on impact and keep your data clean as much as you could have, like a naming convention. Just understand what you're trying to do with it. Those are my big advices for starters. So what were some of the things you prioritized in starting simple when you started applying this tech within your organization? I would say business drivers. And I would not say that we started simple, so this is actually learning lessons. I would say the opposite. We started huge and we had to be like, hey, I don't know if we could boil the ocean right now. Maybe let's slow down and figure out what we can accomplish. So this is actually my own learnings. I would say to people now lawyers, everybody is like learning about tech. It's super exciting. In a legal ops role. You kind of need to learn how to preempt this and kind of help partner on a roadmap and technology strategies. So people are going to come to you saying they want to accomplish a hundred things. Part of having like success in this role is going to be having some discernment and being able to figure out, hey, we can do this now, this can be done in six months and this is not going to happen at all. We should maybe redirect. And I would also say, like, not everything needs to be an agent. So I know that's like the buzzword that's going around, but some things might just be workflow, process improvements or writing a script or stuff like that. So really being able to dig your feet in and say, hey, this is what we can do, this is what's strategic, this is what makes sense is how I would start. So you're starting now. Have a real conversation with your legal team, figure out what their priorities are and help guide them. It's definitely a partnership. And I think Legal Ops is being position now to be a thought partner to legal teams in a way that maybe we weren't a couple years ago. So exciting stuff. But yeah, I can't say I started small at all. So let me just double click on that if I could. What's changed in the last couple of years that you've seen in the relationship of Legal Ops and legal? I would say Legal Ops was a lot of delivery focused. Honestly feel like working in here. We may have been looked at as similar to like paralegal functions or other legal service providers. Now we're like main partners in strategic roadmapping conversations. We're being pulled in when big rocks pop up like immediately and people are really looking at how to optimize. I don't feel like we're in a environment where there's a bunch of resources and players that could help optimize. Processes and scale have become super important. So I think lawyers and legal professionals are really looking like, how can I scale and how can I function in this environment? And also now that we have all these tech pressures, how can I navigate this? And that's been like a mindset change. So now it's like, okay, we want to bring you along for the journey and consult you and get your advice from the start versus coming to us like this is what we want to do. So I would say that's been a big change. Have you seen impacts or connections with your contracting work get more important even beyond legal? Yeah, I mean, we had, like, a huge regulatory project that we worked on, and it just would have never been able to get done without, like, a good CLM and a good legal ops team. Like, we pretty much had a team of a couple lawyers scaling, sending out like 5,000 contracts in like a month. And we made it work, getting super innovative. And I would say, like legal now we're working with like, our integrations team, our engineers. Like, it's kind of all hands on deck. Previously, this would have probably been like an $100,000 invoice to, like, a big law firm, and they would have been like, pencil pushing a bunch of contracts. And I still don't think it would have been hit by this regulatory deadline. So we've had some big tests where we've had to apply tech, and it's kind of changed the game and how teams think about us. So for sure, contracting efforts have allowed us to, like, tackle huge projects that probably would just not have been doable before. So seeing a lot of changes there. Nice. And are you seeing any interest or impact with maybe like, sales or finance or real estate groups or kind of domains that go beyond legal compliance? So I would say that's been the most surprising part. Once you start digging into contract data, you actually learn that there's a lot of stuff there that's not for legal necessarily. So I know legal wants to look at, you know, indemnity. They want to benchmark different risk portfolios. But I'm like, oh, sales op wants to know, hey, we have this new product. How many other products do we have onboarded to this client? Are there any sales ops that we could upsell on here? Which I was like, oh, I never thought that we'd be using our contract analytics to drive, like, who are we reaching out to upsell different products? I've had people reach out saying, hey, we have all these feed tables. We want to do our financial forecasting. Let's like, pull out, like, these different fee tables and figure out how we could feed these downstream. And I was like, oh, okay. So finance, our chief revenue officer is interested in ever sort, or our head of sales and trading is interested in ever sort. I thought it was going to be like, we still do have legal champions, but it's way expanded past that. So, yes, sales is super interested. Even our HR team is interested. People want to pull out a bunch of different insights. And I didn't really realize how meaty our contracts were until we started building these models. So a lot of themes and there's a lot of value to drive there. How did that spread? Was that like a function of, like, legal did this thing, thought it was going to be very specific. And then the other team started getting wind that we've got this capability now to get some real visibility, or did it emerge some other way? Was it like word of mouth or just new requests coming in from the other teams? So our contract analytics rollout actually was not driven by legal. We had our business stakeholders that were like, hey, we have a really big project we want to solve regarding, like, our financing contracts. And our previous vendor actually said they couldn't do it. They didn't have the technology, wasn't there. So it kind of forced us poke around and see what's on the market. And this was brought to us by, like, business people, not the legal side. So that was a really interesting development that, like, it's sales teams and financing teams that are actually looking for these analytics. And legal, of course, is a beneficiary, but that's one portion. I would also say a rollout is really important. So I think we had a really solid, like, PR campaign on what contract analytics is. And like your implementation team, your partners, your CSMs are super important to this and being super organized. I would say, like, we had a really good POC and we were able to move from POC to like, contract signing and implementation within six months. So from building buzz around like, hey, we're trying this out, people like it, to actually executing and delivering and then having a bunch of training has been super important. And I would say word of mouth has really been a big thing. So different teams have liked this. They've been like, hey, can I add my coworker to our channel for this? We want to poke around. I've even had, like, people on financing being like, hey, can you give me API access? I just want to poke around and see what I could do. So word of mouth on top of like, a coordinated rollout has been huge. But yeah, it wasn't even legal that prompted us to go on this journey. So that's been the really interesting part. A lot of folks have had experience on more general LLMs, you know, the chat, GPT Geminis. You've used both those tools, I'm sure, in your personal everyday life, but you're also using more specific like legal tools like a workday contract intelligence to kind of train models in a particular context. Could you talk a little bit about what it's like leveraging more kind of dedicated legal tools versus the generalized ones maybe for folks who are only used to one or the other. Yes, I have a lot of thoughts around this. We've POC'd a lot of tools and I will say a common question you've gotten is how is this not a ChatGPT wrapper? That was especially the thought 6 months ago. I think people are seeing different emerging use cases and I would say having like an enterprise wide AI tool is a whole different ballgame than having like a personal AI agent or GPT or LLM that you're using just because of the data access and the interconnectivity that you could have by having an enterprise tool. We've implemented a couple tools. Some are shared services across the org and we've been able to scope out legal use cases and then we've also evaluated legal specific tools. The thing now is I would say Coinbase has really democratized AI use for the company. So we are not centralizing like we have this person that's building agents for everybody. It's like on your performance review, what agent have you built? You know what I mean? So everybody's encouraged to get in there and roll their sleeves up. That is great. But it's also like the wild wild west right now because having governance across 10 different tools is hard. And circling back to saying like is this a GPT wrapper? Having a big tool that could solve a lot of use cases and creating governance and a model to frame and build and like criteria for success is a lot easier in one tool than having 10 different tools. So if you are at a company that does have the budget for a big enterprise wide tool and you're looking at like do I want to consolidate my tech stack? Do I want to diversify? This is where it really comes into like what are my use cases? How am I scaling this? Does this work in one tool? Like I like evisort because imagine having like an army of contractors having a bunch of different models that they're running. You have to standardize across all of them, review the work or having this dispersing different tools or even trying to build this in an in house tool. And it's like some stuff that we've considered is like we can build in house but because everything is evolving, are we going to have to re engineer in six months based On a new rollout versus investing in a company that has the R and D to kind of keep reiterating and we can just run with it. So we like the big players at Coinbase. We kind of need the shiny tools. We have crazy stuff going on. So. Interesting space right now. You touched on a really interesting issue there. I was just talking to a customer about this yesterday, and a lot of folks are struggling with this. You want to empower the end users to, you know, we can call it making agents, you know, making models, just, you know, creating their own little environments, their own applications, all that sort of stuff, because you get a lot of innovation that way and a lot of good stuff happens. Then again, in the contract space in particular, that can get really unwieldy very quickly because you're creating now data points that have been trained differently and they're populated differently. They can conflict with each other. So we've been seeing a lot of organizations struggle with. Where's that line like, do we want to empower everyone to create models? Do we want that all tightly controlled within our product? We require that to go through the admin so that you do have some governance, some control over those standard data points. But I totally get why you want to empower everyone at the same time. So, like, we empower them with the ask AI to ask their questions, but not to create the data points for everyone. Where do you see that balance? Do you think we're like, we're finding the right place there? I like the balance in evisort. I feel a couple ways. I think that if you want your workforce to stay competitive and you want to upskill, you need to empower people and give them the tools to poke around and play with everything. Like, I feel like we're training a mini army of coders and AI specialists and kind of everybody has a basic proficiency level, which is super important and helps with training. However, legal is not like a place where you could. You can be a little vibey and legal, but you also have to be a little serious and careful about your governance. You know what I mean? You have to strike a balance. I like the balance in evisort because we are controlling, building the data models and we have a central intake point, but once we have them, we're empowering people. Hey, if you want to plug this into an MCP and pull this out for your reporting, have at it. But we're kind of managing the data. And I think one of the things with AI is like, with all of our data projects prior to this it's been humans going through everything. So we had a project where we were working on our security clauses, indexing like all of our commitments, seeing what we're compliant with, what we may have committed to. That was like a huge project with an army of contractors to index that. It's valuable but not super scalable and not sustainable from a pricing standpoint. Now we're able to run analytics two days across 140,000 contracts. But the data integrity is something that people are worried about now and having a QC layer. So I think in terms of extracting data and building data models that other people are using, it's important to have a central intake point that could ensure the quality is there. But democratizing that data and letting people play around with it and building reporting is what I think is great. I don't have the time to, you know, build sales ops reports, financing reports, and honestly, working in legal, it's not super strategic for me moving my workload to be taking on a bunch of AI projects for other teams. But creating the bones for them to run with that data is important. So I think ever sort of strike the right balance. I think other tools are figuring out how to get there, but for me, yeah, I think centralized intake and like having a good clean data source is super important. So you gotta leave that a bit to the experts. But it's in flux. I actually have thoughts. I'm like, I feel like we're also in a transition zone. I don't know if we're gonna be looking at the same landscape in two years. Like I'm building prompts, we might have an agent that's building the prompts and we might have an agent that's building the agent. You know what I mean? So we may also be in a growing pain point where right now we need people that know what's going on to build stuff. But maybe I'll be building something that's going to build stuff down the line and then it's going to totally change. So you also be nimble and like adjust with how the tides are turning. Yeah, I mean, speaking of being nimble, you just talked about an mcp, a model context protocol, you know, an open source kind of standard that helps LLMs connect with data sources. Right. Many folks are just getting up to speed with APIs, right? Application programming interfaces, which is traditionally connecting different softwares. Would you suggest that Legal ops folks in the space get used to and up to speed on MCPs and APIs and kind of connecting systems at that technical level? I mean, if you're not up to speed already. You're falling behind. It's actually crazy how my conversations with vendors have switched from, like, we're having product roadmap conversations, but now I'm like, haggling for 30 minutes with vendors over, hey, what's your API roadmap specifically? And also bringing in partners to look at that. This is super important just because now that we have all these interconnected systems and why I think having enterprise AI is so cool, is that it could contextualize data from 10 different areas. So we could have our CRM, our sourcing tool, our CLM, our contract analytics tool, get our database and pull in answers from that and then make a Google sheet out of it, which is something that, like, you're not doing with your $20 ChatGPT subscription is like a whole different ball game. If you're not proactively figuring out how can I integrate my tech stack, you're already falling behind. So, yes, absolutely. During any discovery, you need to be looking at APIs, or if you have a team that's focused on that, bring them in and ask them questions and poke around. Because that could really be make or break for a product. And then, like, connecting mcps, building everything is also going to be a skill. We're trending towards having engineers and having, like, we have an integrations team that's kind of working on that. I don't know how strategic it is for me to build all of the mcps versus architecting. And that's kind of like a line that people are going to have to balance whether they're going to bring in their own engineers on legal department. And I feel like legal engineering is about to actually blow up, or if you're going to outsource that to, like, Eng. But that's a distinction to make too. But overall, yes, you need to be asking questions that should be like, top five things in rfp, like, what does the API look like? What's the roadmap for that? I'm actually Emily. I asked for, like, what is your API roadmap? Like a sub roadmap conversations. We just had that conversation yesterday, actually. So interesting stuff there. I love what you said before about how you can be a little bit vibey and legal, but you've also got to be serious. I think that's the perfect encapsulation. You didn't come up, you know, initially through legal. You've been now in legal for a number of years. How do you see that difference with this Persona? Like, are they kind of getting Vibey or are they locked into the. No, no, no, no. Only what's absolutely approved, only what's delivered to me. I'm not going to be too creative in how I use the tech. Where are you seeing that? In a pretty high tech company that you work at? So on the surface, I would say lawyers in Vibe maybe is not always something that you would put together. However, now I'm working with like a legal team and I'm like, oh, we're all just kind of winging it and figuring out how to do everything, but just applying legal situations. I feel like our lawyers. Legal is an industry that is totally, I wouldn't say under attack, but it is being transformed by AI. And in order to survive and to adapt and really grow in this environment, like, everybody has to be Vibey. So I'm seeing a big change in personalities. I'm seeing a lot of people that I would never expect to build. Agents bringing me stuff like, hey, can you give me feedback on this? And I'm like, okay, this is a super cool change. And I think, like, back to the democratization of AI and just expanding all of this, I think people like it actually having an arsenal that they could build. Lawyers are actually solving a bunch of problems and you have to be creative for that. So to be able to DIY your own agents or your own workflows or your own solutions, come up with data models that you would have never been able to do before and on the fly is a form of creativity on its own. So I think, like, this is unlocking a lot of value in legal teams and opening up, like, can you just be a pure lawyer now? Do you need to be a little bit of an engineer at Coinbase? To an extent, you got to be able to do both. And I think a lot of people are going to be looking for that. So people that have those personality types are going to soar far. And then there's also people that you know are going to do that for the org, depending on your structure. I think you've touched on something really interesting that often gets overlooked. There's a lot of people that look at the legal Persona and think in terms of the risk aversion is the guiding principle, the attention to detail, all that stuff that gets drilled in, looking at anything that can go wrong. And what they sometimes miss is that legal has never really seen itself as paper pushers or just follow the rules, people. I think a big part of the legal mentality has always been seeing ourselves as artistic. And so when you see those Opportunities for an artful phrasing of something, for a creative approach to a business problem, and now a creative approach to a workflow issue or a technological problem or a bottleneck. I think this is creating a new language for them to be artful in. And I think we will see more and more of this. Yeah, I agree with that 100%. I mean, I think at Coinbase, I don't look at legal at all as a blocker. I feel like it's an enabler, actually. I mean, we're insane. So we're building like new financial products on the fly, stuff that you never heard of. You know, ways to take out equity loans on your crypto, way to tokenize your assets, all of these new things that have never existed. We're meeting clients that are like, hey, I want to do that and for a billion dollar of assets. And we have to find a way to make that work and make it legal. And our legal teams are doing that at scale and figuring out a way to build new products and innovate. And it's like when you're at this forefront of like technology and finance and it's just like completely new emerging sectors, you have to be creative to keep up with this. So I actually, like, applaud people on our legal team for being able to run with all of this stuff. And even now, like, I'm watching the evolution of the crypto space, all of these new products come live and I'm like, yeah, we have to be nimble. So I think they're creative. I think it's super cool, honestly, being in such a kind of fast moving and cutting edge industry. Does leveraging AI internally help you be nimble? Help you kind of react quicker to challenges than you otherwise might be able to? Absolutely. I mean, not a lot of people are going to get sued by the SEC and have to figure out how to flip that and win the next day. So I feel like we have immense challenges and Coinbase specifically has like crypto on their back. So all of the policy work and all the stuff that we're laying the ground for in terms of legitimizing these financial products, creating them, people are kind of taking what we're doing and scaling it, which is fine. Everybody's building, there's enough space in the industry. But yeah, we need AI to do all of that because there's so much to tackle and like, we're not a old behemoth that kind of has the time to not stay nimble. There's a lot of competition and it's like if you need to figure out how to launch this new product, create these new resources through these new sprints, like you need AI to be forward with that. And I think one of our ideals is like top talent. We're really trying to build like a team of super empowered people. I don't think we want to have a bunch of people doing repetitive administrative tasks and I don't think that's the best for the company or even best for morale. So using AI to kind of shape that and give us space so, you know, we could focus our headcount on more strategic areas. We can become more strategic, I think has been a huge benefit of AI and we were all in committing to this. So I love it. It's one of the things that makes me like working here because not a lot of people have exposure to all these sets I kind of look at as like toys now that we have. That's like a big driver to wanting to stay just because who has access to all of this stuff? Not a lot of companies. That's really exciting. And as a system administrator and you've implemented a lot of tools for lawyers, even with the slightly more vibey lawyers that you have at Coinbase, I'm wondering, are there some kind of common pitfalls or issues or things that you might suggest for others, maybe newer in the role to look out for when they're trying to implement new tools for lawyers to adopt? I think in terms of implementing new tools, I've done a whole bunch of POCs and use case development on the front end is super important to the success of implementing a tool. I think we've had scenarios where you've gotten into POCs and we're like, now what? This is great, but what's the impact? So working with teams and figuring out what are they trying to accomplish and working backwards from there has been super important. And I would say also finding champions for the programs is also super critical. So legal ops cannot drive the carriage, I don't know whatever expression. But legal ops can't do everything. You need to have lawyers that are strategically placed that want to make this work and see this as an add on for them. So I would say that's really important in terms of challenges. If you have been in this space for a while and like let's say you have a CLM for, you know, five years, you have a repository, you might have 10 repositories. Cleaning up data and finding a way to kind of evolve in the new age is super complicated. I'm even Watching like some of the companies that we work with, reiterating their products and launching new stuff that would probably be great for new implementations, but are a huge headache to change for us. So I would say like being able to evolve with the space and turning legacy features into stuff that's useful for now is getting super difficult. And managing tech debt, like making the wrong decision on a tool could put you back a year or two years where you need to be. So I think making the right selections, figuring out how to evolve and kind of tracking like what parts of these tools are evolving and how can we evolve with it is super important. If you let something fester for one or two years, like it's going to be really difficult to fix. I think a lot of people are counting on and we're in support of this, we're doing this as part of our company, but they're counting on chat experiences, chat interfaces to be able to speak to the different data sources. And then you don't care so much about how all those things are mapped and fielded and all of that. I think that gets you through to a certain degree. You can get a short term answer where the data was maybe not quite as in line, but it limits your ability to do all the kinds of connections with CLM connected to CRM, et cetera, et cetera, et cetera, accounting. And down the line. We get that a lot in procurement too, where if the data's not mapped well, that is kind of limiting. So do you think I'm reading this right? You kind of need the balance of the two. It's good to get a quick answer. But you really do want these things with the pipes connected properly. I have two thoughts there. So we'll say in terms of like in app agents for sure, having groupings together, identifiers is super important because asking a response for 140,000 contracts versus asking a response on a targeted data set, it's going to give you way different output. So being able to figure out where am I targeting and how can I be more precision is super important. And then I would also say like I spent a bit of time like adding cloud to my terminal, plugging in all of my mcps. What does that look like? And now like we're not even asking questions in the tool for everything. How do we build like a unified profile for like a client across a bunch of different systems and be able to just ask for that and get the full picture as well. So being able to group your data, have identifiers within the tool and Then also thinking about maybe global identifiers is a whole other thing. I know everybody is competing with each other, but I'm like, if you could have some type of protocol, a Geneva protocol and like data and having some type of IDs that would be groundbreaking just in terms of like everything is going to be interconnected. So I think anybody that's thinking of their tool as like the one and only in a stack is going to get left behind. Really thinking out how can I promote integration. That's why most like strategic partnerships between different legal tech companies are super interesting. Because if you're building a tech stack with the best of the best, they're all working together, you can probably build like a God tier data model that could pull out everything. So we're in the early stages of seeing that. But yes, identifiers, global identifiers, linking stuff between tools and also having like, maybe even client profiles that live on a cloud, that pull in data from a bunch of different sources, wherever that lives, is going to be super cool. So we have a lot going on there. I think it's a wait and see game. Right now. Things are changing weekly, monthly, we're implementing new tools, sunsetting other tools, so waiting to see how that all shakes out. I love the idea of a Geneva Convention for technology companies. We call it the Palo Alto Convention where folks just come together, have this unified schema that we're all working off. Yeah, we're going to have international tribunals and all sorts. It could get ugly. But you're clearly an expert and I think one of the top out there, frankly, when it comes to understanding these things in the legal context and at this kind of technology AI level. I mean, you're steeped in contract intelligence, AI agents, but you've also been working in the world before that. Right. And Ivan saw in your past work, you worked with companies dealing with GDPR and CPRA and IPO preparation and insurance renewals. In a world without AI, I'm wondering if you can like look back and it might be painful, but look back at some of those past manual projects and just kind of think about how difficult it was to do manually and what it would look like to do some of those projects now with the tools you have available to you. So I will say I was at a startup where we did a series B, C, D and E without a clm. Hell on earth. Like the amount of time that I spent stitching together like agreements, stock transfer agreements in like a PDF format, this was way before we were dropping everything in DocuSign. That was a nightmare. I don't know how I did it, but at the time I feel like we weren't even visionary to think that this solutions were coming, you know what I mean? So if you ask me looking back at the time, I mean there was no other option. We just had to suffer. And now I guess we don't. Maybe we have a new type of suffering in terms of like I was fighting for my life, downloading all this stuff into my terminal. So new challenges. But there were super complex projects and it was really like having a bunch of people in a room and just churning through it. And honestly a lot of error. I'm kind of like we were talking about your data integrity. We have like OCR contracts from like 2001 and I'm like, what the hell is this? You know, these signatures are not there. Like there's a whole bunch of stuff that was just not above board. So it was a struggle. And I don't think legal teams were able to move as dynamically. And this is when like big law culture was thriving. I think that might be a little bit we're looking to see, hey, how can we trim around our outside counsel spend? That's totally changing. So I would say we were relying a lot more on firms and consultants. But really it was just rough. It was a lot of late nights. There's still late nights but for different projects and bigger rocks. But I'm glad that we've had people that have been able to innovate into this space. And now I'm coming into a space where I could start innovating too and figuring out what the future is. But it was rough, man. So especially at like a startup environment in New York, it was a lot of late nights, a lot of coffee, a lot of errors, renaming files. It was not the glory days at all. But it seems like these technologies allow your teams to do more on their own without the use of big law or even maybe even ALSPs, you know, alternative legal service providers. Yes. But also we're finding hybrid models are actually super important for delivery. So like QC has become a super important component of this and finding a good partner for that. And this is actually opening a space in the industry for like new firms to come in and build out their own models. Like we found somebody that will QC for way cheaper than sending us out to outside counsel for high touch data points. Like stuff that's going to make decisions on financing, staffing decisions, stuff like that. That is really valuable. There does need to be a QC Layer to an extent, while we're getting over this hump of, like, building prompts, testing them, reiterating. So, yes, we're using this to lean less on outside counsel, but also we're asking our outside counsel how we're using AI and mandating them. And then we're also finding new partners to come to the space to partner with us. So I'm looking at everything as a hybrid, integrated solution. Like, I'm not looking at one tool to solve any problem. One tool may be able to solve a problem, but it's also totally open to having contractors still keeping a human in a loop, not necessarily to pencil crunch, but to provide some type of different services that we haven't seen before. How's that working out with big law? You know, turning to them and saying, show us what you're doing with AI or mandating certain kinds of AI use cases. Are they stepping up? Are they confused by it? Are they coming to you with those ideas? I think it's a bit of a science experiment right now in seeing what's sticking and what's not. I know that the big LEO tech tools have, like, the outside council side of the house, and they have, of course, big companies like us, and I think they're being forced to use. I feel like billing practices are popping up. We're like, hey, can you show how much time this has been reduced by using AI and also asking that to come into, like, the billable hours? So that's become a whole conversation that's popping up. But, yeah, firms are having to adapt in terms of setting criteria for success. I feel like that's super challenging and super new right now. It's been a lot of throwing AI at a wall and seeing what's sticking. And I think people in that space are figuring out, okay, these are cutting down our hours. This is delivery that we're seeing. But also, they could kind of be writing whatever. So I think it's right now a testing game. They're implementing their tools at the same time as us, and we're probably ahead of the curve. So more to come there, I would say. I'd say after listening to you today, I'm pretty sure you're ahead of the curve. But I do feel for the law firms because they're dealing with some clients saying, I need you to mandate AI and save me money. They're dealing with other clients saying, do not use AI on any of my things and promise that it won't be touching. And so I do always feel a little bit for them. In that, would you say that alternative legal service providers are also. Are you seeing almost new types of ALSPs into the space or just kind of the traditional ALSPs changing their skill sets? Probably both. I would say also we're looking for things we weren't looking before either. So people may have been doing this heads down and now the technology has just brought them to the forefront. So like we weren't really looking for like these different types of even the contractor roles that we're hiring. We just brought somebody on our team specifically to help us build out AI for legal. We weren't seeing these types of job postings before. In terms of the legal service providers, I just don't think we were looking for this before there wasn't the business need. I'm sure there have been people in their heads down, innovating and waiting for this moment, but now we're seeing it a lot. And I will say when I'm out of the conferences and I'm meeting different people, it's a whole new crop of companies coming up kind of in the way when the whole CLM scene exploded and there was a hundred different CLMs, we're now seeing that. So more companies, more legal services providers. But also we're looking for different things now and I think we're going to continue to look for different things. So to everybody that's been building infrastructure, thank you for being available now and I'm excited to see what comes up now too. Any quick tips for someone who's like, I've just looked at 10 companies, they all look the same. How? Any kind of clues, guidance for folks who might be don't know how to suss those out. Be open to making mistakes and setting expectations with your stakeholders too. I think people need to know that these are new tools. You may try something and it doesn't work. I would also say I want to work with the winning team. So the talent behind it is also really important as well. Product is good, but also having people that will partner with you on a product roadmap and building out the tool has become something super important for us. So I would say if you're at a company like Coinbase looking for strategic partnerships and a deep talent base is super important. We really like companies that have the resources and the backings to somewhat design their products around what we need. I think we're pretty innovative so it's going to help them in the long run. But even if it's not a perfect product, having a willingness to iterate and work and truly Be collaborative is something that I really look for. And also just making sure that like, do I need this tool to solve this? Can I build this? Can this be done with one of my existing tools? Is this on the roadmap? Those are all conversations that we're having now. I think people flooded the market with money and they're like, hey, do we actually need these 10 tools or can we just build this out in house, have a hybrid solution, build a layer on top of this? Those are all questions that I'm asking. And sometimes you make a mistake and you kind of have to be able to level set with your stakeholders and have leadership that'll provide you with COVID for that too. So that's how I'm thinking about this. And Olivier, to close out, anything you tell the folks who are looking five years from the future, like, what do you see this space looking at? What's your view and what this space is going to transform into? I think being humble is a good question here. I have no idea what's to come. I have some predictions. I did think that chatbots were going to take off two years ago. I remember being at a conference, at a roundtable and saying that and I'm like, okay, now we're using chatbots every day. I think agents are the future and kind of turning workflows into agents. So like prompting as is sitting out and building prompts. I think there's a whole economy building around prompt engineering. That could be a bit of a bubble because at a point we're going to be able to build our own tools to prompt on our behalf. So I would see that being something that's evolving. I think the API push and like connecting systems is going to be even bigger. And honestly, if companies want to keep up, I think they're probably going to look for strategic partnerships. Maybe not within like CLM to clm, but CLM to CRM, CLM to analytics tool, elm, clm. I think we're going to see a lot more like consolidation and partnership on that side of the house. And I think we're in a fragmented AI environment, but I think these companies are going to consolidate. I don't know how sustainable it is to onboard different tools, do 10 POCs in a year when one tool may be able to solve everything that you're doing. So I'm expecting we're going to see some big AI behemoths pop up. I think big AI companies in general, anthropic Gemini, are going to start branching out into this space. That being said, I'M not too worried because I think there's very specific niche stuff that legal tech companies can carve out value from. I've seen like the market has dipped a little bit. I'm not too worried about that. I think it's a very specific space. I think there's a lot of R and D that could go specifically to here. But yeah, consolidations, prompting, as we know it, is going to change, agents are going to become huge. And also I have no idea what's coming. I didn't expect LLMs to roll out like this and change the game. Literally my workload and my expectations is so different this year compared to last year and the year before. So I'm just kind of strapping in for the ride, honestly, and kind of vibing. You've got to just go with the vibes to an extent. Sounds like you're staying ahead of it pretty well. We're trying. Well, thank you so much, Olivier. It's been a pleasure. Yeah, this has been great. Do you guys want to invite me again? I'm sure I'll have more thoughts in six months. Awesome. Pleasure talking to you, Olivier. Thanks so much. Meeting of the Minds the legal AI podcast is brought to you by Eversort. To learn more about Eversort and how we can help you contract better with AI, visit Eversort evisort. You can find meaning of the Minds on Apple podcasts, Spotify, or wherever else you listen to podcasts. Don't forget to click subscribe so you won't miss future episodes. On behalf of everyone here at evisort, thanks for tuning in.