AI Beyond Enterprise Speed with Marsh SVP & Chief Information and Operations Officer Paul Beswick

Hi there and welcome to Enterprise AI Innovators, a show where top technology executives share how AI is transforming the enterprise. In each episode, guests uncover the real world applications of AI, from improving products and optimizing operations to redefining the customer experience. I'm Evan Reiser, the founder and CEO of Abnormal AI. And I'm Sam Motamity, a General Partner at Greylock Partners. Today Sam is talking with Paul Beswick, Senior Vice President and Chief Information and Operations Officer at Marshall. Marsh is one of the world's largest professional services firms with four major businesses spanning insurance, brokerage, reinsurance, HR consulting and strategy. Paul has spent nearly 30 years there, first as a consultant writing the tech strategy and now as global CIO actually delivering it across the firm. A few things stuck with me from this conversation. At Marsh, Paul's team is piloting a voice based crowdsourcing tool. Instead of emails, surveys or meetings, employees just dictate two or three minutes of their thinking and AI cleans it up. When a big news event hits, they can ping their specialists and turn scattered expert opinion into a scalable asset set the firm can actually act on. Second, the first version of Len AI, their internal AI tool, took only a couple of days to build and was rolled out to the entire firm in just a few months. That early BET taught them tool routing, MCP and multi step planning before agent harnesses existed. Their home built version of an agentic AI tool now trails the commercial market by only a few months and at a far lower cost. And finally, Paul says the cost of the meeting to discuss what you're going to build is now often more than the cost of building the thing. Len AI itself was built over a weekend before anyone debated whether to do it. When production grade applications take days instead of months, his advice to other CIOs is to break their three big AI projects into 20 small ones and ship them sequentially. Paul, thanks for joining us. Have been looking forward to this conversation and I think just to start, can you give our audience an overview of yourself, your career and your current role at Marsh? So I'm slightly surprised to find myself nearly 30 years at Marsh in various forms, although it's felt like a number of very different things phases. I started as a summer job in the consulting business that we own, Oliver Wyman, and was there for about 23 years, ending up running the global digital practice before slightly to my surprise, ending up deciding that I should go and deliver the tech strategy that I'd helped write as a consultant by becoming the global CIO and then More recently, I picked up our global operations teams as well. So I'm currently run our tech and ops teams globally for Marsh and all of its companies. Amazing. And maybe it'd be helpful actually for our listeners. I think not everyone may appreciate the scope and impact that Marsh has. So can you give us an overview of Marsh and just the company overall, the scope of operations, and then sort of like what falls into your purview and your role as cio? Yeah, our businesses, I think, are well known by people who use them, but are largely B2B and so less well known to the world at large. We have four major businesses. Marsh Risk, which is an insurance broker, and risk advisor, Mercer, which does a variety of work around investments, pensions, benefits, human resources consulting, and so on. Oliver Wyman, the firm I spent a long time at doing strategy consulting. And Guy Carpenter, who are a reinsurance broker and help make sure that the insurance business is also on safe ground as well as all of their clients. Are there any specific examples you can talk about? Either things you've done organizationally to enable the supercharging of this judgment concept specifically, or things you've observed, you know, employees do with the tools that you've given them that really stand out? We have a project that we call Crowdsource of Marsh that we think is an interesting play that we're testing out in a. In a few places. And the idea is pretty simple. The idea is, at the end of the day, if I want to gather this stuff, one of the best ways to do it is to ask people. But the way we ask people today is we either ask them to type a long email or fill in a survey or have a meeting. These are all pretty inefficient approaches that require quite a lot of coordination. What if we just ask them by putting a question in front of them and recognizing it's easier to speak and faster to speak than it is to type. Just get them to dictate an answer, bring it back into a central form, get AI to clean it up and process it. Now I've got an artifact that I can actually scale. So just as a sort of an information capture engine, we're seeing some interesting use cases for that sort of approach. So you take a client team and survey them every month to ask them what is happening in that client. What are the key issues that are on people's minds? What are the hot topics that we need to know about? We know this stuff, but we don't know it in a way you can act on systematically. Some people do that already. A lot of people don't. Right. You systematize that now, you know, in a way you can start feeding other things off. Another example would be some big piece of news hits Strait of Hormuz is blocked. What do our marine people think about this? Well, let's just ask them. So make it a big deal. Let's just ask them to give us two or three minutes of what they think. And the combination of those things, rather than just being some stuff that people said once, becomes the foundation of an asset that actually can scale and create some network effects. So that's a pretty interesting one that we've been finding a lot of use for, and I think it's quite exciting. Are there any other things that are unique or interesting that you're doing? I think about a lot of Waikola jobs and companies as essentially factories that make decisions where that is the output and the raw material is the information and the data, and then there is the decision process. So being able to effectively feed that data in ends up being really important. So moving from, say, dashboard spreadsheets, floating around into something where you just ask and it comes to the point of the decision on its own, I think is an interesting play. I stumbled upon something completely random about two days ago, just in response to a question someone was asking in one of our team's rooms. And there's no, you know, this is not a big play, but it's interesting. I'm sure people will recognize the spreadsheet that they get sent every week or every month that they have to look at and do something with that isn't necessarily particularly user friendly, but at least always looks the same. A nice little use case for a Claude cowork tool. Build me a local HTML file that doesn't need a server, doesn't need to be deployed anywhere, sits on my desktop, but is designed to take that spreadsheet just by dragging it in and give me the dashboard and the breakdown of everything that I actually need to know in that dashboard, just mechanically in half a second or so. So quite a lot of that sort of stuff that sits around the place. Another interesting one, we have a tool called Lennox, which is a browser extension for our core AI tool. Think about data entry problems. Obviously, ideally you engineer them away, but in the meantime, Lennox knows what page it's on. And so writing a little custom module that says for this app that maybe we don't control the source code for, whenever I'm on this page, copy all these fields out and store it temporarily in some sort of cache so that when I go to this page in the following app I can produce a one click button to paste it all in directly. It doesn't solve my rekeying problem, but it does make it a hell of a lot more accurate and a lot faster. So there's lots of little things like that that get quite interesting. Paul, those are both really good examples and I think they illustrate something I often talk to leaders in our portfolio about, which is some of the most consequential things are actually quite small. Like the HTML example you gave is that wouldn't show up on a board level slide of like here's what we're going to do with AI this year. But that could make somebody's life way more productive and change the way they distill and glean insights from that spreadsheet. And there's all sorts of knock on effects that are hard to a priori sort of estimate. This highlights one of the big problems with AI, right? It shows up in so many places, so many of which are only partly scalable and very, very difficult to monetize. The story of it is quite a difficult one to tell. It's fine if you can find $100 million problem and go after that, that's an easy story to tell. But so much of it is is the accumulation of hundreds or thousands of different things that people who do different jobs have figured out to make themselves more effective and put that time into something. It's a much, much harder story to tell than a lot of other types of technology were. I want to talk a little bit about some of the internal tooling you've built at Marsh. In doing research before the show, I did some reading on Len AI which seemed really interesting. So maybe I have a two part question. One is tell us about Len AI as kind of a good concrete example and then two, tell us about the system that underneath allows you guys to build these types of things and the investments you've had to make. Yeah, so we started very early on this. Lenai's first version was created late spring 23 and we had it into pilot by the beginning of the summer to a few hundred people. So that was probably a few weeks after we could get access to OpenAI's model through Microsoft, which was sort of the big security unlock for us at the time. And we added to the entire firm by the end of that summer. And you know, it's sort of grown over time. But the nice thing about it was especially early on, it's incredibly cheap, cost us A small team, not very long to build. First version actually took a couple of days to put into place and get into that pilot group. And now we've sort of built on that and developed the ecosystem around it. And it's been enormously helpful in terms of learning how this stuff works and having really direct hands on experience of it. So we were building kind of tool routing approaches well before MCP came along because we needed to route to tools and figure out a structure for doing that. It is nice that we don't have to do that anymore and now we can just hand that off to mcp. But we learned a lot about the realities of how you do that effectively at that point in time. We're building kind of multi step planning things ourselves before harnesses came along for doing that kind of thing. Again we've sort of started to switch those out as the market has provided standards for those sorts of things. But it's also helped clarify a lot of the stuff that allows us to make this successful. LEN itself is fundamentally a colleague productivity tool and it's very general. It's also a very low cost experimentation space where anyone can try out their ideas, we can see what works and pick up the best and go and do something with them. But in doing that and listening fundamentally to what the organization wanted, it led us to build out document processing capabilities and translation and summarization at very early stages and more recently connect us to data of various kinds and connect us into the office suite and all these sorts of layers of new tools and capabilities that helped expand the number of use cases that we could take on. Regardless of whether LEN AI continues to exist. By the way, this is Len right here over my shoulder. I don't think many companies have a soft toy of their internal tool. Regardless of whether this thing sort of continues to live forever. All of the stuff we've built to connect into the rest of the organization, all the scaffolding we need for the industrialized use cases, productivity and efficiency plays we're running in the OPS organization. If we buy a commercial tool, what we would plug into it to extend its capabilities into our environment. So it's been, you know, a great learning experience and it's built us this sort of ecosystem of the kit of parts that we need to go and do this at scale and in the places where we can really sort of, you know, pin value to it very directly. We've. We continue to find that even if we, we lag by a few months as to what's out there commercially in the Market, we only lag by a few months and it costs a lot less to do it this way. And so for instance, our cowork equivalent is a home built version that we have in pilot now. And we'll, we'll start to roll out to the organization pretty soon. It's taught us a lot about writing great agent harnesses. And that issue is not just one that relates to a general colleague productivity tool. It's really important as we start to put agents into the heart of some of our operational processes as well. Maybe segueing to one part of the business we haven't talked about, which is sort of the insurance and risk management side of the business. What's the role of AI there going to be? And again, we like to tease out the contrarian in our guests. So are there things you think about how those industries are going to reshape over the next three, four, five years that you think are maybe not well understood yet? Yeah. So the insurance space in particular is a very document heavy ecosystem. There's a lot of interchange of information that happens via the medium of PDF and there's an enormous amount of time cost and delay that's caused by the realities of that. So big, big push on document ingestion and systematically capturing, parsing, filing and then routing a lot of that information based on capabilities for reading that stuff that are dramatically better than they were even a couple of years ago. So that is a big play. We see good gains from that. I know lots of people are looking at this sort of space, but it's very fundamental to the way that we operate today. So I expect faster, lower cost, more insight being gathered, because the cost of gathering the marginal piece of insight from the stuff that flows through is all going to help us be a stronger advisor to our clients to have a deeper understanding of the market and to serve them more effectively. So on that side, that's probably the biggest single play. But there are lots of ways in that business and in others that we are transforming the way that our clients interact with us. The insight we have and the data that we have to make it just lower friction, easier to get the stuff at their fingertips, easier to do simple transactions in ways that are lower cost for them. And then probably the third piece focusing on that space specifically would be arming our risk advisors with more insight into what's happening in the marketplace and the ability to build more effective and more strategic coverage programs for our clients because of the data that we're able to bring to that conversation, because of the depth of Insight into the market and all the things we talked about around knowledge management and data allow the corporate enterprise wide scale that we have to turn up at the front line much more systematically and consistently. How do we measure and justify the ROI on AI? We intuitively can feel it. But how do you, and I'm curious for you specifically, but maybe even just more generally, what's the case that one makes to the CEO, the board around continuing investment in AI, given the amount that one can actually spend on these things? If you keep it cheap, it's a lot easier to show an ROI or not have it be a big issue. And one of the things that we've done with the strategy we've followed is it is a dramatically lower cost strategy than especially on the copy productivity side, them buying into tools, you know, that come with very large upfront commitments and big bills which almost force you to start pushing targets down to people to help cover the cost. Because you can't pin it to any one thing. It is general productivity. And the only way you can really make that fly is by making the cost problem someone else's. We haven't had to do that. And I think that's been enormously useful with this exciting new technology coming in with all the excitement and none of the fear. So that's certainly one part of it. But there are different plays that we have here that's productivity hard to pin down. But be smart about the way you're building and you can do it dramatically more cheaply than you think. From a client transformation perspective, we see revenue growth in a number of our products as we've taken them through an AI transformation just because the experience is so much better and more compelling and the value that clients can get from that is much higher. We have a product called ADA in our immersive business that takes a huge amount of insight about managing talent and dealing with rules and regulations in different countries around the world based on a huge database of documents that we've, you know, for ourselves and from other sources. That was incredibly difficult to navigate, but now you can just ask and we will find you the right stuff and bring it back to you in a form that's most useful to you. It's a massively better value proposition that's helping drive revenue growth within my organization. On the operation side, I think there's another point here though, which is I saw someone, I don't know who I can give credit to for this, but I saw a LinkedIn post from someone that resonated with me. Increasingly the cost of the meeting to discuss what you're going to build is more than the cost of building the thing. And one of the things that's even higher cost in a lot of companies is the months of negotiation to get budget to even start thinking about doing anything at all. The way we think about roi, roi, the way we think about funding these things is going to have to change. Because if you take the beginning of Len AI, there was no conversation about that. It was done over a weekend and it was in pilot and conversation wasn't should we do this thing? This thing was done. There was zero cost to doing that. In reality, conversation was are we going to like how the cost grows as people use it? But the cost is proportionate to the use and to the value. So it's a very different place to be if you can knock out production grade applications in weeks and sometimes days. And we have done. The multi month negotiation process at capex is insane. And I think that there are pretty fundamental changes obviously happening in how software is being built. There need to be commensurate changes in how organizations think about where software should be built, by whom, under what constraints, how the money flows for doing this. And so a big topic for us is thinking about how we reorganize our tech team and match it up against the business in a fundamentally different way to reflect that. Well, there's a lot more I want to talk about, but I want to make sure we get to our lightning round before we run out of time. So we'll jump into our lightning round and we're looking for sort of like the one tweet version of each question, which admittedly is a little difficult on some of these. Does that sound good? Give it a go. Okay, awesome. So to kick off, how do you think companies should measure the success of a CIO in today's AI era? That's a pretty tough one to start off with. I think most fundamental is the ability to enable out innovation of your competitors. If you're talking to a new CIO in another industry, so not a personal competitor and you know, he had just started his new job, he or she, and they're in and they say, hey Paul, we can do three projects in our first year. Like what are the three AI projects you would recommend someone start with? Oh, I think there's a problem right there because I think those, that implies big projects, you need to do dozens of projects. So I think my main advice would be break those three into, you know, 20 each and do them sequentially one at a time. We've got to break these big projects. They are designed to fail. You can't show momentum that way. It doesn't fit with what people want to see and they create a dynamic that accumulates risk and drag and increases the odds of it adds cost and risk at the same time. What's a book you've read that's had a big impact on you and why it doesn't have to be work related. I get an astonishing little time to read these days and I mostly read fiction when I do. My favorite book is Titus Groan because it's so wonderfully. The characters are so wonderfully well drawn and I do find myself thinking about that. But I don't know what relevance it has to the rest of my life other than it describes this sort of ancient, fantastical, rambling castle which perhaps is quite like most people's legacy IT systems. What is the best way as a CIO that you can just stay relevant? On the latest in AI and today's enterprise AI trends? I'm a big believer. You have to do it. You have to try it. You haven't vibe coded. That is a problem. I have always liked to have a few side projects going at any point in time and I code probably weekly if not more. I like to try things out and feel it hands on. And I think you've got to touch and feel it to really understand it and get a sense for its potential. Completely agree. And maybe one final question. What do you believe will be true about AI impact on the world that most people listening would still consider science fiction? That is a big question. I'm not sure how to answer that. There are so many possible futures here. I think that what is what I'm reasonably sure of is that by the time my kids are my age they'll be doing jobs that didn't exist right now and we've never thought of. And we will find we have a capacity to innovate and come up with new things to do that will surprise us all. I completely agree and that's why I get quite frustrated by the conversations around there being no more work. I think many jobs we do today, including in some ways the job I do like. If you had 100 years ago asked would this job exist? The answer is no. And human ingenuity will never be defeated. So I think there could be a messy period where there's work transformation happening. But. But I'm very optimistic and I agree with you. I think the next generation will be doing things that will be hard for us to describe. And imagine today. Yep. I'm not sure AI is the most dangerous thing in the world at the moment. Yeah, exactly, exactly, exactly. Awesome, Paul. Well, thanks a lot for joining us. Really excellent conversation. I could have kept going for a few more hours, and I love some of the specific examples you shared. And really excited to get this out to our listeners. Excellent. Well, thank you, Sean. That was Paul Bess, senior vice president and Chief information and Operations Officer at Marsh. Thanks for listening to Enterprise AI Innovators. I'm Sam Motamity, the general partner at Greylock Partners. And I'm Evan Reiser, the founder and CEO of Abnormal AI. Please be sure to subscribe so you never miss an episode. Learn more about Enterprise AI transformation at enterprisesoftware blog. This show is produced by Abnormal Studios. We'll see you next time.