Shifting From AI Experimentation to Enterprise Transformation

Speaker A: CIOs have long been expected to be the technical gurus of the enterprise in service to the business. But today they're visionaries, evangelists and change agents who are the leaders in tech successfully challenging the status quo and driving transformational change across the business. This is techfluential, a podcast series hosted by custom content from WSJ in Deloitte. I'm, um, Brian Kimenetsky, senior Editor custom content from WSJ with our host, Lou Di Lorenzo, principal at Deloitte Consulting llp. Lou, it's great to talk with you again.

Speaker B: You too, Brian. Good to see you.

Speaker A: We're into the ninth episode of techfluential, but many listeners, they are new to the show. Can you explain what it means to be techfluential and why it matters?

Speaker B: There's never been a more important time to explore the role of leadership and technology, Brian, and that's what techfluential is about. When I think of techfluential leaders, it's those that are helping make critical decisions on how to drive value through the use of technology. Technology is a way to solve problems. Technology is a way to help better meet customer expectations. And so those that look at it that way and can get the rest of the organization rallied around that, that's what techfluential means to me. And so I'm excited that we've got a chance to bring in some incredible thought leaders in this space to help make it real for our listeners.

Speaker A: What is the leadership challenge that you are exploring in this episode?

Speaker B: Resisting the temptation to focus on efficiency versus understanding that technology, in particular AI, which is dominating the discussion, can be equally, if not more powerful in Dr. Customer experience, which ultimately is going to lead to better commercial results.

Speaker A: When you think about the deployment of any technology, but especially AI, and that's obviously the moment that we're in. There's so many trapdoors in the process and contradictions. You need to move quickly, but you also have to be deliberate as a leader. How do you manage those forces that are often in direct opposition?

Speaker B: Yeah, Brian, I think any technology, and obviously we're talking about AI. Uh, if you're not sure what you're doing and why you're doing it, you're probably not doing it right. And so focusing on the value statement, what is the problem that we're trying to solve? How do we reimagine work versus automating a current process that is optimized to the best of our ability? Anchoring on some of those first principles around what's the Strategy, what's the objective? What's the business value we expect to get? What are the outcomes of this particular workflow or process? And then how do we thoughtfully use technology to bring it to life? I think those that can focus on that and communicate it relatively simply are those that are going to win at this more than others that can are going to have a harder time.

Speaker A: So with that in mind, who's joining today's episode?

Speaker B: Well, we're thrilled to welcome Liz Santoni, who's the Executive Vice President and Chief Customer Experience Officer from Cisco. Liz is also on the board at Workday and Mercedes Benz. And then Sam Deshpande, who is a board member at finra, was the former CIO at Humana and also an executive at Capital One. So just a wealth of operating and board service. I'm thrilled to have both of them on our show.

Speaker A: And so with that, here's Lou DiLorenza with Liz Santoni and Sam Deshpande.

Speaker B: Liz, welcome. Sam, great to see you. Thanks for being here.

Speaker C: Thank you, Lu.

Speaker D: It is so great to be here.

Speaker B: I'm excited to talk to both of you about the challenge of driving innovation across the enterprise at scale while managing the complex new risks that are inherent in that environment. And so, Liz, I'd love to start with you. When you've tried to move AI into live workflows at actual operating scale, not a proof of concept, but real heart of the business operations, what was the first thing that broke and what did it force you to rethink?

Speaker C: I've been at Cisco for 25 plus years, and when I think about enterprise software, what did we do? We deployed it, we trained people on it, and adoption follows. That mental model completely breaks with AI because it's just not another tool your team uses, because it comes together as the system that's exercising judgment on your behalf, on the user's behalf. And so the people closest to that are, uh, the users. They're the best signal on where the system is right, where it's wrong. So treating the users as part of our tech stack, I think that's a completely different mental model shift that we had to go through and learn through that. I think the other thing that really broke for us is our assumption about workflow design. So it really forced us a rethink on what are some of the first principles? What is this workflow actually trying to do for the customer? What does the customer really need? At the end of the day, the technology is the easy part. Redesigning workflows, who owns what, what gets eliminated, what gets rebuilt, that's the real work. And we weren't ready for it. I don't think most organizations are really ready for it because it's not built into the scope of what they're thinking about as part of an AI project.

Speaker B: Yeah, Liz. I love the turn of phrase of having the employees be part of the tech stack. Sam, as we think about your deep experience in regulated industries, how do you drive deployment, driving deployment at scale, bringing these new ways of working to life, does something feel different to you in this space?

Speaker D: The interesting thing is, if you are having this conversation with someone who hasn't lived in a regulated industry, their reaction might be there are complexities in every single industry. M and that is true in a regulated industry like healthcare or banking. You have all those same business complexities, but then on top of them, you have a whole bunch of laws and regulations at the federal level, at the state level, sometimes at the local level. And how do you make sure that the AI system can actually deal with all of those is part of the challenge. The good news is that AI has the potential of, uh, making all these complex requirements and processes just work better and actually in doing that, improve the economics of the overall business. Just given the regulatory requirements and the compliance with them. You have to really think about all the things that existed before, and those things were typically rules and rules engines. So does the AI, uh, replace the rules engine? Does the AI, uh work with the rules engine? You really need to spend time thinking that through because one of the critical requirements is that you be able to handle, not just in general, all the requirements, but every single corner case needs to be dealt with.

Speaker B: LIZ Knowing the scope of your deployments as you're taking AI and deploying it across the customer experience organization before saying, okay, we're ready to roll an agentic capability in a customer facing workflow. What were the things on your must have list before you gave the okay to move forward?

Speaker C: Lou these are live customer networks, live environments. I'll, um, go back to what I mentioned earlier about the underlying workflow itself. If humans really can't agree on what the right answer is in a given scenario, the agent is not going to be able to resolve that ambiguity. We have to solve for that. What the agent does is it just makes it faster and more visible. The second thing is, and this gets even more exacerbated in, like regulated environments, do we have real human oversight, not just a checkbox. It's not just the agent following the decision tree. Right? But where does it deflect to a human and does the human have enough context to intervene correctly? So you might have the best things laid out like governance and monitoring on paper. Those don't count if you're not actioning it. And the other one that we looked at is just measuring the baseline. I mean you can think about these as core hygiene things that you need to do. You can't know if AI is improving something you've never measured. That's just a very costly pilot. And so that is almost like, I would say it's like a time to relief whether I'm resolving an issue on our end where the customer's hair is on fire and is that the time to relieve. And if AI is really working, the people in there should feel the difference. You have to experience that. And what would make me say not yet is any workflow where the consequence of a wrong decision was irreversible. These are live network live deployments. Or the other one is where we couldn't explain the explainability in terms of the agent's reasoning to the customer if they asked. It isn't just a governance requirement for us, it's a trust requirement. So for us, explainability becomes that governance requirement and it's a uh, trust requirement as well. Because for us, trust is that whole game.

Speaker B: So much of what we're talking about hinges on different leadership perspectives and elements around that. So Sam, where do you think leaders underestimate some of these considerations around uh, monitoring or managing after they're live in the environment?

Speaker D: So what I found is that even experienced leaders can underestimate how a well performing model can suddenly flip from good to bad with a seemingly unrelated change happening somewhere in the connected ecosystem of data and applications. The challenge often isn't that you suddenly go to a big problem. You normally have early signals and you only end up with big problems if you ignore the early signals or don't act on them. So one of the critical requirements is that you need to have a really good monitoring system for the outputs. And what you need to be able to monitor are, uh, one, are there outliers happening that you haven't experienced before? Or two, are you seeing a meaningful shift in the distribution itself? And you need to be able to do this for every single model in production. In companies that get it right, they will not only have all the right groups working together to create the solutions, but they'll have one accountable executive whose responsibility it is to make sure that that system gets built and it remains healthy.

Speaker B: So important.

Speaker C: Hey Lou, are you okay? If I Just add to something, please.

Speaker B: Liz?

Speaker C: Yeah, it almost felt like a mind meld here with Sam. So I felt like I needed to add to that because. So spot on to what Sam just said because this is the thing that we learned the hard way. Because when you look at all of governance frameworks, they're built for the M time when you're actually deploying something. But what we learned is something that we had deployed like six, seven, eight months ago in a different context. When things change, it could be, I don't know, a product change, a policy change. Whatever the change is, the model doesn't know the world has moved. Your monitoring has to catch that. And like Sam said, which I want to emphasize is your monitoring only works if you act on it. And what our instinct was as part of a large organization is we want to avoid the risk because this is customer facing. We wanted to wait for certainty before course correcting it. But the problem is with AI systems, that's just too late. And so what we have to pivot to is have monitoring that services issues early and uh, a leadership culture that gives you the air cover to move on imperfect information. And that's a huge mental model shift for large enterprise organizations.

Speaker B: Fabulous. You both have very significant board of director experiences. We know that boards are asking, management teams are asking, where are we on AI? Are we behind? Are we doing all the things that we should be doing? My view, that's a little bit of a, uh, reactive question. Sam, what are the questions that you think boards should be asking? Maybe looking down the field a little bit as it relates to AI deployment, as we think about areas like risk and speed and long term value.

Speaker D: So, lou, you are 100% right. I can't imagine there's a single board out there. I can't imagine there's a single CEO out there who's not asking the organization, what are we doing related to AI? The places where I feel a board can really focus in making sure that the organization can go from experimentation to actually transforming the organization using the AI tools is one, does the organization have a digital foundation in place? Because otherwise you'll just end up with a whole bunch of AI tools which may or may not actually work well within the overall system. Number two is all related to talent. If you are successful, while AI is complex, you need to build up the internal understanding with your talent and the expertise of adopting these tools. But you can't do it alone. You need to have a handful of strategic partners that you have chosen carefully who work with you. This is not an area that you outsource completely, because if you do, you will not be able to evolve or have the domain expertise to even manage the solutions that have been built. And it is just unrealistic for any organization to think that they can do it all on their own. And number three is, in addition to just thinking about use cases, has the organization really thought about how to transform the entirety of the business and operations using AI over the next several years? And is there a plan? The plan will keep on changing as new tools are developed, or you learn something along the way, or your priorities change. But the fact that you have a plan will allow you to make sure that you're directing your investments to the right areas, the ones with the biggest opportunity, and that the solutions you're building over time actually connect to each other. Because the alternative is that you'll be looking back and you could be drowning in a whole bunch of point solutions that have been adopted over time.

Speaker B: Yeah, great insights there, Sam. Liz, maybe take us a little bit into those conversations. One of the things I can imagine is taking place at the board level is understanding what's a one way door and what's not.

Speaker C: Yeah, I like how you started this out, Lou. And a lot of what Sam said resonated as well. This question often is, geez, especially with AI, the distinction between guiding and overseeing matters. Here, overseeing is kind of backward looking. Guiding means that me as on the board, we're asking the forward looking questions before the answers become obvious. So when you look at what the LLMs have done is in many ways democratized access to building capabilities to take an idea and turn that into value and take it to market. So your competition is not just that large enterprise out there. It's anybody out there with the great ideas and the ambition and with a little bit of investment. So how do you also move at a speed that this moment requires, especially when you are a large enterprise? And then there's this customer question which I think is underrepresented in many of these AI conversations because a lot of it, when we talk about roi, just immediately goes towards. It's about efficiency, it's about cost, it's about speed. My question is how does AI make the experience better where customers actually feel the difference? I do think that this is an outcome that matters quite a bit. It's harder to quantify compared to the efficiency and the cost. But I think it's one of the hardest things to keep in focus because many times the conversations are dominated by the tech stack and the cost and the Efficiency and not as much on, um, the experience.

Speaker D: One of the things you talked about, Liz, which I love, is the speed in this particular case, because we have all seen many transformations in the last several decades. But even if you compare to the mobile wave or the Internet before that, or some of the transformations before that, it just feels like AI is moving at a completely different clock speed. And where the boards have such an important role to play is normally governance comes back to investment decisions, it comes back to operational risks. In this particular case, it almost feels like there's a huge strategic risk sitting out there, which is, are you moving fast enough with a clear purpose to take on the new AI wave so that you can still compete in the future?

Speaker C: Yeah. So spot on. In those other tech waves, we built the infrastructure stack and then we kind of left it alone for like two, three years. We didn't go back and re architect it. This is. The underlying infrastructure is changing. I mean, you're swapping out your models for new reasoning models. This is a completely different speed at which this is moving.

Speaker B: I do think there's a fundamentally different point of view around the basis of competition here. Typically you would focus on cost or focus on efficiency or whatever. You sort of pick a lane. But I do think the velocity of the capabilities coming to life, as well as the nature of them, allows for companies to pursue simultaneous competitive advantages in ways that I would say were previously prohibitive and would lead to a conversation about a lack of focus. I want to thank you both. Tremendous insights on one of the more important topics as we think about technology and how to drive value from it as we close the program. I ask everybody the same question. Liz, in your mind, what does it mean to be a techfluential leader?

Speaker C: I thought about this. Uh, I do think being in this moment means being willing to close the gap between what you say in a keynote and what you say in an operating review. And I do think a lot of leaders live in the gap because there is a pressure to project this confidence about AI. What is helpful in my experience is being specific, being honest about things. I am not afraid to call out that, hey, look, AI won't fix a broken workflow, it will accelerate it. For me, being techfluential is when I look at my support team and I go, they're not just case processors anymore, they're actually advisors. Because we are measuring success by whether the person using the system actually feels the difference. Not by just, oh, I deployed it. 98% of my team is using these tools. So I tell Everybody get close to the work by doing the work and being honest about what you find. Influence that's built on being honest, being candid, that's what compounds influence that's built on performance, theater and positioning doesn't.

Speaker B: Well said, Sam. We'll give you the last word. Take it away.

Speaker D: I feel that we are sitting at a point in time that if you look back in a few years, this might turn out to be a golden age of transformation using AI. And a question any tech leader could ask is, if you are looking back personally, what impact did you have? And I feel like a personal commitment to be techfluential. A personal commitment, first of all, to spend the time, to really understand where the strategic levers in the business lie and to spend the time with your peers to help them imagine how the technology can fundamentally allow them to rethink what they are offering to their customers. If you do those two things, I feel like you can create a really strong handshake and help shape the future.

Speaker B: I want to thank you both. Tremendous insights on one of the more important topics as we think about technology and how to drive value from it.

Speaker C: My pleasure, Sam.

Speaker B: Thanks for being on the program.

Speaker D: Great to be here,

Speaker A: Lou. Thanks. That was a really dynamic conversation. How do listeners take the insights from your conversation here and others in the series and apply it to the specific nature of their business context?

Speaker B: Yeah, I appreciate the question, Brian. I think those that have had a chance to be on the journey with us are going to find a number of things in each of these episodes that get to that idea of how do you lead in a world where technology is the business, whether you sell it or not? How do you help people understand that their jobs are going to be delivered differently through the thoughtful use of technology? And that's exciting, but you got to be good at telling stories in order to do that and making decisions and understanding. You're not going to have perfect information to do that. And so anyway, I like to think there's a lot in here across the episodes and just the tremendous wisdom and experience to provide some insights for our listeners.

Speaker A: Yeah, I agree, Lou. There have been so many valuable takeaways from these conversations and I'm looking forward to our next episode where we'll pull together the themes and thought leadership of our first season of techfluential. And I'm pleased to say as well that we'll be back for a second season and we'll continue to explore the changing nature of of technology leadership and what it means to be techfluential. Inside the enterprise. Thanks to everyone for listening.

Speaker E: The views and opinions expressed by podcast speakers and guests are solely their own and do not reflect the opinions of Deloitte. This podcast provides general and educational information only and is not intended to constitute advice or services of any kind. For additional information about Deloitte, go to deloitte.com us about the information contained in this podcast is for informational purposes only. This content was created by custom content from WSJ, a, uh, unit of the Wall Street Journal. Advertising.