Are We Measuring AI ROI All Wrong?

Speaker A: I'm Cody Stack and this is 0100, the unboring supply chain podcast. Right now, enterprise leaders are staring down a massive dilemma. AI bills are exploding with stories like Uber burning through its annual token budget in just four months. Yet finance leaders are struggling to see where any of that money is actually coming back to the business. It leaves companies wondering, is AI a true savings engine or just an expensive cost sink? But are we making a fundamental mistake by treating IT like just another IT project, getting so trapped in chasing quick savings that we miss the bigger enterprise transformation? Today, we're unpacking how to rethink the true value of AI. Joining me are VP of Research, Garant John and Senior Director of Research and Advisory Justin Gilbeau. Garant, Justin, welcome.

Speaker B: Thanks Cody. Great to be here.

Speaker C: Yeah, looking forward to this. Thanks, Cody.

Speaker A: Let's talk about the boards and the demands on the boards. Every board is demanding massive AI productivity gains, while CFOs are hunting for headcount reductions to justify the ballooning tech bills. But taking a step back, are we making a fundamental mistake by treating AI like a standard IT project that has to pay for itself immediately, rather than a general purpose technology like the steam engine or like electricity? Garrett, why don't you get us started?

Speaker B: Well, Cody, I think there are plenty of examples of standard IT projects. I'm thinking like multi year erp, uh, implementations for instance, that cost millions of dollars and take years to pay. But I think you're right. There is clearly kind of intense pressure on publicly listed company boards and particularly CFOs, to justify the growing spending on AI. But having said that, I think it's wrong to expect rapid ROI at a time when people are really just getting used to this technology and learning how it can boost their personal productivity. And it's certainly a mistake to prematurely cut headcount because of AI investments. In fact, in the past couple of years, we've seen a number of companies, Klarna, IBM, Forge. They've laid off hundreds of staff in different areas and then had to rehire them a few months later when the AI&M the automation proves to be ineffective. Now, let's be clear, AI will need to pay its way and IT is going to reduce the size of supply chain functions in terms of headcount in the coming years. But I think leaders need to get the use cases, the tech, the processes, the governance, all of those things right before that can realistically happen.

Speaker C: Yeah, I think that's right, Garant. Uh, in the calls that we've had recently with 0100 clients, the company is getting traction or not treating AI as another software development. They're treating it as a redesign of how work gets done around a few critical decisions. If you take a broken workflow, weak data, and muddy decision rights, and then drop AI on top, you're not going to get any meaningful transformation. You're to get a more expensive version of the same dysfunction. So, yeah, boards and CFOs are right to press on the economics, but the right question is not did this tool pay for itself in 90 days? It's did we remove a business constraint that matters? You know, if a company can make better sourcing decisions faster, absorb more volatility without adding additional headcount, or even compress planning cycle times in a way that improves service or cash, you know, that's a real value story, even if it doesn't first show up as a clean labor item.

Speaker A: That's a great point, Justin. We've talked in the past about how if you drop AI on top of things, it just scales the dysfunction if you already have dysfunction there. So I really like that point. So let's dig into this. Let's talk about the cost end of this equation. The conversation around AI sticker shock is everywhere right now. In May, it was reported that Uber blew through its entire 2026 token budget in just four months. And companies like Meta and Amazon are now trying to curb staff AI usage due to token costs. How are supply chain leaders getting caught off guard by these hidden operational costs? And how do you govern token spend without just completely stalling innovation?

Speaker C: Yeah, it's like we're catching up to the technology right in front of us. I mean, this definitely came up recently during some 0, 100 community roundtables. What's catching people off guard is that the AI bill is not just the model itself. It's the model usage, bad prompt habits, the wrong model for the wrong job, duplicative experiments, and then all of the hidden cleanup work that has to happen when you have poor pilots. One of the more interesting things we heard recently was that some of the mature operators are already managing this like a utility. So they're metering usage, they're defaulting people to cheaper models, they're creating clearer rules for when a premium model is actually worth it. And that matters because if you don't make those economics visible, people treat every task like it deserves the most expensive reasoning stack available. We found that the trick is not to shut experimentation down. It's put enough structure around it that people learn good habits early. So in Practice that means a tiered model strategy where you have things like visibility into cost per workflow and better guidance on where you want experimentation versus where you want disciplined execution with a clear outcome. Otherwise you get more of this innovation theater on one side and then you get the financial backlash that'll follow it.

Speaker B: Yeah, I think that's absolutely right. And um, one thing I'm absolutely sure about is that we're going to see many more companies having to curb staff usage of LLMs in the coming months as their costs balloon. And I think one of the reasons is many companies are only just now beginning to give employees that access to some models other than just Microsoft copilot. So uh, that seems to me to be inevitable. I think you're absolutely right, Justin. I think, you know, a couple of things need to be done here are firstly to make those costs much more visible to people. I would bet that most staff, and I probably include myself in this, have absolutely no idea what it costs each time they use a particular LLM to help them in their daily work. But more importantly, I think companies have a duty to provide guidance to their staff on when to use different models. I'm thinking particularly there about the very latest models or versions of things like Claude or ChatGPT. Again, I don't think most staff would really understand at the this point which is the right kind of model, more sophisticated, more expensive or older and a bit cheaper, uh, to use for the tasks they happen to be working on. So much greater clarity and visibility around both of those areas I think are needed to help make people make the right choices in how they work.

Speaker A: As the owner of Zero 100's AI budget, I love to hear that you have no idea how much it's costing. No. And I think you both hit some really good points, which is there's a learning curve for using this both like at a personal level and then building on top of it. Right. And building that governance around it. I uh, like the point of making costs visible to people. And yeah, you can drive people down to cheaper models when they don't need it, but you have to show them how to get there and they're going to learn on their own too. But building in that structure can really help. So we've talked a little bit about the cost part of it. Let's talk about the other side of the equation. When companies do try to put a clean financial figure on AI savings, it can get messy really fast. If a team claims an AI implementation is paying for itself, but the proof is mostly faster Cycle times, fewer manual touches or better decisions. Is that real cash on the P and L or are we still relying on faith based ROI?

Speaker B: Well, I think finance and um, CFOs are always going to want, um, to see hard savings that they can show on the bottom line. But the reality, as you say, is that, uh, the business benefits of AI in supply chain and operations are much greater than cost reduction alone. Now, I cover the sourcing domain from a research standpoint and this week I've been judging entries for a leading digital procurement awards program and you're certainly seeing hard savings numbers being quoted in some of those entries. For example, one leading food and beverage company says its AI initiative has generated over 100 million euros in cost savings, around 10 million euros in free cash flow so far. But at the same time it talks about taking out 300,000 hours annually, saving its buyers 45 minutes per day on average, speeding up cost modeling and forecasting by 80%, cutting out 90% of the effort required by business stakeholders to get what they need, and accelerating execution by 25%. So I think it's unfair to describe those kind of things as faith based roi. To me they're kind of real business benefits and they need to be properly accounted for. When companies are assessing the value of

Speaker C: their AI investments, CFOs and boards are always going to press for more clarity on the roi. But our ROI needs to catch up to what the models are actually capable of today. I think this phrase around faith based ROI is only really useful when teams are being sloppy about the outcomes that are actually being produced. Within the supply chain context. You're seeing better decisions, shorter cycle times, fewer manual interventions. These are not fake benefits. You know, in supply chain, these are often the mechanisms through which we eventually see cash, cost and service improve. The problem is that companies are better oftentimes accounting labor than they are valuing things like decision speed or constraint removal if you shorten a cycle time for sourcing or improve supplier responsiveness if you're able to make planning decisions earlier with better context. This may not be a neat headcount story, but this is true ROI that the business will experience. I'd say the mature approach we're seeing seeing amongst leaders is stop forcing every AI use case into an FTE reduction template. A lot of the most important value is in thoroughput quality of execution and the ability to absorb more complexity without scaling the organization at the same rate.

Speaker A: That's a great point, Justin. And I know you've been spending a lot of time on this. You've been building out a framework around how supply chain scorecards need to evolve, moving from basic efficiency to transformational roi. So if operations leaders shouldn't just be looking at headcount or direct spend, what are those metrics that actually prove AI is driving real enterprise value? What should they really be looking at? What should be on their scorecards?

Speaker C: Yeah, Cody, one of the challenges I saw members face as they move from more human activities to more autonomous activities is that their metrics don't always reflect that new operating reality. You know, for example, when I've led planning teams in the past, the expectation was that the planner was taking the forecast and then using their knowledge and understanding, their suppliers and insight to really judge that process. Whereas modern companies with agentic systems, they're increasingly measuring their planners against not touching the plan or adjusting it and really trusting the agentic output. So we mapped metrics along a continuum from baseline metrics where you ask things like is the process healthy? To more advanced and transformational metrics which really reflect if the supply chain is helping the enterprise really as a partner to make better decisions faster. These transformational metrics, you know, they're measuring things like decision latency, the percentage of low value planner or analyst work that you've been able to automate, and the percentage of time your teams are spending on administrative work versus truly value added activities like quarterly business reviews, negotiations, or more critical areas that require a human's judgment. And so as companies move up this metric process, these transformational metrics don't generally show up in things like a weekly staff meeting. But leaders that are increasingly adopting them are seeing the benefit of focusing on what will truly move the needle of the modern supply chain. We have to make sure that our metrics actually catch up, uh, to the technology itself.

Speaker A: I think that's the forward looking perspective, at least in the 0100 view. But Garen, to your point earlier, I don't want to discount the fact that some companies are seeing real financial payback on the P and L right now as well. In fact, Garant, you had a recent field note that highlights high volume repeatable tasks and cost intelligence like should cost modeling over historical price anchors. What are some concrete examples where AI is genuinely moving the needle on spend today?

Speaker B: Well, if I take the food and beverage company example I mentioned a little earlier, the procurement team there highlights 5% savings through the deployment of a contract and analysis agent, 3% direct material savings from a, uh, cost modeling agent. Those are just a couple of specific examples. Elsewhere I think we've seen companies Like Walmart for example, saving on average about 3% by sourcing previously unmanaged tail spend using autonomous agents. Perhaps more significantly though, one of our industrial manufacturing members has saved over $300 million in nine months by using AI to mine commodity cost supplier, ah, forecasted demand data, uh, and then use that to present saving recommendations to their buyers. Now that didn't happen overnight. In fact, they spent more than five years building the data foundation to support this type of savings discovery. So to me, that just kind of underlines the fact that business leaders really do need to take the long view on AI rather than just obsessing about quarterly results.

Speaker C: Yeah, I mean, the places where the math looks most credible are the places with high frequency work, repeatable judgment, and enough contextual data to support better decisions. That's why you see companies starting with things like sourcing cost intelligence and contract analysis. The broader pattern is pretty intuitive for leaders. AI does well where it helps you process more information than a human team can while realistically synthesizing fast enough and turning that recommendation or action inside to an actual workflow. I think some of the challenges around AI ROI that we've seen recently is that companies have bolted on some of these solutions without really adjusting the workflow to be able to shift those decisions from humans to machines. When you bolt on AI without actually redeveloping and building the workflow, you end up scaling up a process that is still dependent on manual intervention. And that's where you're not going to see those ROI pieces come into play. But again, this is why your should cost style, use cases, contract review, doing things like tailspin sourcing. These are showing up as concrete value pools that companies can take advantage of. I also think there's an important nuance here. You know, the real winners, they're not just automating a task, they're redesigning that workflow around the task. And that's the idea of the 0100 power thread where we're looking at these solutions at an end to end. That's where you're going to move from isolated savings to these compounding returns.

Speaker A: Justin, you started talking about what the losers are doing too. Let's talk about the flip side. Where are you both seeing companies fall into the hype trap right now? What are the AI, uh, investments where teams look at the business case and think the mess just doesn't add up.

Speaker C: Yeah, a lot of companies can describe an impressive future state agent architecture, but if the data isn't trustworthy, if you haven't standardized your process you haven't made clear things like governance and escalation paths, then the economics get shaky pretty quick. So you're paying for this intelligence that the organization can't reliably act upon. You've all of a sudden gotten the Ferrari and it's still sitting in the garage because you're not taking full advantage of it. So I'd also be cautious in any area where the value case depends on avoided disaster. But the response process is still weak. So we know that we want to be able to respond to risks faster, be able to take advantage of customer signals that will help improve sales or protect margin. But if you're not doing the work of setting up that governance and process and those decision rights, then you've bought a more expensive alerting layer. One thing I've said to clients recently is at the end of the day, these faster signals, better availability to data is just telling you you missed out an opportunity if your supply chain network is not set up to respond. So really thinking through those challenges proactively and not relying on human heroics while you have that Ferrari, that beautiful model still sitting in the garage.

Speaker B: Yeah, completely agree, Justin. And I think supplier risk is a really good example. Don't get me wrong, I'm a big advocate of using AI based risk sens to monitor events across your global supply base and um, flag potential disruptions, particularly when we're talking about critical materials, sole source suppliers, regulated categories, or indeed suppliers that are tied to revenue critical products. But I can tell you, having worked for one of those software vendors in this space before joining 0100, that uh, the business case doesn't always add up. Now that's partly because many of the alerts these systems generate aren't relevant or precise enough. And it's partly because, as you rightly say, procurement or supply chain teams often don't have the processes or the resources even in place to be able to actually take action on the back of those alerts when they come through and when they are relevant. So if you're not careful, you can end up spending hundreds of thousands or even millions of dollars on the technology, that sort of fancy Ferrari you mentioned, without actually seeing any real tangible benefits. The bottom line here is that, uh, avoided risk value doesn't just kind of magically show up in the P and L, as many organizations have discovered, to their cost.

Speaker A: We've talked a bunch about the costs, we've talked about the roi. Let's talk about how it impacts. Justin, you were talking about storytelling before and let's talk about how it actually impacts your implementation strategy. So the mainstream narrative says start with quick wins, right? Then you can demonstrate that ROI quickly, you can demonstrate some of that savings. But is there a danger that obsessing over tailspend and intake traps companies in minor savings while bigger structural opportunities go untucked? Some of the power threads that you were talking about, when do quick wins become a distraction?

Speaker B: Well, I think quick wins are important. Improving the value and gaining trust among leadership. Uh, and tail spend is a kind of classic example of that because you're talking about essentially low hanging fruit areas or categories that it's much less risky to point AI at as for something like intake orchestration or easier, faster buying routes for business stakeholders. If we want to put it in plainer language. To me, that's actually kind of vital in freeing up time for procurement teams so they can address some of those bigger, more complex opportunities elsewhere. And Most of the CPOs I talk to, uh, are very focused on those kinds of bigger opportunities. But having said that, in the short term they need to get comfortable with AI based capabilities and they need to be able to highlight the benefits of those. So naturally having some quick wins in a bigger sort of three year or five year digital roadmap is really, really essential. I think quick wins are only a distraction if you don't have this mid to long term vision and if you aren't focused on sort of really transforming the operating model in terms of how your function works or is reimagined alongside those kind of AI investments. So that really kind of speaks to the partly to the kind of power threads area that Justin mentioned earlier.

Speaker C: Uh, yeah, I agree wholeheartedly with Garant. You know, they become a distraction, these quick wins, only when they stay disconnected from your broader strategic horizon and those strategic goals. I think CFOs and boards are done with waiting for returns. And what they've seen is that there's been a hundred pilots that various people across organizations have put into place without seeing a lot of fruit from it, without seeing a lot of returns. Right. And so these quick wins should function like footholds, not endpoints. So they should teach you something about governance or data readiness, your model choice, where that workflow really breaks and where you need to invest further. But these small wins need to lead to something. So what I'm hearing in calls is that leading operators are using the small wins to earn the right to tackle bigger, messier, cross functional processes. The weaker pattern is when companies accumulate little pilots that never connect into this kind of broader redesign. That we're talking about. At that point, you haven't really built momentum, you've just built a demo portfolio that ultimately leads to what we're seeing, where companies are starting to roll back AI initiatives not because the tech doesn't work, but because they didn't do the proactive work to build the governance, the structure and the workflow proactively. So, yes, quick wins, but quick wins tied to your strategy and tied to opportunities to build the case for further investment.

Speaker A: Our, uh, founder, Kevin o' Mara calls that a science fair when we're just rolling out all of these individual projects. But I love that. Piecing things together towards orchestration. I like the phrase you said footholds, not endpoints.

Speaker B: That was great.

Speaker A: So, final question for both of you guys. What is the single biggest decision or move a supply chain leader needs to make today if they want to stop treating AI as an expensive experiment and start using it as a real value driver? Justin, why don't you kick us off?

Speaker C: Look, if you want to make an impact here, you got to pick one cross functional decision flow that really matters and redesign it end to end. If you have your teams that are sitting in silos, plan, source, make, deliver, if you build down one AI pilot in one area, you're just going to scale up that one area and, and continue to have a broken process that's not end to end. So you don't need another giant list of AI ideas. You need one decision flow where speed, quality and coordination clearly matter to drive things like cost and cash and growth. And then you need to put the right governance, data ownership model, strategy and the human decision rights around it. What we'll often do with clients is we'll look at one end to end power thread, one end to end workflow, and then start to map the team, the data and the process that sits underneath it. Understanding the tech stack and some of its capabilities, that's the starting framework that leaders need to look at. Once AI is embedded in a real operating thread, that's when those economics will become easier to see. So you can measure what changed, who owns it, and where the bottlenecks are. Because you start with that end to end view, that's the point where AI starts to feel less like an expensive experiment and more like a new layer of operating capability.

Speaker B: Yeah, I think that's exactly right. Justin. Focus on the business problem you're trying to solve and pick something that you can really get stuck into. This is not about asking where can we apply AI, it's, uh, asking which supply chain decision. If we could make it faster, better or more autonomous every week or every day would materially move cost, cash, service, quality, growth, for example, how can we speed up the qualification of alternative suppliers? How do we identify and act on, um, key commodity cost reduction opportunities? If you take that kind of approach and look at how AI can help you make those better decisions or execute those decisions automatically, then that's going to move you away from this just being a costly pilot that, uh, your cfo, CEO or others are going to be on your back about. Really sort of ensure that these are kind of sustainable and scalable solutions.

Speaker A: I, uh, like that. So not an individual use case, not 20 individual use cases. Map an entire workflow. And to Garen's point, how is that actually going to impact your supply chain? Identify the ones that are going to make the most difference on key metrics so your CFO doesn't get angry with you. It's great takeaways, guys. Garen, Justin, thanks so much for the conversation. Special thanks to Alicia Lean and Charlie Smith for production. For more insights from Zero100, find us on LinkedIn@zero100.com or on our members app. And if you enjoyed the episode, leave us a rating or review wherever you get your podcasts. I'm Cody Sack and this is 0100

Speaker B: m m m.