Ep#249 How leaders need to rethink their approach to AI altogether

Speaker A: Welcome to the John Meyer Podcast, powered by Meyer Media, where we bring you thought, leadership and conversations from around the world. Today we're cutting through the noise around agentic AI to ask more fundamental questions. Are leaders approaching AI with the right mental models at all? Joining us today are Christopher and Diego from epiuse, who bring deep experience in AI architecture, governance and enterprise implementation. Today they'll help us reframe AI not as a novel disruption, but as an evolution that demands we return to the first principles, context, tooling, instruction, and before we can scale responsibly. Please join me in welcoming Christopher and Diego to the show. Christopher, thank you so much for joining me.

Speaker B: Thanks John. Pleased to be here.

Speaker A: Diego, thank you so much for joining me.

Speaker C: Thank you, John. It's great to be here.

Speaker A: Diego, There's a lot of confusion around agentic AI right now. For from what you're seeing, what are most companies getting wrong at the very start?

Speaker C: Well, that's interesting. I think as most of the technologies, uh, we are at the fascination phase, as I like to say. We are basically implementing the technology because of the technology. We are not looking into the value proposition that we are aiming to implement that technology. So I think from our experience, most of the projects are getting wrong. The fact that they are not starting from the business perspective, what are they want. What, what is that they wanted to, to generate as a value proposition and as any technology project, I would say that you should start from two very specific aspects. How can you cost reduction and how can you increase your revenue from there you can, you know, work backwards and understand how this technology or any technology could help you to get the result you wanted to. So I think most of the aspects that we've been seeing on agentec AI is a reproduction, uh, it's a repetition actually from m other uh, technology breakthroughs that we had through time.

Speaker A: Do you feel it's a novelty phase?

Speaker C: Well, I think the concept of novelty in technology is a very tricky concept, especially because most of the technology that we implement is uh, a consolidation of uh, a set of different technologies. So when, uh, we talk about AI AI, it's been around for more than 50 years. So of course generative AI, gentech AI are evolutions, evolution from a concept that we already had. So from, from a novelty perspective, I don't think we are, you know, we are, we are talking about that much, uh, recent technology. But definitely what we've been seeing from a business perspective is different and, and I think we have to manage that differently.

Speaker B: The phrase that comes to mind is blinded by the light and there's a lot of noise, a lot of hype cycle, um, overload of information without necessarily a fundamental understanding of what this base technology is. What does it offer? What does it offer as the enterprise? I would say the most common question we get when we're doing discovery or we're scoping out specific, uh, AI use cases is can the AI do X?

Speaker C: Yeah.

Speaker B: And I, you know, I've worked on big projects where you, you'll get 2, 300 questions. Can it do this, can it do that, can it do the next thing? And fundamentally you're asking the wrong question. The question you should be asking is the same as you would, as if you were employing, uh, a human being. Does it have the knowledge, the context, does it have, um, access to the tooling to do the job and can we give articulate instructions for it to execute that job effectively? And I think that we're even, we're talking at the atomic level. So, so at that level saying, okay, well functionally this is, you know, this is what is possible. The bigger problem for the enterprise is to say, well, that's super effective. That's great. How do we now plug this in to the grander landscape, uh, of our data, our information, our governance, all the bits and pieces that go with delivering enterprise solution at scale whilst addressing this, this atomic problem in, to your point, a novel way. And a lot of mistakes get made, uh, you know, down the road in one, having a good fundamental understanding of what it is, um, the technology can offer and what we can do at that level. And we get stuck in this, what do they call it, pilot purg, where a, uh, bunch of new projects and new initiatives stuck in this kind of QA pilot phase that never make it, uh, into production because we haven't considered the grander vision and where this stuff is going to fit.

Speaker A: Christopher, you actually touched on it just shortly about the implementation, what it's going to do, why it's going to do it. Do you feel that everybody should take a step back or some people should take a step back and understand why you would implement AI, why you would implement agentic AI and the value behind it before you even implement it. What are you trying to accomplish rather than just throw it out there?

Speaker B: Absolutely. And we talk about new technology or the novelty of Blinded by the Light, this fascinating stuff. Actually, the first principles, they haven't changed. What problem are we trying to solve and what outcome are we looking for? I think the important thing, you know, when we're talking about having, uh, autonomous A digital autonomous reasoning engine that is now living inside our enterprise architecture. Uh, is what is the fallout of that? I now have a digital worker that has very high graduate level reasoning capability. What can it see? Can I control what it sees? What can it access? How do I manage this digital worker? Um, and that has impact right across, you know, right across the way that we structure delivering these projects. Our engineers are moving from creators to curators and so we're now no longer writing fundamental functionality that's taken care of. Now we have to manage this world where I have all of this functionality. How do I orchestrate it together? And I would say a lot of the failure aren't or failures aren't necessarily. There are fundamental failures. So there are things where um, the context is wrong. They're not intelligence failures. We've set the knowledge base incorrectly, our data is dirty, the data landscape is uh, inconsistent and we get inconsistent results. But at a enterprise scale level it's more to say how are we orchestrating this, how are we managing that orchestration? What does the observability on that orchestration look like? When you deploy a um, uh, artificially intelligent agent into an environment and it starts to, you know, to work consistently on, on repetitive set of set of tasks, it in itself starts to create exceptions and edge cases that you may, may have not. Well, I can guarantee you haven't uh, considered do we have observability into that? Why is the model performing that way? Why are we sitting with, with um, a set of these edge cases um, that we hadn't considered in our initial to. Your question is should we be taking a step back? We absolutely should be taking a step back. But not only that, the consideration of the technology we're using. If you deploy a high reasoning model like say Opus Claude and you give it menial tasks, it's like hiring a highly intelligent um, DNA scientist to do um, office administration. And of course the cost ratio of that in terms of running a business is not very clever. So the, the as it applies in the human world, it applies likewise in, in the digital.

Speaker A: Diego. Do you feel that we're actually implementing true agentic AI or just rebranding a workflow, an existing workflow that we've already implemented but just threw on the AI terminology into it?

Speaker C: Yeah, we do that a lot in technology. Right. Rebrand things and call it differently. But on the concept and the essence is basically the same I think from the agentic AI perspective. Uh, and I would say that uh, from an information and knowledge perspective as a whole, within Technology, uh, we already have a lot of different technologies that implement, you know, some level of intelligence. We can talk about, you know, uh, automation, uh, workflow automations. We can talk about uh, bpm, we can talk about etl, a lot of processes that when you look into it, it's basically the process of getting data, giving some context which is uh, uh, consolidated as an information and generating knowledge. So at some level we already had that before, right? With automation, with workflows and so on and so forth with the AI movement. Now what we've been seeing is that the first thing we have to differentiate from a generative AI perspective to a agentic AI perspective. We are talking about reactive approach to a proactive approach. I like an analogy that it's that we can understand the two concepts in which imagine generative AI as a vending machine in which you can, you know, purchase uh, a uh, uh, soda or uh, a candy bar. You know, it's right there, it's ready. You ask, you, you have it from an agentic AI perspective. You, you are a sous chef, you have to you know, prepare the dish. You have to understand the tools that you have in hand to, to, to prepare that. And not only that, you have to have the policies, the rules related to how to prepare that dish and you know, what, what are the type of tools that you can use at that specific facility. So it's a uh, it's a more evolved concept in which by the way, we see that a lot in technology in general. Technology is basically a discipline, a science of creating abstractions. We are solving problems that we see a lot on a daily basis using some sort of uh, automation or some sort of uh, abstraction as I like to say, to solve that. So again from a uh, generative AI perspective to agentic AI perspective, we are talking about reactive and proactive approach. And of course to be proactive you have to have context, you have to have memory, you have to have tools that you have to use. Of course I'm mixing uh, some technical aspects, but trying to give you the general uh, relation between the technology and the, the you know, the daily basis that we see uh, in the world, not only in technology itself.

Speaker B: It's a, it's a good point, right, if you consider classic workflow. So get, get some data, do um, some operation on that data, pass it along uh, to another ah, function point. And if we, if we think about those as nodes in a workflow, what we see quite common now is model in the loop. So instead of having necessarily a set of functional Rules or hard coded, uh, rules, uh, engine that's doing that work. We'll just drop uh, an LLM in the middle of that, that'll do the work and pass it on. Now there's nothing to say that there's necessarily anything wrong with that other than to say, well really you're not implementing agentic AI, you're just calling a more sophisticated API. When we talk about agentic AI, we're talking about reasoning. I'm going to make a decision rather than I'm going to give you a task and a very specific set of instructions and you're going to pass me back the answer that I was looking for. In the agentic world, I'm going to have numerous things going on, let's say even in an asynchronous workflow where I can carry on working on this while this is um, solving a problem and I can do this in an asynchronous manner. And as those things collate, the agent can make decision where these things need to go. So the approach to the deployment of the technology becomes very important. Um, we don't necessarily need to replace something that is working quite well. And solving the problem for your business where you really want to be looking is saying, well, which areas? And a good example of this is exceptions. So workflows and big software systems like ERP Systems for example, do an excellent job of processing the happy case or the happy use case of any specific workflow. What they don't tend to do a very good job of is when there's a problem. So let's pick a random example. So like invoice processing. So I've got a purchase order, I've got an invoice, I've got a receipt and I've got a bunch of goods on those things I've got matching numbers, all of that good stuff. So in order for that to work, all the numbers have to match, all the line items have to match, everything has to be happy case. Now in the reality we don't live in a perfect world. So the mulched up uh, uh, invoice arrives and there's a mistake made or whatever and we pump that into our workflow system and the numbers don't match. Now I have an exception and that exception needs to be by and large chased down by a human being who's going to expend however many hours trying to solve the problem, enter the fray. Agentic AI. Well actually I can pass that high reasoning to a digital uh, worker who can solve that problem, um, without Involving a human being at all. And so for me those types of use cases certainly in the early days of the deployment of this technology because it's a limited scope, it's a well understood use case, a well understood discipline, you're going to get high value very quickly without deploying a really over complicated uh, uh, architecture that you're not sure how to manage yet.

Speaker C: And from your point uh Christopher, it's interesting how uh, from the enterprise perspective we end up having a design issue because when we are talking about reasoning, when we are talking about how you standardize something or how do you uh, uh, understand the context of that specific domain. You know we are talking about design, we are talking about understanding the business processes, we're talking about understanding uh, the people that are somehow related to execute that, that specific process which we are currently uh, used uh to do manually or on a human uh perspective we now have to design everything and ensure that everything is covered to be somehow automated or at least executed by an agent. So I think in the end what we have is a design problem. Most of the enterprises uh, do have uh, design problems within their ERP systems or within their business processes. And at the point that you have to implement that using agentic AI, using, using AI you have to cover all those aspects which is hard and tricky

Speaker B: aspects of that now that are emerging which I think are quite important. So um, one of the terms that we hear a lot now is governance as an enabler. And so what they're meaning by that is if you feed a human being or any worker terrible information you generally get a terrible result. Garbage in, garbage out. Same uh, uh, analogy uh, for a computer but AI specifically and where the challenges are is that there's a perception that AI will solve this problem. Now the likelihood is AI will solve the problem but it is absolutely the foundation of that house is built on the cleanliness of your data, the policies that live around the management and access uh, of that data, the consistency of the data across your, your application landscape and what, whatever it is that the agent has to do. So when we talk about governance as an enabler, the reason your data was dirty in the first place is because the governance and the, and the um, and the controls were not, were not there to start with. And that's why we see inconsistencies across the landscape, etc. So suddenly a discipline that perhaps not the most popular in it and in computer science circles, governance, risk and compliance, governance becomes the enabler because the cleaner and the more um, clear cut those policies are, the more Effective your deployment of this technology um, uh, is going to be.

Speaker A: Diego, you touched on the policies and the implementation which actually goes to. One of my important questions is when you look at how companies are adopting AI today, what actually feels genuinely new and what feels familiar?

Speaker C: Well uh, definitely feels familiar. Uh how we engage technology, how we engage uh some new uh, technology uh, that we have available. Uh it's interesting how the enterprise usually go for uh, and try it and test it but the value generation, it's way way more tricky. So I think this is something that we saw before. And what feels new is that to implement AI you definitely have to cover a lot of aspects uh, that weren't uh covered before. So I talked about design. How do you understand your business processes or how well governed your applications are. So you have to manage that to implement agentic AI properly. And the thing is you are now not talking about an agentic AI project, you are talking about a governance project in which you have to cover a lot of different aspects to ensure that your engine will actually run as supposed to. So ah, as an agentic AI, I like to say that an agentic AI and um, uh a more abstract technology that we have to implement, which agentic AI of course is you have to cover a lot of different disciplines that weren't covered before or at least were not well managed before. So data management, uh, business, uh, process, uh design, uh ERP implementations, exceptions, how we handle exceptions, uh security. So all of those disciplines you have to uh engage to ensure that you have an agentic AI project in place. So now I think it's new the fact that the enterprise understand that the things that they you know put behind they now they have to address.

Speaker A: Christopher, how important is it to have the foundation in place first before you implement AI? And are some companies rushing in to implement it before understanding what it needs to have in place.

Speaker B: So the rush is that's definitely true. So uh, like I was talking about earlier, you know uh, pilot purgatory, uh stuck in the M mud uh we ran at this. We you know, we thought the foundation model is going to solve our, all of our problems when in fact it is not. And let's unpack that a little. So one of the things that's kind of very prevalent is this belief that uh, uh because of the, the results and, and to your point the results that are delivered, you know, you asked about what's familiar and what's unique on a, on a really successful agentic or, and specifically agentic projects, the ROI or the roai or the roaii return on artificial intelligence investment. Those metrics can become very specific. And so you know, the buzzword is uh, what the FTE reduction and there is social implication that goes with that and the workforce and how that's going to be managed, um, you know, moving into the future. But to go back to the project side of that and the stuck in the mud idea and preparation and all the things that have to go on, um, there is a misconception that the implementation of AI in any particular discipline and uh, is going to be super fast as a result of the AI part of that technology, you know, part of that project. The reality is the AI part is more often than not the most simple part of it. And the integration touch points, the access to data, the policies, the data cleanliness, how we're going to manage that, uh, um, as we start to feed the model the context, give it access to the tooling and uh, you know, here we go, we're going to give it some instructions. That foundational layer can um, be a heavy lift and it generally is a heavy lift. And that lift is no different from any normal uh, large scale integration project where you're going to put a new technology into a large functioning enterprise that's dependent on the digital domain. So the important part of that is to say it's not fairy dust. Yes, it's very, very good and the, the, we can see the results when implemented properly, but it doesn't negate the behavior and the disciplines and the policies and all the bits and pieces that have brought you to this point where you want to, you know, bolt the technology on top of your, on top of your landscape.

Speaker C: And it's interesting how that is, uh, also seen on the data. You know, 40% of the projects, by 27, 40% of the AI projects will fail. And most of them are use cases that are, you know, I must say some weird use cases, uh, on the lack of a better word. And we also have some interesting uh, success stories. One uh, in which I would like to cover. It's a bank that implement 350 use cases of agentic AI. And it's interesting how the most successful ones were the ones talking and implementing simple processes, IT support, you know, HR onboarding, uh, uh, uh, invoice reconciliation. So processes that as I mentioned before, as we, as you pointed uh, to Christopher, uh, are not that, you know, that fancy, they are boring processes but those are the process that we can actually calculate cost reduction, you know, how can we improve our processes time, uh, consumption, FTEs, so I think one of the key aspects to ensure that an agentic AI is successful is understanding the use case. What is the use case that we are covering? Do we actually understand it? Forgetting about agentic or forgetting about AI, do we understand that specific use case? If we do, if we have the tooling and the technology, the foundation technology to cover it, yeah, we'll probably see a, uh, successful agentic AI project.

Speaker A: Diego mentioned 40% fail. There are successful implementations. Christopher, you talked about the pilot phase a couple of times. What actually determines an AI successful adoption versus one that quietly falls apart though?

Speaker B: So, um, it's a good question. The base metrics don't change. So, um, uh, am I doing it faster, cheaper, with less? And is it generating, you know, or is my ability to generate revenue, uh, uh, improved as a result thereof? I don't think there's any fancy metrics that really are, uh, associated uh, with this. I mean, in terms of the business case, uh, in terms of actual successful projects, one of the things I would point out is does it work? Yes, it works great. But we now have 15 people that are managing the models and the edge cases and all of the noise that it's creating, um, uh, as a result of its implementation. Is that a successful implement? No, probably not. And so over time, I like the term quietly failing. Um, so it is doing the job, but we're catching all of the mess that's being created as a result of the deployment. But it raises a valid and an interesting, uh, point. Diego was talking about design, um, one of the features that everybody will be familiar with that works with, uh, agentic AI is the idea human in the loop. So, uh, I will deploy a set of agents, um, but I'm going to have human, uh, beings check their work before we pass it downstream or we move on to the next step. If you think about that from a design perspective, that is an awful design. And the reason it's an awful design is that human beings are not good at mundane, repetitive tasks, accurately. This is why we introduced computing in the first place. Standard automation do a lot of, um, boring work very accurately, consistently, day in, day out. So in the design phase, and particularly if we're talking about HITTLE or human in the loop, we now talk about concept of human on the loop. And what they mean by that is to say, um, instead of putting a human in the loop, where I approve, disapprove, and I'm doing this sort of mundane set of repetitive tasks, I let the agent do its work, uh, autonomously. So let's Pick a use case, ah, agent, uh, drafts, receives email, uh, drafts response. Human in the loop checks response. Send email now. Fine. Downstream. This concept of human on the loop says no, we design for over time. And most enterprises will start with human in the loop until we trust the technology. But we design for ah, email inbound. Agent drafts, uh, response, agent sense response. Human on the loop is looking at that process from an observability perspective. He's watching for sentiment, he's watching for outliers, and he's watching it, if you like, uh, holistically above the process and knows when to intervene, to pull out. I have an outlier with a very funny sentiment response to a customer or client or, you know, whatever the, whatever the situation is that in terms of a design phase is super, super important because otherwise all we're doing is we're creating a queue, a big queue of work done much more efficiently and much faster and a huge pool of human beings that are having to vet that work and then send it on. And so that observability part of deploying agentic AI, we've got complex orchestration, we've got complex tasks that are going on. I want to be able to observe that and that. But to be pulling out exceptions or the outliers and getting involved at that point, not sitting in the chain and, you know, working through this tons and tons of mundane work.

Speaker C: Yeah. And to talk about success on the Gentek AI, uh, projects, uh, this is definitely a business project. This is not a technology project. Don't get me wrong, I'm a computer science. I love technology. But to have a successful project, you have to make sure that you understand the business outcome because you have to understand the business context, as Christopher mentioned. So, uh, we see a pattern in successful, uh, agentic projects in which we have the business outcomes well established. We start from the business, we understand the needs and what are the goals usually related to cost or revenue or, you know, time consumption. And if we have that, those, uh, goals well established and we, we work backwards from there, we definitely see success. What we, what where we don't see success is where we, we have an agentic AI project as a technology project. So we are in love with models. You know, we are in love with building integrations and building interfaces. Uh, so yeah, again, although we use technology, it's not a means in itself. You know, it's a means to an end. So although cliche as it is, uh, that's the pattern that was being seen on successful, uh, agentic AI projects.

Speaker A: Christopher Diego, my last question for you is do we really want true autonomy or accountability?

Speaker C: Well Christopher mentioned the human on the loop and I think that that resonates a lot on the accountability aspect. I think we, we have, we must have accountability on that. Governance uh, is ah, is again it's a ah, a pattern to successful agentic AI projects because without governance we have uh, an imminent incident, we have an imminent problem because we understand, we don't understand what that agent can uh, do you know, so govern. Governance is a uh, is a uh, an aspect that we have to ensure that we cover on those, on those projects. And yeah, this is definitely resonates on the accountability aspect.

Speaker B: I think the simple answer to that is we want both. Um, do we want true autonomy? Absolutely. Um, um, the more autonomous the process is, the more effective it is and the more efficient we become. The accountability aspect of it is a deeper question and that is to say um, if I allow something to make an autonomous decision, who's accountable if that decision is incorrect? Um, what do those structures look like over time when we're running more and more autonomous processes and things start to go wrong or if, and I um, hate to say rather than if, but when those things go wrong, when what is the accountability matrix look like? I, I, the kill switch, the ability to intervene, the ability to be, have that deep observability into you know, of the mechanics and, and how the process is moving forward, um, the balance of that against and I suppose the, the way to think about it would be to say as the risk level it goes higher and higher. It'd be logical to say that the autonomy and the, and the, the amount of human in the loop and the amount of intervention that we require is going to become obviously more and more important. You know there's one thing uh, uh, processing an invoice, it's another thing making a, a deep business decision that's going to affect a just in time manufacturing line and an ordering process and you know, all the things to go with that. So, so in short, I would say the answer is both. I think uh, it's a leading question because depending on where that let's say risk matrice is we can start to touch on okay, what's going to happen in regulated industries now? What happens in the military? What happens in um, in, in public sector services where we are delivering mission critical things to human beings Ultimately what does that autonomy, accountability matrice look like? But in short, I would say the answer is both.

Speaker A: Thank you for joining the John Meyer Podcast. If today's conversation gave you new insights around agentic AI. We'd love to hear from you. Don't forget to subscribe and share this episode. Until next time, keep innovating.