AI readiness: Bridging the gaps in enterprise architecture

Speaker A: Welcome to Deloitte's On Cloud podcast, the show that cuts through the noise to bring you practical insights on cloud, AI, software engineering and more. Tune in for real world tech strategies to lead in the age of disruption. Now, here's your host, Gary Arora.

Speaker B: Welcome back to On Cloud. My name is Gary Arora. I'm a chief architect for cloud and AI solutions at Deloitte. And something big is shifting in enterprise technology. AI is accelerating faster than most architectures can keep up with. And leaders everywhere are asking that one big question, are we actually ready for what's coming? So today we are talking about what is changing. The hidden architecture gaps, the challenges of creating consistency across environments, and what it actually takes to bring AI into legacy systems without breaking everything. To help us unpack it all, we have Kevin Chunks, Chief architect and National Technology Advisor at Red Hat, someone who guides some of the largest enterprises through these exact challenges every day. Kevin, thank you so much for joining the show.

Speaker C: Gary, thank you so much for having me. I'm excited to be here.

Speaker B: Likewise. So let's start right at the top. We are at an unusual moment right now. AI is here. Most architectures are still legacy, unable to support it. Virtualization markets are being disrupted. Cloud strategy is being rewritten in real time. We are seeing a lot of workloads actually move back to. On, um, Prem, from cloud, from your seat, advising these large enterprises. What's the fundamental shift happening right now that technology leaders just cannot afford to misread? And why does this moment matter more than, let's say, the last decade of cloud transformation that we have had?

Speaker C: Yeah, it's a great setup. I mean, it's a big question, right? I think the big things that we're hearing about this inflection moment are there's this move to AI. AI has moved from this point of interest in science fiction and some pretty good predictive models to generative models that suddenly shifted. What that pace of capability is, what the realities are of making that part of your business, and business models are being reconsidered, and how we look at technology and hosting and how we deliver services are all over the place, is really changing very quickly. That said, an awful lot of customers have not really focused, really gotten to the point where they see their specific workload, the specific use case that's going to change things for them. So there's a lot of experimentation that's happening, a lot of innovation that's happening around new AI offerings, new capabilities. How much do you have to build yourself versus how much you could buy in the marketplace, all of that fluctuation is creating this, ooh, am I ready to buy in? Or is it a time to hold and prepare? And so that's one space that we have a lot of conversations around. The other space that we have a lot of conversations around. That is the big driver of enterprise change right now is really the virtualization disruption in the market. The legacy provider that was almost ubiquitous virtualization technology in the space changed hands and changed prices and naturally forced people to rethink, how do I want to move my data center? What's my true enterprise capability going to look like? How do I use this moment to either just do a virtualization to virtualization, a V kind of migration? Is that my only problem, or am I solving a V2V problem and also preparing myself for the future with AI and other technologies coming down? So I think those two drivers, how much, how much do I want to just solve my immediate problem, but how much do I, uh, prepare for the future are the two big drivers that we're excited about and seeing come to bear.

Speaker B: I want to take it one level deeper. When you're in the trenches with the CIOs and the architects, you get a very different view of what their systems can actually handle. A lot of the leaders believe they are cloud ready until they actually try to run AI or modern workloads at scale. Where do you see the biggest gap between what enterprises think their architectures can support versus what it can actually handle?

Speaker C: Yeah, I think the big ones that we see are this idea of AI is a bolt on. AI, uh, is just a sort of plug in concept as opposed to really having it be infused all the way into your processes. And for customers who are just purely experimenting, AI is a bit of a bolt on. It's a, it's a sandbox environment. It's something that's just experimental in terms of what's happening. I think it's those customers that are then saying, ooh, I've got something that I want to take out of the sandbox, out of the science kind of activities and I want to make this really applied. And now suddenly governance and scalability and infrastructure architecture and where am I going to apply this? Am I going to apply this in my data center? Am I going to apply this in an edge environment or is it going to be hybrid with a cloud, cloud piece on that? Those are the questions that people haven't really thought through yet as things come out of the sandboxes. And so the customers who are at that point, some very Large financial firms, some very big NEO clouds and others that are starting to really think about that governance and security and compliance considerations. That's where those conversations are shifting. Not to mention the difference between training initial models and then doing what are called BLLM mods. Right. So when you're, when you're doing the compression around your models, when you're doing the full inferencing and deploying that into production, those are the pieces that I think uh, senior leadership is still getting their arms wrapped around. What does that mean? What's the cost evaluation of that? How much change needs to take place in order to take advantage of that. Good idea that came out of the sandbox.

Speaker B: I uh, loved how you framed it. AI as a bolt on is certainly the mindset that I feel is the elephant in the room. But how do we get out of this? How do we shift from AI as a bolt on to infusing AI into legacy systems? Because a lot of the leaders, they do want AI everywhere but their systems were never designed for it. So how do you help these enterprise customers to realistically integrate AI into 10 or 20 or 30 year old systems without triggering a multi year ah, rewrite.

Speaker C: Yeah. And so there are some timing issues that come with this. Right. So it's not, if it's not an all at once kind of thing, it does take a change over time. Most of that is really finding individual business processes that have a scope limit to them and finding if those applications have they moved on a cloud native journey? Are they, are they starting to modernize in whole or in part? Are they API driven? Is there a way to infuse information into those applications, those legacy business or business applications that can change the real tone of uh, what that business outcome is that it does. Whether that's accelerating the pace of being able to accomplish something. I want to process my invoices faster or I want to detect fraud more effectively or I want to do X, Y, Z, I want to, I want to move ships or goods or to where they need to go faster. Any of those kinds of use cases tend to be wrapped into business applications that are there. If we can find AI solutions that are specific, measurable and identifiable that you can plug in and say, hey, I've built this extra piece of knowledge, whether that's an agentic concept or otherwise. And now I can, I have a way to infuse that incrementally into my application. I can do a small thing that is scope bounding. What we did with microservices and microservice architectures, I can start using those agency AI concepts and start incrementally adding them into my core applications. That gives us a way to say I took an action, I saw what the impact was, I can measure it, I can look at it, I can see how it goes and then I can take another action and another action and with that incremental process, really make a huge transformation, frankly, very quickly, but having good feedback loops along the way.

Speaker B: So you brought up specific agenc solution solving measurable problems and there are a lot of open source tooling in this space. And of course the AI conversation isn't complete without talking about open source. Red Hat does sit at uh, the intersection of open source and enterprise grade reliability. What do you think is something that's misunderstood about open source AI in enterprise? And how do you advise leaders to use open source in places where it speeds up innovation but also without introducing risk?

Speaker C: Yeah, there's always some amount of risk in any move forward.

Speaker A: Right.

Speaker C: So you have to be comfortable with the risks that you're taking when you're at early stage, when you're sandboxing, when you're looking at what are the, what are those use cases, how can I define that problem, how can I apply a specific solution that is then measurable and I can really look at it as part of my business outcome? You know, look, a lot of people are going to find ways to minimize their cost when doing that type of activity. Totally get that. If you don't know the outcome, you want it to cost the least amount possible for your MVP for minimum viable product kind of concept on it. Totally understand that. However, in that transition out of the sandbox, if you'll take my analogy, into a production environment where you have governance and compliance and security requirements and the ability to operate at large scale in an environment and have confidence and trust and consistency with how you do it. That's when there's this shift between pure upstream, fast, super fast innovation and an enterprise supported capability like what Red Hat brings to the table. And for some companies that are saying, hey, this is not my first, my, I've got a series of capabilities that I want to put in AI, AI, uh, power. I know that these are going to go into production. I have high confidence that these are going to move their all the way into production. But in that case you really want to retire the complexity of that interaction, that integration as early as you can. So you start with the technologies that you're going to end up finishing running on. For companies that are saying I really don't know if this thing is going to go to production, you're going to go with more upstream open source for that experimental phase. But as uh, you move to the other side of it, that's when you want to wrap that framework so it doesn't get away from too early.

Speaker B: And speaking about enterprise standard, let's talk about another trend in enterprise technology. For better part of the last decade I helped enterprises move their workloads into cloud, the data into cloud stores. And lately we have been noticing a little bit of the reverse where the workloads uh, are coming back to on prem, especially now with AI as more enterprises are exploring running large language models or even small language models on prem either because of data uh, gravity or closer uh, governance or even cost predictability. The big question we are trying to answer is what does an enterprise AI platform look like today? So from where you sit are there any patterns you're seeing from customers who want on prem AI but still expect cloud like elasticity or security and automation?

Speaker C: Yeah, for sure, they're all right. It really does come all the way down to the hardware that you're using. So to your point, there's no lack of workloads moving to the hyperscalers. They're still growing at an incredible pace. However, with some of these capabilities I think we're hitting a maturity point where some capabilities are coming back on prem. And like you said there's a number of drivers, data gravity, certainly one of them, data sovereignty kind of flows into that same piece when you get some of the national boundaries associated with things, the legal constructs of um, where data actually exists, where things from a, from a legal framework, how liability looks like in prosecuting these different concerns, how much independence certain companies want in certain, certain industries to have those frameworks in place. So I think we're probably starting to see a more, more mature, slightly more sophisticated set of differences. So people are really putting applications and workloads not in a one hit wonder emotion where it's all going to cloud or all going to be on prem or all at the edge, but really this nuanced capability of true hybrid or multi cloud where it's I want to put the right applications and investments in the right location to service up what they're doing. So I think those are, those are a lot of the big drivers that ah, I talk to folks about piece about how to make that work is when you're doing it on prem in particular you're probably going to be as close to the bare metal as possible. Performance Particularly when doing any kind of training activity or any kind of inferencing activity, you want that to be highly performant so that for all the economic reasons of making these things viable. And so what we're seeing a lot of are solutions that are componentized solutions, meaning they're not hci, they're not, they're not like really tightly bundled kind of appliance pieces, but they're more flexible than that. So you want pre baked architectures to retire on a lot of, a lot of complexity in that space. And then from there you want application platform and capabilities to really drive outcomes, give you that flexibility so that you can constantly swap out the components that you're looking at. So if one particular group within your company has a, has a problem space, they have a set of tooling they want to, they want control over their set of tooling, you can still use a consistent app platform capability that gives you the ability for each one of those different divisions, each one of those different projects to really choose their tools wisely and have that flexibility. Many of which can not only run on prem, but then also run into Chrome. So that's, those are usually the conversations that we're having at the, at the C suite level.

Speaker B: So in terms of the next gen AI native architecture, we are seeing more of hybrid pattern where an AI platform might be spread across multiple vendors and it makes sense to use the best of tech choices available and not be locked in. But it does open up the issue of having a consistent experience, not to mention security and governance and the maintenance of all across the M multi public clouds, across edge, across private cloud, on premise, uh, across your virtualized platforms. So first of all in your experience what does consistency actually mean at an architecture level and are there any shortcuts or some secrets here that we can implement so we are not starting from scratch every time?

Speaker C: Yeah, so I think you're right. The number of hybrid experiences, the desire to be able to change where you do things, have that flexibility where I want to run at my own edge or have combinations, all those permutations or it get really complex and they all end up becoming really economically driven decisions more than they should have to be technology driven decisions. And so I think the conversation around that is what gives you the ability to handle the complexity at each one of those different environments. Whether you're on prem and you're on a mixed set of hardware from different vendors, et cetera. If I have something that's consistent across all of those, then I'm really making an economic based decision. And that's exciting, right? Because that means my teams are more portable, I can deploy them wherever they need to go. That's good for both my internal corporate teams, but also my consulting teams that are coming in to advise. They have a uh, common familiar set of environments that they're working through. And I have this ability to really define at different phases of maturity of my application of all training is going to be on prem because of the data gravity. And I want my bike. From a networking perspective, I don't want anything doing ingress and egress across the cloud lines. That's totally fair. But for inference, fee and execution of my model, maybe I want to have the most distributed set of those capabilities. From a security standpoint, from that consistency standpoint that you brought up, having the ability to have uh, the consistency of that platform that you're landing on. In our case that would be Rahat OpenShift. But having that consistency as you use it in all those different areas becomes a very valuable piece. And it's hard to have that kind of a platform that is patched and secure and understandable that is also consistent across all those different environments.

Speaker B: That's something we are noticing that modernization today is becoming an ecosystem sport. Something that no single vendor or product can do alone for the right reasons. What role do partners play in delivering consistent modernization experiences across cloud virtualizations and AI workloads?

Speaker C: Yeah, I use the analogy a lot of any modernization, anything that you're doing, it's like going in for a surgery, like a knee replacement or an upgrade or uh, shoulder hip or shoulder or something like that. You want to try to do the least amount of damage possible so that the customer is really getting the most out of what their specific investment is trying to do. You want to minimize that recovery gap to get across to the other side. That means do the least amount of damage possible. So from a vendor standpoint, this multi vendor kind of concept, you've got networking vendors and storage vendors and data protection vendors and observability vendors and another, this very complex group ecosystem that comes together for these, for these solutions. Your data has to live somewhere. And very few organizations are completely homogeneous when it comes to them. They're going to have different storage providers, they're going to have different hardware providers, they're going to have all these different things. Uh, with all that being said, if you have something that glues all of those together, that becomes a really important piece. It's kind of like having a point guard. Right? If you go with my second analogy and the same answer. It's like having a point guard. They pass the ball, they're able to get the best out of all the different players on the field. And that's where we want, that's where we play a big role in that. But bringing the best of storage, bringing the best of compute, bringing the best of the chipsets and the accelerators, wherever they're from. Right. Having that investment so you can take advantage of all those different innovations because they're all innovating at just a phenomenal pace. But not being too far behind within one group, that's, I'd say what the architectural gaps start to look like and where you're looking at. Okay, I get that there's complexity. I get that I want to do this, but this also gives you that adaptability over time so that you can take advantage of and go, I want more of this in my portfolio. I want more of that in my portfolio. Uh, and change and adapt over time.

Speaker B: Is there a balance one needs to strike here with multiple vendors coming into the ecosystem? Is there something as too many vendors? Because we do see that many enterprises want more vendor choice, but one, their architectures are, uh, usually pushing them towards a single vendor because a single vendor is now offering multiple capabilities and they are launching new capabilities. If you take some of the large data platforms, for example, and then you have separate vendors, which, when you combine, you get one holistic solution with multiple capabilities. Is there a decision tree that supports one or the other? What are you seeing from m. Your vantage point?

Speaker C: Yeah, I think, uh, again, I'd go back to this concept of early innovation activities. Sandboxing activities is the easiest point with which you can control variables and you can have single vendor solutions to prove out a concept. But once you get into any scale out of that concept, almost inevitably you're going to want to be able to evolve it over time. And so I think early on minimizing, minimizing variability is a great place to go. As you start to go into more of a production environment, just know that plan for the fact that you're going to have more people there. You practice on your own, but you ultimately go, you play games as a team. And because it's a team, it's a team sport. It does emphasize one of the big points of intersection in all this, though. Systems integrators and global systems integrators play an incredibly important role in helping navigate through where people are having success. Right. And so groups like systems integrators and the teams that they create the most in that environment really do become this awareful spectrum of balancing. You know, at what point in time do I take on the uh, added complexity of having multiple vendors? And it's like I said, it's usually when you hit that, when you hit that tipping point of we're really going to put this into production and now it's got to work inside of the entire eco center of ecosystem, uh, of my data center and my legacy hardware and my chipsets and the new things that I'm buying along the way. It's got to orchestrate amongst all of that.

Speaker B: So different muscle and rigor needed for proving out concepts than for scaling out for production workloads. Let's wrap up on an optimistic future. Yeah, if you look a couple years out and in technology terms a couple years can be decades, but let's say you look two or three years out beyond today's hybrid cloud shakeup or virtualization shakeup with the early waves of AI integration that we are in, what's the next major architectural breakthrough you think enterprises should prepare for? Is there anything that excites you most about where this is headed?

Speaker C: Yeah, I think there's two things uh, that I'll put out there and it's maybe just outside of your two to three year but not much outside of it. So I think it's worth putting on the long range radar screen. I think quantum compute is going to have a phenomenal impact on certain industries. In particular I think industries like manufacturing, healthcare, any kind of like large transportation planning activities, I think those are going to be and they're on a whole bunch of public sector activities that that's really going to have a big impact on. I mention that one more because I think it creates ah, a real clarity around why we want to make sure we're getting our arms around AI. We're getting our arms around application platforming in that space. Now the amount of data uh, that a quantum system puts out is really not something that humans and human level processes without AI assistants are going to be able to take advantage of. And so that's one that I think is right that, like 20, 29, so 23 years at this point in time. And I think that is a piece that we've got to start thinking about how do we rigorously handle workplace preparation, business process design activities, IT infrastructures to be able to support the virtualized legacy applications, the containerized modern applications for new experiences, the AI infusion into all of that so that we're ready to take advantage of those next, next capabilities. Those are the ones that probably excite me the most. Certainly I think that there's all the things that, that are happening with robotics, more independent robotics that I think will have a really interesting space on that. And I'm still very bullish that someday we will have more autonomous driving vehicles, which I would, I would strangely put into the robotics field. Basically you're just riding inside of a robot. So I think. But I think, uh, I think some of that technology is going down. Really interesting places.

Speaker B: Yeah, really interesting space. And we are already seeing autonomous driving in multiple cities. So a lot of cool new infusions with AI and quantum. That's all, folks. This has been an incredibly grounding conversation. Thank you so much, Kevin, for helping cut through the noise and show what's actually changing in this new wave of platform decisions. If you found this episode helpful, leave us a review and check out our other episodes On Cloud for more conversations on how AI and emerging tech are reshaping the enterprise. I'm Gary Ora. Thank you for joining us and we'll see you on the next episode of On Cloud.

Speaker A: You've been listening to Deloitte's On Cloud podcast. Share this episode, subscribe and join the conversation and tune in next time for more real world tech insights and strategies. Until then, keep innovating, Keep leading. This podcast is produced by Deloitte. The views and opinions expressed by podcast speakers and guests are solely their own and do not reflect the opinions of Deloitte. This podcast provides general information only and is not intended to constitute advice or services of any kind. For additional information about Deloitte, go to Deloitte.com about.