DevOps and observability

Speaker A: M welcome to Next in Tech, an S and P Global podcast where the world of emerging tech lives. I'm your host Eric Hansel and chief analyst for industry research at S and P Global and today we're exploring some recent research on DevOps and observability. And with me to discuss it is Mike Fratto. Mike, welcome back to the podcast.

Speaker B: Thanks Eric. Thanks for having me on.

Speaker A: Well, it's great to have you back in. And you've put some research out recently about what got driven off of our automation and orchestration study. And it's one of those things that, I don't know, it just seems to be that constant struggle of getting organizations to higher levels of automation. And of course, now that we're wading into all the implications of AI and agents and ideas around autonomy, it's right back up into the forefront again. Can you give us a little background about what you're seeing and what's your take on it?

Speaker B: So the study that we, that we just finished up fielded over the winter, it was 700 professionals, from managers up to senior level managers. So it was a pretty good sort of overview both from a technical standpoint and sort of a technical management standpoint. And overall, what's interesting was the respondents understood that they had to improve operations in order to improve what's going on within the business. Right. Be more responsive to business needs, reduce that downtime when it occurs, maintain that uptime and application, reliability and access and all of those things basically get out of the way of the business and support the business. And, and they were looking at automation as a way to do that. And so there's a lot of positivity about using automation. And automation is a way to really sort of help it and help the business as a, as its primary focus. However, there's still a lot of hesitation about, about whether or not they actually get there, get to that sort of, that automation and orchestration nirvana that's being continually pitched to them. And so that was really the top line view of this particular survey.

Speaker A: Well, this is something that enterprises have wrestled with, I don't know, it seems like forever because this is all of the AI ness and autonomous operation kinds of stuff, I think is just the latest iteration. But we've had the tools to do various levels of automation for years and years and it seems like each generation of automation improvements move us forward and yet there still seems to be this fundamental concern from the enterprise side, uh, about the trustworthiness of those automation capabilities. Are we moving the needle or are we getting people Further down the road, I'm curious what has to happen to go to this next level.

Speaker B: So I think we're continually moving the needle a little further or moving that rock down the road a little further. We haven't gotten there yet. And I think fundamentally the challenge comes down to education and experience. If you're an IT person and you're thinking about IT systems and how to operate IT systems, you tend to think atomically. It's like, I've got my network and I need to operate my network, I have my servers, I need to operate my servers and that kind of thing. Automation, relying on automation, building automation systems and understanding how they work is really looking at a systemic view of whatever systems you're looking at. So it's almost like you don't care if it's a switch or a router, it's a networking device which you can now program and you can do things with. So it's that kind of viewpoint that I think needs to shift. And that's a non technological answer. But I think technologically one of the interesting changes that we're seeing, and I'm going to bring up mcp, because we can't talk about automation and AI without talking about mcp. But there's a very fundamental reason that I'm bringing up mcp. In particular, when we start talking about AI sort of managed systems that we're seeing more and more is if MCP is, is deployed, well then that removes a lot of the integration problems that we end up having, which can stym. Which does stymie. I know, it stymies automation. So all of a sudden if your networking company has an MCP server that you can talk to and the other networking company that you're using also has one, you can now talk to these two things in the same language and they'll figure out the details. And that's just one example of some of the things that are going on.

Speaker A: So it seems like we're getting to a point at which maybe that trust factor gets enhanced by, by visibility. Because when you talk about mcp, I mean we've had in some form or fashion there are always APIs in a lot of services. The problem was the availability of that information wasn't great. Everybody had their own specific API. Actually getting it out of something was kind of a pain. But it sounds like what you're heading towards is that getting visibility into that automation flow starts to get you a greater level of trust in that you, you can see what's going on. One of my favorite topics to harp on is the power of technology to raise the level of abstraction and the thing that you're identifying. If it's like if we're doing network automation, the do not really care whether or not it's a switch or a router, but you now have some abstracted controls that let you do very high level things of uh, connect these two things or block this traffic or allow this traffic or whatever. It seems like the missing piece has been the visibility and the observability piece to that. What's the extent to which you see that as actually starting to help maybe overcome some of this resistance? Or is that one more thing that teams have got to understand and trust in order to get there?

Speaker B: They're going to have to understand and trust their observability platforms, more importantly deploy them. But what's interesting about observability is there's sort of the big picture application performance aspects which are useful and beneficial. But when we're talking about IT automation, there's a lot of, I guess I would call it grunt work, foundational work that an observability platform, that that itself is automated does for you. So for example, one of the hindrances of automation is your environment changes and it may change all the time and keeping up with those changes is extremely difficult. I mean you have to be very detailed. Like you change an operating system on a switch somewhere and you've got to log it, document it and make those changes and see if it impacts any of your automation. A lot of the observability platforms can do a lot of that work for you. There's always the change I can integrate it in. You now have this current network model and there are startups that are focusing on exactly this problem. But so now we have this network model or this IT model which is consistent and is always current. So now you don't have to worry about that as much as long as that model is correct. Once that model is correct, now the automation platform, the observability platform, automation platform, whatever it happens to be, can start coming up with Playbooks and scripts and rules and other changes that can be affected because it knows the current state of the system and that starts to unlock a whole lot of things. And you don't even have to go to the point of say writing a playbook. If you've got playbooks for certain actions, the system can just pick those out and go, oh, you want to add an entire new office? Okay, here's the playbook for that. I'm going to pull this over here, I'm going to populate it with the information I need it, I'm going to push it out, I'm going to execute that playbook, you know, that's simple. So it can both use work that you've already done or it can potentially create new playbooks and new scripts that it can then execute. So it takes care of a lot of the basic work.

Speaker A: Well, and it seems like that really addresses what has been one of the big challenges, which is the brittleness of a lot of automation tools. Because you think back to systems administration, Puppet Chef, these were things that back in the day you had specific sets of scripts for specific devices. This server configured this way, you could run these scripts on it and it would do the Windows update on it or add whatever capability was, or whatever was there. But you bought a whole bunch of new systems or you upgraded them, um, you added more capacity, you put new stuff in them. Suddenly you got to change the scripts because now there's more stuff that's in there. And we saw some of this change as we headed to um, tools like Ansible. And you're talking about building Playbooks for some of this. Those are things that started to get better. The cloud automation stuff was always much more expansive. But the nice thing was with the cloud you had a really uniform surface that you were working across because you had a cloud provider running things like Terraform tools and things like that. Hey, they're great because you always have the same means of doing the thing and the services are all reasonably stable and you've got sort of uniform control surface across which to work. But most enterprise environments you don't have that same kind of capability and it's more of a challenge. But I thought it was interesting the point that you were making about using that observability data to do the thing that's really hard, which is one, identify all of the stuff that you've got and two, identify changes that are happening. Because the environment is not static, things are going to change. And the ability to be able to actually integrate that into the automation, that's. I guess we're getting closer to closing loop in both the visibility side and the automation side. But it seems like that that's a big part of starting to automate way some of those challenges.

Speaker B: Yeah, it is. And once organizations start to go down that path of merging observability with their automation platform, with or without AI and that that's already occurring, it starts to unlock a lot of potential, a lot of time for it. For example, some Very simple things that can save hours and hours of time is for example, what happens when an event occurs. An application goes down, there's some kind of disruption. Well, the first thing it has to go to is spend hours gathering up all that data, formatting it, processing it in a way that they can now start to understand it. An observability platform already has that information because it's monitoring the systems so it can create and, or collect automatically and then format that data much more quickly. So it might take five minutes to collect all that data, where it might take a uh, person an hour or two or longer to collect all of it. And it can do it continuously. So you've just saved an hour, 55 minutes in your troubleshooting process. On the back end, if your observability platform is plugged into whatever you're using for workflows, Slack teams, whatever it happens to be ServiceNow, looking at the tickets, looking at the systems, it, it can do things like post mortems, so what happened? And it can write that report. And now all of a sudden not only does it write the report, it can ingest it or send it off to an AI management platform. Now it becomes part of local learning that becomes really useful. And so there's a, ah, bunch of sort of rather seemingly uninteresting things that observability platforms can do today, but actually save IT staff time, allowing them to work on more interesting issues and solving the problem.

Speaker A: It's being able to take that stuff, which is the unsexy part of a lot of those tasks, and being able to tackle them in ways that improve reliability. Because guess what, humans are singularly error prone. One of the reasons why we should be automating is because repetitive tasks that you know how to define and constrain well, automation can do that a whole heck of a lot better than humans can. And that next stage of being able to do all of the tidying, the documenting all those things that, you know, who wants to go write down all the various address changes or configuration changes and what have you. I think also the post mortem piece is another interesting part of that because how many times do teams just get really busy after an event and nobody bothers to write it up because holy cow, we've got one, we spend so much time recovering from the event and then two, we had to catch up with all the other stuff that we weren't able to do because whatever the event was going on and so whatever potential for learning from that event that happened to be just simply drifts out into the ether because we didn't have time to document it. And that's. That has a lot of value in and of itself.

Speaker B: Yeah. Oftentimes people or oftentimes teams don't go back and look at those postmortems. So taking that postmortem and then feeding it back into whatever machine learning system if you have one, all of a sudden that becomes part of the knowledge base automatically and it becomes immediately useful. From the survey that we did, the benefits, some of the benefits are things that we're talking about right now. The benefits that the respondents found, it's things like less time spent on data collection and formatting and that kind of stuff, lowering time to resolution and oftentimes better time to a higher likelihood of figuring out the remediation on the first try. Right. And that's largely because the observability platform has the knowledge and the continual learning where it's able to provide better and better, say, root cause analysis and recommendations the longer you run it and the more that you feed it.

Speaker A: Which brings us to the idea of those learning platforms. AI assists. What, what kind of role can AI play? And it seems like we're also in that early stage of getting to a point of trusting AI capabilities as well, until they get better proven out. Even though we've seemed to have a lot of really great examples. I mean, with my security hat on the ability to ingest vast amounts of data and make sense of what are potentially very rare anomalous events. AI is doing phenomenal things that front. But on operations we seem to be right back into that level of trust concern.

Speaker B: Right. There's a level of trust. It's due largely to lack of experience and to some of the times where AI can be not just wrong, but hilariously wrong. We tend to remember those quite a bit. Organizations are seeing benefits across the board. There's unsexy stuff like feature discovery. So you can go into the gen portion and say, how do I do X? How do I make a report? How do I do this, how do I do that? And it'll surface up that information to you and offer other kinds of features. So there's this just that basic stuff which is just. It saves you reading the manual. Right? Because it can take you right to that portion of the manual that's helpful and that's really useful. But, but more importantly, pulling off your example insecurity by looking at that vast amount of data, there's the potential for an AI platform to see early warning signs that might get missed in that sort of raft of data. So if some temperature probe in Iraq somewhere starts to see an increase in temperature, and then shortly thereafter you start running into other sort of hardware counter errors and then shortly thereafter that you start seeing other sorts of issues. Uh, these are things that may be early warnings of a potential problem. And if you see it once, twice, three times, starts to form this pattern and there's the potential of it can start to alert earlier in the process and say, hey, I've seen these three things. It's led to an outage before, you might want to check and do something about it. There's that kind of early warning, but then it's just making recommendations, helping figuring out, is this an application problem, is this, uh, some kind of code problem, is this some sort of system problem? And at least cutting off the paths that you might think that are not relevant. For example, it may not be the network nor DNS, right. Which always gets blamed.

Speaker A: It's always the network, right.

Speaker B: Everybody hates networking, but think of an AI platform and we're talking in reference specifically to DevOps, but Ops in general, which can go in and look at the application data and say, I see this problem and here's the line of code, here's the database query, here's the whatever, which seems to be causing that issue, then you can now take that and give it to a human and they can go figure it out. Or potentially the platform can say, uh, this is the line of code that keeps breaking or whatever the problem happens to be, here's how to fix it. Then it can actually maybe write the new code block which then gets reviewed and then you can push it live. There's a lot more potential that we're starting to see in and organizations are starting to use those more advanced features as they gain experience with AI platforms.

Speaker A: It's a lot of again, those big picture correlation problems of, okay, we updated this library and rolled out new code. What was the problem? And so we pushed to production and suddenly there was an issue and this one couldn't talk to that one. Well, it must be a network problem. Well, well, if in fact this one had a library update, new code got pushed out to this other system to be able to then actually start to triangulate in ways that might be harder to correlate in other ways if you didn't have that information, if you didn't have the telemetry that was letting you know where changes were being made, where events were taking place, what challenges you actually saw when these things were happening.

Speaker B: Yeah, exactly. It's understanding and monitoring these dynamic systems and being able to pull that information together, analyze it and then point it in the right direction.

Speaker A: Well, uh, I was working on a project many eons ago and seemed like about every other week or so there'd be this one major performance problem. And they were trying to figure out what was going on. And all the systems seemed to be up and things were kind of running and they didn't have a lot of telemetry to actually tell them what was going on. But they knew that maybe there was a network performance problem. And what was happening was in fact the team that was running a warehouse system would update the warehouse system whenever they had new builds. And there was this huge push that impacted both the storage environment and the network environment. And because there were so many disconnected indicators, they couldn't pull them all together. And it wasn't until they got all the teams together to be able to go, think about this. These are the kinds of things that, with greater observability, now you've got a place to be able to go pull all that information together and finally some tools that can ingest and correlate that at scale, which that seems to be the big piece to, uh, my mind. So from an advisory perspective, that seems reasonably straightforward. But you were making the comment that, granted everybody's got an AI fail story somewhere. How long does it take for people to get over those, to get to a point at which they trust the AI in ways, whatever their deployment happens to be, in ways that they start to act on this? Or is this a different way of thinking that's needed around AI augmentation?

Speaker B: I think the trust will build quickly. I think the idea of having a human in the loop is, I think it's going to be with us for a long time, especially for new processes, new issues that are not well understood. It has, you know, they run into issues, it's well understood, it's well documented. So much so that they could give a written document to a junior engineer and trust that when they see these symptoms, they go do this thing. And you don't need to have a senior engineer signing off all the time. It's those kinds of processes that are well understood. You know what it looks like, you know how to fix it. Those are the kinds of things that it builds trust in their own workflows. And so once they give them to an automation platform, they'll build trust that they can just go and automate those things. And then for new events that are occurring or maybe are not as well understood having that human in the loop so the AI platform can go out, it could use the observability platform, whatever data sources it happens to have specialized models that it's using, or general purpose model, it doesn't really matter. Go get all the information from all the IT systems that's current right now, format it, analyze it, present a report, say these are the symptoms that we're seeing. This, this is where I think the problem happens to be. And if there's more than one or two potential root causes, I can list those out and then a human can go, well okay, this makes sense, that makes sense. Can do their own investigation. You now saved all that time that uh, an IT version is spending doing that sort of basic work and then once it goes and says yeah, okay, I think it's this issue, what's the fix? And this is the great part because back when I cared about actually touched IT systems and actually did things, I was a horrible scripter, I was terrible at it. I was good, but I was bad. I was bad. And so having an AI platform which can write a script, suggest fixes can do a lot of that work. It doesn't have to be efficient, it doesn't have to be elegant, it just has to run and run well, that's it, that's all I care about, right? This is not going to be a core loop in my customer facing application. I'm going to run it. All of a sudden that starts to open up a whole lot of possibilities for how to deploy remediations and fix problems more quickly. Take the script that we've been using for the last five years, suggest some changes, somebody uh, can review the changes and then approve them and then push it, execute it and make whatever changes and fix the problem. So, so it starts to unlock a whole lot of really interesting potential. And then we started getting into the possibility of using digital twins to first test fixes and test changes on a digital twin to see what breaks or what the impacts are. That provides another sort of safety net in that whole workflow process, ah, uh,

Speaker A: one more level at which to take it to be able to ensure that you've got sufficient trust. And interesting your point about the digital twin piece because that again, you build the digital twins from all of the telemetry data so that you can identify how it operates. One more reason why you have to have sufficient levels of observability built into that environment to help feed this process, to build trust and to make that whole thing work. Right. Hopefully something that comes full circle Exactly. Well, this has been great, Mike. I, uh, appreciate all the perspectives and hopefully this will help an enterprise or two move a little further up that ladder. But clearly a ways to go, but a lot of benefit if you can get there.

Speaker B: Yeah, there's a lot of benefit. There's a lot of incremental benefit that organizations are seeing. We see it in our survey results and we've continued to see it for the last three or four years that we've been running various observability surveys. It's not like you're going to go from zero and have a fully automated system. There's all those benefits that you pick up along the way. It's a journey with a village

Speaker A: and one that hopefully builds trust along the way. I will point our listeners to the research. It'll be in the show notes, but appreciate all the insights. Thanks for being back.

Speaker B: Great. Thank you for having me.

Speaker A: That is it for this episode of Next in Tech. Thanks to our audience for staying with us and thanks to our production team, including Sophie Carr for odd Mediation. Dylan Schieble on the marketing events teams. If you like this episode, please subscribe or like us. I hope you'll join us for our next episode because there is always something Next in Tech.