K8s Maxxing with AI-Native Platform Engineering Stack with OpenChoreo

1 - > Bret: Welcome to DevOps and Docker Talk, and I am your host, Bret Fisher. 2 - > Technically, the full name of this is Cloud Native DevOps and 3 - > Docker Talk with host Bret Fisher. 4 - > But, you know, DDT is what we call it inside the team. 5 - > So DDT, we're now in our seventh year of podcasting, and if you include 6 - > the livestreams I was doing before we started the podcast, I think 7 - > it's over eight, maybe even nine at this point, which is pretty amazing.

8 - > Uh, not a lot of podcasts get to last a decade, so I'm anxious to 9 - > get there, and still have it called DevOps and Docker Talk, which DevOps 10 - > never died, Docker never died. 11 - > Like, we didn't have to change the name 12 - > But on here, we talk about all cloud native things, and another Kubernetes 13 - > project has begun and has now showed up as a CNCF sandbox project 14 - > this year, and that is OpenChoreo. 15 - > Hence, open… 16 - > Which stands for Open Choreography, and that's where Choreo… I was 17 - > actually mispronouncing it as choreo or something with a hard H, but it's choreo.

18 - > And this project is all about adding on AI agents and essentially a new abstraction 19 - > on Kubernetes itself that interfaces between what we're all now using as our 20 - > primary interface to everything software, which is agent harnesses, and what the 21 - > Kubernetes API and other APIs provide that are typically on Kubernetes, like 22 - > your monitoring or your observability or whatever else you might need in there. 23 - > There's a lot that goes on. 24 - > But this project is adding a new abstraction layer, I'm calling it an 25 - > abstraction layer, because it provides API, MCP, CLI, and web UI all to 26 - > front end everything else you typically deal with, whether that's your GitOps 27 - > tools like Argo, your orchestration 28 - > Your storage, your observability, Kubernetes itself, your 29 - > authentication, like all that stuff.

30 - > And we are now in the world where we're all playing around with agent SDKs and 31 - > maybe trying to figure out what can we automate in our infrastructure safely 32 - > with reliable LLM prompts like this. 33 - > These are all things we're probably thinking about, right? 34 - > If you're not doing it, you're thinking about it, you're waiting for someone 35 - > to tell you how to get it done. 36 - > And OpenChoreo is a new project that came out of WSO2, the company, and 37 - > they were running it on their own SaaS infrastructure and realized that it might 38 - > be useful to others, which by the way, was also how the Docker project got started 39 - > by a SaaS company or a cloud company realizing that their infrastructure 40 - > tooling was maybe the most useful part of their product, and released it for free.

41 - > So OpenChoreo came out of a business hosting a legitimate 42 - > tool for Kubernetes users and felt like it needed to be open source. 43 - > So now they're gonna be running that on their own platform 44 - > for customers eventually. 45 - > But for now, they're just focused on getting it open source. 46 - > And now that it's officially a CNCF sandbox project, that means they've met 47 - > the minimum level of sort of maturity to explore whether this could succeed in 48 - > the marketplace, explore whether or not those of us that are building open source 49 - > tools or running open source tools on top of Kubernetes would be interested in 50 - > an agent platform on top of Kubernetes.

51 - > And specifically what this is about is about us administrating and managing 52 - > these environments with agent harnesses through the OpenChoreo abstraction, 53 - > so through their MCP API CLI. 54 - > This is not about running inference or building custom models. 55 - > This isn't about being an AIOps or MLOps person, and this isn't 56 - > about running custom agents that we built in pods inside of a cluster, 57 - > although that's certainly possible. 58 - > You can do that.

59 - > But the main focus here is on exactly what I talk about on this show, as well 60 - > as my other podcast, Agentic DevOps. 61 - > And quite frankly, this podcast could probably be on both. 62 - > I'm trying to prevent that from happening because they're, you 63 - > know, we've all got podcast players. 64 - > You can add both of my podcasts to the, your player, right?

65 - > So you know that that's a thing. 66 - > Um, and this one I'm trying to keep focused on CNCF projects, Kubernetes and 67 - > Docker projects, and the Agentics DevOps podcast is really about the patterns, 68 - > behaviors, and tools for just being a DevOps or platform engineer or SRE 69 - > who's using AI to get their job done. 70 - > That's what that podcast is about. 71 - > Technically, this could be on both.

72 - > But, uh, anyway, On the show this time, I have Lakmal and Sameera from 73 - > the project team over at wso2 talking about this, getting into the details 74 - > of what it does and doesn't do, and also explaining to me how it's not 75 - > what I actually thought it was when we first started talking about it. 76 - > And we even on the livestream, which you can link in the show notes, there's 77 - > links to the actual demos that we tend to cut out of the podcast version of this, 78 - > but the demos are from the livestream.

79 - > You can see how the product works, how you can use agents to interface with 80 - > Kubernetes, which we, could use agents before, but we didn't necessarily 81 - > have all of that infrastructure in place besides just a raw API. 82 - > So this helps solve a lot of that with a new level of abstraction 83 - > that we get into through this show. 84 - > So let's get into it Welcome to the show. 85 - > in the middle there, we got Lakmal, and on the right, Samira, both from WSO2.

86 - > Lakmal, tell us about yourself. 87 - > Who are you? 88 - > Why are you here? 89 - > Lakmal: Yeah, I'm, working as a VP and distinguished engineer at WS02, 90 - > also a Choreo, DM before Choreo BU.

91 - > yeah, I started my career as a system engineer, in 2000, one, 92 - > and last 25 years, I engaging with building many, many platform. 93 - > Yeah, here we are today. 94 - > Bret: Nice. 95 - > Sameera, how about you?

96 - > Sameera: Hey, Bret. 97 - > I'm good. 98 - > I'm Sameera Jaswal. 99 - > I'm one of the core maintainers of OpenChoreo at WSO2, too.

100 - > I started my career in the middle as a, you know, you can think of programming 101 - > language design, compile engineer, platform architect, and then you'll 102 - > hear more about how abstractions and compilation throughout this talk. 103 - > me. 104 - > Bret: a gist by the end of what you really are into. 105 - > I'll get it.

106 - > Okay. 107 - > all right. 108 - > So y- you all reached out. 109 - > This is a new project.

110 - > I'm always excited about new CNCF projects because it kind of… It - 111 - > Once they've accepted into the sandbox process, we kind of get an idea. 112 - > I feel like it's reading the tea leaves a little bit. 113 - > Not everything is a winner, not every project makes it to graduation, but it's 114 - > an interesting way, I think, to, like, separate sort of the hobby projects or 115 - > the experimentations from things that teams are taking seriously, 'cause 116 - > you don't really go for sandbox status unless you're taking things seriously.

117 - > That's where I sort of like wake up and start paying attention to 118 - > the new projects, and I'm excited to talk about this today because 119 - > I'm obsessed with AI agents. 120 - > I'm obsessed with, like, how we manage infrastructure with agents involved. 121 - > How do we put MCP in our clusters safely? 122 - > how do we sit on our harness as our primary mechanism for interacting 123 - > with, I don't know, 20 to 40 years of complexity that we've all built up?

124 - > Like, the number of abstractions we all have to deal with today is insane, 125 - > and it's amazing that anyone can actually go to a conference and have 126 - > a common conversation, 'cause there's a thousand tools we're all using. 127 - > So I'm excited to get into this. 128 - > how did the project get started? 129 - > who's gonna pick the first question?

130 - > Like, how did this project come out of the company? 131 - > Lakmal: Yeah, we started this, software offering called 132 - > Choreo SaaS, five years back. 133 - > it's, it start as a, the IPaaS initially, like a integration platform as a service. 134 - > Then it evolved into the… You can run all other services 135 - > within the platform itself.

136 - > So it's, it evolved into the internal developer platform, couple of years later. 137 - > Name is also coming as a choreography. 138 - > It's, we shorten it to the Choreo. 139 - > It's like a choreography your microservices in a certain way, like a 140 - > Kubernetes do the orchestration, we do choreography for all your microservices.

141 - > That, that how the name also came. 142 - > so we were offering this SaaS offering to our customers, enterprise customers. 143 - > It's fully managed by WS02. 144 - > we were running Choreo version 2, and two years back we started 145 - > Choreo version 3, project.

146 - > So next version of the, our current offering. 147 - > Then eventually it's become the OpenChoreo and in WS02, all our software open source. 148 - > So we thought of, "Okay, why not we are running… we having this software 149 - > running as our SaaS. Why not open source it?"

That's how this OpenChoreo, started. 150 - > Then, this year, January, we have donated into the CNCf, Foundation. 151 - > Now we are a sandbox project. 152 - > Bret: Yeah.

153 - > All happening so fast. 154 - > that's a similar path, by the way, to how Docker was invented, 'cause it 155 - > was a platform company saying, "Hey, this piece of our platform is pretty 156 - > interesting, but it's also c- at this point maybe a utility. let's just give 157 - > it away." Turned out that that worked.

158 - > so it's not a bad idea to do this. 159 - > what was the unique problem we were trying to solve with this? 160 - > Like, I say we. 161 - > I'm, I'm a part of the team now.

162 - > how is it, how does the team look at it from, like, this is a unique 163 - > thing in a world where we've got dozens of Kubernetes distros? 164 - > In fact, first off, I should say, do you even call, do you call 165 - > this a, do you think of this as like a distribution of Kubernetes? 166 - > Sameera: N-not really. 167 - > Bret: No?

168 - > Like, do you see it as like a vanilla Kubernetes with some things bolted on? 169 - > Or like how is this different from like a Rancher? 170 - > Sameera: okay, I'll, I'll take that question. 171 - > in my opinion, so OpenChoreo is something you, something we 172 - > have built on top of Kubernetes.

173 - > Bret: Mm-hmm. 174 - > Sameera: So you can use any of Kubernetes distro, Rancher, EKS, AKS, 175 - > like any Kubernetes, and then you can just install OpenChoreo on top of it. 176 - > So the idea behind is that, so you have Kubernetes and all the 177 - > other tools, and then, so you have a developer platform, I think. 178 - > what we have seen in the industry is, so you ca- you have this developer 179 - > platform with Kubernetes and Argo CD and all the things, glued on, 180 - > and then you expose that layer to developers, and you build a developer 181 - > experience around it, portals, MCPs.

182 - > then we have figured out that, that way you expose the platform 183 - > complexity to your developers. 184 - > So they'll have to learn Kubernetes, they'll have to learn all the tools. 185 - > I think what's unique about here is, I don't know whether this will work out, we 186 - > build a middle layer, abstraction layer on top of Kubernetes and other tools. 187 - > And that's what Lakshman said earlier.

188 - > Through like six years, we have figured out the abstraction layer. 189 - > That took some time for us to build. 190 - > And then that abstraction layer helps for us to build a developer experience layer. 191 - > So developers, they're like kind of abstracted away from Kubernetes, per se.

192 - > They can still see what's going on, but they may not to learn a lot of things 193 - > about Kubernetes and other tools. 194 - > Bret: Yeah 195 - > Sameera: so that's about one of the unique layers in the platform. 196 - > Bret: So when we talk specifically about Open- OpenChoreo, 197 - > is it installing all the parts of this? 198 - > Okay, so we're looking at it for the, for the audio audience.

199 - > We're looking at a visualization. 200 - > I love diagrams 'cause they really help me I l- the conceptual-ness of like what a 201 - > thing is, is sometimes instead of a bullet list, I love these, so, I brought that 202 - > up, and there's all sorts of com- It's… 203 - > To me, it looks like a full-fledged, fully built out Kubernetes cluster 204 - > with all the extras, right? 205 - > Like, all the necessary things that a sort of an enterprise cluster would have. 206 - > it's got these data components.

207 - > You know, you're mentioning Keda for scaling. 208 - > You've got Cilium in the networking. 209 - > You've, you've got, you know, API gateways in there, and then you 210 - > showed part of the diagram, there's the workflow part where it's Argo 211 - > workflows and build packs in there. 212 - > And then you've got observability layers where you've got probably , I don't 213 - > see Loki in there, but like Prometheus and all the OpenTels, OTel stuff.

214 - > And then over to the side, there's OpenChoreo as this control plane. 215 - > So it's almost like to me it's like the control plane on top of the control plane. 216 - > so is this really just focused about those things on the left and you're showing how 217 - > it interacts, or is this actually helping us implement the rest of this puzzle? 218 - > I was a little confused by that.

219 - > Lakmal: Yeah. 220 - > I think the main idea behind the, having the unified control plane 221 - > across all these, capabilities. 222 - > p-platform engineers or developers or even now AI agents can interact with 223 - > control plane via our different protocols. 224 - > And then control plane do the, all the orchestration across all these other 225 - > planes, like you, you mentioned the data plane with the Cilium network policy, 226 - > with the S-Scale Zero, with the Keda.

227 - > All these thing can orchestrate using a, a control plane with the governance 228 - > enforce through the control plane. 229 - > That's the main idea behind. 230 - > So what, what have seen as a problem, even we expose all these services to 231 - > the end user, it can be a platform engineer, it can developer, then 232 - > you can't enforce the governance. 233 - > But having this single control layer, we can have the guardrails, 234 - > policy enforcement, and y- single, pane of glass, we can see 235 - > everything within the control plane.

236 - > So that help to orchestrate across entire different pl-planes. 237 - > That planes, built b- using the modular architect in OpenChoreo. 238 - > So for example, i-if you take a API gateway, it can be come from the, WSO2 239 - > gateway, K gateway, so, Co-Kong gateway. 240 - > It doesn't tie to a single vendor.

241 - > So it, it's building an entire ecosystem around it. 242 - > it's not only the platform itself. 243 - > Users can pick and choose what they want to use in their data plane. 244 - > so that, the abstraction layer help to, os- orchestrate all these 245 - > different, vendor tooling that they want to use in the, their planes 246 - > Bret: so, so is it - If I'm a platform engineer, I - and I'm listening 247 - > to this show, that probably means I have platforms today, right?

248 - > Like I've already got Kubernetes in production. 249 - > I've already got a lot of these components. 250 - > So is the idea h-here that this particular project is for implementing the left 251 - > side of my screen of the specific OpenChoreo control plane components, 252 - > and then including those system components that would link or integrate 253 - > me with the APIs of those other things? 254 - > Is that how - Is that sort of where you're drawing your boundary 255 - > of where your project ends?

256 - > Sameera: Yeah. 257 - > yeah, I think that- that's a good way to put it, Bret. 258 - > So our main component would be the control plane, that you 259 - > see on the left of the screen. 260 - > And then at the same time, there are certain system components 261 - > that are running in all the other planes as well, data planes, 262 - > observability plane, workflow planes.

263 - > So if you're already running these projects, then it's just a matter 264 - > of like installing our system components and organizing your sys- 265 - > platform architecture in this way, 266 - > Bret: Yeah 267 - > Sameera: right? 268 - > and then I think what Lakmal said about abstractions is that the… 269 - > When you think about these developer platforms, primary users are platform 270 - > engineers, as you said, and then, primary consumers are developers, 271 - > Bret: Right 272 - > Sameera: So, so they will perhaps, they will use the experience plane, 273 - > UI, MCP server and things like that.

274 - > They will say, "I, I want a project, I want a component. I want component to 275 - > de- deploy on dev environment," right? 276 - > And the control plane understands that and executes and compile that into 277 - > Kubernetes and other projects, so in a way that other projects understand, right? 278 - > So that's how these arrows and everything, that's the idea behind these arrows.

279 - > Bret: Yeah. 280 - > And when I'm looking at this diagram, I mean, from s- from sort of a 281 - > platform team perspective, I mean, I recognize all the names, right? 282 - > It's a lot of terminology, from different, CNCF projects. 283 - > But, are you thinking of this as AI is the sort of main interaction point?

284 - > I see here that we've got, like there's, I think there's a, a web console or 285 - > something, So, do you think about this from the perspective of like AI 286 - > is going to probably be increasingly the way we interact with Kubernetes? 287 - > So this thing, you know, we've got these arrows in here of how things are 288 - > interacting with the Kubernetes cluster, and it feels like it's all going through 289 - > the OpenChoreo control plane to get there. 290 - > 'Cause you've got this list of like, we could use the U- Backstage UI, 291 - > presumably that would be for humans only.

292 - > unless maybe there's some pretty graphs for the AI to look at. 293 - > I'm not sure why it would use that. 294 - > But then you've got MCP, CLI, and API as these other interaction endpoints, and 295 - > they're not… When I think, when I see this diagram, I think, "Oh, this allows me 296 - > to put another layer of abstraction for my AI to work through, and so I'm not running 297 - > my AI directly against a kubectl API, or, you know, the Kubernetes native API." 298 - > I've sort of got this enhanced layer, with additional functionality, presumably 299 - > maybe some… Yeah, I see you have authorization and access control, 300 - > which is a big thing for me lately.

301 - > Like how do we minimize the blast radius of giving agents access to different 302 - > tokens and like not giving them root, you know, God rights to the cluster every day? 303 - > which I see a lot of people doing by just giving them the kubectl admin abilities. 304 - > is that kind of the, the goal of, of where you see agents being 305 - > responsible for all these clusters? 306 - > Lakmal: Yeah.

307 - > E-exactly that, you describe is correctly, Bret. 308 - > So, what we want to have is, now in the developer platform, we give the 309 - > golden path, the guardrails, the policies to our human developer. 310 - > The same way we want to expose this golden path to the AI agents 311 - > that interact within the platform. 312 - > So in, in OpenChoreo, the agents are in first class citizens, so 313 - > they can, at the moment, current, release, we exposing two MCP servers.

314 - > One is the OpenChoreo control pane MCP server, other one is the 315 - > OpenChoreo observability MCP server. 316 - > So even external agents can interact with this MCP as well 317 - > as internal agent, inbuilt agents can interact with MCP C- servers. 318 - > Now, when they interact with MCP servers, they have the permissions 319 - > and all the guardrails will apply same as the developer, using the platform. 320 - > It's a unified experience, whether it's a human developer, human SRE, or a agent, 321 - > working with the, SRE a- or as a developer 322 - > Bret: Very cool.

323 - > All right, so this shows up as a whole bunch of MCP tools that I 324 - > could plug in locally to my local… So if I've got Claude Code running, 325 - > I plug the MCP endpoints in. 326 - > do you know, like, the estimated tool count? 327 - > Like, what are we dealing with here? 328 - > 100, 100 tools?

329 - > 20 tools? 330 - > Lakmal: yeah. 331 - > we have, depending on the user persona interact with MCP. 332 - > So we have the developer persona, we have the SRE persona.

333 - > They give different, tooling based on the, what their role. 334 - > Yeah, so based on that they can do different, different, activity. 335 - > Sameera: roughly 100. 336 - > Bret: Okay.

337 - > Okay. 338 - > Sameera: say 100 tools altogether 339 - > Bret: a not insignificant amount of tools, but that's a good point. 340 - > So what you're saying is based on who I'm au- authing as, which role I'm authing 341 - > as, I, I get a scoped set of MCP tools based on my permissions essentially? 342 - > Yeah.

343 - > Sameera: exactly 344 - > Bret: So, all right. 345 - > so these agents are coming, you're listing AI agent modules here. 346 - > You got SRE, FinOps, and architect. 347 - > are these things that are, that are coming defined?

348 - > Like, and what does that agent mean to me? 349 - > Like, if I'm on my local harness, does this mean that those are long-running 350 - > agents that are sitting in the cluster? 351 - > Like, how do I interact with those agents? 352 - > Sameera: like Lakmal said earlier, we have internal agents and external agents.

353 - > They both use the same experience, plain MCP servers, tools. 354 - > Claude Code is the external agent. 355 - > these FinOps, SRE, and architecture agent, they're like platform 356 - > agents running in the cluster, 357 - > Bret: Yeah 358 - > Sameera: and they are reacting to certain events in a way. 359 - > For example, if you, if there's an alert, like lot of i00, errors are 360 - > going on, then our SRE agent will react, and then it'll give you some report on 361 - > what's going on, why this is happening.

362 - > So you can configure that way. 363 - > You can configure the agent with certain alerts, types of alerts, 364 - > memory, memory, like if you are consuming high memory, certain limit 365 - > that SRE agent will again react. 366 - > Bret: Sure 367 - > Sameera: FinOps agent is also similar in that way. 368 - > Yeah.

369 - > Bret: I see. 370 - > So each of those agents comes sort of predefined, it sounds like. 371 - > Like, they have a scope to them. 372 - > And then presumably, I guess I'm somehow through the control plane, I'm 373 - > plugging those into, like, my messaging apps or other… Like, into Slack or 374 - > whatever, and that those are how those agents reach out to me is through… 375 - > Or do they go through a middle layer of, like, an alerting system 376 - > like Prometheus has for alerts?

377 - > Like, how does that path look for the agents letting 378 - > us all know of the problems? 379 - > Lakmal: Yeah. 380 - > at the moment it's going a middle layer, the alerting system. 381 - > we have the log-based alerting, with the OpenSearch and other layer, 382 - > and the Prometheus for the metrics.

383 - > So agents also ke-going the same layer. 384 - > this alerting system can be, configured with the PagerDuty or 385 - > Slack channel or the other channels. 386 - > That how the architecture currently working. 387 - > Yeah 388 - > Bret: Yeah.

389 - > And so this feels like it's designed to be sort of, I mean, we, I think the website 390 - > or something said batteries included. 391 - > I think I included that in the, in the thumbnail. 392 - > It, 'cause it, a lot of the, the conversations that I have, 393 - > I mean, I'm lucky to be in this, this DevOps, Agentic DevOps guild 394 - > that I run, and we meet weekly. 395 - > So I get to talk with dozens of teams around, like, what they're trying to 396 - > do right now in terms of everything agents related to infrastructure 397 - > and this exact stuff, right?

398 - > So people are sort of in that middle mode right now where 399 - > we've - the local agent harness has kind of become common knowledge. 400 - > Like, we've all experimented with GUIs and TUIs and multi-agent 401 - > management and sessions and skills. 402 - > Like, we're all, you know, this is the year of s- playing with skills. 403 - > And, but then the minute that I have to start running agents, I'm, I just 404 - > call these things server agents b- to try to define the difference.

405 - > These 20, these long-running, always looping agents that are sitting there 406 - > checking something or, continually polling for something or waiting for a 407 - > webhook, maybe if they're event-driven. 408 - > Like, these things are probably gonna be everywhere. 409 - > We're gonna have them all over the place. 410 - > And y- you're providing th- a few of those that are sort of out of the box defined 411 - > to do specific things, so I don't have to go find an agent SDK and make up my ideas 412 - > of what an agent might do in my cluster.

413 - > 'Cause that kind of feels like what a lot of us are doing right now, 414 - > is we're sort of making up, we're writing little a- agents everywhere. 415 - > We're making it up as we go along 'cause we don't really know what 416 - > parts that they, we would automate or which parts they would be good at. 417 - > You know, what parts of summarization and human judgment do they do well, 418 - > and what context do they need? 419 - > And that's… So, but it feels like the rest of this system, is 420 - > the rest of this system to help them with the context problem?

421 - > 'Cause that always feels like a really hard problem right now around 422 - > how, what context do I need to give my agent to make it useful and not 423 - > hallucinate, and how do I get at that stuff at the time it needs it? 424 - > Lakmal: Yeah, exactly. 425 - > So I think the main idea behind we have this ecosystem, within 426 - > the ecosystem we have the agents. 427 - > So, in community or even, we, OpenChoreo maintainers will release a agent, then 428 - > our users can pick and choose this agent run in their OpenChoreo system.

429 - > So for example, for SRE agent, the intention behind that, okay, when I say 430 - > when something happen, SRE agent can, triggered and we feeding, all the logs, 431 - > matrices, config changes, code changes to the SRE agent within that window. 432 - > Then, what happen, so it can generate the RCA report, root cause analysis report, 433 - > and also alerting to the SRE, human SRE. 434 - > When they come into the system, they already have the root cause 435 - > analysis report there, that is providing to the SRE agent.

436 - > And also we are not stopping there. 437 - > this agent can provide the re-remediation action as well. 438 - > Okay, this is a quick fix if you want to, fix this issue. 439 - > So then at the moment it's a human in the loop, SRE, human 440 - > SRE can apply these fixes.

441 - > But, I mean, we have the permission model where we can give some of the, 442 - > this agent to automatically apply these fixes, themself, rather waiting to SRE. 443 - > We can say, "Okay, is it only config change? I will allow my SRE agent to 444 - > apply that change and fix the issue." So eventually it can take the more, 445 - > more like activity, but it's again, it's had the human control, in the loop.

446 - > So that's how the, we have designed it. 447 - > Like you said, the main i- intention behind it, so we running a system 448 - > agent, you can, you say, survey agent. 449 - > We know what are the context we need, feed into this agent to 450 - > provide a better, better result. 451 - > So that's how these inbuilt agents are, acting.

452 - > Sameera: Yeah, and I also think, like, if you're running something, 453 - > this is, these agents are a good starting point for you to come up 454 - > with your own agents as well using. 455 - > And then you can replicate with, you can, come up with your own agents because the 456 - > context problem is kinda solved i- in, within the platform, what Lakmal said. 457 - > So you can… Because these are all about not exposing Kubernetes details 458 - > into the agent, but exposing the OpenChoreo abstraction layer, so that 459 - > way it's what we have figured out.

460 - > I mean, not sure whether it'll work. 461 - > Bret: that's the hard part. 462 - > Yeah. 463 - > Sameera: Yeah.

464 - > Figure out is this abstraction field helped us to give the right context also 465 - > Bret: Yeah. 466 - > all right. 467 - > so I was writing this article over the last week around what does an 468 - > AI Kubernetes platform even mean? 469 - > And what we're really talking about here is something that I had to define 470 - > in this newsletter as what I'm calling type three AI on Kubernetes .

It sounds 471 - > really nerdy, but if you've been going to KubeCons since the invention of 472 - > ChatGPT, for most of the history of what we now call gen AI or whatever - we 473 - > are not talking about AIOps and MLOps. 474 - > I mean, technically you could use this thing to manage that too, but for those of 475 - > us that go to KubeCon, there's been - I've been ranting a little bit for a couple 476 - > of years now that there's been this weird disconnect where when a lot of people 477 - > in the industry talk about Kubernetes and AI, they're actually talking about 478 - > running AI or making AI, building models, reinforcement, learning, running 479 - > inference, and that, that is not my world.

480 - > Like, I am not that person, right? 481 - > It's interesting from an architecture and engineering perspective, but 482 - > I don't think I'll ever… that to me, that's a speci-specialty and it 483 - > might get easier to do, and some of us might have to have that new role. 484 - > But I feel like there's still a significant, maybe even majority 485 - > of us that won't be doing that. 486 - > our new job is to build out these agents and understand where the, you 487 - > know, the features and the edges of models can be implemented so that 488 - > we can automate and build more.

489 - > Because I see this whole thing as just like VMs, the cloud, the invention 490 - > of the PC, you… I-over my 30-year career, we've had major pivoting points 491 - > in technology that essentially allowed us to scale ourselves as a human. 492 - > We went from managing 10 servers in the '90s to 100 servers with VMs to 493 - > a, a thousand servers with the cloud. 494 - > Maybe containers allowed us to run 10,000 pods per, per admin, and 495 - > then now this is the next level.

496 - > And to me, what you all are building is exactly that layer of abstraction, which 497 - > is the agents and their context management and permission management essentially. 498 - > that layer is now this new level of abstraction that will 499 - > hide some of the complexity. 500 - > Like, we don't have to know every kubectl command in the world anymore. 501 - > In fact, I don't even know how we all get Kubernetes certified because if 502 - > we're not gonna be running kubectl and have to know every option, what 503 - > is a Kubernetes admin test other than just understanding architecture?

504 - > But my rant here is that like I had to go through… This post took me a re-week 505 - > to write because I was trying to theorize around we're really talking about agentic 506 - > operations, and that's where I'm - that's the reason I have the new podcast. 507 - > That's the reason this project is exciting, is this is that new layer. 508 - > This is the abstraction that we're all kind of searching for 509 - > because I think we've understood the local harness now after having 510 - > those for a little over a year.

511 - > I think a lot of us are like, "Yeah, I, I get this." We have all sorts of neat 512 - > ways of managing these little local agents, but this new nebulous layer that's 513 - > sitting in front of our infrastructure is still not fully realized. 514 - > So I feel like you all are pretty early in the game in terms of saying, we can help. 515 - > We have this defined component, and we can shove it in there."

And, I'm even trying 516 - > to draw out a diagram of what- If we're all on a maturity path to, we started 517 - > with the agent harness locally, and the end goal is we've maximized the agent 518 - > assistance that we're all gonna have. 519 - > Like the dozen agents we're gonna have, two dozen agents, we've maximized 520 - > all the functions and features of what models can do for us as DevOps 521 - > platform engineers, SREs, right? 522 - > all of us are managing this infrastructure and there's a dozen tools and f- the 523 - > thousand features we all now need to learn, and we're learning skills 524 - > and agent files and MCPs and where all that can go awry and go crazy.

525 - > But eventually the idea is we're improving our productivity, we're improving our 526 - > management, we're hopefully making things more secure and reducing, 527 - > outages and everything's getting better. 528 - > But at the end of it, what are we doing? 529 - > And I think a year ago we were all kind of nervous like, are our jobs going away? 530 - > that's been a big conversation.

531 - > are we - Is there only gonna be one DevOps person in the company? 532 - > But to me, this is the job, like this is the new job, is you all built 533 - > this, now we've got to implement it. 534 - > Now we've got to understand it, we got to understand the components. 535 - > Where do the agents help?

536 - > where do they not help? 537 - > where do we still need the human in the loop, like you said? 538 - > and I just wondered if that's, if any of that is anything that you would 539 - > agree with or that I'm… Am I crazy? 540 - > Does this all sound like a good idea to talk about?

541 - > Lakmal: No, I think, this is great, Bret. 542 - > I think you're, you are correct. 543 - > So yeah, it's evolved last one year, within the one year, we can run agent in 544 - > production, like helping to the different personas engaging within the platform. 545 - > I believe the generating code problem is almost solved now with the Claude 546 - > Code or other Cursor or Codex, right?

547 - > So people can generate code, write application within minutes So we 548 - > call it as vibe coding, right? 549 - > Everyone call it as vibe coding. 550 - > When the moment they hit the deployment, their vibe is ending. 551 - > Because now with vibe coded, application has to be promoted into the production.

552 - > But still, the platform doesn't support - they have to m- go away from their agent. 553 - > They have to either create a ticket or like, go to the self-service 554 - > portal, and deploy the application. 555 - > But what we have seen, within the, our, skills and MCP servers, 556 - > they can use the same, agent they are using to develop, their code. 557 - > They can say, "Okay," I can say, "I want to deploy, the, my application 558 - > into the development environment.

559 - > Just you figure it out how to do it." So now it's, we call it vibe deployment. 560 - > So OpenChoreo try to fix, fill the gap, vibe deployment part, 561 - > how the operations side of it. 562 - > So you can write, agent, you can write your code using agent.

563 - > Now you use, you can use the same agent to deploy your application into the 564 - > production and operation, with support of OpenChoreo, section as well as the 565 - > tools supporting to the MCP servers. 566 - > that's, where we are, we want to play, around this, platform 567 - > engineering side of it. 568 - > Sameera: Yeah. 569 - > And Bret, I think you mentioned, some-something about how 570 - > abstractions help agents, right?

571 - > I think the way I think about that is more abstractions, fewer tokens, 572 - > Bret: Hmm. 573 - > Sameera: so they don't have to learn Kubernetes deep YAMLs and kubectl. 574 - > like, I think the same applies to programming languages in a way, 575 - > writing assembly versus writing in Java or C# or Go, right? 576 - > fewer tokens in that way.

577 - > So they, they have more context with less tokens. 578 - > I think that is also, that's what I'm, working on these days, see 579 - > whether we can establish that. 580 - > Bret: Yeah, that's a, I think we're all very quickly, especially these last three 581 - > months when it comes to tokens, we're very quickly understanding that this is finite, 582 - > and that, just kind of like Kubernetes itself, we've all had that moment where, 583 - > early in our Kubernetes journey, we would realize how much infrastructure 584 - > we actually need to just run the apps.

585 - > Like we, you know, the, when you build out a full-scale Kubernetes 586 - > cluster, th- there's a s- there's a non-insignificant amount of infrastructure 587 - > to manage the infrastructure. 588 - > And that, back when we first started the journey on Kubernetes, I don't 589 - > think we all really understood, the eventuality of dozens of different 590 - > controllers and dedicated nodes in the control plane that we were gonna have 591 - > to have just to organize and Uh, to, to herd the cats, as, they might say.

592 - > And, I feel like now the same thing is with tokens. 593 - > It's like we got all excited, and we're like, "We're gonna put it everywhere." 594 - > And now we're gonna very quickly end up in a world where we've got budgets, 595 - > and now we've got to optimize our agents, and we have to care about the 596 - > model, and we have to, we have to… 597 - > One of the conversations we're having in the Agentic DevOps Guild is around 598 - > evals and how can we use evals, to sort of figure out which model, if 599 - > I can apply a bunch of skills, if my agents are really just behaving with a 600 - > bunch of skills and context management stuff, how can I, sort of evaluate or 601 - > programmatically determine or test that maybe I could use Kimi or, MiniMax or 602 - > Qwen or some, sort of open weight model that's super cheap for my very specific 603 - > little, you know, FinOps niche agent.

604 - > Whereas it maybe doesn't need Opus because maybe that's the one that 605 - > I need because I have dumb prompts and I s- I type dumb things. 606 - > And maybe my agent in the cluster can be running really cheap because none 607 - > of us wanna put these in production and then find out a month later that it's 608 - > too expensive to run, and now I have to shut it down and make the humans 609 - > do it again because I don't think that's a world I, I wanna go back to. 610 - > Lakmal: That's true 611 - > Bret: I know a couple of companies that are playing in this space.

612 - > We've had Anyshift on the show, we've had, Mendral on the show, 613 - > and these are both AI startups that are building agents to help with 614 - > infrastructure, CI/CD, sort of helping, helping you with fault remediation. 615 - > and the pro- approach that I've been seeing everybody take is, 616 - > like, it's just sort of the normal pattern of engineering approach. 617 - > We walk before we run, you know, we r- we run before we sprint, and that means 618 - > on day one, you might be read-only.

619 - > agents can't do things, they can just go look up things, and that 620 - > they're just a little helper assistant, and they're not gonna… 621 - > They have no control over anything. 622 - > They're just here to help you find information in the sea of, 623 - > of infinite information that we have about our infrastructure. 624 - > And then, then, like, with Mendral, the team with Mendral that's been on 625 - > the show, they're now to the point where, very specific use cases, they're 626 - > letting the agent have a tiny bit of control because it's predictable.

627 - > They can test and make sure that it's, that it works, that it- 628 - > it's as declarative as possible so that it is predictable, and 629 - > they start to, to let that happen. 630 - > is that something we can do with OpenChoreo? 631 - > Like, does this have sort of a default stance of, "I'm only gonna be read-only"? 632 - > how does, how do the permissions work that way?

633 - > Lakmal: Yeah, it's not only the read-only at the moment, but you can control it. 634 - > we have the permission model where, you can give the, different 635 - > tool to the UI, your agents. 636 - > It can be read-only tools, it can be write operational tools as well. 637 - > Based on the, what we want to do, you can allow this permission, because agent 638 - > have their own identity, and they, within the identity they have the permission.

639 - > So based on the permission, they can use different tools. 640 - > if you are confident enough that your agent is acting, performing well, you 641 - > can give the right permission to some s- some level of factory, for example. 642 - > you can give like, okay, if there are configuration issue, agent can go and 643 - > fix it itself, the configuration, fixes. 644 - > So you can fine-tune how you, your agent can interact with your platform 645 - > Bret: Yeah.

646 - > I've been wondering about a lot of this around, like, how is per- 647 - > how are permissions gonna work? 648 - > so much of our permissions models today feel like they're not designed for an 649 - > agentic world, and, how are we gonna track the actions of these things? 650 - > Like, every little step they take, is hopefully gonna be put into some 651 - > sort of observability platform. 652 - > So I'm, I am very curious to see how, what your all's vision is.

653 - > Okay. 654 - > Yeah, I'm trying to envision, what ends up… Like, how to correlate this to 655 - > the things we know today, like GitOps repos with Argo CD definition files 656 - > in them that, that get, you know, pulled in because Argo's watching 657 - > or Flux is watching a different repo and it's looking for YAML changes. 658 - > And, I'm assuming, does this change that workflow? 659 - > Does… I remember seeing GitOps on the diagram, so I'm kinda thinking, 660 - > like, how does this correlate to sort of that, that GitOps loop that 661 - > we've seen so many teams adopt?

662 - > Sameera: Yeah, I think these, the CRDs, right? 663 - > so they, you can put all of these in your, in your GitHub repo, and then you 664 - > can configure Argo CD, configure Flux. 665 - > the normal workflow will work as it is, right? 666 - > Bret: Okay 667 - > Sameera: And so there are two modes, right?

668 - > Here, I think we are using more ClickOps, ClickOps approach, but you can configure 669 - > the same thing with, your GitOps. 670 - > Bret: Oh, 671 - > Sameera: will talk to… Yeah 672 - > Bret: I see. 673 - > S- so yeah, so in this case, what we're saying is like, I mean, yeah, 674 - > we're looking at a web dashboard, but we're not really… We're j- 675 - > I mean, for the audio listeners, we're, we're looking at a dashboard. 676 - > We're not really, we're vibe opsing, I guess.

677 - > Not ClickOpsing, 'cause we're not clicking on the 678 - > Lakmal: Yeah 679 - > Bret: dashboard. 680 - > we're using the agent, but the agent is causing… Just to be 681 - > clear, this is actually a question. 682 - > So the agent is causing OpenChoreo to create new resources inside the 683 - > cluster for deploying this inf- this, the Google, microservice demo. 684 - > And I, that is an alternative to having an agent that I just say, "Hey, we're gonna 685 - > make a new PR in YAML, and you're gonna push that to GitHub, and then Argo or Flux 686 - > is gonna pick that up later."

because I guess I could, I could tell these agents, 687 - > or I could give them skills or change the skills somehow so that it knows that 688 - > that's my workflow, and that I'm just using it as sort of a read-only partner. 689 - > But kind of just like we're not supposed to go into the AWS console 690 - > and ClickOps away there, like, I don't necessarily want OpenChoreo changing my 691 - > infrastructure outside of my GitOps loop. 692 - > I'm actually just curious, do you think that the GitOps approach is less 693 - > important if we have this sort of chain of events and… or do you, like, do 694 - > you see enterprise teams possibly mo- shifting to more of this vibe approach 695 - > since we have something in the middle?

696 - > Or do you think that, you think that GitOps is, like, here to 697 - > stay and that's still the s- sort of more mature, safe approach? 698 - > Lakmal: Yeah. 699 - > I would say, when you come to the lower environment, this a-agentic 700 - > directly creating this re-custom resources will play a big role. 701 - > because, but maybe in, in production, when you promote in the production, people will 702 - > use, GitOps or declarative way of defining these things, in-into the production.

703 - > but eventually if the human will confident enough with your agents 704 - > a-and, trust your, agent, eventually I would say, people agent can directly 705 - > call to the MCPs and create the, like, the create custom resources 706 - > directly within the class itself. 707 - > Bret: Yeah 708 - > Sameera: way I think, exactly. 709 - > Lakmal: in the this year, but maybe in future. 710 - > Sameera: I mean, one, one adv- the way I think about that is, I 711 - > don't know, one advantage of GitOps is you have the tracing, right?

712 - > you know what, what happened exactly over the years. 713 - > Probably, yeah, dev environment probably you don't need GitOps, 714 - > like you can give developers full capability, do whatever you want. 715 - > But for other environments, I don't know, my view, like you would be… I would 716 - > still use GitOps to control the workflow. 717 - > Yeah 718 - > Bret: Yeah, I've started to wonder if, if the agents will eventually sort of 719 - > like- We're interacting with the local agent on our Claude Code, and we're using 720 - > these skills and, I think everyone's already writing the YAML with agents.

721 - > Like, we're already at the point where teams that have u- are using 722 - > Claude Code, they're not handwriting Kubernetes YAML anymore, right? 723 - > they have the agent do that. 724 - > Why would, why would I wanna do that anymore? 725 - > And, I wondered if there was m- 'cause there is a mode in Argo where, like, 726 - > you can sorta do things retroactively, where maybe you make changes, 727 - > but then you document them after you're, after you're, you're done.

728 - > And I wondered if agents possibly created a future where GitOps was more re- m-maybe 729 - > not the first thing we did, but maybe more of a, a system of record, but done 730 - > after and if agents sort of were pushing us into a faster evolution of this. 731 - > a constant conversation we're all having is around sandboxing and, 732 - > specifically, our local agents. 733 - > one of the challenges is, I'm just doing stuff on my local machine all 734 - > day, and I don't necessarily… And every AI that I open up by default, even 735 - > if I use, like, Claude Code built-in sandboxing, chances are it has access 736 - > to my AWS, my, my Docker CLI, my kubectl CLI, my Terraform, all these things, 737 - > and those keys are all already authed, and it could just deploy to things.

738 - > And so one of the, one of the things that I'm trying to work on with 739 - > some friends is, where could we draw a boundary around sandboxes? 740 - > And this almost feels like a very perfect scenario of these MCP tools and 741 - > these skills are maybe all in a Docker sandbox, which is more of like a VM with 742 - > a, with the harness running inside it. 743 - > And I only use that particular harness because it has these particular, 744 - > permissions, and I only spin that one up when I want to… It's almost 745 - > like per environment, configuration, and each Docker sandbox tends 746 - > to have its own configuration.

747 - > the, it's… The way at least it works now is it's very, things are very isolated, 748 - > including all your keys and everything. 749 - > It, it doesn't really bring a lot into it, so you're sort of isolating 750 - > everything into different harnesses. 751 - > So I'd have, like, my prod harness, my, my staging harness, and I can interact 752 - > with those environments, and keep them safe away from, like, my normal day-to-day 753 - > harness that kind of has maybe too many… a lot of small teams, too many 754 - > keys are on their local machines, right?

755 - > they know it. 756 - > They're out there. 757 - > if you're a team of three, chances are you probably got the production Terraform key 758 - > on your machine, and you're just like, you know not to type the Terraform commands. 759 - > you know that's the human thing.

760 - > But now that we have the agents, I think people are getting, starting 761 - > to get a little more concerned. 762 - > They're like: What if it accidentally picks the wrong environment? 763 - > Or if it - I accidentally have the wrong key and it just starts going? 764 - > so this feels like a very, very interesting approach of Each environment, 765 - > or, I'm assuming this thing manages, can manage multiple clusters in one 766 - > OpenChoreo, or is it a per cluster thing?

767 - > Lakmal: It's one multiple clusters you can manage within OpenChoreo. 768 - > Bret: Yeah. 769 - > so yeah, so I, I almost treat it like an environmental gateway t- for my agents 770 - > where I'm like, I call this production and I have this special Docker sandbox or what 771 - > other sandbox I wanna apply, and the keys to production, even if it's read-only, 772 - > are only accessible from that thing. 773 - > And maybe also I'm gonna have some GitHub keys in there so that my agents i- in that 774 - > sandbox can also write to the GitOps repo.

775 - > Maybe they can make the PRs for me. 776 - > but I'm now realizing how these things are coming together, where I can have 777 - > the one local agent that can see the infrastructure through OpenChoreo, 778 - > but can also implement GitOps changes based on the information that it's 779 - > getting out of the infrastructure. 780 - > So if the pods and, you know, a pull back off loop, I can see that 781 - > the agent can determine that, write me the GitOps in the same session.

782 - > It writes the GitOps b- diff. 783 - > We're pushing that to a PR, and that's how I'm gonna be iterating on my cluster now. 784 - > Or maybe I'm doing it in Slack while I'm at lunch, and it's all just through a 785 - > Slack bot, and I'm not even… I'll ask you these questions to make sure that we 786 - > have them, have them at top of mind here. 787 - > "I like it's Claude Code that can interpret or understand, 788 - > promote into staging.

Can we use a private LLM to interact with 789 - > OpenChoreo instead of Claude Code?" 790 - > Lakmal: yeah, so it's, it's totally, in Claude Code, it's, 791 - > it's your, your model, right? 792 - > So it's a, Anthropic model. 793 - > You can use any Codex or any other agent.

794 - > It's just MCP servers you are interacting with. 795 - > You can use any LLM, in your local agent. 796 - > And also in the inbuilt agent also, you can configure your LLM. 797 - > you can either OpenAI or whatever model you can configure it.

798 - > it's a configuration, for us 799 - > Bret: Right. 800 - > Yeah, picking the model for the agents, is, is a good thing 'cause we're gonna, 801 - > we're, we're - I got a feeling we're gonna need to get cheaper models for the, for 802 - > the cluster, for the always-on activities. 803 - > yeah. 804 - > and then, does the agent have visibility into inter-cell dependency traffic, or 805 - > is it context window completely blinded to anything outside its assigned project?

806 - > Lakmal: i-it's depending on the different agent interact with, different contexts. 807 - > So for example, say, in this case, SRE agent, when that troubleshooting, it 808 - > can, go beyond the one project, because some component will interact with the 809 - > other component within other project. 810 - > So, so if there are interaction, this agent can look at, what 811 - > happening in the other, other component, in the other project. 812 - > So it's b-based on the context that agent are interacting with, they can go beyond 813 - > the single project or multiple project 814 - > Bret: Right.

815 - > Okay. 816 - > And just to be clear on, the scope of this thing, it doesn't appear like you're 817 - > trying to boil the ocean here, where OpenChoreo is the single control plane 818 - > for AWS, for GCP, for all the other things that we might have to manage. 819 - > it, it's trying to remain Kubernetes focused, right? 820 - > So, like, the lowest layer in our infrastructure stack is, I guess m- 821 - > maybe it could possibly get some of the technically OS level stuff underneath 822 - > Kubernetes, but that's, that feels like that's the s- the dropping off point, 823 - > 'cause I don't see, clouds listed in here.

824 - > Lakmal: Yeah. 825 - > we have, we are running on top of Kubernetes. 826 - > We are not even managing the Kubernetes with OpenChoreo. 827 - > we, or we build on top of the Kubernetes, so you can run 828 - > OpenChoreo on E- EKS or EKS, but we are not managing the EKS or EKS.

829 - > So that how we architect. 830 - > But, OpenChoreo has this resource abstraction where you can manage the 831 - > cloud resources, within the OpenChoreo. 832 - > So we have integrated with, Crossplane at the moment. 833 - > So OpenChoreo control plane can in- talk with the Crossplane integration 834 - > and via the Crossplane, it can create a, say, S3 bucket in AWS, and it can, 835 - > do the lifecycle manager for S3 bucket.

836 - > So it provide a single, unified control plane to manage it, but, 837 - > it's up to the users if they want to, use, manage in different way. 838 - > That's totally up to the users 839 - > Sameera: Yeah. 840 - > Just to add to that and then, Bret you saw like multiple planes, right? 841 - > Control plane, data plane, observability plane.

842 - > And you can run each plane in its own Kubernetes cluster, or you 843 - > can run all planes in one cluster. 844 - > And you can run your control plane locally, you can run data plane in AWS. 845 - > So it's, as Lakmal said, it's, the way it's architected is so it's, 846 - > you can configure, the way you want. 847 - > And then like, like for example, if the data plane is in AWS and control plane is 848 - > somewhere else, you don't have to expose the Kubernetes API server to the internet.

849 - > There, there's a certain agent running in the, in the data plane that creates an 850 - > outbound connection to the control plane. 851 - > that's sort of the standard these days 852 - > Bret: Right. 853 - > So yeah, basically, if you're gonna use this to help manage something, it needs 854 - > to be something running in Kubernetes, and it needs to be probably like a s- 855 - > something around the CNCF ecosystem of tooling or something like that. 856 - > So this isn't trying to replace Terraform or, you know, control my 857 - > cloud formation, and stuff like that.

858 - > Yeah, yeah, or run my, my, my other infrastructure like 859 - > Vercel or something like that. 860 - > Yeah. 861 - > that gives me a nice boundary because like when we talk about the future of 862 - > what it's m- means to be an SRE, DevOps platform engineer, whatever, like where 863 - > do the edges of these tools exist? 864 - > Because, sure, I can manage everything from Claude Code in theory at this point, 865 - > but, it probably, I probably need very scoped things in order to keep it in line.

866 - > And having this thing w- well defined is like it runs on top of 867 - > Kubernetes, so things that you can do in Kubernetes, it can help with. 868 - > But don't - you maybe can shoehorn in a bunch of things like, for AWS or 869 - > whatever, but maybe it's not the right tool to do everything all in one place. 870 - > 'Cause obviously there's a ton of context outside of Kubernetes that 871 - > it would probably need and, yeah. 872 - > So there's probably other tools for that.

873 - > At the end of the day, we're gonna have dozens of agents, I feel like, and we're 874 - > just gonna, we're gonna have an agent management plane where we're all just 875 - > staring at agent configs and skill files. 876 - > And that's like, that's our new job. 877 - > More markdown. 878 - > So how can people get started?

879 - > They go to openChoreo.dev and or is this a Helm chart? 880 - > Like how do we get started with implementing this? 881 - > We, well, step one, you gotta have Kubernetes some- running somewhere, right?

882 - > Sameera: So yeah, if you, if you go to OpenChoreo dev, documentation 883 - > page, there are m- couple of ways for you to get started. 884 - > First thing, if you wanna just try out OpenChoreo, so we have this quick 885 - > start guide that'll basically… The quick start guide that'll basically… 886 - > You just need Docker for that. 887 - > So Docker and Kubernetes, we will do the, we will… 888 - > There's one command that'll install Kubernetes in Docker, and 889 - > then that'll install OpenChoreo.

890 - > So you get a… Once it is done, you get a UI, you get MCP access, all that. 891 - > That's just 10 minutes. 892 - > won't, pollute your local environment in that way. 893 - > You can just destroy it after that is done.

894 - > The other option is you need to have Kubernetes cluster on your 895 - > local machine, install OpenChoreo. 896 - > There are, then the third option is install OpenChoreo in cloud environment. 897 - > That's so, that's sort of the way how we have structured getting started. 898 - > Bret: Okay, so it's got some persistence to it.

899 - > I'm assuming this thing, does this thing have, like, databases 900 - > as a part of its deployment? 901 - > Does it have, like, a Redis or, like, what - When we talk about the 902 - > infrastructure inside of OpenChoreo, I mean, we've got a web portal, we've 903 - > got some, obviously some MCP endpoints, we've got some system components based 904 - > on other things you're installing. 905 - > what's that persistence layer look like for us that are gonna have 906 - > to deploy this on our clusters?

907 - > Sameera: for the, I think for the whole OpenChoreo control plan, I would say 908 - > right now the main persistent layer is ETCD, whatever the Kubernetes use. 909 - > For Backstage, you can configure, Postgres or any other database 910 - > that Backstage, recommends. 911 - > Right now it uses… Yeah. 912 - > Yeah.

913 - > I think apart from that, there's no persistent, 914 - > Lakmal: Also, for a, like a, the SRE agent, you can configure, it's 915 - > coming with SQLite, for the saving the incident and providing the 916 - > context in the future incidents. 917 - > But you can configure a Post- a Postgres or any other database. 918 - > Bret: Yeah. 919 - > I was actually wondering too, I was trying to imagine, this thing is doing 920 - > things, these agents are doing things, and we probably want a history of that.

921 - > Is this thing able to, log into your existing… Like, is it logging 922 - > through Kubernetes to your existing… So it's essentially creating more 923 - > m- monitoring and logging data in your infrastructure by the act of it 924 - > just being there and doing things. 925 - > Is that, is that where I would go to see, hey, what does my 926 - > agent do in the last 24 hours? 927 - > Is it something like I would just look at my normal monitoring 928 - > and logging platform for that?

929 - > Lakmal: Yes. 930 - > it will do all the audit logs, to the, log itself like we are using, depending on the 931 - > lo- log module, it can be, OpenSearch with a different index with, for the, audit 932 - > logs, what agent do, whether human do. 933 - > Everything is tracked, within the logs itself 934 - > Bret: Okay. 935 - > and all the data, about the infrastructure, are the agents 936 - > just pulling that real time?

937 - > Or do we, do you have to, deal with caching, you know, infrastruc- 938 - > infrastructure graphs or any… Do you have to do anything like that and 939 - > to optimize in the OpenChoreo layer? 940 - > Lakmal: Yeah. 941 - > At the moment, it's a real time. 942 - > we are not using any, VectorDB or something like that, for the 943 - > caching or something like that.

944 - > At the moment, it's, real time pulling the data, but we give the 945 - > time interval, the context saying, "Okay, within this time interval, you 946 - > have to pull the logs, matrices." it will just look in that interval and, 947 - > get the, all the data to troubleshoot 948 - > Bret: Right. 949 - > So it's, so it, once it sees an event, it already knows exactly the second 950 - > that event happened, so then it can limit itself to th- so that you're not 951 - > burning a bunch of tokens by boiling the ocean and scanning every log on 952 - > Lakmal: Yeah.

953 - > Bret: infinity. 954 - > Okay. 955 - > Lakmal: the idea 956 - > Bret: Yeah. 957 - > Perfect.

958 - > is it, does it need to look at, like, Grafana graphs? 959 - > Like, or is it just simply doing prom queries and just getting 960 - > that data real time as well? 961 - > Lakmal: It's the data to the, observability MCP server. 962 - > It's, just going to the MC- MCP servers 963 - > Bret: Yeah.

964 - > RIP Grafana. 965 - > Sameera: Yeah. 966 - > Bret: Yeah. 967 - > Oh, sorry, what would you say?

968 - > I was talking over top of you 969 - > Sameera: No, this is directly from, observability MPV service talk to 970 - > the OpenSearch and get the data. 971 - > Bret: Yeah. 972 - > Yeah, yeah. 973 - > Yeah, so that it's a little bit simpler than it having to 974 - > constantly, figure out everything and store it as some sort of cache.

975 - > Which, I mean, in theory, that would make the agent faster so that it didn't 976 - > have to query everything itself when you start to work with it, but it also adds 977 - > a lot of complexity and, and resource utilization, which gets back to the 978 - > point of at some point, everybody has had that infrastructure where they 979 - > realize they, they have a, a cluster that has more resources used for the 980 - > infrastructure than the actual apps themselves, which, you know, has happened.

981 - > sometimes when we have small clusters and we're getting started, like we are 982 - > like, "Yeah, you know, the, the app only costs $100 a month to run, but 983 - > the infrastructure costs 10,000 a day." 984 - > Lakmal: That's true. 985 - > Bret: yeah. 986 - > Well, this has been very cool.

987 - > we could honestly, I could talk to you for another hour about this because 988 - > I'm really, I'm really interested in the patterns and the, and the ar- the 989 - > architecture design that you've learned, because I think these, these are things 990 - > that we're gonna have to do the same thing for our cloud infrastructure. 991 - > We're gonna have to do the same thing for our line of business apps. 992 - > We're gonna have to s- figure out how to give these agents under- a deeper 993 - > understanding of that infrastructure so that we can depend on them more and 994 - > rely less on the human, the human tribal knowledge that we all have about how 995 - > these systems are put together and where all the bodies are buried and where all 996 - > the, My, the friends, my friends of mine, they call it the sins of the data center.

997 - > Where are the, where are the sins of the data center at? 998 - > And this thing is gonna have to know all that. 999 - > this thing is really, when I say that, I mean like dozens of agents that all 1000 - > have certain context and certain control. 1001 - > but it all kind of feels like it's coming back to this still, this, Viktor Farcic, 1002 - > who runs the DevOps Toolkit channel, a friend of mine, he talks about that, 1003 - > like to me, to him, Claude Code is the center of the universe and he manages 1004 - > everything through it, and that anybody who's creating a project that isn't 1005 - > expected to be local harness first as my interface is creating an outdated project.

1006 - > So this kind of feels like it's, this, is doing exactly what his, he was 1007 - > predicting over the last year, which is, you know, the harness is my gateway to 1008 - > all this infrastructure, and everything else needs to be there to help me 1009 - > be like local agent harness first. 1010 - > and it's pretty cool to see that, to see that take place. 1011 - > all right, so people get started at the openChoreo.dev website, and 1012 - > then I think you're on, LinkedIn.

1013 - > I'm trying to think what the project is on LinkedIn and X. 1014 - > Lakmal: Yeah. 1015 - > We have the X and LinkedIn pages, and also we have the Slack 1016 - > channel, CNCF Slack channel. 1017 - > we were actively, engaging with there 1018 - > Bret: Yes, 1019 - > Sameera: And we use GitHub discussions a lot.

1020 - > If you want to engage with us, we'll be there. 1021 - > Bret: I love to, I love GitHub discussions. 1022 - > I d- wish they were used more, r- rather than issues. 1023 - > Like, I feel like a lot of things just need to start in discussions before they 1024 - > go to issues, and maybe some projects do that now, where they not, they don't 1025 - > even let you post things in issues.

1026 - > You have to, you have to start the discussions first. 1027 - > yay, AI ruining open source for us. 1028 - > at least for now. 1029 - > We're figuring it out.

1030 - > thank you both so much for being here. 1031 - > you can follow the project, like s- we said, on LinkedIn or on X or, join the 1032 - > Slack and the… Is it the, is it the Kubernetes Slack or the CNCF Slack? 1033 - > Lakmal: CSCF lab 1034 - > Bret: CNCF 1035 - > Sameera: Slack OpenChoreo channel, I would say. 1036 - > Bret: Yeah.

1037 - > Okay. 1038 - > I'm in both, and I can never remember which projects are in which and, 1039 - > or if the Kubernetes… Yeah. 1040 - > So it's in the CNCF Slack. 1041 - > You can, probably just Google that and find that.

1042 - > tell your agent to sign you up. 1043 - > Like, don't, you know, don't go do it yourself. 1044 - > thank you so much, Lakmal and Sameera, for being here, I'm looking forward 1045 - > to seeing what's next in the project. 1046 - > last quick thing.

1047 - > What's next? 1048 - > What's, what big PRs are coming down the pike since we're talking open source here? 1049 - > Lakmal: More agents? 1050 - > Bret: More agents.

1051 - > Could - I could have guessed it. 1052 - > I could have guessed it. 1053 - > Sameera: All about AI. 1054 - > Bret: Yeah.

1055 - > More, more of the AI overlords. 1056 - > All right. 1057 - > Thank you so much for being here. 1058 - > thanks for everybody in chat for being with us, and we'll 1059 - > see you in the next one.

1060 - > Bye, everybody. 1061 - > Thanks so much for watching. 1062 - > I'll see you in the next one.