
DevOps and Docker Talk: Cloud Native Interviews and Tooling · 2025-08-12 · 1h 5m
Key moments - from our scoring
Substance score
60 / 100
Five dimensions, 20 points each
Docker has released a comprehensive suite of AI capabilities that work independently or together as building blocks for developers. Gordon AI, accessible to all Docker Desktop users, serves as a context-aware assistant that understands Docker concepts and can translate between docker run commands and Compose files, convert Dockerfiles to hardened images, and provide help through magic icons on containers, images, and volumes. Docker Model Runner enables running open models like Devstral and Qwen locally on Mac (using unified memory), Windows (with NVIDIA or Adreno GPUs), while acknowledging the tradeoff between local experimentation and production-grade models. The newer MCP toolkit and Docker Hub's MCP server catalogs let developers containerize and manage Model Context Protocol servers for connecting to applications like Claude Desktop and VS Code Copilot, handling credential injection and lifecycle management. Michael Irwin emphasizes that enterprises are still experimenting with hybrid approaches - smaller fine-tuned models locally for development, larger models in cloud - while smaller models increasingly become viable for specific tasks. The conversation highlights that this tooling democratizes AI experimentation for developers who may lack ML expertise, similar to how containers abstracted away infrastructure complexity.
Gordon AI is available to everyone running Docker Desktop for free, with the only limitation being that business organizations must enable it due to enterprise caution around AI features.
On Mac it uses unified system memory, on Windows it requires NVIDIA or Adreno GPUs, and quantized models can also run on CPU though performance is limited.
While possible, Docker recommends caution since a smaller quantized model used locally will produce different outputs than a larger version in production; some enterprises use different fine-tuned models for different tasks instead.
Docker Model Package CLI can wrap existing GGUF files into OCI format for Docker Hub, but Docker doesn't provide tooling to create models from scratch; most model training uses separate container environments.
Docker's MCP toolkit containerizes Model Context Protocol servers and manages their lifecycle with secure credential injection, allowing integration with Claude Desktop, VS Code Copilot, Cursor, and other agentic applications.
Our reviewer’s read on each dimension, with quotes from the episode.
The episode covers substantial technical territory - Docker Model Runner, MCP Gateway, Compose specifications, and Offload - with concrete explanations of how these tools work. However, much of the content is feature-by-feature walkthrough rather than novel insights. The discussion on MCP tools' context window impact and the gateway's proxy pattern provide useful technical clarity, but most claims are relatively straightforward (e.g., 'models are different from databases,' 'smaller models behave differently than larger ones'). There is genuine insight in the conversation about hybrid local/cloud models and dynamic tool selection, but these represent a minority of the runtime.
So models are a little different...if I'm using the four billion parameter version during local development and then I deploy and I'm using the 32 billion parameter version in production, those are very different models and you're going to get very different interactions and different outputs from the models there.
the way that tools work is basically, it's taking all the tool descriptions...it's sending that to the model on every request...as Nirmal just pointed out, like that takes context window there
The episode is largely a feature announcement and product demo wrapped in conversation. The core ideas - containerizing AI workloads, using gateways to manage services, delegating to cloud for compute - are established patterns applied to new Docker tools. The MCP Gateway-as-proxy concept is derivative of API Gateway patterns. Nirmal's comparison to Google's search appliance from 20 years ago is a good analogy but comes late and doesn't drive novel thinking. The discussion of tool selection and context windows is sensible but not counterintuitive. No first-principles arguments or contrarian takes are present.
So an API gateway...An API gateway, single endpoint...the MCP gateway serves in a similar fashion in which it's going to be routing to the right MCP server
This just feels like one giant list of tools. And in one MCP server, which is just the nature of proxy, right? But behind one IP address is a whole bunch of websites, like you don't realize it, right?
Michael Irwin is a Docker employee with 3.5 years tenure in Developer Success and has credibility as a practitioner/educator who breaks down Docker's tooling. He is not a founder or deep technical architect, but he is an informed company insider with real responsibility for developer education and product explanation. Nirmal Mehta from AWS adds cloud infrastructure perspective and real-world judgment about enterprise use cases. Both are relevant practitioners rather than pure thought leaders, though neither is a senior IC or founder at the level that would score 17+.
I'm Michael Irwin, I work at Docker. I'm in our developer success team, and do a lot of teaching, training, education, breaking down the stuff
I was in, the city of Virginia beach running the IT and the data center there, or at least running the engineering groups.
The episode provides specific tool names (Docker Model Runner, MCP Gateway, Gordon AI, Offload, Open Web UI) and concrete examples (Devstral, Qwen3, Alibaba models, 141 MCP tools in toolkit, 48GB RAM, 72 GitHub MCP tools). However, there are few concrete numbers on performance, pricing (only 'GPU minutes' and '300 free minutes'), or real usage metrics. The Compose YAML example is referenced but not shown. The MCP Gateway's filtering and permissioning are described in principle but not with specific implementation examples. Real customer stories are teased ('stay tuned') but not delivered.
141 different tools and growing that Docker has added
I've got a 40, 46 gig model. I think it's the Devstral new model
The host (Bret Fisher) asks clear setup questions and requests explanations ('Tell me about that,' 'Can we build, can we make our own yet?'). Michael provides patient walkthrough responses. However, the conversation is largely linear feature review with limited pushback, disagreement, or probing. When Bret asks about hybrid offload per-service, Michael's response is lukewarm ('tell us more about the use cases'), and there's no hard follow-up. The discussion of MCP server overload and tool selection is good, but the host does not press on tradeoffs, limitations, or when *not* to use these tools. Nirmal's interjections add some judgment but don't create genuine tension.
What's cool about the model now is with the specification evolution, you can now specify, hey, this is the environment variable
Tell me about that. Nirmal and Michael, what are you seeing? are people coming to Docker, like trying to run the biggest models possible?
Computed from the transcript - who did the talking, and the words that came up most.
Bret and Nirmal are joined by Michael Irwin to discuss Docker's comprehensive AI toolkit, covering everything from local model deployment to cloud-based container orchestration across multiple interconnected tools and services. The Agentic DevOps Guild has launched! It's a training + community + mentorship program for engineers wanting to learn the latest CI/CD automation and dive into Agentic DevOps. Meetups are happening now, with new course videos dropping every few weeks. Join the Guild and become your team's leader in AI for infrastructure automation We run through almost the entire Docker AI ecosystem, including the Model Runner for hosting open-source models locally or remotely, the Hub model catalog, Gordon AI chatbot integration within Docker Desktop and CLI, how MCP toolkit and catalog enable seamless tool integration with local AI systems, Docker's open-source MCP gateway for managing these connections, plus Docker's new Offload service for running containers and models entirely in Docker Cloud and much, much more!
Transcribed and scored by The B2B Podcast Index.
This is DevOps and Docker talk, and I am your host, Bret Fisher. This was a fun episode this week. Nirmal Mehta is back from AWS and our guest this week was Michael Irwin of Docker. He is a recurring guest if you've been listening to this podcast for any length of time, I think we had him on earlier this year, and he's been a friend for a decade, former Docker captain, now Docker employee advocating for Docker everywhere.
He's all over the place and, I always have him on because he breaks stuff down to help us all understand what's going on at Docker. And there is a giant list of things we had to talk about this week, all AI related, all separate pieces of the puzzle that can be used independently or together. So in this episode, we will be covering the Docker Model Runner, which I've talked about a lot on this show over the last four months, for running open or free models locally or remotely on servers or on your machine, wherever.
Basically you get to run your own models and Docker will host them for you, Docker will encapsulate and allow you to use the Docker CLI to manage it. Then we have the hub model catalogs, so you can pick one of the dozens of models I guess, that are available in Docker Hub and we talk about Gordon ai, which is their chat bot built into Docker Desktop and the Docker CLI. We then get into MCP toolkit and the hubs MCP catalog, and how to bring all our tools into our local ai or like, using some other AI and you're just wanting to use your own MCP tools.
We talk about how Docker manages all that and it uses the MCP gateway that they recently open sourced to front end all those tools and to help you manage your MCP tools. We also then get into compose and how you add models and the MCP gateway plus MCP tools, all into your composed files. Then we talk about how to use that for building agents a little bit And then Offload, which allows you to build, run containers, run Docker models, all in Docker Cloud, and they call it Offload It actually is a pretty unique name.
Most other people just called it Docker Cloud, but they call it Docker Offload. Great name. You just toggle inside your Docker UI and you're good to go. Everything's remote.
And then we at the very end talked about Compose Now works at least the YAML files of Compose, now work inside of Google Cloud Run. Probably coming to other places soon. So we go for almost 90 minutes on this show because for good reason, we talk about the use cases of all these different various AI parts of the puzzle. And I'm excited because it paints, I hope for you a complete picture of what Docker has released, how it all works together, when you would choose each part for problem solving, different problems and all that.
So please enjoy this episode with Nirmal and Michael Irwin of Docker. Hi! Hey, Awesome. I'm doing all right.
we have a special guest Michael, another Virginian. Hello. Hello, we all have known each other a decade. So Michael, tell us who you are so I'm, Michael Irwin, I work at Docker.
I'm in our developer success team, and do a lot of teaching, training, education, breaking down the stuff, trying to make it somewhat understandable. I've been with Docker for almost three and a half years now, so it's gone pretty quickly, but, it's fun to spend time with the community and just help developers out. Learn about this stuff, but also how do you actually take advantage and do cool stuff with it? That's awesome.
I mean, this is like a new wave of, essentially development tooling and, it's pretty exciting. Yeah. we've got a list people. I only make lists on this show a couple of times a year.
Yeah, it does seem like every time I'm on the show, there's a list involved. yeah, that's a pretty decent list. let's break it down because we got a lot to get through and we realize that on Docker's channel and on this channel, They've been talking about all these fun features. I've talked at length about Docker Model Runner.
Hub Catalog, I don't have a bunch of videos on MCP or the MCP tools, but actually we've had streams on it before. But, I think the MCP toolkit and the MCP gateway I'm most excited about that. So we're going to get to that in a little bit. I think, technically, the first thing out of the gate was Gordon AI.
this to me is. From a user's perspective, it's a Docker focused or maybe a developer focused AI chatbot, essentially similar to ChatGPT, but it's in my Docker interface. if I'm staring at the Docker desktop interface, there's an ask Gordon button. And if you've never clicked that, or if you clicked it once a couple years ago, it has changed a lot.
There have been enhancements so o now we have memory and threads and it saves me from having to go to ChatGPT when I'm working, specifically around Docker stuff, but anything it feels like I can ask it developer related stuff Can you tell me now, is this free for everyone? How does this play out in terms of who can use this Yeah, so everybody can access it right now. The only limitations may be some of our business orgs. those orgs have to, enable it.
We're just not going to roll out all the AI stuff as most organizations are pretty cautious about that. but yeah, Gordon's available to everybody. it started off actually mostly as a documentation help of just help me answer questions and keep up with new features and that kind of stuff. But, as you've noted, we've added new capabilities and new tools along the way.
I was doing some experiments just the other day of Hey Gordon, help me convert this Dockerfile to use the new Docker hardened images. And it would do the conversion, find the images that my organization has that match the one that I'm using this Dockerfile. you're starting to see more and more capabilities built into it as well it's a pretty fun little assistant there. Yeah, highly recommend folks that are listening that if you've not touched Docker and you just open it up, check out Gordon and ask those questions that you probably have, compose all the things you could probably put that in there and Gordon will probably try to compose all the things.
you know what one of my common uses for this is I need to translate a docker run command into a compose file or back and forth like, please give me the docker run equivalent of this compose file or please turn these docker run commands or this docker build command into a compose file and it saves me. It's not, I mean, I've been doing this stuff almost every day for a decade, so it's not like I needed to do it for me, but it's still faster than a pro typing it in from memory, I'm at the point now where I rarely need to refer to the docs, but it's still faster than me at writing a Compose file out and saving it to the hard drive.
You know, because sometimes we debate around whether AI tools are useful for senior versus junior and blah, blah, blah, and I don't know, as a senior, it saves me keystrokes, it saves me time, unlike when it first came out a couple years ago, I have Tracked it, given feedback to the team and it no longer makes Compose files with old Compose information. Like it, at least in the the version tag. Yeah. Cause it, the version tag.
Yeah. I was a big proponent of Hey, this hasn't been in there for five years. Why is it recommending it? and they fixed that.
So, I'm very appreciative of it. We're going to save it, we're going to talk about MCP tools, and then we're going to come back to this, because it gets, it's gotten better, and one of the reasons it's gotten better is it can talk to tools. So I don't want to really spoil that, But yeah, I love the suggestions because sometimes when you're staring at a blank chat box, it feels kind of like being a writer and staring at a blank document. It's like, I know this is supposed to be a cool tool, but I don't know what to do with it.
do I start? Yeah. analogy there's even some cool spots, like if you go to the containers tab, You'll see on the right side a little like magic icon there, and then you can ask questions about that container. So if you have a container that's failing to start or whatever, you can basically start a thread around the context of why is this thing not starting.
Interesting. I have not tried that. it actually reminds me of the little star we get for Gemini that's in every single Google app that I'm opening. I guess we're all, slowly converging on the starlight AI icons now, I would have thought it would have been a robot face, but we chose this vague collection of stars and pluses.
I and that is pretty cool. I, so is there more, is it like an images as well? So yeah, anywhere you see that icon, you can start a conversation around that thing. I'm going to start clicking on that more often to see what it tells me.
Like, Hey, what's the, what data. Ooh, How do I back up this volume? That's, that is pretty cool. So they're that is the number one question I love that they're context aware.
Like they know that this is a volume, or at least the prompts. it's like prompt suggestions, which This feels a little meta. Is the AI suggesting prompts for the ai? It's not in this case, but it certainly could.
All right. so that's ask Gordon, Woo! Alright. And that is what you said.
that is available to everyone running Docker Desktop. Is that, to be clear, that's not available for people running Docker Engine on Linux, right? That is not There is a CLI to it, isn't there? There's Docker AI.
but that's part of the Docker Desktop installation. that's a CLI plugin and it's going to talk to the components that are bundled in Docker Desktop there. Yeah. so we have our, this is an AI that's outsourced to Docker.
It's not running locally. it's just calling APIs from the Docker desktop interface. So then if I don't want to use Gordon AI, but I want to run my own AI models locally, which I feel like this is pretty niche because a lot of people I talk to, their GPU is underwhelming, and they don't have an M4 Mac or a NVIDIA desktop tower with a giant GPU in it. granted, I guess there are models that run on CPUs, but I am not the model expert here.
I've been trying to catch up this year because I actually do have a decent laptop now with an M4 in it so I can run at least some of the Apple models. But this Docker model runner. it's a pretty cool feature, but you can pick your models, so you can pick a really small one. On Windows, it has to be an NVIDIA GPU on Windows to work.
Is that right? So it supports both NVIDIA and Adreno, so if you've got a Qualcomm chip, it'll work there. Okay, and on Mac, it just uses system memory because that's how Macs work. They have the universal memory, right?
And I mean, for me, it's been great because I have a brand new machine with 48 gig of RAM. I know that is not normal. but these models, Mac, it can be a little problematic because it's like, if I have a bunch of browser tabs open, I can't run the full size model that I want to run, because that's one of the problems is like, it's all using the same thing. So I have to do like we did back with VMs, I have to shut all my apps down and then run, because I always want to run the biggest model possible.
So I have this like 40, 46 gig model. I think it's the Devstral new model that's supposed to be great for local development. So for those of you listening, if you want to go in depth, we're not going to have time for that, but it is, Docker Model Runner lets you run models locally. you can technically run this now on Docker infrastructure, right?
Which we're going to get to in a little bit. dun dun. Okay. yeah, there is this thing called offload.
Michael's going to know way more about that. Cause that's a really new feature, but you don't have to actually run these models locally now. Like you could offload them. Right.
Okay. What are you seeing? are people coming to Docker, like trying to run the biggest models possible? They got like multiple GPUs or are you just seeing people kind of tinkering out?
What's some analogies there? there's a little bit of everything. I'd say a lot of folks are still exploring the space and figure out what's the right ways of doing things as well too. So, one of the interesting things is, and one of the things to keep in mind is that Let's break outta the AI space.
when you say, for example, a database. Okay. I'm using Postgres database. If I'm using a Postgres database, a managed offering out in the cloud.
Okay. Let's just say RDS. Okay. during development I can run a Postgres container and yeah, it's a smaller version and it's not a managed offering.
and that works because all the binary protocols are the same. I can basically just swap out. My connection URL, and it just works. But models are a little different.
And so, I've seen some folks that have been like, I'm going to use a smaller version of a model during local development, but then when I deploy, I'm going to use the larger hosted model that my cloud provider provides. Okay, so maybe it's a parameter change, I'm using fewer parameters so it can actually run locally, and then use the larger version. The analogy of, okay, me running a local PostgreSQL container and me running a local model, if it's a different model, it's a different model.
And the results that you get from that model are going to vary quite a bit. And so that's one of the things that we have to kind of keep reminding people if you use for example a four billion parameter, So, we're going to talk a little bit about how you can use this model during local development. Yes, it can fit on your machine, but then if you deploy and you're using the 32 billion parameter version in production, those are very different models and you're going to get very different interactions and different outputs from the models there.
So, it's something to keep in mind as folks are looking at building their applications. So, where are we seeing folks use this? Well Of course, if you can use the same model across your entire software development life cycle, then that works out pretty well. but we're also starting to see a little bit of a rise of, using basically fine tuned use case specific models or folks training their own models.
and using those for the specific application. and again, that's tends to be a little bit smaller, more use case specific. And then, yes, that makes sense to use there. okay, I need to be able to run that on my own.
et cetera. again, I think a lot of folks are still feeling out the space and figuring out exactly how should I think about how should I use this? and of course, the tooling has to exist kind of before you can actually do a lot of those experiments. So, in many ways, we've been building out that tooling to help support that experimentation.
but I think in many ways, folks are still figuring out exactly what this is going to look like for them going forward. Yeah. I'm trying to imagine what enterprises are doing and building out and I'm imagining it's not like this, but one of my imaginations is reminding me of 20 years ago, buying a Google box, which I don't remember the name of, they called it, but it was this appliance you would put in your data center that was, it was yellow. Racked search appliance.
you go. Yeah. And I don't know if Nirmal, if you had any customers back then with, if you were a consultant back then, but that, I can't name those customers. so I can, I was in, the city of Virginia beach running the IT and the data center there, or at least running the engineering groups.
And those were back in the days where we didn't want Google indexing our internal infrastructure. You don't want your internal data to be. accessed or used potentially by this big IT conglomerate. Yeah.
And so you bring, they, they sell you a, on prem box and you put it in your data center and it would scan and have all access to everything you could give it, back when apps didn't really have their own search, Google was providing that for us, and they would index our email and index our file servers and databases if we wanted to give it access to them I'm not sure that's around anymore, but, at a time, Google wasn't going to give away their software and we didn't all know how to run it.
And, I'm sure it was running on Linux. And at the time in the mid 2000s, we weren't yet Linux at the city. For different reasons. This kind of feels like the same moment for enterprise where.
They're going to have to buy GPUs probably for the first time if they're going to run it on prem, they're going to want to keep it separate. They're going to either buy something for the first, probably GPUs if they're not training models, they just want to run things internally to access their internal data, or maybe they're doing it in the cloud and Nirmal's company, AWS, is providing them the GPUs and then presumably because they're getting dedicated hardware, they won't have to worry about OpenAI or Anthropic having access to all their stuff so with respect to ModelRunner and again, just a reminder, these are my own opinions and not that of my employer, Amazon Web Services.
There's still a lot of use cases. what Michael was talking about with respect to choosing lots of different models for different types of tasks, I think there's probably a hybrid model at some point where folks are using different fine tuned niche models for specific tasks that they're doing locally, and then as hardware improves, and hopefully, you know, maybe your 3 year old developer laptop that you get at your corporation has enough, or the models get, you know, Get, optimize it enough that they can run on CPU or the GPUs that you have on a corporate laptop, but there'll be some tooling, probably embedded into the development tooling and, or you can choose your own models.
and then those, there'll be other models where you need to reach out to the hyperscalers for those that just you're not going to get the depth of reasoning you're not going to get the depth of knowledge that you will from something like Claude in the cloud versus something like deep seek running a quantized deep seek running on your mac but again this is all changing very very fast and Right now, we're in a different state where folks have to spend money to access those larger models, which hasn't been the pattern with respect to software development in a long time, right?
Not everyone has that advantage, right? I mean, if you want to use Claude Code and, not hit any major limits, then you got to pay 200 a month, which, not everyone's going to be able to afford 2, 400 a year, to do software development, there's also edge use cases, right? So IoT devices, just trying to figure it out. Also just kicking the tires.
I think that's probably the main, like Michael said, I think that's just, everyone is just trying to kick the tires as cheaply as possible. model runner feels like a gateway, like it feels like a gateway drug to get me Hooked on like the idea, cause I, I mean, I wasn't paying attention to, I wasn't a machine, I wasn't an ML person or I wasn't building AI infrastructure, but Docker model runner and, well, and, you know, to a lesser extent, Ollama, Ollama always felt like it was more for those people that were doing that, but bringing this capability into a tool that I'm already using actually felt like, Okay, this is meant for me now, like this is, I don't have to be, I don't have to understand weights and all the intricacies of how models are built and work and the different, I don't have to, I don't have to understand the different file format and whether that works on my particular thing.
it just all kind of works. It's Docker easy at that point for me. think that's the key point here of just, let's try to increase the access to these capabilities and let folks start to experiment and I think Again, we, as an industry, are trying to still figure out exactly how to use a lot of these tools and, okay, what is the right size model for the job at hand? and so, again, in order to be able to do that experimentation, you have to increase the access to it.
and so, but that's still kind of, you know, Where we are in many ways. So did we check off another thing on the list? about to I'm gonna have to keep saying this We're not a news podcast. We don't talk about the latest things like in general in tech but you know, I got to give a shout out to Fireship because I have Between that and a few other channels, I've really learned a lot this year about models.
And I was, there's this little, diagram of sort of the state of a lot of the open, waiter open, or free, I'm just going to say free models. They're free in some capacity, that you can download. And, I was using Devstral. We talked about it a couple of times on this show already, which is a model that came out in May.
I actually had a newsletter on that. Shout out to, Bret. news, there's this guy, Bret, he makes a newsletter, Bret. news, you can go check that out.
and I talked about this, that maybe this was the sweet spot because it was small enough, you could run it with a modern GPU or modern Mac, and it wasn't the worst, nothing like the frontier models that we get with OpenAI and Anthropic, but it was something that was better. And, that's called Devstral. And then we had Quinn three just come out, I don't know, a week ago or something that is I think it was earlier this week. Oh, see, time warp of AI, I think it three days is like three weeks.
It was two days ago. Yeah, Oh gosh. I always assume that I'm seeing a Fireship videos late, but, yeah, one day ago, so yeah, that's true. One day ago, there's a newer model coming out from Alibaba, right?
And it is even better, although it does take more GPU memory, I that's hard to run like on a laptop. I don't think you can, so that's a perfect encapsulation of like why Docker model run is there, but also why it's going to be a mix of models going forward. so we've, we have an AI that helps you use Docker and get started. We have a tool that helps you run models locally, and understand that what's the next step in this journey?
by the way, you go to Docker Hub to look for models, or you can do it in Docker Desktop, or you can look at things in the CLI. there's many ways to see the models. you can pull things from HuggingFace now. That's pretty sweet.
Can we build, can we make our own yet? Or do we have that? We can so, you can package, but you would have to already have the gguff file and all that kind of stuff. So we don't have a lot of the tooling there to help you actually create the model itself.
Although a lot of folks will use container based environments to do that. we don't have any specific tooling around that ourselves. So is there, you're saying there's a doc, oh, there, there is. Oh, I didn't realize.
So there is now a Docker model package CLI for creating, basically wrapping the GGUF or, other models or whatever, into the Docker OCI standard format for shipping and pulling and pushing essentially Docker. models that are in Docker hubs that are in the Docker format. OCI format. Yep.
So when we look at agentic applications, step one is, you need models. It's kind of the brains of the operations. and then you need tools. And so you've got highlighted here the MCP toolkit.
That was kind of the first adventure into the MCP space. And that one was focused a little bit more on how do we provide tools to the other agentic applications that you are already running on your machine. So Claude Desktop or using VS Code Copilot on my machine or Cursor, etc. How do we provide those MCP servers?
And containerized ways, secure credential injection, etc. Basically manage the life cycle of those MCP servers. Again, in the use case of connecting them to your other agentic applications. And so, again, this is kind of where we started our MCP journey.
if you see flipping through a couple of these, actually we just released a Docker Hub MCP server that allows you to Search for images on hub or, you know, those within your organization, which is super helpful for like maybe a write me a Dockerfile that does X. Well, cool. Let's go find that the right image that should be used for that. so again, starts to open up some of these, additional capabilities here.
Yeah. And inside the Docker desktop UI, there is now a beta tab, essentially, that's called MCP Toolkit. And it is a GUI that allows me to explore and enable one of, 141 different tools and growing that Docker has added. So like a lot of the other places on the internet that either they host models like Anthropic or OpenAI, or They're a place where you can create AI applications and all those places have started to create their own little portals for finding tools and they may or may not, I mean, most of them all now settle on MCP, but before we had really MCP as the protocol standard, they were already doing like OpenAI was doing this before, but they were very, it was proprietary.
You don't know how Evernote or Notion got, showed up as a tool feature in ChatGPT, it did. But we just assumed that was their custom integration and now we have this standard called MCP that everything should interact with everything properly the way that they should. At least right now it's the one that's winning. We don't know whether we'll still be talking about MCP in five years, but it's here now.
It's what we're talking about now. And this lights up a lot of capabilities. In other words, you turn on an MCP tool. And that sits behind something called MCP Gateway.
So tell me, what is MCP Gateway? Yeah. So at the end of the day, the toolkit is a combination of several different things. the MCP gateway is actually a component that we just open source at WeAreDeveloper.
So you can actually run this gateway directly. In a container, completely on its own. And the MCP gateway is what's actually responsible for managing the lifecycle of these MCP servers. it itself is an MCP server.
think of it more like an MCP proxy. it exposes itself as an MCP server that then can connect to your applications. But when you ask that server, Hey, what tools do you have? It's really delegating, or I mean, it's using cache versions of, okay, what are the downstream MCP servers?
And so it's acting as basically a proxy here. so when requests come in and say, hey, you know, from the agents at GAP, if I want to execute this tool, go do a search on DuckDuckGo at that point, the MCP gateway will actually spin up the container, that DuckDuckGo MCP server, and then delegate the request to that container, which then does the search, and then the MCP gateway returns the results. So kind of think of it as a proxy that's managing the lifecycle of all those containers, but also, you know, injecting the credentials, configuration.
it also does other things like actually looking at what's going in and out of the prompts going through the proxy to make sure, you know, secrets aren't being leaked or, that kind of stuff as well too. and we're even starting to do some further explorations of what are other ways to kind of secure those MCP servers? You know, for example, a file system one should never have network access. Cool, so let's, when that container starts, you get no network access, or, the GitHub MCP server, you know, it's talking to the GitHub APIs.
Let's only authorize those host names that it can communicate with. So, you know, start to do a little bit more of a kind of permissioning model around these MCP servers, which is where a lot of people are kind of most cautious and nervous about MCP servers, because it's, they're completely autonomous for the most part, and you have to trust what's going on there. this exact feature is both necessary and also Solomon Hyke's prediction from a month ago. He was on this show and was saying that we're going to see all these infrastructure companies and all these tooling companies that are going to offer to lock this shit down.
I think it's the quote I have to get from him. that sounds right. he compared the origin of containers and how it started with developers. And then eventually it took it and sort of managed it in the infrastructure layer and provided all the restrictions and security and limitations and configuration and all this stuff.
And the same thing's happening to MCP. Where it started out as a developer tool to empower developers to do all these cool things with AI that they couldn't do and let the AI actually do stuff for us. And now very quickly in a matter of months, IT is coming in and saying, okay, we're going to lock this down. It's crazy.
You can, you know, your prompts can delete your, drop your databases, your, as we just saw happen on the internet recently this week. I want to, on this gateway topic though, it can sound complicated. And maybe the internals are a little, and there was obviously code built into this program. But for those of us that aren't maybe building agents yet, or like really getting into building apps that use AIs in the app, this just appears as kind of magic.
Like it, you go into the Docker desktop UI, I enable, I go through the toolkit, there's all these suggestions, everything from, the MCP server for GitHub itself to an MCP server that could give me access to Grafana data to accessing the Heroku API and you're looking at all these things and you're just like, I'm enabling them. it's like a kid in a candy store. I'm just going check, check, check. Yeah, I want Notion.
I want Stripe. I get them in a list, they're enabled, which means they're not actually running, right? they're waiting to be called before the gateway runs them in memory. That's all transparent to me, I don't realize that's happening.
And if I choose to use this toolkit with Gordon, if I just go into Gordon, And in the Gordon AI, if I don't want to run a local model myself, or I'm not using Claude Desktop or something that gives me the ability to enable MCP tools, I can just go in here and say, enable all my MCP tools, all 34 of them. I've got get ones and I've got, and so now what that means is the Gordon AI can now use these tools, which makes this free AI even smarter. And I can say, is there a Docker hub image NGINX?
I don't know if there is. Let's see. I've never even tested this. So it's kind of a, what could go wrong?
let's use Google live on the internet and see what happens. yeah, I was just saying, look at it, going out and checking, using the Docker, the newly created Docker hub, MCB, tool That you had, just, released. so is this going through a, MCP gateway or is this not with the MCP gateway yet? when you and Gordon AI flip the switch to say, yes, I want to use the MCP toolkit, basically what that's doing is, in the Gordon AI application here, it's enrolling the MCP toolkit as an MCP server.
And so then it's going to ask The MCP toolkit, hey, what tools do you have available? And so when you saw that list of tools, that's again coming from the gateway. Gordon is just simply treating the MCP toolkit as An MCP server, which in itself is going to launch MCP server. So that's kind of why I mentioned, it's kind of thinking of it like a proxy there.
Yeah, it does feel like one. Yeah. And this, for those watching, like it didn't actually work because I don't actually have access to hardened images, but, I just wanted to see what it would, what it'd say, but it, the, the UI Which is the right answer? Which is the right answer for did the right thing.
it didn't expose a vulnerability in the MCP server. But yeah, so it basically, I can give Gordon AI more. And I can do a lot more functionality, more abilities to do things without having to run my own model, without having to figure out Claude Desktop. But I will say, because I'm in love with this toolkit so much, because I love this idea of one place for my MCP tools, for me to enter in the API secrets so it can access my notion and my Gmail but I don't want to have to do that in Claude Desktop and then in warp and then in VS code and then in Docker and then in Ollama and like every place I might run.
A tool that needs MCP tools or access to an LLM. So I did it in Docker Desktop. I enabled the ones and set them up the way I wanted. And then inside of my tools around my computer that all support AI MCPs, they all have now added MCP client functionality that lets me talk to another MCP, any MCP server that speaks proper MCP through their API.
And in this case, what I've done in the warp terminal, because it does support MCP, is I just tell it, the command it's going to run is docker mcp gateway run. And it uses the standard in and standard out, which is one of the ways that you can use mcp. And then I suddenly have all 34 tools that we enabled in my docker desktop. Available in Warp, just as long as Docker Desktop's running, that's all I gotta do.
And then, because Warp is using Claude Sonnet 4, or whatever I told Warp to do, Docker doesn't care about that, because I'm not asking it to use, think that, we talked about this it's called bring your own key, I guess that's what everybody's talking about. Uh, bring your own key is when you want to bring your own Model to whatever to access, you're a key to access the model. Yeah, but in warp in particular, like this is nuanced, but in warp, you can't usually, you can't yet access your own models yet.
I think they're going to make that an enterprise feature or something. But if I open up VS code, I could do the same thing. If I opened up a ChatGPT desktop, I could do the same thing. And, Aider or like any of the CLI tools, although anything that accepts MCP so far, I've gotten to work this way.
And it's been awesome because all these, all these different IDEs and AI tools all set up a little bit different. Goose you set up differently than Claude Desktop. So Client. they're all, and all they, all of them have different knobs and, ways of controlling MCP servers and at varying degrees of control and flexibility.
So this is really nice because then you can also just have all those tools running if you right. They look like one giant MCP server. Yeah, because normally I would have to add each MCP tool as its own server, I think, it feels like some of the tools are all standardizing on Claude Desktop as, that config file, which I the mcp. json, Yeah, it feels like that, like everyone's settling on just using that one file.
Which is, I guess it's kind of feels kind of hacky, but I guess it's fine. it feels like every editor using my VIM settings. It's like, no, no, no, no. Calm down.
I don't necessarily want you all to use the same file. I don't want you all overwriting each other and changing the same file. so you're bringing up a really important thing, which is, since we're new to this, the number of MCP servers is not too many yet, even though it does feel like there's probably like a lot of MCP servers. I've been using 10 new MCP servers since we've started this conversation.
But it's still like a number of tools that we can still rationalize about. But probably in another month or two at this rate, there is a limit to how many MCP tools One single client, I guess you could say, or one instantiation of a task that you're using like Claude Code or Cline, it's context window can only use a certain amount of tools. And so, is there some ideas about breaking up in the MCP gateway, like having maybe like sets of tools that have like specific supersets or tools or something like that?
Yeah. Good question. And so that's a good call out. And so I actually want to, zoom in on that just a tiny bit there, because for folks that may be new to this, they may not quite understand that the way that tools work is basically, it's taking all the tool descriptions.
Okay, here's a tool name. Here's when I'm going to use this tool. Here's the parameters that are needed to invoke this tool. And it's sending that to the model on every request.
And so the model's having to read all that and basically say, hey, based on this conversation, hey, here's a toolbox of stuff that I may or may not be able to use. But as Nirmal just pointed out, like that takes context window there and granted yes some of the newer models have incredibly huge context windows but depending on the use case, it's going to affect your speed, it's going to affect the quality, and so yeah, you do want to be careful of like, okay, I'm not just going to go in there and just flip the box on all the MCP servers.
Now you have access to everything. Like, you do want to be a little conscious of that as well. in fact, I found it funny in a, I was playing in Cursor not long ago, and you know, they have even just YOLO mode, just go crazy with it. But even they have A warning once you, I think it's after you enable the 31st tool of like, hey, heads up, you're getting a little crazy.
So like, I'm like, for the one that has YOLO mode to call me out for being crazy for too many tools, like it was, it's again, kind of a reminder of just, okay, you do want to be conscious of the number of tools that you're using. so to actually answer the question. Yeah. It's been something that we've been exploring and kind of waiting to just see what the feedback is gonna be on that.
Are there separate tool sets that, clients can connect to. you know, that's certainly a possibility as well, since this MCP gateway is an open source container, when you run this for your application, not only can you say, these are the servers I want, but then you can even further filter through, these are the tools from those servers that I actually want to expose. So, for example, I think the GitHub official one is up to 72 tools now or something. It's a crazy number.
but most of the time, I only need maybe three or four of them. So, I want to filter that. And that's why you see, Cloud and VS Code and many of these others. Even though you're pulling in these MCP servers, many of those provide client side functionality to kind of filter that list as well, Yeah.
I wonder if we get to a state because the MC, this is getting a little bit meta, but everything when you talk about agentic AI gets meta really quickly. So, I wonder if the MCP gateway itself is an MCP server, so it can rationalize about itself, I wonder if we get into the pattern of, okay, there's this new task that I want this agent to do, and the first thing, after it comes up with its task list, the steps it wants to take, is go through that list and then, ask the MCP gateway to reconfigure itself on each task and turn on Only the ones that it identified as likely the ones that it needs for that task.
And just dynamically, at any given time, don't have anything more than five running. So figure it out. You can choose whatever five you want, but only have five. We've done some experiments with that, not quite to that full dynamicness, but I've even done some ones of a Okay, here's a tool to enable other tools, is basically what it is.
And, okay, and give me parameters of, okay, do you need, GitHub? Do you need, Slack? You know, tell me what it is that you need, and then I'll enable those specific things. And then what's cool then is, as part of the MCP protocol, there's also notifications.
So the MCP server can then notify, The client says, hey, there's a new list of tools available, and then the next API request to the model then has this new set of tools. I think we're almost capability is there, but, I think that's likely the next step. but it's also kind of like a, yeah, how do you safeguard that? So it's, Yeah, it's an interesting time period, for sure.
we got an interesting question, is MCP Gateways intent to replace an API Gateway or in parallel to it? Great question. Michael, you want to take that one? yeah, great question.
I'd say in some ways that there's similar functionality, but they serve very different purposes. So an API gateway, I'll just take the most basic example, but I know there's lots of different ones, An API gateway, single endpoint, and I may have lots of different microservices. Let's just pick a catalog. Okay, so for product related ones, it's going to go to this microservice.
Users, it's going to go to this other one. Cart, another service, whatever. And the API gateway is routing all those different requests and rate limiting, etc. In many ways, like this MCP gateway serves in a similar fashion in which it's going to be routing to the right MCP server to actually handle the tool execution and whatnot.
But again, it's only for the MCP protocol. So it's not going to be replacing an API gateway because it's not doing normal API requests, etc. It's only for MCP related workloads and requests. different protocols at play here.
I think that's probably the best way to describe it. otherwise, you could also say that MCP and API Gateway are likely going to be running in parallel. and so probably what I would see would be, I have an API gateway that routes a request to an endpoint, and then that particular application, let's just say it's an agentic application, can then have its own MCP gateway to satisfy whatever agentic flow it needs to use there. I wanted to, while you guys were having an awesome conversation, I was trying to draw up, just a visualization to try to represent, okay, so just so people understand, because this MCP, we could make a whole show on MCP tools, honestly, from an infrastructure perspective, how do these things talk?
How do they integrate? The fact that you're talking about that they're just really adding to the context window is a fantastic fact that, A lot of people could go months or years using MCP tools day to day and never know that, right? a normal non engineer could use MCP tools, not understand how these things are all working. for those that are into this, are playing around with MCP tools elsewhere and understanding a little bit of MCP server functionality and client versus server versus host and all that stuff.
Before Docker's gateway, the MCP gateway, you would have like an MCP client That whether it's your IDE, your terminal, or AI, chat, desktop, or whatever you've got, that is acting as an MCP client. Assuming it supports MCP servers, you can add them one at a time. So I would add GitHub's MCP server, then I would add DuckDuckGo's MCP server. I might add Notion's MCP server, since I'm a big Notion fan.
And each one of those servers has One to infinity, tools, which are, I look at as like API routes. and each one has its own very niche purpose. depending on the tool, and this is part of the frustration with the ecosystem right now is we're only months into this, but it's amazing that all these tools are all starting to support each other, tools have different ways where you manage this. Some of them you can disable and enable specific servers.
Some, you can actually choose the tools individually, which is like choosing API routes. And to me, it's you're always trying to get down to the smallest amount of tools that you need to prevent confusion. Cause I'm, my biggest problem is I enable all the tools because I get tired of managing them. I just want them to work.
I just want them all to work when they need to work. And then I, so I enable them all. I end up with 50 plus tools. And then when I'm asking AI to do things, it chooses the wrong tool because I wasn't precise enough in my ask to trigger the right words that are written in the system prompt of that MCP server.
So actually, maybe an easier update might be to put another layer on top of the MCP server and kind of an in between. so I'm connecting the MCP gateway now to multiple other MCP servers. So I get, yeah, you're right. I need another layer here.
That's actually MCP servers. so, there's now this gateway in the middle, and it, the only negative of this approach is for right now, because we don't have this futuristic utopia yet, is that to my terminal, or my IDE, it all looks like one giant list of tools. And in one MCP server, which is just the nature of proxy, right? But behind one IP address is a whole bunch of websites, like you don't realize it, right?
So it is, the analogy still works, I believe, there. But in this case, because it's connecting all of them together into one proxy, and the nice thing is, it's, I can see in the memory usage and the containers. In fact, when Michael was on weeks ago, We saw the MCP gateway spinning up servers dynamically and then shutting them down and you could see the container launch run, you know, run the curl command or whatever, and then close. And it was so quick, we couldn't, capture and swat toggle windows to see the tools launching.
And, I mean, it's beautiful. It's exactly what containers were for. It's, it's ephemeral, it's wonderful. But, if your IDE or if your chat desktop or whatever is acting as your MCP client, the agent thing, if that doesn't let you choose individual tools, then this approach is a little hard because the only way from my IDE that I can, I have to turn off all of Docker or none of Docker.
It I think. This gets us to a conversation of eventually we will have this. I'm thinking of it as like the model, the plan model before the model that will go, okay, you used all these keywords. I'm going to pick out the right tools and I'm going to hand those off to the next model, which is going to do the actual work.
That's probably already here. Solomon predicted it a month ago. Yeah, I'm sorry. What?
so that's what Michael and I, while you were drawing this Oh, is that what you were just talking about? that's what Michael and I, we were talking about. So the gateway itself has its own MCP server that controls itself. And so we're a few months away from exactly what you were just talking about.
Bret, because of context windows, because there's too many tools, because of all the things that you did, all the challenges you just mentioned, Bret. the first step might be the client going to the MCP gateway, MCP server first and saying, hey, these are the things I'm about to go do. Out of the list, check your MCP gateway and tell me the list of, MCP tools that I actually need for that. And then only turn those on for the next task.
Yeah. and then it'll just repeat that cycle again. and then winnow down that list of MCP tools to the only things that are needed for that task at hand. So there's another layer here, which we're, and Michael and I, we were discussing while you were building that beautiful diagram.
It's, people are experimenting with that. All the pieces are in place, but that this pattern isn't quite there just yet, but it will likely be, I'm pretty sure this is what we're going to be doing pretty Nobody wants to go manually choose every MCP server that they're going to need before every AI request. almost feels like it takes away the speed advantage of using the MCP tool to go get the data for me. if I have to do all this work in each tool independently, Because I often will have an IDE accessing AI, acting as the MCP client, and then I'll have a terminal acting as an MCP client.
At the same time, I've got ChatGPT desktop running over here, also while VS Code, I think a lot of us, eventually evolve to the point where we've got two or three tools all at the same time managing MCP tools. We've got, I guess we have multiple IDEs, I should say. and trying to understand how all this comes together is only interesting right now, but in six months, we're not going to want to be messing with all this stuff. We're just going to want this part to work so we can work on building agents, All right.
So Compose, my favorite tool, a lot of people's favorite Docker tool, other than the fact that Docker exists. you announced at, we are developers that Compose is getting more. There's functionality in the YAML specifically, where I guess we're talking about the YAML configuration that drives the Compose command line, that in just three months ago, you were adding model support, and that was like an early alpha idea of what if I could specify the model I wanted Docker Model Runner to run when I launch my app that maybe needs a model, a local model, and I use the example and I have a An actual demo over on GIST, that people can pick up that you simply, you, you write your Compose file, you use something called open web, open, what is it?
Open web, web, open web UI, I think. Yeah, horrible name. Extremely generic name for what is a ChatGPT clone, essentially. the open source variant, which can use any models or more than one model.
It actually lets you choose it in the interface. And all you need is a little bit of compose file. So, I created a 29 lines, and it probably needs to be updated because it's probably outdated, but, 29 lines of Compose that's half comments that allows me to spin up an open web UI container while also spinning up the models or making sure, basically, that I have the models locally that I need to run it. And this gives me a ChatGPT experience without ChatGPT.
Thank you. And you guys, you enable this. Now you're not creating the models. You're not creating the open web UI.
you're simply providing the glue for it to all come together in a very easy way locally. Yeah. as we agentic apps need three things. They need models, they need tools, and then the code that glues it all together.
What the Compose file lets us do now is define all three of those in a single document. here's the models that my app is going to need for MCP gateway that I'm just going to run as another containerized service. And then the code, the custom code, can be really any agentic framework. this example is, Open web UI, but that Compose snippet what we've done is we've evolved the specification now models are a top level element in the Compose file, which is pretty cool.
This just dropped in the last couple of weeks, so this is brand new. Gotta update my gist. yep, and so where before, yeah, you had to use this provider syntax, and that still works. now it's actually part of the specification.
defining a model, This is going to pull from Docker Hub. again, You can have your own models and your own container registry. It's just an OCI artifact. You can specify that anywhere.
then we've got the services, And then the app itself. What's cool about the model now is with the specification evolution, you can now specify, hey, this is the environment variable I want you to basically inject into my container. to specify what's the endpoint, where's the base URL that I should use to access this model. And then what's the model name as well.
So the cool thing then is I can go back up to the top level model specification, I can swap that out and the environment variables will be automatically updated and as assuming that my app is using those environment variables, Everything just works. So again, think of Compose as, it's the glue that's making sure that everything is there for the application to actually be able to leverage it. Yeah. the gateway part here was pretty cool to me.
That I can add in my tools, my MCP tools inside of the YAML file. when I saw that part, I was like, yes. that is like my vision, my dream is that I can pass a composed file to someone else and it'll use their keys. Presuming, my.
Team is all using the same provider that we would have the same. Because open, well, open AI, base URL, open AI model, and then open AI API key or whatever. if you're going to use ones in the SAS, like those are all pretty generic. Even if you're not using OpenAI, they're all pretty generic, environment variables.
So I guess this would work across teams or across people well, and that's a good point to call it. One of the things that OpenAI did when they released their APIs was basically, hey, here's a specification on how to interact with models that pretty much, Everybody else has adopted and used. and so the Docker model runner exposes an OpenAI compatible API. And so that's why you see these environment variables kind of using the OpenAI prefix.
Because again, I can use now any agentic application that can talk to OpenAI or use the OpenAI libraries. And it's just a configuration change at this point. All right. Now, the coup de gras.
Piece la resistance. I can't even do my pretend French, All this stuff has been running locally. Like when we think of Docker desktop, we think of everything locally. And then a year or two ago, Docker launched, Docker build cloud, which was like getting back to Docker's roots.
I almost feel like of providing more a SaaS service that essentially is. Doing something in a container for me. And in that case, it was just building containers using an outsourced build kit. So it was better for parallelization and multi architecture.
it was sweet. And I love it for when I need to build like enterprise tools or big business things that take 20 minutes to build. None of my sample little examples do that, but anything in the real world takes that long and you need to build multi architecture and generally it's going to be faster in a cloud environment. So you provided that.
Now it feels like you've upgraded, like it's beyond just building, and it does any image or any container I want to run, any model I want to run, I guess, not maybe any, I don't know if there's a limitation there, but, bigger models, then maybe I can run locally, and then, also builds. So it can do building the image, hosting the container, running the model endpoint for a, QIN3 or whatever. I can now do all that in something called Offload. So tell me about that.
Docker Offload, the way I explain it to people is, hey, you need more resources? Burst into the cloud. And so it's basically, I'm going to offload this into the cloud, but yet it's still, everything still works as if it were local. So if I've got bind mounts, okay, great, we're going to automatically set up the synchronized file shares.
And so all that's going to work the way using mutagen and some of the other tools behind the scenes to make that work. Port publishing that still works as you would expect it to. So again, it gives that local experience, but using remote resources. I'm just offloading this to the cloud, but yet it's still.
My environment. and so, yeah, to make it clear, like, this is not a production runtime environment, I can't share this environment out, or, I can't, you know, create a URL and say, hey, check this out, colleague, or whatever, it's still for your personal use. Now, of course, can you make a, Cloudflare tunnels, and I'm going to make it production, sure, but I wouldn't could hack that. but yeah.
Yeah. So what is the intent? so what is the use case? the big should I use Docker offload for first?
Yeah, so, okay, great. You're wanting to play around these agentic apps and, you know, we were talking about not everybody has access to high end GPUs or, you know, M4 machines and whatnot. great with the flip of a switch, and you had it there in Docker Desktop, but at the top you just flip a switch, and now you're using offload. and so now you've got access to a pretty significant NVIDIA GPU, and additional resources, and so yeah, as you're, We see the use case, especially more for the agent applications, because that's where those resources are needed.
It does open up some interesting doors for, maybe I'm just on a super lightweight laptop that I'm using for school and I don't have the ability to even run a lot of my containerized workloads. Great, I can use that for offload, you know, offload that to the cloud. it does open up some interesting opportunities for, Use cases beyond agentic apps, but that's kind of where the big focus is right now. So if you're like a Docker insider, or if you're someone who's used Docker a while, it's the Docker context command that we've had forever, augmenting or changing the environment variable docker underscore host, which we've had since almost the beginning, and it allows you from your local Docker CLI, And even the GUI works this way too, because I could always set up a Docker remote engine and then create a new context in the Docker CLI.
That would use SSH tunneling to go to that server, and then I could run my Docker CLI locally, my Compose CLI locally, and it would technically be accessing and running against, the remote host that I had set up, but that was never really a cloud service, like it was never, no one provides Docker API access as a service that I'm aware of, and, the context command, while it's easy to use, and you can actually use it on any command, you can use docker run dash dash context, I believe, or docker dash dash context run, I can't remember the order, but, you can change that on any command.
these are all things that existed, but you made it, stupid easy. It's just, like you said, it's a toggle, it's so easy. You just click that button and then the UI changes, the colors change, so now you know. You're now remote.
Yeah, and so I'll go ahead and say too, behind the scenes, it's using context. It's using those exact things. The tricky part is, because I've done similar, development environments where, I'm going to work against a Raspberry Pi at home or, whatever else it might be. the tricky part is when you want to get into bind mounts, file sharing kind of stuff, or port publishing, and I want to be able to access that port from my machine, like automating all those different pieces.
That's not trivial. I mean, it's possible. a separate tool, yeah, you gotta download ngrok or something, And so this brings all that together into a single offering here. That's pretty amazing.
Like there's a lot going on underneath the hood that, switch is hiding a lot of different functionality. Like it's. To make that very transparent And this supports builds too, right? So like.
When I toggle this in the UI, or is there a CLI toggle? Yeah, there is. okay. So if I toggle this, it's, yeah, you're like, you're saying it's a context change, but it's UI aware, and it takes in all the other little things that we don't think about until they don't work.
And then we're like, oh, yeah, it's not really running locally anymore. So now I can't use localhost colon. Well, that all just, I'm going to show you how this works and you don't even have to know kind of like the rest of the Dockery because you don't really have to know how it works underneath. but if you think it's too much magic, I like to break it down and say, it's just really the Docker context.
I don't actually look anything at the code. I don't know really how it's working. But to me, when I went and checked, it does change the context for me. It actually injects it and then removes it.
I did notice it from the CLI. I could change context. And it would retain the context, but if I use the toggle button, it deletes the context and then re adds it. Regardless, it is in the background, it's doing cool things.
I think the immediate request from the captains was, can I do both? Can I have, Per workload or per service offload so that just my model's remote and maybe that really big database server and then all my apps are local. I don't know why I would care, but like it, that's something that people ask for. I'm not sure that I care that to that level.
I think I'm fine with either or, but I can understand that if I'm running some things locally already and I just want to add on something in addition, it would be neat if I could just choose for one service. Yeah. So as of right now, it is an all or nothing. you're doing everything local.
You're doing everything out in the cloud. there's not a way to. Split that up yet. it's something that we've heard from a couple of folks, but again, it's that same thing of, tell us more about the use cases.
So if that's a use case you have, feel free to reach out to us and help us better understand, why you might want to split runtime. hosting, split environment, hybrid environment. That's the correct term. Why do you say it like that?
and just to be clear, offload has its own cost. like this isn't free forever for infinity. You can't just take up a bunch of GPUs. I was asking the team a little bit and without getting too nerdy, it sounds like it isolates, it spins up a VM or there's maybe some hot VMs.
And I get a dedicated. OS essentially it sounds like so that I can get the GPU if I need. And you kind of get an option of, do I want GPU, servers with GPUs or not? do I, am I going to run GPU workloads or not?
And that affects pricing do we get anything out of the box with a Docker subscription or is it just an, a completely separate So actually it's a. Kind of private beta, but yet people can sign up for it and that kind of stuff. folks will get the 300 GPU minutes, which isn't a ton, but it's enough to experiment and play around with it. and then, start giving us feedback, etc.
Yeah, if you spin up the GPU instance and then go to lunch, by the time you get back, you'll have probably used up your free minutes. It's a long lunch, hey, that's my kind of lunch. but yeah, so we went an hour, and we barely scratched the surface, do we cover it all? did we list at least all the announcements of Major features and tools.
I don't even want to say we've covered all the features because there's probably some stuff with MCP we missed. So you open source MCP gateway, but we should point out you don't actually have to know, like you can just use Docker desktop and MCP tools locally. But the reason you provide an MCP gateway is open source is so we could put it in the compose file and then run it on servers. think about it this way.
the MCP toolkit bundled with Docker Desktop is going to be more for, I'm consuming, I'm just wanting to use MCP servers and connect them to my other agentic applications. And the MCP Gateway is going to be more for, now I want to build my own agentic applications and connect those tools to, Those, those applications that we're running there. Yeah. Do you see people using MCP Gateway in production?
Do you see that as like a. Not that you provide support or anything like that, but is it designed so that We've got a couple of folks that are already starting to do so. stay tuned for some use case stories around that. Yeah.
Awesome. well, this is a lot. I feel like I need to launch another 10 Docker YouTube uploads just to cover each tool specifically, each use case specifically. there's a lot here, but this is Amazing work.
I mean, I don't know if you have a fleet of AI robots working for you yet, but certainly feels like a lot of different products that are all coming together very quickly that are all somehow related to each other, but also independently usable. And, I'm having you on the show as usual is a great way to break it down into the real usable bits What do we really care about without all the marketing, hype of general AI hype, which is always a problem on the internet. But this feels like really useful stuff.
Um, Eivor, another podcast. I don't know, Eivor, what's up? Are you requesting yet another podcast? Um, a whole new show about Compose provider services?
Oh, yes. Also, you can now run Compose directly from, well, you can use Compose YAML from directly inside of cloud tools. The first one was Google Cloud Run. So I could technically spin up Google, which I love, Google Cloud Run is fantastic.
Um, it would be my first choice for running any containers in Google if I was using Google. Um, and so now they're, accepting the Compose YAML, spec, essentially, inside of their command line. So this is like, this feels like the opposite of what Docker used to do. Docker used to build in cloud functionality into the Docker tooling.
But now we're saying, Hey, let's partner with those tools, those companies, and let them build in cloud or compose specification into their tool. So we can have basically file reuse. YAML reuse. Is that right?
yeah. So this is the first time exactly in which it's not Docker tooling that's providing the cloud support, but it's cloud native. They're the ones building the tooling and consuming the Compose file. yeah, it's a big moment.
And as we work with Google Cloud on this, yeah, you can deploy the normal container workloads, etc. But they already have support for Model Runner to be able to run the models there as well, it's pretty exciting And I know, the provider services this is how we started with models. having support in Compose, where That was another service in which the service wasn't backed by a normal container. the old method.
yes, but what's cool about this is, so first off, these hooks still are in place. So that a Compose file can basically delegate off to this additional provider plugin to say, hey, this is how you're going to spin up a model. But it starts to open up a whole ecosystem where anybody can make a provider or, okay, hey, I've got this cloud based database, just as an example. And, okay, Now I can still use Compose and it's going to spin up my containers, but also create this cloud based container and then inject environment variables into my app.
again, it starts to open up some pretty cool extensibility capabilities of Compose as well. I think we, yeah, we need to bring Michael back just to dig into that because, it's essentially like extensions or, Plugins Yeah. for Yeah, so Compose is about to get a whole lot more love. It feels like it's already, I mean, it's been years since we've added a root extension or like a, top level, top level build.
it's not every day that Docker decides there's a whole new type of thing that we deploy. Now we have models, we'll see if providers someday become something. that'll be cool. and this is all due to the Compose spec, which now allows other tools to use the Compose standard.
And that's just great for everybody, because everybody uses Compose. it's like the most universal YAML out there, in my opinion. great. Well, I think we've covered it all.
Nirmal and I need, another month to digest all this, and then we'll invite you back on. do it. but yeah, we've checked the box of, Everything Docker first half of the year, stay tuned for the second half of the year. I actually sincerely hope you don't have as busy of a second half, just because it's, these are a lot of videos I got to make, you're putting a lot of work into my, inbox, We're helping you have content to create.
I know, yeah, there's no shortage of content to create right now with Docker. I am very excited to play with all these things. I sound excited because I am excited. This is real stuff that I think is beneficial and largely free, largely, Like almost all of this stuff is really just extra functionality that already exists that in our tooling without adding a whole bunch of SaaS services we have to buy on top of it.
yeah, so congrats. People can find out more docker.com. docs.
Docker.com, dockers got videos on YouTube now they're putting up YouTube videos, so check that out. I saw Michael putting up some videos recently on LinkedIn. it's all over the place.
You can follow Michael Irwin on LinkedIn. he's on BlueSky. I think you're on BlueSky. I think you're on BlueSky.
Um, or, or, where, yeah, figured out where I'm hanging out. Thanks so much for being here. Thank you, Michael. Thank you, Nirmal, for joining and staying so long.
I'll see you in the next one.
Other episodes covering the same guests and topics, from across The B2B Podcast Index.