WTF is an AI Factory

Speaker A: Foreign. Hello, I'm, um, Rob Hirschfeld, CEO and co founder of Rack N and your host for the Cloud 2030 podcast. In this episode, we try, desperately try to untangle what is an AI factory? What does it mean, what's required? Are there different levels of AI factory? Ah, we have so many questions, um, and we acknowledge it's a marketing term and also something people seem to want. Um, so we're going to have a great discussion about what it is and hopefully you'll start with a little bit of an opinion and listen to where we get. Uh, I'm happy to start with news because the hugging face, that hack is transformed and people are cashing in their, um, OpenAI and Anthropic shares and moving to the hills, apparently over the threat of unmanaged AGI. So, um, I'm happy to spend some time talking about that. Um, the ultimate topic for today is, um, wtf, uh, is an, uh, AI factory. That term has been making it in the news quite a bit. It was in the VMware announcement. Um, the analysis I've seen of the VMware event was they are trying to remake VMware as AI, as an AI factory platform. Uh, it begs the question of what is an AI factory? Uh,

Speaker B: it's a little bit frustrating for me because, you know, I'm looking at enterprise AI usage, uh, and, you know, I don't have. What do they call B2 hundreds or whatever they're called, you know, by us. It's, you know, can we scrounge up some, can we scrounge up some, some graphics cards or maybe get a DGX somewhere? But the AGI factory is, is a different thing.

Speaker A: Uh, uh, this is, this is the confusion. What, what is an AGI or an AI AI factory? An AGI factory. That, uh, that's a whole different. That's a whole different kettle of fish. But, um, yeah, um, why isn't it a single GPU system? Why can't we have a one node AI factory?

Speaker C: Actually, there was an announcement this morning. M. Excuse me. About the first AI PCs. Um, I'm just trying to pull it up because I thought it was quite interesting

Speaker A: somebody. Because it feels like, I mean, from my team's perspective, Apple and Dell have both been selling AI machines for a while. What makes this one different?

Speaker C: Um, apparently capacity. I took a very, very quick read, so I'm trying to get my facts straight before I speak.

Speaker A: I know on my team I can fill in while you look. Um, we have engineers increasingly asking to buy high, um, end desktops that they can run local models on.

Speaker C: Yeah,

Speaker A: um, I don't think they would consider those AI factories. I would think they would. They consider them dev desktops, um, where they can run. Or agentic workstations maybe. Um,

Speaker B: well, it's one step below an agentic workstation. So the idea is nowadays we give everybody a laptop with 48 gigs of memory so they can run uh, local LLMs just

Speaker A: right.

Speaker B: You know, but then that's not the same thing as buying again a DGX or a, um, uh, or a STRIX Halo. That's a different beast. And that's starting where you start getting into AI, ah, factories. But I think AI factories is when you start looking at being able to provide AI services on demand. Right. AI resources on demand.

Speaker A: I, this is actually part of where I was hoping we would, we would start putting some stakes in the ground. Um, not that, you know, four people's opinion is going to shape the industry, but um, like trying to define cloud back in the late, in the early, early aughts. Um, but you know, it'd be nice for, it'd be nice to actually, you know, set some grounds, Some grounds on it. Um, so. Right. I, I generally agree with you and AI Factory should provide an API to run AI reasoning requests, uh, from remote systems. That's, that's been sort of the way I've thought about it, but I don't, I don't know that that's like if I look at OpenShift AI, um, it, I think it does, it does prevent. Offer an API endpoint that you would, you could use as the reasoning engine via an AI as, as a. Via remote API as opposed to running a local model? Does it matter?

Speaker C: Say that again.

Speaker A: Does it, does it matter if you, does having a, presenting a remote API to be able to, you know, separate the LLM reasoning from that, the requester matter. Right. If my developer is running a local LLM on their desktop and created an agentic workstation. Although Michael, you were saying that's not an agentic workstation to you. Um, you know, where, where does, where does that jump into AI factory?

Speaker C: Well, I think, I think the words AI factory probably have 20 definitions at this point

Speaker A: at least.

Speaker B: Yes.

Speaker C: And so I would kind of look at it from the point of view of what is the. If we were to try and put a standard definition around what AI factoring means and then look at what, what's required for it hardware, you know, like bare metal up all the way through the stacks, what would that look like? And I don't know that tpu, gpu, um, CPU is necessarily going to get us where we would want to go in that, in, in that kind of domain definition. Because are you going uh, to look at dedicated boards for inference? Are you going to look at dedicated boards for embeddings, you know like how work. Because I think that in, in some respects that's probably where it needs to go. Division of labor across boards that somehow can interact with each other. That may sound a little far fetched but I think the way the models, the frontier models run versus how people are building agents are completely different things.

Speaker D: Yeah. Two things that would seem to make a borderline here is the functionality. Um, you would be hard pressed to do a full on training of a, of a, of an, of an LLM on even a bunch of stacked DGXs. I mean it's that, that is, that's just not feasible. Certainly not feasible. Um, at you know kind of the full. It would have to be a very, very small, small LLM to, for you to be able to do that. The workstation that you're talking about is predominantly for inference. Can be for um, the kind of embeddings that you're using for local um, like RAG and local processing. It can also be uh utilized uh to enhance the harness. So customization various types of, I'll uh, call it fine tuning for the time being. Cause it's not strictly fine tuning. There's a lot of work that's now going on locally managed by the harness or some overlay on it that is getting into latent space that is munging about with the uh, with what's going on that is not necessarily visible any other way. When we talk about factories I think you are looking at um, the hardcore creation of new LLMs or new versions of them. Um, refining them on the basis of the first, first line training. Not post training modifications necessarily. Now it can be in. Is it fact here, here's a question. Is there a big difference between an AI factory and a Neo cloud today?

Speaker A: Uh, in the way I'm seeing them. Marketed. Yes,

Speaker D: marketed. What about. I don't, I don't,

Speaker A: I don't see. So this is. You're bringing up something that I surprises me but I think is, is matching what the marketing is. Uh, some of this um, I'm interested in the marketing. I do not see AI factories as. And training actually coupled together the way you were doing. I think when I hear vendors talking about AI factories they are not actually talking about training models. They are only talking about it as an inference system.

Speaker D: Where's. What about, what about fine tuning? What about the modifications to the customization of these models? For example,

Speaker A: Strangely when I, in the marketing speak, I see, I don't see it. They are assuming that it's an inference platform that's using open weight models.

Speaker C: Wow.

Speaker A: Which I, which I actually think is strangely restrictive.

Speaker D: Yeah. Uh, the factories are extending the ability for a customer to customize or purpose build a model for their exclusive use perhaps or not um, possibly for resale. That is special has a specialized um, access to um, a variety of different kinds of information. We're looking at, um, if you look at the discussions of AI factories that are coming out of Nvidia in particular your point about open weights. Yes. You're not correct. I think about training and uh, you are not considering what is required for a post training, learning and enhancement of an LLM of a, of a foundation of a foundation model.

Speaker A: Actually you're hitting something that I wanted, where I wanted to go. I like what you're adding into my definition into, into what I was seeing as a definition. Um, because there's a software platform capability that, what you're, that you're describing

Speaker D: that.

Speaker A: So it's not just I put kubernetes on bare metal and AI factory requires you to have some additional capabilities on top. You know, basically kubernetes is a, is seems like the lingua franca, uh, under underlay for it. But every AI factory is capabilities stacked on top of it. Um, you know, so a model router and then. Plus. Go ahead, Sean.

Speaker C: No, I, I, I agree with Rich, but I raised the question of what is customization of a frontier model versus creation of a small language model for a purpose and what impact would that have on the definition of the factory? Because if you look at the factory, I know Rich is shaking his head. I'm watching him shake his head. Um, I look at the factory as the way in which small language models will emerge more heavily. Not because it's a hugging face type thing that you know, put a model out and people will come to it or whatever, but rather from the point of view of um, I'm seeing this from a different perspective which is around business processes to make them more tangible. So if I look at an slm, uh, as a specialized language model with a very particular ontology and a very particular purpose that to me would be something that factories would create.

Speaker D: You're going to get into the problem of what's an SLM now? Uh,

Speaker C: yes, I

Speaker D: know a small language model

Speaker C: versus a specialized.

Speaker D: How do you make a small model? You can quantize it. For example think about how a term is used but you can quantize it, you can uh, establish it, uh, with an overlay. You can add ah, features that are effectively rags. Um, and the rag can be you know, pure semantic, uh, could be graph oriented, could be very structured databases. The other thing that we haven't talked about is the operational, the construction, the construction tools and then the operational tools of putting even a small language model in place. For example, if you need um, the kinds of data governance for the specialized knowledge eval, uh, ongoing testing, drift, you know, the list goes on. The question here is what does a factory provide and can you do some portion of it? Can you elect not to do it on the factory and do it in house and whatever in house is, is it on a workstation or an in house kind of implementation of the model? I think there are um, you know we're going to get into carving it up but it really, I think at the end of the day is going to be um, the definitions are going to be built on the functionality that's offered that can be offered because of uh, the economies of scale predominantly.

Speaker A: Who, how often do you think people are going to really be training their own model? Some of this from an AI factory. I mean you're describing to me AI factory plus or AI Factory, you know, gold, Gold, Gold version. Like uh, can we, can we have a basic AI factory that only does a model router and certain like guardrails or security? Is there, you know, AI factory gold that enables you to say, you know what, we're gonna, you have enough capacity, we're gonna train and refine models for you.

Speaker D: Uh,

Speaker B: you referring to rags? If you're talking referring to rags, then yes, companies are making rags all the time.

Speaker A: Interesting. I, Rags have not been in my, in my lexicon for a while. But you're right. I uh, this is so I think there's an interesting, I think we're all saying there's a software component that's a platform that does AI things, that's a necessary piece of an AI factory. And then there's a spectrum of core features which I'm arguing is a model router. Right. And additional features which training and refinement. M I, it's, I would put that not at base, but it, you know, pro. I'm going to use that as a um. Well

Speaker D: I mean again we're talking, we, we're getting almost to the point of kind of defining somebody's business model. And you know at, at that point we, where um, everybody that's in the factory business um, decides hey, multi tenancy got to be you know, there because um, people are using the factory not just for kind of the basics but actually for you know, serving for delivery. Uh, it's um, you know I have a specialized legal, uh, legal, uh, information business or specialized um, tracking business. Um, and yeah, I'm reselling it, I'm selling it to customers to make use of um, as they would a specialized, any other specialized frontier model. So how about this? Is there a different customer for a factory versus a workstation versus uh, an in house kind of dedicated.

Speaker A: I mean because I'm watching. This is where the things start to go so weird, right? I'm watching Cisco, um, I'm interested in Joanne's definition. I'm watching Cisco, right. And Dell and other sell you know, five to six machine units, basically a chassis half rack as an AI factory. Um, and they're selling it. They're almost just stuffing the hardware and shipping the hardware and calling it an AI factory without any of this. This is where my head starts. So it's like giving somebody a half rack of AI machines. It's a necessary component.

Speaker D: Yeah, but it, you know, let's put it this way. If I ship you, I mean in the physical world if I, if I ship you a bunch of uh, um, shop machinery and you know, you're in the business of making you know, wooden, wooden, you know, house siding. I don't know. But you know, is it a factory? Because I've shipped you all of the goodies and you have to put it in place on a, on a shop floor somewhere and have to hook it all up, have to put it in process and so forth. It's the basis for a factory. Is it a, you know, is it a prefab or is it the foundation for a factory? Yeah, it's a use of a term I think that's going to get you know, specialized.

Speaker A: It's

Speaker D: walking

Speaker A: around

Speaker D: a space specialized or it's going to uh, it's going to turn into you know, a no, a uh, nothing.

Speaker A: I think, I think it already, I think it already. This is, this is. I think what's funny, it already is cloud. It's you know, it's the cloud of our, of our era. Um, because, because I think you some, you know, Dell, Dell would happily tell you, you know, oh, I'm walking through a ah, factory with no people in it and all the machines are off. But it's Clearly, I'm clearly walking through a factory. It could be a lumber mill if only somebody turned on the switch and you know, brought in, you know, brought in the lumber. Um,

Speaker C: but I think the issue is more around the life cycle discipline a bit. And that's where the term factory has been, like digital and transformation, the oxy of morons. Um, the need is I have to do this at scale. So the term factory comes to mind because it's, you know, a model T for the AI era. But irrespective of that, it's about the life cycle and about being able to make, massively reproduce or continue the life cycle of AI in a particular, um, industry or sector or something purposeful. Right. It's about that life cycle. And, and that's why I don't like the term AI factory, because no factory in the world has zero customization. There's always something that's

Speaker A: always

Speaker D: and

Speaker C: right. And we're going to have to live with that. So from, from that perspective. And by the way, this is not my personal definition of an AI factory, um, but it's the one that seems to resonate with me because it talks to life cycle and because I see the whole thing coming from a very different angle, which is the inflection point of being purposeful with AI and people recognizing that M. You can't look at a single process. You need to look at things as having dependencies and co dependencies and cross references and whatever. So when you get to the hardware side, you have to be very clear about what is the purpose. Otherwise you're going to commission for the wrong thing, if you will. So what resonates with, for me with this definition is this notion of putting the pieces together to perpetuate a life cycle. Read over.

Speaker D: Does it cover.

Speaker A: I love that addition, actually, I think it's important.

Speaker D: Does it cover

Speaker C: over. I'll say more often then.

Speaker D: Does it cover process? Then does it cover. Um, because

Speaker A: the.

Speaker D: Now it's it. Let me ask you this. If you don't like factory as a term, what would you prefer it be called? Is it a utility?

Speaker C: Yes.

Speaker D: Okay, that makes the most sense to me. And utilities come in, you know, all shapes and sizes. They can be lupus built, they can general purpose, they can be scaled at different times and they are put in place for reasonably well defined purpose. End to end. To your point, with service levels established, it starts to feel more like a, you know, how do you build a utility? An AI, uh, utility as opposed to a factory.

Speaker C: Yeah. And I do see it as A utility. To me that makes the most sense because of uh, the way you can combine utilities to gain efficiencies, optimizations, whatever word you choose. Enhancement.

Speaker D: Yeah.

Speaker B: So we're treating AI like a commodity then.

Speaker D: Uh,

Speaker B: in that way

Speaker D: parts of it are commodities. I mean, well,

Speaker B: that's

Speaker D: what the

Speaker B: factory does.

Speaker D: Commodity means in. Tell me what aspect of it is commodity. Is it because it's fungible and there and interchangeable? Is it a commodity because it's a, it's kind of universally of a building block that is universal? Is it its function or is it the way you've, you, you price it and buy it

Speaker B: in the way that like a cloud, you have a la carte menu of things that are provided in the factory and you integrate them together into what you need in a repeatable pattern. So it's like an orchestrator, something like that.

Speaker D: Yeah. Or router, um, or, or a model router. Yeah, sure. Um, I, I think, I mean I, I, I think of the, of the, what people have been calling AI factories as being utilities and they're, I, I think we, I, I duck this, this problem of, of naming and, and the semantics of that, of the, of the factory term. But I, um, it feels more and more like that's what, that's what's being built. It's, you know, if you want to look at analogs, it is some of the cloud services, um, like the cloud services, um, or you know, various kinds of SaaS services in the, uh, in the, that predated. So yeah,

Speaker C: what appeals to me about the utility notion is you can swap them out, you can enhance them, you can upgrade them. Um, they're not so tied together that you can't choose one part utility A and swap it out for utility B. Because utility B is now more efficient or optimized or quantized or whatever you want, whatever the function being approved and have it become part of that life cycle. The notion of the life cycle and the fact that things are utilities, routers, or not, that makes a lot of sense to me going forward because I think if you look at the cost and the issues that arise around. I built this on model X1 and now they've changed the model to X3 and every single time they do an upgrade I have to go through patch hell, et cetera. Whereas the utility is self contained.

Speaker D: Is there a moral equivalent or an analog in this in, in AI utilities, um, that would correspond to appliances. We've seen, uh, specialized virtualized firewalls, uh, routers, things like that show up in the Cloud, uh, in cloud operations do we have and do we want to think about kind uh, of appliances as building blocks or kind of um, enhancements that I, that could be added to a base utility? AI utility. Will they be packaged that way?

Speaker C: I think so. I think some of them will be. I was talking with someone yesterday where their, their product serves as a, it's in Kubernetes. It is, is kind of like a proxy service. Around Kubernetes, between the API or the MCP and the Kubernetes container, they've built a form of proxy. From a security perspective that's where their product is coming from. And to me that's kind of the appliance notion that Rich is describing and I'm seeing more and more of that come up.

Speaker D: What eventually distinguishes an AI factory then from a hyperscaler cloud provider? I mean uh, does AWS eventually become not just a cloud service provider but uh,

Speaker C: an AI

Speaker D: factory? Does, does Oracle, does Microsoft?

Speaker B: I think by their definitions they already call themselves AI factories.

Speaker A: Yeah, there's the cloud.

Speaker C: Yeah,

Speaker A: I think that the cloud, the, the cloud. We're going to have cloud AI factory services. By this definition eoclouds are going to offer rental or token AI. I mean would you consider uh, anthropic an AI factory? I'm not sure I would.

Speaker D: No necessarily. No. I mean degree of

Speaker A: uh, autonomy in the AI factory versus an AI. A degree of autonomy, like I'm running the infrastructure and the, like all of those pieces that, that becomes my AI. I think there to me there is uh, you don't look at as somebody providing you a model and say there's an AI factory, they're just providing you with a token source.

Speaker B: There's orchestration and pipelines in an AI factory.

Speaker D: Mhm.

Speaker A: And a degree of governance and controls and process. Right. Lifecycle is part of all these pieces. So we keep coming back to the software side of this, not the hardware side, but go ahead.

Speaker C: So my question, given the comment about pipelines and so forth and orchestration, is what is the AI factory producing? Is it producing a capability of AI or aig? Is it producing agents? Is it producing what is it producing? And if I can understand better what it's actually producing, then I could probably be more coherent in terms of my comments around good, bad or indifferent. Because the life cycle issue of the factory is you're taking what that piece of AI or model is doing and you're putting the components together, you're creating the pipelines, you're creating the orchestration and everything else. What is the outcome of it? What comes out the other side of the factory door? Because that would better inform the hardware distinctions.

Speaker D: If you ask the same question, if you ask the same question of a utility, would you ask it about uh, what comes out the door as a thing? I mean a utility doesn't produce cans of soup or chairs or what have you. It is a utility. It's a, it is a. And to your um, to your life cycle model, ah, it is um, something that is possibly customized, purpose built, depending on, you know, the

Speaker C: right,

Speaker D: you know, what's open to you from the, from the vendor. Um, again, factory, you might use it as a factory to kick out applications, to pick it, to kick out publications, to kick out data products of various kinds. But this is why I think it's useful, perhaps more useful to think of it first and foremost as a utility that uh, can be possibly, very possibly reconfigured or set up, configured at the outset to do some specific job. Some of it will be production, some of it will be service.

Speaker A: I, but the utility to me strikes me as something is very, is very consumption oriented. Like I can see, um, a model API as a utility, anthropic to me as a utility, I mean I have a harness. The harness accesses tokens and I can, it consumes tokens. I think that's a utility model. To me. When I see people defining AI factories, they are laying in process governance, um, places to run, agents, you know, it's, they, they. There's actually a lot of tooling going, staying in the factory model. Right. So you know there's, you'd almost say that there's a utility model would be I have power, I have, you know, raw materials in for my factory. I have, right. Uh, potentially even the tools themselves. And I need all of that stuff set up so I can then go produce my thing in the, in the factory. And so it's, it's multidimensional. The, the hardware aspect really is just providing you know, resources in the form of, of you know, running an LLM for you. You could, you could set up an AI factory. This is where I get weird. You could set up an AI factory without any um, AI infrastructure at all and just use uh, you know, reasoning via API models. If we're going to say that you could have an outsourced AI factory.

Speaker C: Yeah, you could. And here's the question. At uh, what point do you run out of APIs or does your cohort of APIs get so large that you have to go a different way? In that factory as one example. I mean to me the utility aspect of it is not based on the consumption. Like, you know, you pay X amount of dollars to get X amount of electricity because the standard Delivery is always 12 kilowatt into a particular plug. Right. I mean you get a feed from, from a town or a utility, no pun intended, and you're paying. Even if you're not using the bare minimum of what's being delivered, you're still getting charged for it. So to me the notion of utility on a consumption payment model, that's irrelevant. I look at the utility aspect of this is delivering value in a number of different ways. And I can repurpose that same utility over and over and over again. It's like a router.

Speaker D: Mhm. Um,

Speaker C: because my basis for my agents in my factory may always be the same. The one thing that may change is the system of record that it may access.

Speaker D: Yeah, you ask a question a little bit in jest, but maybe not. Um, and that is when you start dealing with APIs and it gets so um, Byzantine and arcane. That's not the right way. That is no longer the right way to do things.

Speaker C: Right.

Speaker D: And that starts to, once again starts to in my mind, establish these facilities, call them factories, call them utilities. But they also start to look like more like trading prints, trading pits or transaction. Transaction, um, forums if you will, where you know, agents, agents seek out and do agent to agent protocols, agent to agent transactions. M in a, um, managed and quote, safe, um, safe arena. It's a, you know, it becomes, it becomes closer to um, a Chicago Board of Trade than uh, it does um, a cloud service with APIs.

Speaker A: Interesting.

Speaker D: And rather than thinking about this as being, you know, APIs and, and living in the, in the last century, you know, which is, you know, remote procedure calls, um, this starts to become a good deal more like agentic transaction or agentic commerce for want of a better word. And that has to exist someplace. It has to be built and operated someplace. Those strike me as utilities. So, um, you know, I think we're, I think you're onto something with the whole, you know, APIs get kind of over, we get overrun by them. How do you, how do you deal with it? We still haven't come up with anything in my mind that's clear about how agentics actually do a lot more direct interactions, multiple interactions. Simultaneous interactions. Collaborative interactions doesn't exist yet.

Speaker A: I thought you could just give them artifactory and let them go.

Speaker D: Uh, you know, a couple weeks ago you might have been able to get away with saying that, but no longer.

Speaker A: I, I think the interesting thing I'm listening to this is that the, the vendors are speaking to definitely, uh, a zeitgeist for enterprise users in,

Speaker D: uh,

Speaker A: trying to capture this idea of an AI factory. I think that, I think part of what we've been talking to and I think is useful is just running, just a whole bunch of people running a cloud desktop session or a open claw session is not enterprise. And I think that the term AI factory is showing up because we're trying to invoke this idea of process and controls and you know, some type of machines and systems that there's Right. It's, it's an aspirational statement of saying you want to use AI. I am going to give you the process controls and parts and all the pieces. Right. You can't just use raw AI. You need, you need systems and you know, you know, it's, it's, you can't, you know, raw AI is not enough. Do you need all this other, all these other pieces? I mean it's. Joanne, it's what you've been working towards, uh, very much. I think you shaped my definition of this quite a bit. Um, just from practice, um, the marketing of these, you know, watching VMwares pivot towards an AI AI factory messaging. I've been watching Red Hat do this for a year with OpenShift AI. Um, you know, it, it's those things are talking to that enterprise buyer. Yeah.

Speaker D: Can I, can I ask you a question? Because I, I've yet to, I, I still get a head scratch on this one. What is OpenShift AI?

Speaker A: OpenShift AI is OpenShift plus, um, uh, a uh, huge suite of products to help manage AI workloads. So they have a open model router, they have a, um, pipelining system, they have a network, a, ah, notebook LLM, uh, interface. They have um, guidance and controls to monitor the pipeline. So it's really a pipelining orchestration platform with a whole bunch of additional services very similar to what a CICD pipeline would look like.

Speaker D: And when they sell that to you with all the bells and whistles, what do they call it? Do they call it a factory the way Dell calls selling half a rack and a bunch of iron a factory?

Speaker A: Um, I'm looking, I haven't seen them call it an AI factory at the moment. Um, it's, let's see. AI overview says it's an ML, ah, ops platform. Um, let's see. So they, they haven't quite jumped on the AI factory messaging at the moment. Um, which I actually find a little surprising because, because it's such a popular term but they were here before people were using it. Um, when VMware's announcement came out, they definitely positioned it as an AI factory. Let um, me see if Rancher is doing it. Um, and Rafa, or let's see. Suse AI Factory is a headline. Um, so their AI factory is an open infrastructure platform for private enterprise. AI private. Um, and it shows a whole bunch of pipe. I, I could put this on screen. But um, they have a standard automated software stack, pre validated blueprints sandbox for production parity. They have an Nvidia version that embeds Nvidia AI Enterprise. So Nvidia's, um, pulling some of these pieces in there. I wonder if Rafay, um, Ti. Um, they're not using the um, they're really selling to the neoclouds.

Speaker D: Okay. And

Speaker A: I don't, I don't, I don't see them using the AI factory as

Speaker D: equipping the neoclouds to be in their minds or in their parlance factories. AI factories.

Speaker A: They're, they're really equipping the neoclouds. Rafa is to, to be a token to sell models that sell access to models.

Speaker D: Okay.

Speaker A: Um, which is a different, A different thing. Um, I look at Dell AI factory. Um, they have an Nvidia AI factory and they're playing off Nvidia. Ah. Um, they have an agentic AI platform.

Speaker D: When Nvidia talks about the, the, the deal that they're putting together with OpenAI for

Speaker A: uh,

Speaker D: you know, enormous facilities on which AI factories will be built, what do you think, you know, what, what shows up in the package? What, what's getting wrapped up here? Because it sounds

Speaker A: their, their, their data, their Nvidia's is um, AI factories are, um. I was reading a summary. Purpose built. Um, they're selling you the servers. Purpose built hardware.

Speaker D: Well, um, but they've got a whole stack. I mean you look at what Nvidia sells you. Yeah, they're selling you servers, they're selling you hardware, but they're also selling you a stack of software that is designed specifically for that hardware.

Speaker A: Yeah, they

Speaker D: run

Speaker A: AI

Speaker D: all the nematron. You know, they've got a, they've got a five, five layers of, of goodies that they deliver to the party and that includes the kinds of tools that you use when you're training, when you're fine tuning, when you're deliver, when you are um, also serving up it Fronts. So you know, it's a, it's get it all here, here. It's all in one nice package. You know, the truck rolls up and drops it on your, on your front porch.

Speaker A: That's that they're trying to get there. Yes. That's how they're positioning what they're, what they, they offer for. They, so they're trying to build a very integrated AI stack. This is to me the essence of where this question comes from is I'm watching Nvidia and then all the surrounding hardware OEMs talk about a AI factory. And they're basically giving you, you know, reference, uh, architecture, set of gear. Um, you know the tales I hear from the field are those that gear shows up and people don't know what to put on it and so they end up running slurm on it or something like that. Which you know, I, but right, that's the, because they don't, they're like ah, ah, what do I do? Um, and so, but then I watch, you know, the software vendors come from the other side and say okay, we have a, AI, you know, tooling, tooling set up with pipelines and controls and model routers and models. Right. You know, IBM spends a lot of times to be a red hat talking about, you know, they have their own open weight models that are verified safe copyright, um, you know, safe all that, you know, all that. Um, although concerns about that seem to have dropped off, dropped down through the floor. Um, the, you know, and so um, you know there's, there's definitely a software, the software side and the place where I was trying to scratch my head is like all these software vendors are talking about AI factories. The hardware vendors are selling you kit to create an AI factory. Um, you know, I, I, I look at as a, as a many, many problem in the middle. Yeah.

Speaker D: And, and some, something that looks kind of like an MSP or a NeoCloud is actually buying the hardware kit, buying the software kit and putting it in place. It's like changing the whole nature of restaurants and turning them all into ghost kitchens. Right. Somebody, somebody, somebody puts, puts the uh, real estate there. Somebody else comes in, puts all the institutional um, grade stoves and you know, big appliances, refrigerators, freezers, everything else. And then somebody else comes in, buys up the specific um, knives, prep all the specialized tools that they need for that cuisine and then go to somebody like the other Cisco and uh, get food delivered.

Speaker A: I think that's a fascinating analogy. Um, um, I think the neoclouds have a vision of getting to that point, but the ghosts. Um, and it's a really fascinating analogy because somebody who knows what, what food they want to cook does not make a restaurant. Um, so, you know, I think the neoclouds have the challenge of current. Their current customers are, you know, are already chefs, and they're just being industrial equipment suppliers.

Speaker D: Yeah, well, the neoclouds are, you know, living the good, Living their best life because, you know, OpenAI and Anthropic and, uh, and

Speaker A: Microsoft

Speaker D: need as many, you know, GPUs out there serving up. Serving up inference as they can possibly get.

Speaker A: Agreed. And that does not translate into, um, composable AI factories, um, or a repeatable business model because.

Speaker D: Well,

Speaker A: I mean, it's working at the moment

Speaker D: for a while,

Speaker A: and until, you know, the gold rush. All right. Uh, I don't necessarily feel we got closer, but I think it was a. It was a healthy, spirited conversation. I mean, the market's going to flap around, however it wants to flap around. So,

Speaker D: uh, it's going to flap around and it's also going to be the case where people are going to play with the word factory, and it's going to mean so many different things to so many different people, but it's going to be whatever you're selling, you're selling a factory. Guess what?

Speaker A: Yes. No, that's. That's right. No, that is. That. And we'll be at AI Infrastructure Summit next week, and we can. We can count up informally, we can over under. On the number of people saying AI

Speaker D: Factory number. How many. How many of the. How many of the vendors there will have AI Factory on their. On their. On

Speaker A: their booth?

Speaker D: I

Speaker A: think. I think. I think 30% or more, I think.

Speaker D: Yeah. Um, I'd say it's going to be somewhere between 30 and it won't be quite half.

Speaker A: Okay. 30 to 50. I think you and I are. We'll come back and report.

Speaker D: We will. All right.

Speaker A: All right, I see. On the record. See y'. All.

Speaker D: Bye.

Speaker A: This conversation is one of those ones that is as much about the aspirations of the market as anything else. Um, but I found it really helpful. It didn't necessarily lock in a definition to me, and it also helped clarify what people want, how. How confused people are about how to build an AI infrastructure that works for them. Uh, so I think we really turned over some very interesting components to this conversation, and, um, I hope you are finding it interesting and informative. Please let us know. Or better yet, come join the conversation. Thank you for listening to the Cloud 2030 podcast. It is sponsored by Rack Nice, where we are really working to build a community of people who are using and thinking about infrastructure differently. Because that's what Rack N does. We write software that helps put, uh, operators back in control of distributed infrastructure, really thinking about how things should be run and building software that makes that possible. If this is interesting to you, uh, please try out the software. We would love to get your opinion and hear how you think this could transform infrastructure more broadly. Or just keep enjoying the podcast and coming to the discussions, uh, and laying out your thoughts and how you see the future unfolding. It's all part of building a better infrastructure operations community. Thank you.