Mozilla CTO: Why Most Enterprises Don't Control Their AI

Ravi Krikorian: Pinterest deployed open models instead of closed and saved something in the order of $10 million that quarter alone.

Michael Krigsman: Open source software built the Internet, but it's losing the AI war. Ravi Krikorian is CTO of Mozilla, which makes the Firefox browser.

Ravi Krikorian: Most enterprises are renters, not owners. They're talking to either big platform companies, they're making API calls that are off prem, they're doing all these things. You're basically turning over your, your destiny over to a different system that be making a call to. So in that definition, yes, most of them are renters. And we're starting to see a bunch of this even inside of Mozilla of like how that renters versus owners situation is playing out. Like you know, I was just looking at one of our Claude code leaderboards within Mozilla. We're using it on one of our subsidiary companies, MozVC, and one of the engineers was effectively could have racked up a $10,000 API bill uh, this month. But thankfully we're paying for the $200 a month subscription. But you know that subscription cost could change at any moment, will probably change post ipo. So all those things I think in my mind collapse into this renters and not owners framing. But Michael, I want to touch on that first one. I think the question of whether open source is losing the AI war is more of a question of just like what do you think the war is like? Is it losing on a bunch of leaderboards then? Sure, it's losing on a bunch of demo cases, the extreme cases. The question for me is really like, like is it going to win on the everyday use cases? It's going to win on the things that most enterprises care about, most companies care about, most regular users care about. Like it's probably losing or uh, definitely losing on the frontier battle. But like in the same way that you know, we would have said at some point that Linux lost the desktop, but at the same time Linux effectively runs every computer on the planet. And then there's a few iPhones lying around like kind of thing. And so like I think we could be in a similar situation when it comes to open source, open weights and the AI world. And like on top of that I really think that, you know, I think there's a lot of evidence that shows that the open models are compressing their difference between where the closed models are. So I think this battle is long from over. This war is long from over. I think you're right in maybe in today's parlance it might be losing, but like, I think the curves are starting to compress somewhere.

Michael Krigsman: Walk us through the issues, the problems. When an enterprise is essentially beholden to the major model makers to closed systems.

Ravi Krikorian: A lot of this comes down to just a question of what does it mean to control your destiny? What does it mean to control the optionality that you're sort of working in? So if I were to rewind the clock, like I think that when we were all talking about cloud and cloud deployments, I think we all got into this position where we're just like, well, we can't just have one cloud provider like we in the enterprise case or uh, the people who are trying to deploy large systems case. Like we need the ability to hedge, we need the ability to actually choose and be able to play off vendors from each other, do all the things that we want in order to make sure we're getting the right value for our cash, we're getting the right features that we want. So like, you know, in a lot of ways we didn't build raw to people like AWS and others. You use tools like Terraform or something else in order to build an abstraction layer so you could quickly move between them. So when AWS shows up one day and it's like, I want to give you a huge bill this month, we're going to be like, well, fine, we'll pay it, but next month we're moving and then that starts a negotiation. And I think we're in the same place right now when it comes to all these open, sorry, uh, like these AI tools and the enterprise. You know, Zapier just did. I mentioned this in my newsletter this week. Zapier just did a survey of talking to people, talking to enterprises about whether or not they feel that they could switch off their AI providers. Something like 85, 86% of them thought they could, but when they actually tried to do it, only about 30% of them could actually pull it off. So uh, there's already so much lock in happening. And so you get into a situation where like these things aren't under my control and then, and then the providers can just change model behavior at any given moment. If they're having capacity issues, they could shuttle you off to a different model that might behave differently. So therefore I've lost control of even just how I'm programming against this. And then that thing I mentioned earlier, which is costs right now I feel are both a little, both kind of inflated, but also a little under control. I would have predicted a few months ago that Token Costs were going to go down, they kind of have. But token usage has skyrocketed and a lot of that could also be because of model change. And uh, new models and new harnesses just need more tokens, which is in the benefit of the people I'm buying from. So I feel like all these situations, if I were an enterprise buyer would be like this seems a little out of my control right now. And so I think that's what I really mean.

Michael Krigsman: So would it be accurate to say the issues that you've brought up are loss of control, uh, of the model, which is ultimately all important here, loss of control on uh, pricing and obviously not token usage because that's going to be driven by the user. But essentially you're at the mercy of the large platform providers.

Ravi Krikorian: Precisely. I think like the models could be changing at any given moment, so therefore writing code against them or. Right. Or trying to codify your behavior against them, it's not under your control, it's under control of the provider. That seems not great. It might be fine in a prototyping sense, but not uh, like a large scale deployment sense. I think what you, what we mentioned on the costs, I think that's definitely one thing that's a little bit out of control. And I also do think token usage is a little bit not fully in your control. Sure you can decide how much or how complicated a task you're asking, but like once I set off my, once I set off my initial prompt, all the different things that happen in the harness could just exponentially grow. And that is also a little bit out of my control.

Michael Krigsman: Folks, we're talking with Rafi Krikorian who is the CTO of Mozilla. You can ask questions if you're watching on LinkedIn. Just pop your question into the chat. So Raffi, one of the points that you mentioned was the lock in is how big an issue is that and how does open source solve it and what are the solutions in general to this lock in platform problem that you were describing?

Ravi Krikorian: I do think it's a big problem in the sense that if I'm running software within my business, I want to know that I have optionality. I want to know that I can move between different people, depending on feature sets, depending on exactly what I want to go do. I don't think it makes sense for people to assume that one provider is going to do everything for me. I want to be able to finely grain and tune it so I can use the features of a provider that match the things I need to get done. So in some way I do think locking is a big deal. I think the Zapier survey is a good example of just like how people are implicitly getting locked in. Like they. I don't think they willingly walked into that situation. In fact, they believe they could walk out of it. It turns out to be quite hard to go do for a variety of different reasons. And so I think open source is one of the solutions, is clearly not the only one. I think it's one of the different solutions. Like, you know, the open source models are both getting really quite good at what they can do in the enterprise work cases. Like we've been experimenting with the open models within our Mozilla AI group and that group is building developer facing tooling and enterprise facing tooling. So they've been understanding and better using what those models look like. I'm using an open model on my day to day coding tasks. Like I run Quinn 30B on my laptop, especially when I'm on United WI Fi and that thing doesn't work. So it provides me a really good experience for my laptop. So I can pick and choose the kinds of things I want to go do. So if you already live in a world where your assumption is I should be able to pick and choose, then the open model sort of like give you that ability to do it. And then we're been working on a set of products, one of them that we call Otari, it used to be called any LLM and it's basically a router that you can install on your desktop or you can run in that cloud environment. If you point to it, it can then switch rapidly between different models depending on exactly what you want to do or what particular workload looks like right now. So just even embracing the idea that you should be talking multiple vendor, multiple model, and just going into that mentality I think gets you into a more stable spot. And then you can start really having a conversation. Just like this model for this use case, this model for this use case, this provider for that use case. It just seems like a more resilient place to be. And if you're in that space, then Open source is clearly going to be part of that solution. Might not be the entire thing, or for some people it might be the entire thing. You can then be the one that chooses.

Michael Krigsman: The focus then for you is on the user's ability to make those choice decisions as opposed to leaving it to the third party vendor, essentially.

Ravi Krikorian: Right. I mean, if you were to like abstract away my entire position here, it's basically some question of choice and um, competition, like how can you actually have a user, end user be the one that's making the decisions and how can they have a vibrant ecosystem that they can choosing from and actually selecting the things that they want to be using from that menu of different things. So if we can get to that world of open systems combined with open source, that allows for choice and competition, I think my job here is done. Uh, we are very far away from that world right now. There are lots of things between here and there, but that's the world I'm trying to get to.

Michael Krigsman: We have some questions that are coming in and so let's jump to those. The first question is on Twitter X from Anthony Scriffignano, who is a prominent data data scientist. He does a lot of work with, consulting, uh, for various agencies with three letters and things like that. And Anthony says this, he's been a guest on CxOTalk a number of times. He says, drilling down on your comments about model change without notice. Please talk a bit about the challenge of regression testing. Simply proving that things still work the way they used to work. Are we losing the battle?

Ravi Krikorian: Last year there was a really big push into building evaluation suites, building toolings to do evaluations, et cetera. Evaluations are incredibly important. The academic community does it all the time. And I think what we've lost the muscle of and we need to figure out how to bring back is for all of us to be doing our own evaluations on the workloads that we really care about. So in some ways we look at evals all the time, right? Like we look at the coding bench. We look at all those things, understanding how well models are doing. But those are very much in abstract cases. And frankly they can be, I mean, I'm hesitant to be saying gamed, but they can be tuned for. You can tune yourself to be able to pass a particular benchmark. But if instead what we were doing is recording all the different prompts and all the different tasks that we're doing locally and then creating tooling that allows for model evaluation ourselves, then we could easily figure out, or maybe not easily, but at least have a framework to figure out whether or not a modeled selection choice or an agentic harness choice, whatever, how that's going to perform. On my actual personal workload, I've actually been thinking a lot about this. I actually wrote you can find it on GitHub. I built this thing I called Morph, which is basically git, but for prompt based workflows if you think about what Git does, Git allowed me to take computer syntax that was then compiled down to bytes and stuff those bytes into a version control system. I don't think that really works anymore on these LLM based coding tasks. And what you instead need to do is you need to record the prompt and then record the output of the prompt in that case, which is code like bytes might be at the wrong abstraction level. So if you start doing that, so if you start using version control to manage what our prompts look like, like what's the prompt, what's the harness, what's the model, when I ran it, all the settings and stuff like that, then we can have reproducible use cases of understanding what my real workload looks like. And then I can start playing that back against other models or other versions. Like for example, Opus 4.8 just came out. Everyone is super excited by it, I'm super excited by it. But, uh, one of the first things I'm going to do this weekend is I've been recording all the prompts of all the different pieces of code I've been tinkering on and I'm going to run those prompts against Opus 4. 8. So I can just then start doing comparison of just like actually if I had this model six months ago, this is the direction this piece of code would have gone into. We can start thinking about how to build those kind of workflows both in the enterprise case and in the personal coding case. Then I think we can get to a more intelligent conversation of like, this model is actually better because at some point you might need the opus 4.8 or in some cases Qin 3530B might be just good enough. And so understanding of what that balance looks like, we can only really do that with data. So we just need the right tools to record that data.

Michael Krigsman: I think I'll say amend to that. I mean, this whole situation is very ad hoc right now, choosing models and figuring it out. I personally, right at the moment Pay for the $200 plan for anthropic and the $200 plan for perplexity. Um, and I say to myself, who really knows anyway?

Ravi Krikorian: Exactly. I mean, I mean, if you look at me, I mean, I'm spending more money on different AI plans than I spend on my cable bill or like my television streaming bill right now. Right? Because like, to your point, I'm just like, I guess I'll try them all and just see which one works best. But like, we need a better definition of what works Best means for Rafi, what works best means for Michael, so that we can actually then have the tooling that can figure it out for us, stuff like that.

Michael Krigsman: We have some more questions coming in and this is from Chris Peterson and I encourage you guys ask your questions, take advantage of this opportunity. All right, uh, Chris Peterson again on Twitter X says, do you think that tiered usage plans will continue long term or will all the big AI providers be forced into per token pricing as venture capital wants to exit and not keep funding?

Ravi Krikorian: There is no clear exact answer, but I try to look for equivalent analogies, right? So I used to work at Uber. I ran the self driving car division there and you could see it even in Uber that like post IPO the prices changed, right? Like all the subsidization that we did of uh, Gen Z just basically disappeared and the real true pricing showed up in that situation. And I'm likely to believe that this is a similar marketplace that we all that subsidization that like $200 plan that my mods engine, my Moz AI engineer who then did analysis, like I would have spent $10,000 in API calls, I think it won't go to the $10,000 case because I think that sort of locks the market, um, sort of like limits the market. But I do think it's not going to be these $200 plans. I think that that price either floats up significantly or we go into this per API model. At which point you need to then start making the decision of like, do I use Opus, do I use Sonnet, do I run it locally, do I do something else? So I think that like, I think the best situation is to start thinking about this, evaluate those kind of questions now to try to find the right model for the tasks that you're doing. Because I do think the price will change. I think you're right. It's just unclear exactly how it's going to change. And if you think that I'm wrong and I might be, then you can make other different arguments of just like, do I really need to spin up an entire data center's worth of GPUs? If I'm just like, write me a piece of code that says hello world, right? Like then just as like good engineers, that feels kind of icky. And so trying to figure out like what do we think is like the right engineering size for the right tasks I'm doing could be another angle to look at it.

Michael Krigsman: I always want to use the best possible model because if I'm uh, asking an LLM for a solution. I have some intention, some reason behind it. I want the best possible result.

Ravi Krikorian: I don't disagree with you. I mean like I've spoken to, you know, I was speaking to an engineer the other day who was making the argument that his bosses only allowed them to use the frontier models because they have a belief that it'll make them more efficient, that the frontier model will probably give them the right answer the first time and they're not going to sit there and whittle it away. And I guess like my response to that is just like, it's a good belief but where's the data behind it? So like this is, this is why I'm really harping on the. We need to be recording our prompts, we need to record what models are on, we need to personally do our own evals of this. So we need good tooling to do that so we can actually then make these real logical trade off decisions of just like, I think I agree with you Michael. Like I would like to be using Opus 4.8 the entire time until I get the bill. And which point I'm just like, okay, but like, was that really worth $1500 this month or could I have done that with $500, like kind of thing? So I think it's just when like the rubber meets the road, like fitting the right size is going to be the thing that we all need to do.

Michael Krigsman: Here's a question on LinkedIn from Tim Cook Crawford, who is a, uh, major CIO advisor and he says on the topic of pricing, it seems that outcome based pricing is not granular enough, but token based pricing is harder to understand. Is there a good alternative? It seems this is part of what is causing the drive to locally sourced AI platforms.

Ravi Krikorian: We've been facing this challenge as we figure out pricing on our different products as well. So the Moz AI team has something that they've released that they call Optimus. It's a fully open source, um, agentic workflow system. So you know, think of it as like a really good agent with really good UI to help you inside your enterprise. Connect your email to Notion to zapier, etcetera, etc, etcetera. And right now their pricing model is a bit wonky because we were trying to figure out how to do outcome based pricing as well. It's just like what does it cost to price an action as opposed to all the thought and thinking needed in order to make that action go and what if the action is super complicated and stuff like that. So I, I would Love an answer to that of uh, like if you understand a better way to go, do that, like I'm fully open to it, we're just facing the exact same problem. But uh, token based pricing is so crazy in my opinion. Like I was actually having this conversation, you know, Mozilla has a partnership with Mila, the Canadian research lab. And I was up in Montreal just, you know, last week and I was having this conversation with a bunch of AI researchers and I was like, why do the models effectively, when they're doing their internal thinking, do it in English? Like why is it, why am I looking at a train of thought in English? And then it talks to me in English or whatever is the native language at the point that clearly can't be the most efficient way for it to be doing itself. And they all looked at me and they're like, no, that makes a lot of sense. You can probably come up with a better thing. You're doing it for inspectability. But then I'm like, but then why am I paying for the model thought to be happening? English would be told me there's a better way to do it. So I think this whole per token pricing is just frankly too confusing. It's probably not going to be the end run, but I don't think anyone has solved how do you tier actions, how do you tier impacts? I don't think anyone has solved that just yet, but I'm very excited if someone does because I'm going to copy it, just open source it.

Michael Krigsman: One of the things I find most fascinating about all of this, this token discussion, is although the cost per token is coming down, the expense of tokens is going up because we realize, hey, we can do all of these new things. And so I'm willing to spend more money. I mean, it's kind of a crazy set of economics that way.

Ravi Krikorian: I don't think a lot of people understand like the impact of the different harnesses that you might choose in which to run your models in. Like open code is going to have a slightly different behavior than Cursor is going to have. Just have a slightly different behavior than Terminix is going to have. Like all those things. I don't think most people understand that, but like in a lot of ways, like those harnesses are also part of that. Um, I can call the problem, but part of what's causing this token explosion is because of the way that they're going to do their internal calls, are they going to allow for a call depth of 20 or a call depth of 100 and stuff like that. MCP is part of the issue of just like using natural language to figure out, like, should I even make this call in the first place? And having to do all that thinking, like we've built a whole almost like Jenga pyramid of tools that's all trying to do communication via, uh, English. And so that token cost is getting a little, it's getting a little crazy. There must be better solutions. I'm not smart enough to know what there is, but there must be better solutions.

Michael Krigsman: On the subject of tools and the tool that you were describing to help keep track of and audit prompts and results, it seems to me that it needs to be automated in some way because otherwise it simply becomes, uh, too complicated and then you end up with an entire set of privacy issues.

Ravi Krikorian: If you look at for under my username on GitHub, which is just r, but under r on GitHub is a project called Morph. The way Morph works is that when you init your directory, you then tell it which is the agentic framework that you're going to be working in. So you can do morph set up cursor, or morph set up open code so it'll hook itself in and then just start recording all the prompts as you put them out there. But the nice thing of doing that is that you get in a standardized format. If you're working in a team scenario and someone uses cursor, someone uses open code, it all normalizes. You get them all in a single database format, stuff like that. But you're right, the problem is that a lot of people are, I'm, um, not going to call them lazy, but they don't switch out of the tool. Depending on what they're doing, at one point they might be actually typing in a prompt to do a, uh, programming, but then they might also type in a prompt because they just want to ask a question. I'll just like, well, I'm already here here, so I'm going to keep on going. Um, so yeah, there's a massive privacy question that comes out of that. The issue there is that you can't really scrub it, because if you scrub it, then you're fundamentally changing what the prompt is, which might make the eval a bit problematic. So I haven't really thought about how this works outside of personal context, but we clearly need to figure this out on the enterprise context, etc. So maybe it's a little more of a social question of like, when you're doing an enterprise, you can promote the prompts that we should be using as part of the eval set as opposed to just doing it across everything that happened on my computer. So maybe there are things like that. But, you know, I feel like this will fall into the same problem of, you know, people use their web computers, their work computers all the time to surf the Internet and they have bookmarks on them. So we're having a similar problem. The obvious issue being that people ask more detailed questions and prompts than they do the web. But we'll just have to solve that.

Michael Krigsman: This would be an excellent time, too. To subscribe to the CxOTalk newsletter, go to CxOTalk.com so we can notify you of upcoming shows. Okay, so this is from Arsalan Khan, and he says all these AI agents are creating confusion for the employees. Should there be an AI agent that addresses this confusion? Should we have a limit of how many AI agents can be created in an organization? I like this, a very enterprise type of question.

Ravi Krikorian: There's the analogy that I'm, uh, very much a fan of, which is the IT team is slowly becoming the HR team for agents, effectively. So I do think that, like, if I were inside one of these enterprise environments right now, I would be putting a lot of weight onto the IT team to actually figure out, like, what are the standards that we want to go with, what are the permission models we want to work with? How do we do observability on what all the different things that are happening are? And so I think if we can figure out how to centralize a bunch of those inside the enterprise environment, then I think we can have good policies that then can control the way that things are being rolled out. This is a new frontier that we're all experiencing, not just on the builder side, but like you're saying, on the deployment side. So understanding and figuring out how to open source those policies would actually be a really good thing. The question, uh, I keep on having people ask me is just like, how do you manage write versus read permissions? Like the dangerous triangle that sort of happens within, within agentic frameworks of like, they can read things and they can actuate things and how if you can't. If there are some transactional boundaries across my entire enterprise, how do I make sure I can roll back and. Or how do I make sure these are actually idempotent? So an agent does it three times. I don't accidentally bill myself $1,000 each time, like, kind of thing. And all open questions, like, I think we're all racing into this because we see what the potential power looks like. But no, it is not just a coding problem. This is like a whole socio, technical, economic, commercial thing that we need to be solving together.

Michael Krigsman: On that topic, Arsalan Khan comes right back and he says who should care about these token prices in the enterprise? The cfo, the cio, Nobody, Everyone.

Ravi Krikorian: The responsible thing to do is to sort of like everyone sort of needs to care about it but I think you need to be a little bit open minded about it. So I think that like people going on their whole like token maxing spree, obviously that could be problematic on my end, my end bill, but I think like we need to understand, you know, I think of myself, if we just look at myself, I'd like, I think of myself as a pretty decent engineer, maybe not the best engineer. I'm um, maybe a little out of date. But these systems have made me incredibly productive. A lot of it's because I have a lot of experience in software architecture already and these systems are amplifying my work and I don't think I'm token maxing like crazy but there is a cost to doing it. So I do think that like a CFO in a similar way that he or she is looking at what the headcount costs look like. What is the value we're actually getting out of like the employee workforce? Are we getting the right value for the number of employees on what the output and the outputs of the organization look like? I think they are a part of the conversation but I think that given that the world is moving so quickly, I don't think it's purely the cfo. I think it's CFO in partnership with whatever is the right CXO function, the cio, the cto, whoever is the right thing I think is a partnership there of understanding where we are today, how the trend lines are going to look tomorrow, what the money impact looks like. So I think that almost requires a full C suite, potentially even at some point a board level conversation just like how do we want this to work inside that organization? Yeah.

Michael Krigsman: And to your point we don't have great tools for aligning the inputs with what we ultimately with the data that we need in order to make those evaluations.

Ravi Krikorian: No, I mean I think there's going to be a whole cottage industry that can potentially pop up here of just building HR like tools but for agentic workflows. So just like how much are we actually spending here? What's actually coming out? How many are ah, we hiring every single quarter how many returning off every single quarter? Who are they reporting to? What are the like? I think there's some actually pretty good metaphors that we can use. We shouldn't obviously follow them to the T, but there's some pretty good starting points and you can imagine a whole cottage industry of like building those tools so that CIOs, CFOs, etc. Can manage their new agent.

Michael Krigsman: Let's grab another question, uh, again from LinkedIn and this is from Swami VI. And uh, he has an interesting question. He says, would enterprises develop or should enterprises develop a hybrid approach between renting commercial models and also managing locally hosted models and leverage them based on usage, case complexity, cost, risk and other factors? How do you envision the target operating model? And I think, Rafi, you were getting to this earlier describing deploying, uh, multiple models based on cost, use case and so on.

Ravi Krikorian: I definitely think that a responsible organization should get to that. I think when you are just starting off, like, if you're a startup, then I think you should do the most inefficient thing you can, but move very quickly to figure it out. Like, I like this analogy all the time of, um, you know, I used to run a big part of the Twitter engineering team, the entire infrastructure group back in the day. And my team is the team that did the transition from Ruby on Rails to the JVM for a lot of the code base, like let's call this in 2010, 2011. And a lot of what happened after that migration, when we were giving public talks about it, is we have new startup engineers come to us all the time being like, I want to start writing code on the JVM today. Can you help me? And I was like, no, no, no, no, no. Like, you should be lucky to get to the place that we had this conversation about how to optimize everything. Like, you should do the thing that you can do the fastest. I'm not saying Ruby on Rails was a bad thing. I just said Ruby on Rails is a bad thing at this moment of Twitter's evolution. So back, uh, in the beginning stages, you should be doing the thing you can do the fastest and you can learn the most from. And in those cases it probably is just use the frontier models, figure out what you're up to, uh, figure out what you're doing. And then when you reach some point of like scaling or efficiency or stuff like that, that for me is the point that we start thinking about what's the right mixture of models, what's the right mixture of providers? Can I do something local and not. And like you know, a good place to start frankly is that like if you're buying your, if you're buying your engineers, you know, MacBook M, M4S, MacBook M5 laptops, they can probably run a pretty good local model to do coding. I do it all the time. I live on planes and I do it all the time because United WI FI never works. And so like I'm always running a local model on my laptop and that's usually my first resort, my first interaction when I'm doing my prompt based stuff is with this Quinn model that I probably have on my laptop running right now. And but then if I have something more complex or if it seems stuck or and I have a good network connection, I might then make a call out to Claude to Opus 4.7, Opus 4.8 to just be like it's stuck. Help me out here for a second. And I just go back to my day to day workflow. So I think you can find small ways that it's showing up already and that you can make it possible to use already. But I also just being pragmatic about it, I really encourage that when you're off starting, just choose something that lets you move quickly and then figure it out from there.

Michael Krigsman: That's very interesting. You're bringing in an element of governance that we don't usually think about. CIOs uh, typically think about governance as being cost control, uh, security, privacy, things like that. But you're really bringing in another dimension as to based on organizational maturity, size and goals. Thinking about the type of model which gets to both the results, efficiency and uh, cost savings.

Ravi Krikorian: All those things matter a lot and it's actually one of the things to think about like as you're deploying on prem versus using a platform provider. Like I want to be protect, I want my data to be protected by architecture, not legal handshakes. So like the platform provider's privacy policy might say we will never read your code, but I want to know for sure it's not going to happen. So therefore a local model, locally hosted model could be a path in that situation. But I do think, I don't think it's necessarily like a unique thing to say that we should be working on this evolutionary curve. I just generally think of it as like what an engineer would do. You shouldn't prematurely optimize your code base. You want to get something working to an MVP as quickly as possible because that'll help you understand the shape of the problem, help you understand the bounds to get to that MVP that you can ship quickly and then we figure out how to optimize from there.

Michael Krigsman: Let's get to another question, and here's an interesting one from Noah Crow on LinkedIn who says what has to happen before an individual or a small organization can realistically own and operate a capable AI system end to end, including the model, memory, data orchestration and inference without dependence on a major cloud provider?

Ravi Krikorian: The analogy I like to talk about all the time is like when I'm just doing my weekend hacking and I just want to do a quick prototype of something, I'm probably calling ChatGPT or I'm calling, you know, GPT 5 5, um, as part of the API call, because it's just the simplest thing to do that I can just make sure something's up and running, I don't have to debug anything else. And I think that's a vast majority of developers out there. And it sounds like it's a question for you to do. And like right now it's because there is no good credible alternatives to the entire system that someone like an OpenAI or an anthropic provides to you. Like, it's not just API, right? Like really think about it. It's like they already have the gpu, they already know how to orchestrate it. They've built the rag layer, they have tool calling, they've done all this stuff. Um, they're very just abstracted behind the scenes for you. So just having the model is not enough. Now, if just having the model was enough, you could do a lot like MLX on Apple. Silicon works really great. Ollama works really great. There are ways to go do this. But if you want to really think about this in an SMB or an enterprise situation, which is like, I have a rack of servers in my closet. Fun fact. It turns out there's something like 70% of enterprise GPUs sit idle most of the time. So I think there's a big opportunity to do something. But if you have a rack of GPU sitting in your closet, I'm, um, pointing to my closet right now. A rack of GPU sitting in your closet. Or you have a rack sitting somewhere inside the building and you want all your developers to be able to make use of it. It's not easy to set it up. But just being very brutally honest, you're probably going to do kubernetes. But then you got to choose what's going to be your rag layer. Maybe it's deepset you got to choose all the different components and then you got to do like a weekend or a week's worth of like sre ah work in order to get this uh, all working properly. And so this is actually one of the problems I'm working on right now of just like how can Mozilla help figure out how to get this? Like I don't know what's a good analogy for it? Like the LAMP stack for inference or like a J2EE stack for inference. Like how do we actually build like a opinionated system that could be installed that then someone can just get off the ground and start using and when there are small things they want to change, you just apt get, install something else and just swap out that layer. It doesn't really exist right now, but I think it could. If you think about what was the brilliance of Ubuntu, there is a world of code out there in order to make a Linux distribution. But Ubuntu had a little bit of tastemaking of what are the packages that we can fit into a CD that someone can pop into their machine and just get up and running. And I think someone needs to build something similar to that for an open source stack of just like let's just choose the right components that make up this LAMP stack for AI, let's just bundle it together. It's something someone can just pip, install, apage, install, whatever, um, onto their server and it's up and running and then if we want to tweak it, customize it, et cetera, then you can do that next. Um, but this is the thing that one of the things that Mozilla is working on right now is like how do we just build that like container code base that then someone can just app, get, install. But I wish it was done already. Not done yet.

Michael Krigsman: Chris Peterson on Twitter says is Mozilla ah, providing support for open source model and harness developers or organizations that are building open standards to empower agents?

Ravi Krikorian: We are not playing as much in the open model world as we are in the rest of the stack, and only because a lot of people are already playing in the open model world. So like I really view that our job right now is to try filling the gaps in so we can rapidly get to that thing that you can get installed and have just working inside your rack, in your data center. And so I don't think models are currently the problem there. I think the problems are the other things you mentioned. Just like how do we build an open system so we have interoperability between all these different layers of the stack, what truly are the missing layers that need to get invested in? I think the agentic harness is this, this new thing that we're now all talking about. That's an interesting opportunity, but then that still needs a fully open stack underneath it for you to have real trustworthiness around what's truly going on inside that system. So I think it's that open system, the interoperability things that we're going to be leaning in on the most and trying to make sure the community is doing the rest of the pieces. If you look at one of my newsletters recently, I write a newsletter called Owners Not Renter. We did a whole analysis of what an open stack could look like. So we took everything from bare metal all the way up to, at that point, providing an OpenAI compatible API. I think we just released another version of newsletter that also had the agentic harness. But what that layer looked like from hardware all the way to an OpenAI compatible API, broke it down to all the individual components and then literally did a catalog of all the open source projects that fit into every single one of those layers. And it turns out almost all the layers are covered. You could do this today. I mean, the map is kind of red and screaming at us when it comes to enterprise readiness, but all the pieces exist. What's missing right now is making sure they're all compatible with each other, that we can all tie it together in a very simplistic and easy to use kind of way. And then some battle hardening so that an IT team wouldn't vomit when they saw this thing show up on their doorstep. And so I think that's the kind of stuff that I think is Mozilla's unique secret sauce, like can we play an orchestrator in the ecosystem to pull that together.

Michael Krigsman: Anthony Scriffignano comes back and he says using the best model to get the best result can lead to confirmation bias. He always tries to get a dissenting opinion and also, uh, provenance. Why is this the best answer?

Ravi Krikorian: Answer? I wrote this piece in the Atlantic a few months ago called the Validation Machines and it was specifically targeting what that, what you were just saying of just like you get lulled into believing these are the best results because of the confidence in the way that they talk about it. And what they are not doing is like, you know, main analogy here is like, we've all underestimated, I think what do the 10 blue links at Google have actually done? Like, my mom doesn't really understand what page rank is, but when she sees the 10 blue links she's just like, I kind of understand what's going on. Um, and we, we are, we are potentially falling into that world of like when you ask for the best model to give you the best result, like how did that best result come out? Like there was actually just a paper recently maybe some minor divergence from enterprise use case, but I think it's good illustratively. There's a paper recently that came out of I think joint between Princeton and UW up in Seattle, um, where they did an analysis of asking all these agentic systems, um, purchasing decisions like I need to buy xyz. And it turns out maybe unsurprisingly in retrospect, a lot of the platform systems recommend um, products which have been sponsored more often than not. And in some situations there seems to be a lot of evidence, again I defer to very smart researchers at Princeton and uw, a lot of evidence that these systems then do also a little bit of social economic status polling of you to then try to figure out how to maximize the number of dollars they can extract from me when I'm asking this question. So you're right, it's not just confirmation bias. Like it's a clear example of these systems are not under your side, they're on someone else's side. Um, so if you bring that back to coding or if you bring that back to workflows, then I think really the question is just like how do you have trust that it's done the right thing? And so like on the philosophical level, do you have trust because you're using an open system that you're running yourself? Like you know, maybe someone else's biases were embedded. But it's not like making a network call to figure out which is the promoted thing that might get some kind of dividend or cut out of. So there's a little bit of trust you can gain by running it locally. There's a little bit of trust that you can gain by it being open source. But I think Anthony, you're right. Like provenance being backed up with data being understand sourcing of the information. That's where true trust is going to come in. And I just don't believe that we're going to get to a place where the larger providers are going to be incentivized to give that uh, information to you. But the open systems could be. So I want to figure out how to make that happen.

Michael Krigsman: Elizabeth Shaw, who says how does creating your own open source AI infrastructure address the issues you've talked about, except for avoiding token pricing and cost a lot of ways.

Ravi Krikorian: It's not just token pricing and cost. I think when you create, I'm not sure I'm advocating for create, but when you use one of the open systems, um, then at least you know full control over what's going on and what's happening. Like we saw this in like some of the ChatGPT migrations where like you know, they bumped a version Number on, on ChatGPT a lot of people had their code pinned to latest as opposed to being code pinned to a specific version and all of a sudden the behavior just fundamentally changed. And so like yes, pinning it to a version number might have alleviated some of that problem, but there's also a lot of evidence that when they're under capacity load that the behavior of these systems are changing. Who knows what's going on behind the scenes. I have many theories but none of them are necessarily backed up. But like we've noticed that their behavior changes that happen to these systems when they're under load versus when they're, when they're under, not versus. And so like I think having control of your own destiny I think is one of the biggest gains that you can get by running one of these systems yourself. Now you know there are obviously going to be trade offs when you do it. Like when I was at, when I was doing a lot of work in politics, a lot of the reasons why emails got hacked is because people are running on self hosted servers versus running on hard earned servers that the big platforms provide. So there are trade offs that are have to be made in these systems. But I believe that at least for most CIOs, CTOs, et cetera, we can navigate the trade offs if we're just told what they are and if we know there are viable alternatives that we can be working with. And I think a lot of situations, especially in this world of AI that we're working under, we just assume that the only option is to give our credit cards to one of the platform providers. It's a good option, but then you should just understand what all the trade offs are.

Michael Krigsman: This is from Nate angel. And Nate angel says how can Mozilla help ensure that AI is always human centered, always in human centered loops, rather than the cliche we keep hearing about having humans in the AI loops. I'm so glad he, he asked that because we do hear human in the loop has become this meaningless buzzword. And uh, so it's an interesting question.

Ravi Krikorian: I do actually think Mozilla is one of the organizations that's uniquely able to do this. One of the reasons I gave up my board seat in order to do this work as an operator within this company is because I think this combination of our nonprofit status, which means that we can do things that benefit people and benefit humanity as opposed to a bottom line. And um, we need to be sustainable. Don't, don't get me wrong on that, but the fact that my top line every day is driven by a mission and not by a bottom line, I think makes us a, uh, unique organization that can go tackle this problem. And we have a history of doing things like this. Firefox was effectively created as a way to push back against the monopolization of the web by Microsoft in the 90s. And because of Firefox existence and because of the way it was deployed and used throughout the world, and because of the way that people got trust in, forced the web to become open in a bunch of different ways. So I'm looking to do a very similar play when it comes to how we think about AI and how we get to human centered AI. So our Mozilla foundation, um, the foundation side of the organization is doing all this programmatic work around, right? Sizing AI about making sure AI is built on the side of humanity. How do we think about augmentation instead of automation? So all those kind of thinking and programmatic work trickle back down to the products that we're building now. The real thing is it's harder than just an ethos thing. Most companies don't have the ethos we have, so I think we already have a leg up there. But it comes down to actual product decisions that are just hard, uh, especially in this agentic world. You can imagine a world where we build all these agent systems. Optimus being one example from the Moz AI, uh, team, the workflow builder. What it could do is it could literally ask you for your permission every single time I want to take an action. The problem with that is we're just going to get into cookie banners all over again for a while. People will think about it and eventually just hit approve, approve, approve, approve. And then they'll put a rubber duck that sits on my keyboard and just hits the approve button over and over and over again. I think think we are in this position that like, we want to make this ethos work. And now we are actually transitioning to really actively thinking about the UI ux, the actual human interaction in order to pull that off. Like, do we want to baseline the way Agents work and then try to flag a human down when an anomaly happens. Do we need to figure out what transactional boundaries look like so you can actually roll back things in case a mistake was made? There are huge computational, architectural, UI UX problems that need to be solved. But I like to believe that Mozilla is one of the few places that like are really incentivized to lean in and solve it, whereas everyone else might just go do the willy nilly thing. So I don't know, I mean like I like to believe we're the trustworthy people that we've demonstrated over the past 25 years that we're trying to do things on the side of humans, m using technology to make us better, trying to push for openness in a bunch of systems. We're just going to have to do it again. It's just really freaking hard. But we're going to just have to do it again.

Michael Krigsman: As you said, Mozilla beat Microsoft in the browser wars. Can open source beat big tech in AI? And what does winning look like?

Ravi Krikorian: I think it's going to be a hard just being really honest. Hard to win. On the frontier level stuff, I think on the frontier level stuff I think we get to a world of the amount of compute needed, the amount of data needed, the amount of money needed just becomes a little out of the reach. Like I like to jokingly say to people that Mozilla, uh, only has a billion dollars to deploy in the grand scheme of things. But like a billion dollars is a drop in the bucket when you look at the amount of money that the anthropics or the OpenAI's are raising out there in the world. So I think in the frontier world the battle is going to be very, very hard. But I think in the everyday use case world of, of I want to just do things, I want to do actions in the world, I want to do web queries, I want to synthesize information, I want to add things to my calendar. I want to figure out how to coordinate the summer camps between my sons and my son's best friends. I think that the open source models are already proving that they can do those type of things. So I think that it's possible that we get into a world where uh, the closed big frontier models are doing scientifically for breakthroughs, are being the ones that are the co pilots inside a wet lab are doing the things that are trying to invent new forms of mathematics for us. I think for the everyday use like you know, I want to live in A world that we have 7 billion AI agents, one for each of us or like doing things on my behalf. I think open source can get us there pretty fast and pretty well if we focus on it. So I'm optimistic about those kinds of things.

Michael Krigsman: So you're optimistic that we can eventually live in a world where enterprises don't have agentic lock in 100%.

Ravi Krikorian: I think we could get there today if we really wanted to, but these things take time to play out. But I'm 100% optimistic we can get to that world. We're sort of in this rounded corners moment. Uh, when cell phones first came out they all had sharp corners and they look like clunky boxes. And then Steve Jobs polished the corners and made them really usable for the rest of us and now they're indispensable in our day to day life. I think open source AI is kind of in this rounded corners moment. Like it's just a little hard, too hard to use. That small amount of friction deters most developers from actually tinkering with it. If we can deliver them a rounded corners version of it, then it can be a viable alternative. And I think that will change the curve in a similar way that like Firefox, the vast majority of the web sadly does not use Firefox, but enough of the web does that has caused an entire industry to move. I m think that's the play.

Michael Krigsman: But also I do have to say that it's not just an issue of cost and control and ease of use, but also demonstrating the caliber of uh, result that you can get from the major frontier models. Because if you don't have that then you will be relegated to relatively low value problems.

Ravi Krikorian: I think we do need a bunch of like, what's the HBS case study of how these things are being deployed and making actually useful inside organizations. I agree, we need things like that. Mozilla sometime in July will be releasing what we're calling the State of Open Source AI report. And part of it will have some case studies of just like how this stuff is being used into enterprises and how it's actually making a difference. Pinterest being the canonical example that everyone throws around Q3 of last year, Pinterest deployed open models instead of closed and saved something in the order of $10 million that quarter alone by just making that switch. So I think we can tell those stories and find all other stories of just like if companies like that are willing to do it on those type of workflows and it makes economic sense for them to do it becomes a no brainer. The rest of us should be doing it too. But what's the difference is that Pinterest has a bunch of A plus engineers that can make it happen. The question is, how do we make it so that every SMB, every enterprise, every Fortune 500 and make use of the exact same transformation?

Michael Krigsman: You've spoken a little bit about Mozilla AI. Tell us, uh, we have only a few minutes left. Tell us briefly what you're building there and also what is the gap that Mozilla AI is filling that no one else will?

Ravi Krikorian: Mozilla has undergone a pretty big restructuring itself. And so now we're a portfolio of companies all with slightly different business models, all tackling different parts of the Internet and AI ecosystem. There's the Mozilla Corporation, which works on Firefox that everyone knows and loves. Mozilla AI is now one of these newer companies that we're specifically focusing on, the developer ecosystem. So we have actually a few set of products that are open right now. We are in this mode of rapidly creating ideas and figuring out which ones are actually needed in the space and then doing GTM after that. But Mozilla AI has a few things, Otari being one of them, that's they're locally deployable open router like uh, system where you can then point your code at Otari and it can then figure out the best model to use the right model use. It'll track token counts, track costing for you, give you gorgeous dashboards, and then we can then do so much more after that point. Once we're installed in that way it becomes your model routing layer. We have something that we call CQ that I jokingly call stack overflow for agents. So imagine a world where Rafi is using his agent in an enterprise work case and Michael's using his agent in an enterprise workplace. The question is, how do we get institutional learning? So like in, in the, in the regular enterprise case we have whole departments just figuring out how to like share knowledge between people and how do we bring new training in and stuff like that. CQ is meant to do that for the agentic work workflow. So Rafi's agent, Michael's agent can then be talking to CQ as like the common place to share the tasks that they've been developing, share the way that they've been manipulating the work, the code base so that everyone's agent is rising. It's like a rising tide altogether across the entire enterprise. So it's CQ and then we have a Last one that we call Optimus, which is our open source workflow development system. So we've built really good tools to be able to orchestrate a whole bunch of like it friendly things. So, you know, your slacks, your notions, your emails, et cetera. We have some really good open source tools there that can then be tied into either a CQ or an Atari to make it work really well. So we're really going after the developer niche on Mozilla AI. So that's one of our AI bets. Another one is, you know, on the uh, Thunderbird team, the team that makes the email client, they've just created something that they're calling Thunderbolt, which is basically the open source chat client that can be working again in an enterprise use case. Because in a lot of situations a lot of these companies don't want to just have all their documents indexed and put up into CLAUDE for then claude to serve. So if you use a locally hosted model inside your environment, you can then run Thunderbolt on top of it, which will then start figuring out how to access all the different documents, build rag databases off of all the internal company knowledge and then expose it through a Thunderbolt interface so that you can actually have really contextually aware chats with the entire knowledge system that's going inside of your enterprise. So those are just two examples of ways that we're trying to make a dent into being like locally hosted human centered systems that can really work and still bring these benefits of AI technology up, but under a way that's more under your control.

Michael Krigsman: David Quirk on LinkedIn says Mark Sermon from the Mozilla foundation has put out the idea of creating a rebel alliance, which he says is such a great message. How is the whole Mozilla portfolio ecosystem approaching this? And very quickly please.

Ravi Krikorian: We have a whole Moz VC team, a ventures team that we're doing a bunch of investing across the space, but we're also just building partnerships like we're starting to work with the mistrals of the world. We're starting to work with all the hugging faces of the world who's actually in the VC portfolio and try to figure out ways for us to all work together as we build that single open source AI stack. That rebel alliance I think can then sum up to the greater than the big platform providers.

Michael Krigsman: And with that we are out of time. A huge thank you to Rafi Krikorian. He is the Chief Technology officer of Mozilla. Rafi, thank you so much for being with us. I'm very grateful to you.

Ravi Krikorian: No, thank you, Michael. I hope you have me back one day.

Michael Krigsman: I hope you'll come back. Everybody, thank you for watching you guys, as always. You guys are amazing. The questions you ask before you go subscribe to the CxOTalk newsletter. Go to CxOTalk.com we want you to join us. We have truly incredible shows coming up. I always say that. And you know what? It's really true. We have great guests. Okay, folks, have a great day and we'll see you again next time. Take care. It.