Small Models, Massive Wins: The New Shopify AI Formula

Speaker A: Let me tell you a story. So let's say we want to build a feature and let's say you scope out the feature. It's going to take two or three weeks to build, or we could spend two or three months building the infrastructure so that you could build that feature in one hour. Which one should we choose? Right. And Shopify almost always chose the latter. We'd almost always want to build the infrastructure such that the feature can be built quickly. And the reason for that is once you build that infrastructure, many more things can be built on top of it that you never thought about. Whether that's internal first party developers at Shopify or third party developers in our ecosystem. In the past that was like a trade off, right? Two weeks versus two months. But now with AI, you've got these, you know, lovingly, we call them clankers, right? These AI tools to actually just go and take the idea that you have with your steering around the architecture, how we want to build it, how it should fit into our infrastructure, how we want it to appear to the merchants. All of those things can now be embedded in a way that you can almost build the infrastructure at the same speed as the feature. And so if you are building something and you're like, wow, I really wish I had this API or I really wish I had a way to get this type of information, it is your duty at Shopify to actually build out that infrastructure for yourself, to enable that for your feature. Because many more people after you can build on top of it. The experiment is literally what I mentioned. You would build a prototype. You want to see if something's going to resonate. You might give it out to a, uh, small segment of merchants, you might give it out to a small segment of geographies, you might give it it out to a small segment of buyers because you're really testing something. But we really do pride ourselves on taking the information from the test, deleting the code behind the prototype, and actually reimagining the solution from scratch with all of the learnings that we just got from that test.

Speaker B: So which of the three are uh, you building more now because of AI? Infra.

Speaker A: It's always been infra.

Speaker B: It's always been infra.

Speaker A: Always been infra. I mean, one of the great examples of our infrastructure focus was actually during COVID when we were able to quickly, quickly launch things for many more markets, for many more merchants, because we had the infra in place versus many other places I've worked at where you have people kind of build feature by feature We've always built more infra. We will continue to always build more infra.

Speaker B: Today we're talking with Farhan Fauer, head of engineering at Shopify, who runs AI across one of the world's largest commerce platforms. Farhan is deeply familiar, not just with the tech of AI, but with the challenges of the current token economy. And with 29% of companies cited in the VentureBeat Pulse survey saying that it's actually token costs that are killing projects, not model failure. When do you reach for the most intelligent proprietary model you can get? And when you distill that intelligence into a narrow open source model tuned for one specific job? This is Venture Beats beyond the pilot Enterprise AI in action. I'm Sam Wierdeveen. Today's episode is presented by Outshift by Cisco, Cisco's emerging tech incubation engine and driver of agentic AI quantum, um, next gen, infra and beyond. One of the cool things with Shopify is you have, uh, a leader with a lot of vision. His memo about AI, which I thought was really kind of interesting from an engineering perspective. Our task at Shopify is to make our software unquestionably the best canvas for which to develop the best businesses of the future. We do this by keeping everyone cutting edge and bringing the best tools to bear so our merchants can be more successful. So you're in charge of building a lot of those tools. Right? What is the development that you're seeing from these recent frontier models? You know, what's been unlocked for the Shopify team? What is that being made possible that just wasn't possible a year or two ago?

Speaker A: That tweet. Right. In 2025. Right. I call it the tweet that was heard around the world where we said that AI will, uh, be the baseline expectation of Shopify was really a way to solidify the thinking that you're going to have this tool available to you that you could use to remove toil. And you can use it or not use it, but we're going to just, you know, gauge your impact as if you had access to this tool. We have this line at Shopify, which is AI replaces tasks, not jobs. You can just really get to focus on the best parts of the, your working style and all this toil and drudgery. Maybe you had to like go over to this system and grab information and convert into a format to go to that system. You can use AI to do all that toil and now, uh, you can focus on the more higher leverage parts of your job. We really want all of Our employees to embrace that in a way such that we can also bring that to all of our merchants so that they can focus on the thing we want them to focus on. Right. Like you mentioned us building the best tools for entrepreneurs to like, you know, use Shopify as a canvas for their, for their products. We really think that the more entrepreneurship is better in the world and we just believe that all of these entrepreneurs, they want to build amazing products and talk to their customers and all of the toil around that we can take from them. Having to have a storefront that is up worrying about fraud, payments, inventory, fulfillment, loyalty, discounts, taxes, like all those things we want to take away from them so they can just focus on build the best product and talk to your customers. We really do think that by having these frontier models in the development of our employees to build things at Shopify and inside the product itself will allow those folks not have, they don't have to keep up with the latest developments and ChatGPT and Opus and they can just focus on their products and we can bring them all of those innovations into the product directly so they again, they can just focus on amazing experience for their customers.

Speaker B: From what I understand, Shopify has always had this really good ecosystem of people building plugins, people building apps around it which benefits from you guys building the infra. In this sort of day of, uh, you know, anyone can sort of vibe code up something. Is the danger for you to just sort of build out all those things? Do you consciously have to sort of think, you know, no, we got to pull back a little bit and leave something for the ecosystem or we're focused

Speaker A: on more entrepreneurs in the world. Right. Whether it's us, it's someone else on top of our infra, on top of other infra that we never thought about, that is all net good for Shopify. Right. Like we want more entrepreneurs in the world full stop. Right. Does that mean, um, you know, it has to be on Shopify? No, but we want to make sure the infra is so easy to use and so seamless and frictionless for the merchants on our platform and for buyers to transact that. We just believe that it's, it's going to be much, it's going to improve the ecosystem and have more entrepreneurs in general. And so when we build infra we always try to think about the problems that we want to solve and potentially problems other could solve, uh, on top of the Shopify platform.

Speaker B: Okay, so how does this actually show up with your developers? I know you've been a Huge proponent of pairs programming going back quite a way. Is that still valid now in the days of cord code? What tools are in your stack stack for your developers now?

Speaker A: We, uh, deployed GitHub Copilot in 2021, which is a year before ChatGPT. And that was very much like the pair programming vibe, right? Like, I don't remember like even the announcements around then. They're like, it's your AI pair programmer because you could write code and feel like you're not by yourself anymore. You have a, you'd type in part of a line and it would complete the line, you would write a comment and it would complete the whole function. Like you would literally be pair programming with the AI. And at that time still we felt like having two humans at the keyboard and pairing with the AI was going to increase the learning rate of what we were trying to learn about our customers and of course would increase the velocity of uh, shipping things to our merchants and buyers. And so net net, uh, for like two years we had this like AI pair programming. And then you hit this like December Opus 4.5GPT 52 moment in 2025 where all of a sudden we started seeing that actually the agents were better at writing code than us. And I still feel like having two people at the keyboard is going to increase your learning rate because you're going to learn different things from the LLM that's giving you feedback as you're building and different things from the human beside you, another smart person, while you pair across all of the work that you are doing for, uh, our merchants. And so both those things together, we think like the ideal, like AI Centaur is not just human and LLM. It's like it could be two humans pairing, um, and the LLM helping you kind of remove the toil from your workflow. And today we still have a very strong culture at Shopify. We don't pair like 40 hours a week like in some companies and what I've seen in my past as well. But the teams do regularly get together, get together and pair on certain parts of building out their infra or incidents. And they, they bring those tools to bear to make sure they can move forward on the AI tooling. So we're big fans of um, of using all of the frontier models, right? We actually have an LLM proxy internally and what that allows us to do is we can buy tokens in bulk from all the providers. We can have a proxy that everyone in the company connects through and it gives them access to the latest models. It Gives us access to reporting. It gives us access to, um, failover. Right. So you might be using a model on one provider and there's uh, an availability issue. We can automatically seamlessly transfer you to another provider. Same thing with our, um, in production products. And you can just seamlessly feel like, oh, I'm using Opus. And it's spraying across other providers behind the scenes and you don't have to care that, you know, one of them is having an availability issue. It also allows us internally to really ask like, interesting questions, like, not just on like token spend and token and numbers of tokens, but also, hey, who's using the most expensive tokens, who's spending more times and reasoning what types of models are being used and what disciplines and by levels. And like, we can kind of get a, uh, good understanding of like, what's going on from a token usage perspective, uh, inside the company.

Speaker B: What about the actual. So are, uh, you guys a. Claude. Claude code, Codex?

Speaker A: Ah.

Speaker B: Or is it like just anyone? People can choose whatever they want.

Speaker A: So we're big fans of having people having choice. So once I buy the tokens, I don't care what harness you want to use. We expose everyone to the different harnesses so they can kind of get a feel for what may or may not work in their workflow. So cloud code, codex, PI, cursor, GitHub, copilot, inside VS code, like all these things inside VS Code. People can choose however they want to use it. And then we also have, uh, our own agentic platform like river, which again, switches between models, our products switch between models, and then we even have open source internally. And so there's lots of ways to get your hands on different AI from whatever agentic harness you want to use. And many people, by the way, maybe this counterintuitive use different harnesses for different things. You might pull up cursor for something, you might pull up cloud code for something else. You might be in PI, you might write an agent. Um, all of those things are available to you.

Speaker B: Yeah, that seems definitely to be a trend that we saw just recently with the omnigent that got released where it's a meta harness that uses multiple harnesses and stuff like that.

Speaker A: I would say the more interesting thing is that we're seeing just this growth outside of R and D. Right. Like, my surprise was that having folks in not in R and D start using like a terminal or start using a cursor window or using Claude code. Um, and then, you know, and Codex, using the Codex app. And so I see you know, it's not just R and D, right? It's people in talent, hr, finance, sales, building out tools for themselves. We also just recently went public with our Qwik platform. Right. I'm from like the, the early 90s of the Internet and we had geocities. Remember, you could put up any website anywhere and it's our, it's really, it's very much our own internal geocities, which lets you drag and drop an HTML file or JavaScript and it hosts it for you. Because what would happen is you, Sam, might build a tool and be like, look, everyone, I built this tool for me. And they're like, can I see it? Like, how do I, how do I see it? You're like sending screenshots and, and now you can just literally deploy it and be like, cool, here's sam.quick.shopify.IO and I can show you share it with everybody and I can keep it updated with my current workflow and everybody, um, can see this, uh, tool that I built.

Speaker B: So how do you manage when everyone's trying to ship code? We did some interviews, for example, with LinkedIn, where they talked about the idea of a full stack builder and that what they were finding was that product managers and other people, designers, everyone is now starting to contribute code. And that causes a lot of challenges, you know, for engineering, where you have to start, start thinking about, okay, is that code safe, is that code secure? Or how are you guys doing that

Speaker A: at Shopify, there's different tiers of applications here, right? So when we talk about things that land on Qwik, just n of one software, I'm building something for myself or I'm building some for my team. I'm in sales, like all that stuff. Um, there's no, like, it just goes to Qwik and there's no reviews. Right? It's internal only. It's, we're not worried about it. We have a secure version of Qwik which allows you to say, okay, I only want to have four people read this or be able to access this tool. So I want to launch it there. But when you're talking about production code that's going to our, our m, our merchants, which are our customers and our brands that still the onus and the responsibility still falls on the engineer. So anyone's allowed to prototype. And actually we see this now in the R and D cycle where designer pm, they're prototyping in high fidelity the experience they want to show to our customers. And it's so much better, right than showing a screenshot or, uh, you know, a deck or something on the experience. So they show a working prototype. It might even connect directly into our APIs or might be fake in the back end. But it shows you a higher fidelity view of what we want to ship. Then we take that, um, inspiration and then we try to build it for real. And I mentioned we don't just try to build a feature. We try to figure out what's the right infrastructure. That feature should live on. Does it exist? Great. Use it. Does it not exist? Well, maybe we should not. Maybe we should build the infrastructure first so that we can enable it. And those are still owned by engineers. And we have a saying internally that you can't, like the bot can't be responsible. Right. For the things that go into production. Of course you're going to be working with AI. Of course. We might even tag it with Claude code, wrote this with cloud code. Great. But you're still responsible. The human still has to be responsible of what goes to production. With the volume of AI code, we of course have checks that go through the code before they go to production and we have these things in place. But that doesn't mean anyone can just vibe code something and try to get it through our pipeline. It still has the responsibility layer of, uh, an engineer having to own it.

Speaker B: Everyone's been talking about token maxing. How are you sort of thinking about that now? The bills are going up.

Speaker A: Yeah, I mean, there's two ways to think about it. So in 2025, again, when Toby put that tweet out, right, AI is going to be reflexive. At Shopify, it was all about just closing the distance between you running into a problem that has toil and you reaching for AI. Right. Like the sooner you reach for AI, the faster you can get unblocked, the faster you can, uh, remove the toil from your, your path and then you can actually work on the thing that you want to work on. It wasn't about using the most tokens. It was just about reaching for it as soon as possible. And then of course, yeah, turn into like, oh my God, like people were using tokens and then leaderboards would appear and people wanted to be on the leaderboard. And of course, like, the incentives were like, misaligned. We don't have a leaderboard anymore. We have a usage dashboard. We renamed it, but. But we also now tried to highlight the, the utility behind like, what we want to learn from it versus, like who's in the top 10. Right. We also moved in 2026 away from AI reflexivity to AI leverage. Meaning, like, it's not just about now, now that everybody's reaching for AI all the time, we don't have that problem anymore. The idea is to get people to really think about where they can get leverage from AI. Right. I think I call it like, how can I brag about being so lazy that I had this tool build the. Build out what I need to do, need, um, to build out or remove this toil from my workflow. We also do have like circuit breakers in place. So we don't have limits on our tokens. We have circuit breakers in place which basically allow someone to get a message if something they're doing is long running and spending a lot of tokens. And they will get pinged and said, hey, you are you just spent X dollars. Did you mean to spend this? And sometimes the person is like, oh, absolutely. Press the button, unblock, keep going. Other times they're like, whoa, I didn't know that doing this task, it was running in the background. I totally forgot about it. It's gone. It's been running for 10 hours. Actually. I don't think it's valuable. I'd rather stop it now. And, uh, then we can kind of catch things that are runaway at that point. But we don't have that. We don't have people wanting to be in the top 10 or anything like that. In terms of token spend, this series

Speaker B: is brought to you by Outshift, Cisco's incubation engine. By creating an open interoperable infrastructure and Outshift is enabling agents and humans to share intent, context and reasoning. The cognitive evolution for agents is here. Explore the Internet of cognition@outshift.com. is it engineers that are not aware that they're spending a lot or is it other departments? You talked about your proxy system. I want to come back to that. I imagine that gives you very high uptime for everything, which is pretty rare in LLMs at the moment, uh, but also gives you really interesting insights into how the tokens are being used. And it sounds like that is sort of like a big plus versus token maxing.

Speaker A: Any person in the company can see their spend in AI. Like, that's easy for employees to see. And I think it's good to know, hey, I spend, you know, $10 a day or I spend $100 a day. Like, I think it's worth people understanding that idea. Um, and it is true, you're right. Like engineers spend the bulk of the tokens because of course they're, you Know the code writing and architecture and reading of code and understanding of code and all of the things around the actual building of the product are where a lot of the token spend happens. And you're right, the proxy does give us a lot of insight, but the insight is it's a learning insight, right? So if someone spends a lot of tokens, my reaction isn't to go to them and say, like, oh my God, you spend so many tokens. Like, what are you doing? Like, yeah, it's like, hey, you're doing. It seems like you're doing something interesting. Like, what are you actually trying to do here? I want to understand what you are. How are. You are using AI as a lever to improve the quality of the output here. And I find amazing things, right? Sometimes I find things, I'm like, oh, I didn't. I wouldn't have even tried that. And people are seeing the ability to learn from others by seeing that. But I would say the bigger unlock on the Cross team collaboration around AI has really been river, right? River, the, you know, the product I mentioned earlier, which is our Agentix substrate across the company, river can like see Slack messages. It can, it can query into our data warehouse, it can see our product management system. And more interestingly, river only works in public, which means you can't go into a private channel and say, hey, river, like, help me build, you know, or, uh, ask. Ask this question. You have to go into a public channel and ask it. And what's happened is everybody has a public channel. I have a Furhan river channel. And in the channel I ask all these questions and people can see. And what's cool is I might ask a question and it might not. River might not be able to figure it out. Someone might jump into the channel and say, hey, river, how come you didn't check like this table in the data warehouse? That's cool. Clearly where this data lives, river goes, oh, you're absolutely right. It finds that table. But now any future query to river, river learns and will never have that mistake again. It will never not find that table. And in future people will be like, oh, I just asked a question to river and it came back with the answer because somebody jumped into my thread, uh, to answer it. So we are seeing token. To go back to your original question, the growth of tokens being spent by river is going up. The number of PR is being submitted to our code base from river is going up because people are starting to see that as a way to kind of get their work done because it has a Substrate of information across the whole company.

Speaker B: And my guess is that you're all sort of, you're benefiting from everyone else being human in the loop for it, uh, you know, not just yourself kind of thing.

Speaker A: By the way, I think it was an unintended consequence. Right. Like putting river in public was, was intended. But the fact that other people are jumping in to um, help river understand a problem a different way and then it changing the now way river answers that question in the future I think was an unintended benefit. Uh, uh, fascinating.

Speaker B: One of the things that we've been seeing at VentureBeat is we've been doing a lot of different sort of research when people are looking at the different sort of failures of projects. People talk about both, uh, the model, but also the sort of infrastructure around it. You know, how do they balance this open versus closed things? Perhaps just very quickly explain to people what distillation is. Sure.

Speaker A: Uh, yeah. So distillation is this idea that you can have two models like a teacher model and a student model. And you can train a student model usually on a narrower task. Right. It doesn't have to be. Some people I think try to distill into a generalized model, but for us usually a narrow task and what happens is the student model ends up being very specialized for the narrow task and you get a bunch of benefits from that. One, you typically, it's a smaller model so it tends to be faster. Two, it's a smaller model so it tends to be cheaper. And then three, this is the, I would say the non intuitive part. It can actually get better at the narrow task than the teacher model. Right. And so, you know, let's get like a dumb example. Let's say you wanted to like have a model that find like your favorite movie, whatever. You might want to train a smaller model just on that one subtask of finding really good movies. And you can make it again smaller so it's faster, smaller, so it's cheaper. But also maybe because it's just focused on that it does better than the general model on finding those amazing, uh, movies. Now the downside is, is one, you got to train it. So if you change your, if I said, oh, I want not just movies, but I also want favorite songs, you kind of got to retrain it because it's not really designed for that. And then two, you lose generalization. So that frontier general model, you can ask it everything from like the weather and other things. This smaller teacher model because it's been fine tuned, it only really is good at that um, that narrow task. And so at Shopify we've done this a few, in a few places whereby we have, you know, obviously we use the frontier models in lots of places, but we have these narrow tasks, right? So for example, inside a Sidekick. Sidekick is our flagship, um, AI product inside of our administration. It does lots of subtasks for, uh, merchants so that it helps them remove toil. And some of these subtasks don't need to call the frontier model. We were able to distill some of these things to be smaller, um, models. So again they're faster, better for the user, they're cheaper, and then again, uh, not counterintuitively for us anymore, better at that subtask than the generalized model. And we see savings, uh, in size from like 2x, which is huge, um, down sometimes down to like 30x, like 30 times cheaper, faster and more and of higher quality. Um, and then of course we still use the frontier models for a lot of the steering.

Speaker B: How does someone go, hey, we're doing this task. Maybe we should distill this task. Like this is a very specific narrow task. Maybe it's one feature on something. How do you actually have a pipeline that goes ahead and does it?

Speaker A: So instead of just building this one time and like, oh, I'm going to take this, you know, part of Sidekick and distill it by hand. We are early on decided to build infrastructure so that anyone could build a distilled version of their subtask. And so we, we call it UDP Universal Distillation Platform. And what it allows us to do is you give it the teacher model, you give it data, you give it the evals and you give it the target model. You're like, okay, I have Opus 4. 8 and I'm going to, I want to distill this subtask down just for this narrow thing to um, when, right. Quinn 35, which is a, which is an open source model and the pipeline will run through the data and the evals, takes I think about a day or so to come back and then comes back and says, hey, here is the output, the evals of what we were able to find and what we were able to fine tune the weights of this open source model. And then it gives you an evaluation. It'll say, okay, here's what we got. And it can be, you know, faster, hopefully cheaper, hopefully, um, and better at the subtask sometimes, not always. And then we can then decide, we'll say, actually this is a good trade off. We want to deploy this into production but because we built it as infra, there's no like, what's the right word? There's no like approval process or anything. There's no like, there's. It's literally there. We have, we have other pieces of infra. We have a, an internal platform called Tangle, which I think we talked about. It's open source now that allows you to um, visualize the actual running of this uh, of this universal, uh, distillation pipeline as well and makes it easy for anyone to take any sort of task that they're running now with any sort of uh, you know, model in the cloud to distill um, it down to something they might want to run locally in our own, on our own gps.

Speaker B: So I actually saw some of your people give a talk at ICLR last year about exactly this. How often is that happening? Is that happening every day in Shopify? Is it just engineers that are doing that or is there now, is now this making its way across to marketing, saying, hey, I need a model that does this super well?

Speaker A: My dream, and I've talked to the team about this already, is to actually not give the distillation pipeline a target model at all. Like what I'd love to do is give it the data, the evals and the teacher model and say based on, you know, your learnings over time, I want you to look at a different class of model, different sizes, different types, and for you to tell me what the right distillation target is. Because maybe we'll get surprised, maybe it'll be a such a small model like run on like a phone based on your problem. And other times maybe it comes back and says there isn't a way to distill this down to anything better than what we have at the frontier. Like I'd love it to be, to get. And we're moving that direction so it gets even smarter. You can kind of give it less data and it, it can learn from that. It's a good question on the non R and D versus R and D. So far we've only seen really R and D folks using the distillation pipeline. It's a great idea though to think about it as like uh, a marketing or other types of tools that potentially you could say, hey, for this use case, we do all the time in this other discipline, what if we trained a specific model to help us with that task versus having to use the frontier, especially if we can get the accuracy to be higher on that task. So I will write that down as a feature, but we don't we don't have that today.

Speaker B: One of the big questions we always get when we talk about things like this is how do our people know what to do for evals? So you know, everyone just says evals, evals, evals. But what, what does that actually mean? And how does that actually sort of look in Shopify?

Speaker A: Yeah, so actually on the it again depends per product. But evals are actually a really important part of the process because if you don't know, if you don't have a repeatable way to actually judge how a new model might affect the output of what you're working on, then you're in this world of like everything's a one off, right? Like you build it and then you're not able to plug and play a different model, a different size, different reasoning levels. And the evals are the way that we do that typically starts with us having like a golden data set of uh, data that we can use per problem. We also um, work to generate new data. Right. So we actually work with a firm called Toloka that helps us like generate new data around this stuff as well. Because what happens is you might be doing an evaluation for something that isn't necessarily well served by a frontier model. So you know, let me give you like a shopping example. Let's say you wanted to build something that's a good at recommending, like based on uh, you know, T shirts that you buy, like other types of clothes that you might want to buy that might not be automatically well served by a frontier model because it doesn't have that much training data. But we could fine tune a model specifically good at that. And you have to have evals, that meaning a golden data set of like, hey, when somebody looks for this T shirt, we tend to also look for those types of T shirts. And some of it can come from our data and some of it can come from training data. Then what happens is we might use that evals either in a distillation or another fine tuning, uh, approach to figure out, hey, we've now come up with a way and evals to feel like this. If it can satisfy the evals to this level, we think that it's worth actually shipping to production. Right? Using that, using that kind of pipeline, it allows us to then figure out, hey, this is past this threshold, right? 70% we have different evaluation. I'm just going to use like percentages. 70% of the recommendations are good. That could be something that we want to deploy. The more interesting thing is let's say a new frontier model comes out or a new open source model or new data. Having the pipeline that goes through this eval framework allows us to quickly iterate through this set to come back and say, hey, we found the new QEN model is actually better. Oh, we found the new, um, this new data set is better. We've got new training data. We've now updated our evals because if you don't have that, you're kind of doing things one off. But it gives us a very good way to judge the LLMs and even just classical machine learning on the things that are going to production and whether they're worth, uh, they're improved or not.

Speaker B: Do you foresee like a time where you'll be doing that for your customers? Like for entrepreneurs with actual specific shops where they're like, hey, I want a model for my shop that Rick does, perhaps like a rec system or something like that where uh, it's very specific to their particular kind of shop and their particular kind of use case.

Speaker A: Yeah, so we have some, we have something called Sim Gym that we launched, which is like simulation gym. And what it does is it actually helps merchants who. So in the, in the world of AB testing, if you have lots of traffic, it's easy to AB test, right? Because you just be like, okay, 1% of the traffic is going to see this thing. 99 sees that thing. I have control. We can, we can figure those things out. If you're a small merchant, you don't have this traffic. So doing an AB test when you only have like 10 orders a week or 100 orders a week is really hard because you don't have enough traffic. So Sim Gym uses all of the amazing data we have around buyer behavior for your store, actually simulate a change that you make in your, in your, on, uh, your store. So you might say, I'm going to change the size of this, uh, you know, add to cart button. And I want to make sure that it's something uh, that is going to increase conversion. Or I think that I want to test this, this different theme because I think like having a summer theme is going to do work better than having a fall theme. And so now Sim Gym will simulate buyers going through your store and give you a readout as if it was an AB test. So you could say, wow, this did perform better, or oh, no, it didn't perform better, I don't want to launch it. And that's literally what we're doing inside of Shopify today because it lets you kind of do that. The other thing we're doing is we have something called Pulse which helps you really think through from an async perspective. Just look at your store and it'll say things like, hey, did you notice that you're making these changes and it's slowly been increasing the latency of your site? I'm not sure if you notice, but we know that latency is uh, a conversion killer and you should always be as fast as possible. And so we give you these async insights into your store in the background based on all the amazing data we have about buyer behavior and themes and conversion and performance, etc. And so we are doing a lot of these things and they are personalized directly for the store.

Speaker B: What are the trade offs of the distillation? Like, is this something that's costing you more in, you know, now you need big racks of GPUs for doing INF. You, you know, you're really taking on the whole inference thing yourself by doing this. What are the challenges that, that caused?

Speaker A: The one challenge is, you know, just calling the frontier model for everything is like easy, right? Just call the API, of course you have the evals and everything else, but you don't do anything, you just like do it there. The other downside is anytime you make a big change, like I mentioned, I want to find, you know, great movies. Now I want to find great songs. If you make a change, you do have to retrain. And these models, you brought up a good point, which is on infra you do, I mean we have a uh, GPU cloud with our own open source models running. We use the GPUs to train and run these distillations. And then also we have the GPUs to serve the inference for the resulting models. And so yeah, you kind of have to build out a little bit of uh, the infrastructure to then train and serve these models as well. Which by the way, some companies don't want to have. Maybe they just have inference in the cloud, they don't have GPUs directly or from anywhere. And they're like thinking, do I want to worry about this? Um, in which case, uh, they may not do it at all. So but for us at our scale, um, we do have access to that type of inference.

Speaker B: What sort of split do you see at the moment between in production use of the Frontier LLMs versus distilled models? I imagine for development and stuff like that internally it's very much the frontier stuff. But for in production stuff where you're doing stuff at scale. Can you talk a Little bit about the split. Even if you can't say exact figures.

Speaker A: I don't even know if I know exact figures because it's not something I crack that way. But you're right. So dev tokens for sure at this point cannot be distilled. It'll be interesting to see as more and more, you know, open source or local dev models, um, you know, appear to see if they can compete on the coding models against like the frontier models. Because of course all of us would love to see, you know, the cost of that come down. But also like having it run locally means you could run it like on your laptop when you're on a plane, like all those kinds of things. As we find narrow use cases, we do then try to distill them because again, it wasn't just about cost and latency, which are big, it was about accuracy. In some cases we're able to get the accuracy to be higher than the generalized model. So for us it's like again, win, win, win. You kind of got to run the process. My guess is we have, I'm going to guess like maybe like half to a dozen of these distillations running, um, like in production and we're just going to see more and more over time as uh, as we do a better job there.

Speaker B: Ah, are you guys keeping the traces from your, you know, your developers with the aim of eventually being able to train your own coding model, your own, you know, like, like what we've seen, you know, cursor do. And I imagine you've got the scale to actually be able to do it. That's the kind of interesting thing I

Speaker A: never thought it about, uh, in uh, a way to like train our own development model. Because I'm pretty sure every, everything that we're doing is like, it's different in a way. I mean if you take it all the way to its limit. Right. If that were true, then in theory I could run everything in like a local model, right? Because I could just train my own coding model. But I think coding is such a wide open space and even just an innocuous one letter change of code somewhere could have like rippling effects across the whole infrastructure. Which is why we don't even allow our engineers to use smaller models for this. So I think the answer would be, um, that I don't think that that's possible at this stage. It may be again, maybe in the future we, we come up with techniques to have like frontier style models that are even just local, which allows me to then maybe take My internal development data, that would be interesting. But instead I'm more thinking about how do I get them better. Most intelligence to my engineers such that they can be on the forefront of building as much as possible on the roadmap as fast as possible versus trying to be like, well, let me constrain them down to this like very cheap and fast model, but, uh, at the expense of like frontier intelligence.

Speaker B: Shopify is, is one of the leaders with the, the UCP protocol. I think you guys were, were very key in developing it, uh, along with Google, if that's right. Yes.

Speaker A: Yeah. Um, yeah.

Speaker B: Where do you see things going with agents? Like if we project out a year from now, am I literally just telling my agent, hey, I want a new set of hiking boots. I want them, um, you know, for this kind of terrain and I want them to be waterproof and these are the brands I like. Go make it happen. And then it's got my credit card and it goes off. You know. Where do you see the balance of this?

Speaker A: It's so funny because in 2025 when people would say 2026 is the year of agents, I would always like eye roll because I never understood like what they meant, like, what does an age. And it wasn't until I installed OpenClaw in January that I'm like, holy shit. This is like a whole new, like uh, a whole new interaction model. There's two ways to think about agents. Yeah, there's like the Open Claw style, like personal agent, and then there's obviously like, like agentic commerce, which is like, where are buyers going? Right? So you mentioned like one interaction model. As a buyer, maybe you're going to message your agent, wherever that lives in one of the LLMs or open claw. I need new hiking boots. Maybe you have to tell it by the way the brands you like. Or maybe it already knows because it knows you. Right? My Open Claw knows me pretty well. And so it's possible that the, the claw has my, has my um, my interests already embedded and says, uh, hey, I know you like this brand and I know your size already and I know you're in, in my case in Toronto. So I'm going to find things that are in that local, um, area versus maybe you go to chat CBT or Claude or Gemini and you ask it, hey, I'm looking for like new boots. And by the way, those things have memory too, right? So they might be like, hey, by the way, I know you live in Toronto. Um, let me tell you. And so what we, what we see here is that, uh, this, this thread is running through the whole conversation is like, we want to build the infrastructure to enable anything to happen, right? Meaning the reason we developed UCP was we are trying to get everything set up such that no matter which direction the buyer goes, the infrastructure is there for all of the amazing products to show up on the surface, right? So if you're in cloud and you ask, hey, I want to buy a gift for my grade eight daughter because she's graduating, uh, you know, from grade eight, you want amazing products to appear there, right? From all over the Internet, including of course, the amazing Shopify catalog. And so we built the catalog at Shopify to enable anyone, including all the LLM providers, to have access to all of the amazing things that our, our, our entrepreneurs are building. So that's one stage which is like, it just shows up there, right? And the old model was, oh, cool. Here's like, you know, uh, you mentioned like, hiking boots. Maybe your daughter's into hiking. You press the button and it just takes you to that website, right? That was one interaction model. And then with ucp, what we wanted to do is bring that closer to the experience inside the lm. And what UCP allows you to do is not only can, you know, do your transaction right inside of that surface, but bring the branding and the full checkout experience to that surface such that it looks and feels like the brand you're interacting with. So it's not just a generic buy it now button, because guess what? People care about sizes and variants and colors and taxes and duties and loyalty and like all the things that make that brand experience their brand. And you want that to appear all the way through inside of that surface because not only does it get increased conversion, because it does, but it's like it feels like you're, you're transacting with that brand. And then of course, if you're like me, because sometimes I still want to be like, well, what else you got in the store? You can click on the store link and have the full experience there as well. Uh, because we're building the infrastructure, we're enabling all sorts of commerce to happen. And it's up to see where. It's up to us to, to watch where the buyers are going to move to if they're going to go to your world, which is just tell my agent, I need boots, buy the same ones as last time or find me something cool. And by the way, we see that people giving money to their agent and saying, up to this amount, go spend it. Um, or if they're going to do it inside of an LLM surface like a chatgpt or cloud or a Gemini or some other one. Or they're going to just do like, you know what I said, which is they're going to research and then they're going to go into the store or they're going to click on the link and go to the website. Like we do see the, the early signals around like a gentic commerce like it is. The numbers are small, but the growth rate is huge.

Speaker B: What other changes, I guess, do you think that both developers, entrepreneurs, people involved in this area should be thinking about now that, you know, agentic commerce is growing at such a large rate? It does seem in some ways that perhaps search is kind of being replaced. SEO is being replaced.

Speaker A: Yeah, I think a couple of things, like one, you know, we always think about building infrastructure because we don't like to guess where we think people are going. We want to enable all of the different variations. Right. We also think that like commerce is multichannel. Right? You might see something on Instagram, you might walk by the store, you might, your agent might recommend something, your friend might tell you something. Like, there's so many ways that you learn about, um, commerce. And so we want to make sure all of those are available to you. So I tell developers, hey, like, building infra at this stage is probably the right thing because you don't know where things are going to go. That's one. And then two, we want to make sure that your products appear in those surfaces. What's amazing about Shopify is uh, that because we've built this catalog, once you're on Shopify and once you know the signals, um, are in the positive direction, your products automatically will appear in those places. Because, you know, we work with all those providers to make sure they do. And that allows you to have like, again, like SEO, whatever it turns out to be, geo, whatever they're calling it really allows folks to be agnostic. Right. If I'm an entrepreneur, I just want to build amazing products. I don't want to think in my mind, oh, wait a sec, I need to advertise on TikTok, Amazon, maybe eBay. What about, uh, Instagram? Oh, wait a sec. What about chat tbt? How do I get it in there? Like, you don't want to think about that. The whole reason Shopify exists is so you don't. You just have to be like, I'm going to build great products. And you can just press buttons. There's literally a button inside. Shopify is like agentic commerce, like on, off, and then it just shows up and then you don't have to worry about it.

Speaker B: Do you see agents as becoming a key thing? As a coworker and stuff like that? It really sounds like you're, you're going down that path already with river, right? That sounds to me like it's basically perhaps not a personal agent, but a, ah, uh, hive mind agent that helps everyone in the organization.

Speaker A: River is an agent in that you can talk to it, but it doesn't talk to you. Meaning it doesn't proactively message you. Like the thing that I would say the difference between like the, the agentic definition when I think about an agent and of course I just mentioned like agent E commerce that's also not yet messaging you. When you look at uh, the world and you see like what the, like the really wealthy people have and that's coming to the consumer is how you know, like where innovation happens. So like wealthy people have like a buyer, right, for their clothes and they message them like, hey, remember you bought that jacket last week. I found like perfect pair of jeans. Like they're messaging you, right? So right now we're not at the point where, you know, your chatbots messaging you, although they have like the Daily Pulse and things like that. My open clot does message me, by the way. The thing that you know, got my like uh, uh, the hair on my arms going up was when, when I was traveling and my agent messaged me and said, how's your trip? And I was like, what the hell? Like my call messaged me and I was like really freaked out because well one, how did you know I was on a trip? And of course it had my calendar. So then it figured it out but, but just the fact that it was just checking in on me, I was like, okay, that's agentic. So when we get to the point where like a river or you uh, know whether a clot or something is messaging you and giving you like, hey, by the way, I know you like, you know, Vessi, um shoes and I don't know if you know, but Vessi just released like a brand new color for this shoe and in your size is available and ships to you. And then you're like, whoa, that's cool. And maybe in the future you go, hey, next time it does that, like just buy it. And then you'd be at home and all of a sudden like Vessi shoes show up. Like that's kind of. Then I think we're in this future world where we start trusting the recommendations, the payment rails, the budgeting. And it does change everything about the surface area. And so, again, that's why we're building the infrastructure to enable it all. It's not too long. I think in, like, we're talking like two to three years. We will have people giving budgets to their agents. We will have agents buying things on their own. Things would just show up. I'm pretty close to it myself. Like, I'm pretty close to just giving my, like, an agent a budget and be like, hey, 200 bucks a month. Things that you think I need, like, you've seen me buy, right? Like, I mean, I had a question of my, of my. Of my agent yesterday. I asked it something and it said, because you have a Tesla. And I was like, Jesus. It. Like, it really? Because I have a Tesla, it knows. And it said something about that, like, and use that context to try to weave it together. And, uh, say, oh, because you have a Tesla, you should get this thing. And I was like, Jesus. Oh, you know what it was? It was a pool skimmer. And it said, because, you know, because you have a Tesla, you like technology. This pool skimmer is like a Tesla, like, pool skimmer. And I was like, okay. It kind of sold me on the, on the product, and I did buy it.

Speaker B: You are the founder of a very sort of famous or infamous WhatsApp group of, uh, CTOs and engineers in Silicon Valley of SAS companies. Currently, you know, Fable has been disabled for pretty much everyone, even though the intention was for, you know, just people outside the US or people that were non US citizens. I'm curious, in that. That WhatsApp group, what people were actually saying, I think you guys were already one of the project glasswing partners. What happens when that sort of intelligence is taken away if you're sort of getting used to developing with that kind of thing?

Speaker A: Yeah, so a couple of things, right? One, like, the reason we have the LLM proxy is. Allows us to kind of go across the frontier models and open source models, and it allows us to kind of have access to the plethora of tools when a model comes and then it goes. Or it could be as in Oculus, there's like an update, right? Like, hey, like, uh, OpenAI is not, not going to like, let you use GPT 5.2 anymore, right? Like any of those things really. Uh, the proxy allows us to kind of spray across the different providers. So, yes, we, you know, we saw Fable come out. Fable looks amazing. You know, we use it, of course. Um, and then it went away. And you know, the, of course the workflows of people kind of change because they're like, oh my God, I don't have access to anymore. But they go back to opus, they go back to GPT 55 like we go back to work. And so I think what I would tell companies is that you probably want to have um, some sort of solution that allows you to go across models so you're not super tied to anyone. Only because who knows what's going to happen in terms of like regulatory uh, things or just being able to have like uh, a little bit of um, agnosticism or abstraction away from that so you can focus. The other thing is like I mentioned, like we are, you know, we continue to like research and spend time in the open source models because we want to see what's happening in that ecosystem as well. There are already people who are, who are focused on understanding the frontier of open source. Way before, uh, they were worried about like a regulatory or a company turning off access for some people or anything like that. They were always like, hey, I believe in open source. I'm a very big fan that these are fast followers. I'm a fan of like what they're offering, their price point, like how to host it, like all those things. And so those folks continue to, you know, to see that being a um, a thing that they can follow for a long time. And um, they don't even, sometimes they don't even have to like host it themselves. Right? They use it. So like open router or something else. There's like, I can use open router, I can pick a closed source model, I can pick a open source model. It's their LLM proxy, right? They don't have to think about having to host it themselves. And I don't think that has changed with what happened this week. Like I get, I mean I agree with you, it's unprecedented and like you know, a new thing. But I don't think it's changed the full, I don't think it necessarily converted anyone's philosophy. What I think it did do though is it did force people to be like, hey, do I only have one provider in my stack? And I think if for those people who are like I only use one provider, they're probably like, wait a sec, what if something changes in a way such that this provider becomes inaccessible to me for whatever reason. I think every, you're right on that front. Everyone's probably thinking, what is, what is my backup plan? And by the way, we saw this on our own in our own way two or three years ago. Just on the availability side, we could have like a Claude go down or an open ad go down. I'm like, holy shit, I'm building products. They go to my customers that rely on these systems. How do I, um, make sure that I can be available? And that's when the idea of having multiple providers in our proxy allowed us to spray across multiple back ends so that now we don't have to think about it. So now I think a lot of companies will, will be thinking about being multi provider now.

Speaker B: Fascinating. I uh, think on that note for Hun, thank you very much. It's been a very interesting conversation. We've covered a bunch of uh, things and it's fascinating to to hear what you're doing at Shopify. Thanks for joining us.

Speaker A: Thanks for having me.

Speaker B: This series is brought to you by Outshift, Cisco's incubation engine. By creating an open interoperable infrastructure, Outshift is enabling agents and humans to share intent, context and reasoning. The cognitive evolution for agents is here. Explore the Internet of cognition@outshift.com For for more stories about the AI revolution like and subscribe to the podcast and check out venturebeat.com to sign up for our newsletters.