Inside Instacart's AI-Powered Smart Shopping Cart | NVIDIA AI Podcast Ep. 302

Speaker A: Our view is that in five to 10 years, customers shouldn't have to think about shopping in store or online. There will be one single unified mode powered by this continuously learning AI system that incorporates, uh, what customers are doing in store, online states of the shelf to build a fully personalized experience.

Speaker B: Welcome to the Nvidia AI podcast. Our guest today is David Macintosh. David is the Chief Connected Stores Officer at Instacart, and we're here to talk about the present and future of grocery shopping and AI in retail. David, welcome to the podcast. Thanks so much for joining us.

Speaker A: Thank you for having me.

Speaker B: Um, so maybe we can start, uh, with the basics. You can tell us a little bit about yourself, uh, your role at Instacart and kind of your journey that brought you here.

Speaker A: Yeah, happy to. So, fundamentally, I'm a technology entrepreneur. Uh, my prior company, co founder, CEO of Tenor, which is a expression search company, an animated GIF search company. Um, if you're an animated GIF person, you probably use the product that, um, is embedded in all the major messengers, keyboard companies, and so forth. When Google bought the company, we had a couple hundred million users, several hundred million queries per day. And then over three years at Google, we grew it to a billion users, over a billion, uh, queries a day. And what attracted me to Instacart was that I saw a company that is the leader in delivery online, but had an even broader market opportunity to really digitize the grocery industry, to bring technology to, to all of our grocery partners. And so in my first year at Instacart, I led what's called our enterprise business, the Instacart platform, and launched that. And you probably know instacart as an app on your phone or marketplace. But, uh, what's less well known about the company is we have a very significant enterprise business. So, for example, if you go to sprouts.com in the US that entire experience, website, fulfillment, uh, ads, all powered by Instacart end to end. And so as a result, I would talk to retailers very frequently. And what I heard was retailers saying, hey, instacart, you brought me online, you brought my business online, uh, website, uh, E comm, loyalty, et cetera. But I have all these problems in store. Uh, I'm dealing with how do I get more of my customers to sign up for loyalty in store, how should I think about retail media in the store? How do I create a more personalized experience? And then on the other side, we had a lot of customers, a lot of users saying, instacart I love the convenience and delivery of the online experience, but I also like going to the store. I mean, I'm an Omnichannel customer. Um, how can you take what I love about the online experience, the convenience, the personalization, and bring it into the store? And so the birth of Connected Store really sat at the intersection of those two insights. And so really our vision for the store is to digitize it end to end so that it becomes unified. Our view is that in five to 10 years, customers shouldn't have to think about shopping in store or online. It'll be one single unified mode powered by this continuously learning AI system that incorporates what customers are doing in store online States of the Shelf to build a fully personalized experience.

Speaker B: So you teased at this a little bit, but maybe we can kind of double click down what does the term connected store mean at Instacart? And kind of along with that, maybe you can talk a little bit about how AI is being used. You know, as you said, Instacart, not just the grocery shopping app, but kind of across the enterprise services that you offer.

Speaker A: Think about Connected Store as really digitizing the offline store today. So much of a, ah, grocery store today is run very similar to the way it was running decades ago. If you think about the checkout experience, it hasn't fundamentally changed or evolved, um, all that much, but technology has fundamentally shifted in that period of time. Um, and so it's everything from our AI powered smart cards, caper cards, which we'll talk about in a minute, um, to digitizing the deli, putting a screen behind the deli so that it's both easier for associates at the store to prepare an order, but also easier for customers to order. Um, it's connecting to electronic shelf labels so that it's easier for e comm shoppers, people picking orders to light up the shelf tag to more easily find what's on shelf. Um, it's about giving users an in store mode so they can plan their trip to the store, create a shopping list and then sync that with technologies, uh, in the store it's inventory technologies that build an understanding of the shelf to prevent out of stock so that consumers, when they come to the store, could get exactly what they want. So you can think about it as really digitizing all of the components of a store and connecting them together. So for example, you can order from the deli, from the caper cart. Capercart connects to our offering called Food Store, which digitizes the deli. Um, a shopper going into the store can activate those shelf tags. But you could also imagine the caper cart activating the shelf tag to help you find something more easily in the store. The shopping list that you build online, you can then sync with the caper cart in the store. And it reminds you in fact what to get in the store so you don't miss anything and realize it when you're all the way home. So all these technologies are connected together. And at the very center of that ecosystem is Kaper, which is a $350 million acquisition I led a number of years ago. And the caper cart you can think about as a set of sensors in a cart connected with an Nvidia Jetson board in every single cart. So the cart has a weights and measures certified scale. You need weights and measures certification in the US to do, ah, produce, weighing. It's got camera sensors, multiple camera sensors that not only look at the basket, but also face the shelf so you can understand what's on the shelf, which I'll talk about more. Um, you've got location sensors on the cart so you can understand where the cart is in the store. It's both a slam approach also leverages visuals of what's on the shelf. And then it's a sensor fusion system. So you can imagine, um, in a grocery environment, one of the big problems is that often WI fi is spotty. You know, it drops in and out. And often, by the way, there isn't good cell, cell reception as you're going through, you know, these, these.

Speaker B: Yeah. A lot of the stores, once you get in. Yeah. At least I found the cell drops out.

Speaker A: It's, it's, it's a big problem. And, and when you think about it, customers want, uh, experience that responds immediately.

Speaker B: Sure. Right.

Speaker A: So if you think about the way people use the cart, they're putting items in the cart and then it creates this running total. Right. So what people love about the cart is the fact that they can keep track of their, their, their spend. Right. They don't have to go to the checkout line and then put things back because they miscalculated how much they're going to spend.

Speaker B: I'm laughing because I'm thinking, as you're describing this, I'm thinking of one of your, one, uh, of Instacart's videos or ads that I saw where, you know, it's mom and daughter shopping and mom puts something in and it rings up and the daughter puts something in and mom grabs it and takes it out. And you know, it shows how the card automatically deducts it from your total, which is amazing. And then the daughter puts it back in, and Mom's like, yeah, okay, you deserve it. But, yeah. So maybe we can kind of walk through what happens kind of behind the scenes when I'm shopping. And I want to set this up because I think this part just sounds so cool. These little simple things, right? But they really elevate the experience. And as a consumer, when you get a lot of little things together, you're like, oh, this is a great experience. But the idea that I can be shopping, I can have a bunch of stuff in my caper cart, it's keeping track of everything. And I can put something in a piece of produce that needs to be weighed. And so if I already have stuff in the cart and I grab an apple and I put it in the cart, what happens?

Speaker A: Yeah, great question. So the entire basket is a scale. Uh, and so when you put an item into the cart, um, it can understand the weight because it already knows the cumulative weight of everything in the cart. Um, but as you're alluding to, it's not that simple because you can imagine that the grocery environment is very complex, right? And so wifi, as we talked about, is one of the issues. But look, in these stores, there's a variety of lighting conditions. There's tens of thousands of SKUs. The SKUs, by the way, change store to store. So even if it's the same banner, right. Even if it's the same retail, uh, store, it's going to be very different catalogs, very different items. Um, the items change by seasonal differences and so forth. There's very subtle difference, um, in sizes of these items. And then the way that people shop is different, right? So going back to the question on the scale, some people are leaning on the side of the cart, right? Different arms are going in and out. It could be going over bumps in the store. And so it's a very complicated problem. And the expectation from a consumer standpoint is that the responsiveness has to be in hundreds of milliseconds, right? And so if you think about a lot of the AI systems that exist in the cloud, the response time might be seconds, right? And so key to our approach is doing a lot of these calculations around what's in the basket and recommendations at the edge.

Speaker B: Right.

Speaker A: Um, and so the way it works is that we have several sensors coming together. We have camera sensors, we have the weight sensor, we have the location sensor, and then we have that sensor fusion system. So we have an edge encoder that processes a Lot of those signals at the edge. We also, for longer sessions, longer analysis, we will look at those sessions in the cloud with a separate decoder and then we'll put the two together in what we call an overall shopping decoder, that shopping experience decoder that builds the best understanding of the customer's basket. And you could imagine that in this environment there's thousands and thousands of edge cases that can emerge in terms of the way that people are shopping, uh, with this product. And so we've really found that it's important to have multiple signals all coming together. In fact we published a blog post on this, announced it at GTC and actually walked through this in a gtc, uh, a talk. Um, if you look at just camera alone, there's all kinds of ways that the camera can easily be tricked. And I don't use the word trick to imply necessarily an intent on the user. It's just in the natural way that people are shopping. Pulling things out different speeds, a camera getting blocked, the cart getting full, all those things. We found you absolutely need those multiple sensors coming together. The weight of the cart, that basket is almost sort of like an X ray that builds that ground truth understanding of what's actually happening with this thing with all of the camera uh, inputs, uh, informing it. So that's the basket understanding side. There's also then an understanding of the shelf. So going back to what do customers love about the product? You mentioned you like going to grocery stores. One of my favorite things to do is I'll go to grocery uh, stores where Caper's live. We're now live, 100 cities, tripled year over year, um, in the US globally, primarily US but we're also live internationally as well, um, with Kohl's in Australia. And uh, Morrison's recently announced that they're bringing Caper to the uk so we have an international presence but primarily we launched in the U.S. um, and I'll go to these stores and um, this one in particular is uh, Wakefern, ah, Wakefern, um, is ah, a co op on the east coast. They have Shoprite Fairway. We're live in about 20% of all of uh, all of Wakefront stores and growing quickly there. And so I actually will bring typically uh, my wife with me if she's in town, um, because I'll go up to customers and just ask them about how they use the experience. And I'll tell you, I get a lot better responses when I'm shopping with someone versus if this many people sort of look at me like, what are you doing? But of course they won't know I'm with Instacart, but I'll ask them, um, why are you using this product? How did you hear about it? Um, what's the value you're getting out of it? When I talk to people, I hear number one, it's the running total. Number one is I can keep track of what I'm spending. Um, second, I hear the deals, the discounts, the recommendations on the cart. Three, I hear convenience. You can bag as you go. You can actually take the cart all the way to the car. The carts can get rained on, they can get snowed on. Um, they're highly durable. In fact, they often will sit outside charging, so customers can just pull them. They're highly modular, integrate into existing store operations.

Speaker B: And as a customer, your running total is just checked out when you leave the store or hit the button or whatever happens.

Speaker A: That's absolutely right. You can check out either directly on the cart. Some of our retailers, um, have payment, uh, terminals directly on the cart. So you can just tap your credit card or maybe you have a wallet, Apple Pay and so forth. You can be directly on the cart and then in other cases you can check out by transferring it, um, to a existing payment terminal. So for example, if you want to pay with cash, um, or you want to pay with a, uh, variety of alternative payment methods, sometimes that can be easier to do that transfer. And that speaks to the modularity of our approach, which is that, uh, some retailers would prefer to funnel users through existing systems and exits. They have. Some want a brand new lane. But going back to the point around, um, the, you know how it works, right? The end to end environment recommendations. So how do recommendations work? Well, what we found is that most retailers don't have an accurate planogram of the store. What is a planogram? Planogram is a layout of where all the things are at. And keep in mind, even within a given retailer banner, the layout varies often store to store.

Speaker B: Sure.

Speaker A: Right. And the problem is that you might know exactly where the cart is two feet in a particular aisle. But if the planogram says that there's cereal next to you, but in fact the cereal is two aisles over, uh, the recommendation is going to be wrong. And not only is it going to be wrong in that moment, but then users will start to look at the screen less and less over time.

Speaker B: Right.

Speaker A: And so the problem with that is that then the utility of the product gets slowly eroded because the recommendations just

Speaker B: start out trust roads, utility. Yeah.

Speaker A: And so the way we've solved that problem, um, is with the combination of AI and Nvidia jets. So we're actually using the side facing cameras 1 as a signal to inform the location system. Because SLAM is good. SLAM locations, uh, systems are good but in an environment where the aisles are close together there might be some ambiguity on are they aisle two or are they aisle three? And obviously that makes a huge difference on the recommendation. So one signal with the sensor fusion system is okay, where is the cart in the store? The second is what's actually on the shelf. The side facing cameras are building an understanding of what's actually on the shelf so that it can inform the recommendation system. Then the recommendation system connects with cloud systems. So over the last decade we have 1.6 billion plus lifetime grocery orders on the online side. So we've been able to make the recommendation algorithms online very, very, very, very good.

Speaker B: Yeah, I can only imagine.

Speaker A: So we're taking a lot of that same technology, that same approach and we're now bringing it in store and the results are extremely exciting. So many um, of our retailers have, have, have shared that they're seeing double digit sales lifts from, from KPRO from the carts. Right. People are spending double digit percentage more. And then we've even been able to more recently add on top of that with recommendation features that further bring online and in store signals together. So for example, um, you're about to check out, you're heading towards the checkout line, there's a screen that pops up, it says did you forget? And it surfaces the yogurt that you normally buy, but you just forgot to buy this trip. That feature alone, that one feature alone drove a 1% absolute uh, increase.

Speaker B: Wow.

Speaker A: In sales lift in store statsic nearly 1%.

Speaker B: And that's just to kind of make sure that's a personalized recommendation. Right. That's not a like, oh, you're in aisle four where the chips are and chips are on sale on this store today. This is a like Noah, I have to change it to be honest. Noah, you eat a lot of ice cream. So before you leave, did you get the ice cream? And that's my.

Speaker A: Yeah, that's exactly right. Right. And so I think we're just scratching the surface of the new types of experiences. The win win wins we can create by understanding location, by having the customer engage with the screen, by having the recommendations be relevant to them. It's good for retailers, it's good for CPGs, it's good for users because a lot of users complain. Hey I got all the way home. I forgot the one thing I came to the store for. It's really, really annoying. Um, another example is when you've, not

Speaker B: to interrupt, but when you've been ticking off some of these features and talking about your grocery list and having it there and everything. I'm like, you're peering into my shopping brain and my like four different apps with fragmented grocery lists and I forget stuff and everything. And you know, even just the idea of I'm in the store and I don't know where the ketchup is in this particular, you know, being able to

Speaker A: access from the cart, like that's absolutely right.

Speaker B: Yeah.

Speaker A: Um, I mean it's, I think all these little pain points that people have increasingly, because so many people are shopping online, right. They're saying, look, I sort of expect a lot of the things that I do online to be in store. Right. You know, another example of this is we shipped a new recommendation algorithm or we saw, um, another 1% plus absolute improvement in sales lift in the store. Right. That's a whole step on top of all everything we had done before. Right. And what that was doing was better incorporating a lot of the online signals from online delivery into the store. Right. Now when you take a step back and you say, go back to your question, how did that all work? This system would not work without the sensor fusion physical AI, uh, system running on the Jetson board at the edge. It would not work with the understanding of the shelf, the side facing cameras and that routing through the Nvidia Jetson board. And both the edge AI system and cloud AI systems we have. Right. Um, and then that all that would not work with our ability to take signals about what a user is doing online. The history we have in depth in building recommendation algorithms online for the last decade into the store. And so the magic of the product really sits at all of those three things coming together to deliver that value to users. And I think in general we're just scratching the surface of the experiences we can deliver. Like we've got gamification capabilities, for example, where uh, customers can get additional deals and discounts. And you know, imagine for a CPG or retailer, they might say, look, this is a customer who shops, you know, once every two weeks if I give them $2 off their next order if they shop within a week, probably worth it for them, right. And so those are the types of win win wins. You can construct with an understanding of the user, a screen in front of them that they're highly engaged with the understanding of the basket, the AI system that we're deploying, uh, you know, at the edge, really scratching the surface of the new things we can deliver.

Speaker B: Um, you've talked in depth about the benefits for the shopper as well as the retailer. How are the store employees taking to the new systems? And are there benefits of the cart and all of the data flywheel stuff you have going on as you described? How does that trickle down to improve the employee experience?

Speaker A: Yeah, absolutely, A couple different ways. One thing I will say is that we found adoption of the product absolutely relies on employee support. Employees have to be enthusiastic and supportive to help really drive adoption in the store. And so there's some things that sound simple but are incredibly critical, like stackable charging. So if you think about it, the carts stack into each other exactly like traditional shopping carts, so they can fit modularly into the store.

Speaker B: And when you say stack, I'm thinking I push the cart. It goes across the whole parking lot perfectly lines up in the next cart.

Speaker A: Exactly.

Speaker B: That's what you're talking about. Okay.

Speaker A: And so you don't need to plug in each card to charge.

Speaker B: They charge.

Speaker A: And so that's huge because otherwise the staff would be burdened. They say, this work for me. I have to plug in each card individually. Right. So that element, uh, is extremely key. Um, we now have the ability to deploy the carts outside, so you can have a stack charger of, uh, stacked set of carts outside in the cold winter weather. And so stores don't have to do anything differently in terms of their operations. The modularity element is incredibly critical to success. And just stepping back, when I talk about the complexity of the physical AI deployment of Caper, it's these types of things I'm talking about. Okay, let's say you deploy the carts outside. The WI FI signal's probably pretty weak outside. What do you do about poor WI fi at the start of a session when somebody's trying to log in with their phone number to get the loyalty rewards? It's like all those types of things, really. Getting the system to work at the frontier is very, very challenging.

Speaker B: I keep, when you're talk talking about these challenges, I keep going back to the mental image of being at the self check and somehow I put something on when I wasn't supposed to and it says remove the item and I remove it, but I didn't do it right. Or something happens and you're in that loop. So with the complexities of everything you've described, moving around the store and everything.

Speaker A: That's right. You Know what you described too? That loop that you get into in traditional self checkout, you've got typically an associate that's standing a couple feet away from you that can come over and help. With a smart cart, you're in aisle 10, there's nobody around you to help. And so it makes it even more important to get the basket accuracy systems right, to understand what customers are adding, what they're removing from the cart and you know, to your, to maybe a final point on the trust of associates. This is why basket accuracy is so important because the weight system acts as sort of an X ray, so to speak, for the cart context that works in tandem, um, with the visual signals coming in. Again, the sensor fusion system. That's really important because ultimately associates want to make sure that as customers are leaving the store that the total is accurate. Um, and so building that depth of technology along with retailer facing tools where they can keep an eye on the carts as they go throughout the store and they know the system is highly accurate and performant. That's all part of how you make these deployments really successful at scale, even with the chaos of a store environment. Uh, another thing I'll share with you, um, that is pretty interesting is we announced a technology called Store View where Instacart, uh, shoppers that are shopping in order for another customer can actually scan the shelves of a store with their phone to build an understanding of what's on the shelf. Um, Caper, also with the side facing cameras as I described, can build an understanding of what's on the shelf. And so that understanding could then feed notifications to the store to employees about things that are running out of stock. So then they don't have to be reactive, they don't have to wait for a customer to tell them or they don't have to go and do these lengthy checks of the store. It comes to them, uh, proactively. And so that's another example of how this understanding of the store of the shelf can really improve the customer experience. Prove it for users, improve it for retailers, improve it for associates. And so that's really the ecosystem that we're building. Whenever we build features, we've got to think about the users, the retailers, the CPGs, the store associates, how does it all come together to deliver a win for each component of the ecosystem? Sure.

Speaker B: Um, David, with all the data you're gathering and everything you're able to do with it, as you said, from Instacart's depth of years of serving billion plus orders to building store views, and this kind of thing, one of the great things about AI is that, you know, the more good data you feed in, the more it learns, the better it gets. How do you like, do you do system updates for the stores? How do you deliver this, you know, these continual improvements without, you know, the store having to close down for, for an evening for sort of inventory type things?

Speaker A: That's a great question. And by the way, I think that's one of the benefits of digitizing the store. As you digitize the store, you make it measurable.

Speaker B: Yeah.

Speaker A: Which means you can start to optimize it like software. Right. And again, going back to an observation I made at the end, beginning. One of the things that was astounding to me when I first joined Instacart was that most retailers don't really have a good sense of what's on the shelf. Sure, right. And you know, I naively asked, well, okay, don't, don't they have a point of sale? Don't they have an inventory system? You know, you can, you can track all the sales when people leave. So what's the problem? Right? Not so fast. The problem you have is people are pulling things off the shelf and putting them in their basket. So the item might have been available a minute ago, but then it isn't available the next minute. Um, you have ah, what's called DSD vendors, the CPGs often will have their own employees. And you see that big truck next to a store, that employee will be willing things on the shelf totally independently from the employees in the store. So it's an inherently chaotic environment.

Speaker B: They're always the ones I ask where something is, you know, and they're always super nice about it. It's like, I don't actually work here.

Speaker A: Yeah, it's, it's exactly that. Right. And so um, what we're doing is we're building the best understanding of the store. Right. You think about this caper cart with the slam location system, the side facing cameras. We actually had a slide in our GTC presentation showing the 3D map of the store that we're constructing. Right. So think about all the things you can build on top of that. You know, one, one, um, one area we're going is we've got a suite called AI Solutions. So what we're doing is we are bringing AI to our retailer partners, things like assistance on their website. And so for example, both Kroger and Sprouts recently announced that they're going to launch um, Instacart powered Card Assistant on their, on their Websites. Right. But also, um, analytics tools. So for example, you could imagine a world in which we understand that something is missing from the shelf and an AI agent in the background kicks off and starts to communicate with the merchant and say, hey, this is the fifth time this week this thing has been out at 3pm but the delivery comes at 4pm how could we either make the delivery come earlier or maybe we need to adjust the question quantity. Right. Um, and so you can imagine that once you make the store measurable. Right. Once it's observable, you can start to optimize it and even take the caper carts as an example. Um, it's a software system. It's sort of like getting an update to an app or your phone. Um, we ship regular updates to our retailer partners. So the software is getting better all the time. And then this is probably a little bit, you know, in the weeds. But from a, from a, from a, from a technical perspective, some of these models, um, about the store might refresh as frequently as 15 minutes.

Speaker B: Okay. Right.

Speaker A: M. And so they're really designed to get a, as close as we can up to the minute view of what's happening. Because again, going back to the example of out of stock, the um, item might have been on the shelf 15 minutes ago. But then when somebody comes to the shelf either to grab it and put it in their own cart or to pick the order for a customer online, it may not, it may not be there. And so that, that real time understanding of the store that we're building from the more than half a million instacart shoppers that go into a store every day and from these caper carts that go through the store and are continuously scanning the shelf with the side facing cameras. That's unlocking a foundation for us to build new agentic experiences. On top of that, start to automate the optimization of the store to then ultimately deliver a better experience to users. It makes the associate's job easier, uh, and ultimately drives, um, value for the retailer.

Speaker B: Yeah, uh, you said the a word, agentic. Um, what's your approach to using agentic AI? Right now you mentioned the one agent kicking off and doing something in the background, but, um, is instacart. Are you deploying agentic systems as part of what you're doing in stores? Are you looking at agents for sort of different expertise kind of tasks and lines of thinking? What's your philosophy on using agents?

Speaker A: Yeah, I think our vision is, I think consistent uh, with Nvidia's vision, which is we believe that there's going to be experts for different tasks. Right. And so the way that we're approaching it is that we're building the foundation model for grocery. And so that foundation model is taking in a couple different sources. It's taking in the 1.6 billion lifetime plus online grocery delivery orders, uh, the 2 billion item catalog that we have, and then it's taking all the in store data. It's taking this understanding of not just the shelves but also the clickstream. Where is a user pausing in the store? Going back to your example, a kid taking something out of the cart. What convinces you to take something out of your cart? What convinces you to, to put that thing in? Where are you in the store when you're adding something that you normally don't add to the store?

Speaker B: Right.

Speaker A: And so it's that triangulation of where you are, what you're doing, building the best understanding of what is the most personalized experience for you in store and online that we're then feeding into this grocery foundational model that's building the best understanding of users and the store, which you then can stack on top of uh, agentic applications, whether it's a cart assistant that lets you plan your trip to the store and then take that plan and sync it to the cart when you're in the store or shop it within store mode from an app. Right. Um, or it's building associate facing tools like the example I described, or even CPG facing tools. So another example, um, there, if you, if you look at a lot of the uh, shoprite, uh, wakefront stores we're in, you will find Brett from a particular vendor be in let's say half a dozen places throughout the store that bread company wants to understand. Where in the store is my bread being bought?

Speaker B: Sure.

Speaker A: Where are people actually taking it off the shelf? And so today what we can do is give that CPG an understanding of stack ranked lists. Here's the top six places people buy that bread in the store. Right. And so then you can imagine the next step in that process is to use all the location data, the heat map, the 3D representation of the store, and, and expert agents on top to say, you know what, uh, uh, uh, bread cpg, you should put a seventh location and here's where you should put it in the store. You should put an eighth location by the way, and it'll weigh that against the cost of a person not putting those locations in another store because they're burdened by those additional two locations. Or it might say, look in this Store, you only need four. Take away those two locations and put that in another store. And so again, those are all the types of things that can start to be unlocked as you build that really rich understanding of not only the experience in store, but then online and really marry the two data sources together. That's the continuously learning system.

Speaker B: What's something that's maybe an insight, um, that's been uncovered through, I mean through all of this work, but using AI with grocers, with retailers, an insight that surfaced that maybe surprised you?

Speaker A: Yeah. Well, I'd say the first thing that was surprising to me is often the consumer value props that resonate the strongest seem very simple, but are very difficult to execute against. So take the running total as an example. It's amazing that the running total is often the number one thing that users cite when they use the cart. But to actually deliver that, as I stepped through before, how do you build an understanding of the basket and make sure you know what's happening? You remove things, add things, the cart hits bumps throughout the store. Just delivering that is very, very challenging technical problem. But that, that output itself was very, very simple. Um, is extremely, um, extremely impactful for users. I think. The second thing that's been interesting to see is how people, how their behavior changes depending on these different contexts. So what we've seen, for example with assistance in the cloud, people are planning a trip to the store using it differently than they would a traditional online grocery experience. So online, this is probably your experience with Instacart, right? You're searching, adding, searching, adding. Here's a thing I bought before, right? You basically know what you want, you're building a list, you're checking out versus an assistant online user might say, hey, got a family of five. I have this X dollar grocery budget. You know, my oldest child has this allergy, right. Second, uh, one doesn't like fish. Uh, the third one only eats fish. Um, can you make me a meal plan, uh, with that budget for this family? And can you do it for the next two weeks? Those are the types of behaviors and queries that you start to unlock with these new agentic workflows. And then by the way, the behavior gets even richer because you then can take what you've done online and bring it in store. And so that cart assistant I mentioned that, um, Kroger and Sprouts announced they're rolling out that will exist both online in the e commerce website that we're powering for those retailers and it will exist in cabrer, right in the smart car. Which will then start to remind you as you go through the store and actually start to shape your behavior.

Speaker B: Yeah, right.

Speaker A: And what you do in store will then start to improve the online experience, um, and vice versa. And so in general we look for places where we can really push the envelope on user behavior and add new value to users with these new capabilities. Um, it's the same. Another example would be take uh, um, the bag as you go. Use case with caper. Often people will put their bags in the cart, reusable bags, and then because they can take the cart directly to their car, they'll unload directly in the car. Those people might have shopped with plastic bags before, store provided bags before. But because of this brand new experience where they don't have to actually take anything out at checkout and put it on the conveyor belt or take it out and rescan it because it goes into their bag once, they then start to shop with, with uh, with reusable bags. So there's a lot of things like that what we've seen where because you use AI to make the experience more convenient, more personal for customers, it has second, third order effects in the way that they behave.

Speaker B: Yeah, no, that's really interesting. Um, the example of make me a meal plan just makes me think back to like one of the first use cases that I remember seeing kind of anecdotally, I guess when ChatGPT first broke and everybody got on Geni was, you know, here's a picture of my fridge, what can I make for dinner? Right. And so that kind of assistance, you know, really resonates. I mean you're the expert, not me, but it resonates with me of that whole, like, oh, you know, we had chicken twice this week already, like what do I do? Kind of thing. And that, that can be so helpful in the moment, for sure.

Speaker A: Yeah. You know, and I think I've been talking more about the user facing benefits as well. One of the things that we launched at GTC or announced at GTC was that migrating workflows from CPUs to GPUs with respect to ads had really big benefits. We reduced latency significantly. We actually increased click through rate in the experiments that we ran.

Speaker B: Oh, no kidding.

Speaker A: Right. And so there's, there's, you know, everything here again we've been talking about is very visible to consumers, brand new use cases. But when we think about the impact that AI is having at the edge and at the cloud, it's really across the board. Right. It's, we're pushing the frontier with physical AI, with these carts in the store, with shoppers scanning the shelf, you can understand what's happening at the edge. But also by bringing these systems online, there's also impact that you can drive to recommendation systems, ad systems and so forth. Right. And we're really seeing those two pieces come together. That's what we mean when we say, look, we're collecting millions of sensor inputs daily now. That was one of the things we announced at GTC and then combining with um, our online set of data.

Speaker B: Um, so David, what's next?

Speaker A: Yeah, when I think about it, it's really the vision I laid out for you at the beginning of the call. It's on a ten year horizon. We think that customers won't have to choose between shopping in store or online. There won't be uh, a solid line between the physical store and the online store. It's going to be one single unified mode. What you do online will plug into in store and vice versa, uh, just continuously. And it's going to be a continuously improving loop behind the whole system that ultimately makes the shopping experience more personalized, more seamless.

Speaker B: So what does it say about me that as you're describing this, I want to like slow down as I drive by the store, hit the button that opens my hatchback, the groceries to get pulled in. Like, you know you're talking about that kind of. There's no line between online and offline. Right.

Speaker A: I'm like, yeah, yeah. I mean look, if you want to talk even longer horizon, I do think, um, you know, if you think about the data that we're collecting and the 3D maps for example of the store, the SLAM system, you could easily imagine that that starts to, in the now, I'm talking distant future, that could start to unlock robotics workflows. Right? Because again, you know, location of things in the store, where they're at. You even know by the way, how heavy is each item? Why? Because the card is weight, every single item that you put in the cart. Um, you've got multiple cameras looking at the item as it goes in. So you can imagine starting to create reconstructions of that item over time, potentially in 3D. You can imagine with that understanding it could even start to unlock those types of use cases. But that by the way really goes back to the vision of connected store, which is how do you start to unify the experience? For some people it's going to be delivery, it's going to be pickup. For some people, sometimes they're going to want to go into the store, they're going to grab a cartoon. They might even grab a portion of their order via pickup. It might be a hybrid experience where, you know, as you approach the store you've already ordered, you say, look, I know I want to get, um, you know, this type of cereal, the second type of cereal, but I want to pick my own produce. So you can even imagine maybe that picked order is actually waiting for you in the cart. Cart has your name on the front. You grab it from the, from the line and then shop the produce yourself. Um, and so that, that, that totally is consistent with, with the future we imagine, which is again, that we're really breaking down that barrier between in store and online. We're really turning into one single unified, personalized mode for customers.

Speaker B: It's exciting stuff.

Speaker A: It's incredible.

Speaker B: Um, David, for listeners who want to learn more, um, about Instacart, about the caper carts, about everything you've talked about in the future, um, best place to go online, Instacart website, is there technical, uh, blogs, something specific about the carts, social media accounts to follow? Where would you direct them?

Speaker A: Yeah, we've got a lot of this information on the institutecart.com website, um, both pages, uh, around our enterprise technology. Also our blog where we posted the announcement with uh, Nvidia, uh, for gtc. And so for an audience that's looking for more of the technical details, we've got in that blog post, uh, a pretty lengthy description of what we're doing. And by the way, given the reaction we saw at GTC from the audience, we're planning to do a lot more of that actually in partnership with Nvidia, because there is a broad appetite from the audience and really going deeper. Um, but that, that blog post already has a fair amount of detail on really how we're pushing at the frontier of physical AI. Um, so that's, that's one resource. Um, and then I would say for a more consumer heavy audience, we've got videos that you can search on YouTube. Both, uh, videos we've shot of the cart, but also there's been a number of, um, TV broadcasters that have covered caper and done their own segments.

Speaker B: Yeah, excellent. David, this has been great. Thank you so much for joining the pod. And um, you know, as I said, I've always been a fan of the grocery store. I know where it is. I just like going in. So, uh, I'm looking forward to getting my hands on a caper cart myself.

Speaker A: Awesome, thanks for having us. Appreciate it.