Inside Target’s approach to enterprise AI deployment with Sowmya Podila

Speaker A: Hey, demos actually need one good run and you kind of think everything is working but when you want to take it into production, actually would judge the quality of production by one FIU across even a million points.

Speaker B: Welcome to another episode of uh, the E Commerce Toolbox, AI in Retail. Joining us today we have a very special guest in Samia Badali and she is the senior data scientist of Generative AI over at Target. So welcome Samia.

Speaker A: Yeah, I'm glad to be here Kailyn. I have seen a lot of interesting conversations on E Commerce Toolbox podcast.

Speaker B: Yeah, exactly. We just rebranded it actually we're now just a full blown AI in retail podcast. But we always like to start off soumya, uh, by asking a bit about your career journey. You've had a really cool career, you've worked at places like Garner, AWS before Target. So maybe take us through your career journey and how you ended up in your current role.

Speaker A: I have started my career actually at Tata uh, Consultancy Services TCS back in India and I am an engineer at that time. Moved to us to do my master's in technology management. That's where I got introduced to data science as a sweet spot between, you know, tech and business. And since then I have been in love with data science and started actually working in machine learning. And very interestingly I worked in HR analytics and did machine learning and HR at Gartner, which was a very unique niche space at the time. And later I wanted to diversify my skillset in terms of both the tech and also working in different domains and that brought me to aws. Working on the cloud stack and working across a lot of cloud services. Giving the depth insight into the technology stack and working with various of AWS clients across domains helped me gain understanding into how each domain could be different from each other and how we can still apply tech across domains. So that was a very strong experience and I was working in natural language processing and machine learning and then when all those kind of generative AI technologies became mainstream that become a natural extension to my work on those lines. Target came as a new opportunity for me where they were building a centralized advanced AI team where you gathered a bunch of experts and you are doing enterprise wide AI strategies and embedding into various product and business teams inside Target, helping them enable with gen AI upskilling and use cases and doing like uh, a test run grill of POC Style and MVP M and then taking use cases that kind of live up all the expectations and move into production. So that was the approach we were doing at Target and that kind of planned couple of years ago on how they want to tackle this solutioning in this AI era really helped, really impressed me. And that's why I'm here at Target doing it from the last two years. So from computer science undergrad AI, ML M now onto Gen AI has been the speed of my career. Uh, arc.

Speaker B: Yeah, that's really cool. And you were doing AI before AI was really cool. Now everybody wants to do AI. Tell us a bit about what you're seeing actually work in the enterprise. So we're seeing a lot of hype, obviously in the space and demos and AI, they all kind of look the same. You click a button and then all these keyboards start to stroke themselves. Code's being written, changes are being made, everyone's making money. How are you seeing that play out in the field and enterprise? And how is that different from what you've seen in maybe some smaller retailers?

Speaker A: Definitely no. Um, that's kind of my perception of how a lot of AI hype is generated as well in the industry as a whole. I kind of had a line saying that, hey, demos actually need one good run and you kind of think everything is working, but when you want to take it into production, you actually would judge the quality of production by one for you across even a million points. So, you know, that's how it is. The difference is when you want to take a demo into production. So that's where a lot of use cases are getting stalled. So if you have seen a lot of reports that are released outside, they talk about 70 to 80% of the use cases getting stalled in POC phase or even rolled out and then rolled back in because they are not meeting the expectations at a production level or not meeting the governance standards or security standards as well. So yeah, that's been the reality for sure. And now there has been a lot of emphasis or focus on understanding this as an engineering infrastructure or engineering practice that would work at an enterprise scale. So what are the evaluations that you would need to place in poster demo to ensure that this would work consistently at scale and in production? So what is the infrastructure scaling needed for it to run performantly with low latency when you put it in front of a big set of users? And what is the observability and guardrails needed to ensure that we monitor the solution in production and catch before the agent goes rope with any new kind of failure modes? And also what are the governance pieces needed when something, when something is not being right? Who is a responsible party or owner for IT and other kinds of responsible AI concerns, ethical concerns, how do we ensure we comply with the federal state regulations as well. So all these aspects are now thought through. So one of the other aspects is coming up with the agent taxonomy defining what is AI? What is an AI agent, what is an AI system, what is a super agent, what is a sub agency? All these kinds of taxonomies ah, are also getting now defined at an enterprise level. So all of these enterprise level engineering perspective things that need for an AI demo to be successful in production is what now enterprises are focusing on and companies focusing on that are seeing more success and stability and maybe seeing roi. At the end of the day our efficiency gains at least partially. So that's what these practices are, uh, what are making the use cases successful?

Speaker B: Yeah, it makes sense and like just diving into the categories. So in at least what we're seeing, we're building a lot of E commerce agents and they're being used across SMB over to enterprise. But to your point, enterprise is a lot more governance. So in some of our smaller clients like a traditional Shopify store, they might not even have a development team, maybe they're using someone offshore. It's not uncommon that they run our agents. The agents comes up with a PR and GitHub, um, they click the button, it releases it, that's it. Like they're just making changes. And then obviously as you climb up the ladder from mid market into enterprise, we're seeing people start to have CIDC pipelines, a staging environment. Some of these smaller guys are just launching it to the wild, making changes. And as they go up to enterprise, more and more humans get brought into the loop. And it's not uncommon that some of our end to end workflows in enterprise, it goes to a staging website, it has multiple human interventions. So I guess my question for you is like in which categories are you seeing in enterprise the most adoption? Is it for example regression testing, is it like bug fixing, is it compliance? Like where are you seeing, is it content generation, is it customer support? Or are you seeing the most use cases out of curiosity in the enterprise specifically?

Speaker A: So I'm kind of seeing more success with well scoped projects such as domain agents. For example, you take one department such as inventory management and for one specific use case, even inside that inventory management such as you want to understand the items that you never want to run out of stock. So how do we ensure that those items are always in stock and if they went out of stock how can we do root cause analysis with AI Uh, agents, for example, that is one specific use case. Another use case could be like within fashion designing or creative designing. How can we ensure efficient understanding of trends or efficient understanding of generating an ad campaign through AI generated imagery? So very well scoped, specific use case in any of the domains I kind of see success rather than trying to do it all with one agent or do too many things at once. So those are the kind of domain agent kind of use cases is where I am seeing a lot of success. And inside the domain, like I said to expand on the scope, redundant use cases where you already have established processes and what good looks like or golden data sets. You know, hey, humans have done this for five years. This is what they have done across these five years. We already have the data set, this is the input, this is the output, this is what good looks like. That is all defined. Now if we already have that established process, how can we now add an AI layer on top of it to automate some of this? And now we have all the golden data sets to run it through this AI pipeline to ensure that the human output versus AI output, how synergetic are these, how well aligned are these, right? So wherever you have these well scoped existing data sets or processes or methodology to evaluate what good looks like, those are the use cases where we see a lot of success. So in terms of smaller retailers, where are other organizations that do not have that big of a bandwidth to kind of build at an enterprise scale? What I would suggest is pick low risk things. For example, you do not want to put an AI assistant in front of your customers that would damage your reputation or uh, trust or uh, you do not want to commit quotes on your production grade database that might bring your whole shopping website down that you might never be able to recover. You do not want to play that big of a risk, right? But if you're generating an ad campaign or if you're clicking instead of clicking pictures of your products, you're designing with the AI. So it might not be the M product with most conversion, but you are not running into bigger risks for yourself, right? So I would suggest choosing low risk, well scoped problem for those kind of organizations as well.

Speaker C: If you're listening to the E Commerce toolbox, you're entitled to a podcast exclusive website audit. Go to Noibu.com podcast audit for a free scan that uncovers the hidden friction blocking your conversions and shows you where you're leaking revenue.

Speaker B: So I was reading ahead of time that you had actually built an SRE agent, so maybe Talk to me a bit about that and how that drove productivity lift.

Speaker A: So I think every other company, especially who has a bigger, uh, public facing infrastructure, would have the site reliability engineering team. Right? So there would be a lot of failures in staging or production that these engineers need to address. And there is a lot of material for it kind of that helps them trickle down when they see an error or failure on where it could be arising from. So there's a lot of work, uh, that goes into root cause analysis within the SLA to figure out where the production failure is coming from and to be able to fix it. A lot of times there are, for organizations that are big, there are a lot of products and one core team is supporting a lot of products at the same time where they would not have all the needed insights or guides in their head. So they have to open a guide to understand or a playbook that they walk through by each step to figure out the root cause analysis. First they go to a dashboard, try to see where the failures has happened and drill down on one failure point and then keep drilling down till they figure out the answer. Right. And a lot of people actually work around the clock. There is an on site team and offshore team and a lot of handoffs happening as well. So that's where we kind of thought, you know, why don't we take all these guides and playbooks that exist across the countries for a lot of products together and put them behind an AI assistant and integrate them with where they're already working. So they do this root cause analysis as a team together, probably on Slack. And when the clock tickles down, they kind of hand it out to the offshore team and sign it off. So we built an AI assistant or a bot and integrated into these platforms. So whenever there's a failure that's been triaged and the bot acts as the first point of response, guiding the SRE engineers to kind of help them track down the production failure or the root cause faster. So it takes into input all these playbooks and then helps them, hey, I already saw some of these dashboards. I kind of see that this might be the issue. These are the next few steps. Who do you want to start with these or. I already did two of them, but two of them need human intervention mentioned. Would you like to perform these two steps? So it's kind of that kind of assistant for the engineers to be able to effectively chase down production failures and also hand off across country teams. Basically, AI will generate the summary of what an engineer has done and hand it off. You know, to the next team person. So that's the kind of intent behind these kind of initiatives that's honestly really powerful.

Speaker B: And I think you're right in saying like a lot of enterprise has just too much risk with their reputation to effectively just have like some customer facing agent go to town. Where do you think agents are going to get the least amount of adoption in the next year in the enterprise?

Speaker A: That's a very good question. So I would say that areas where a lot of subject matter, expertise, human decision are, is needed is one area. Where we have tried it, it's not living up to the performance expectations is one area. And the other area is yeah, AA is able to do it, but for a job that costs $10 to do it manually, the AI is spending $15. Actually there is not actually a, uh, revenue gained from actually implementing AI. So initially when we were all in experimentation phase, even the vendors were pricing these models so low to kind of boost the adoption. Now that there is a good amount of adoption, the prices are uh, the token costs, uh, or the tokenomics is also not adding up, the prices are going up and the solutions are kind of built around using excessive context, making too many LLM calls to arrive at an answer and the costs of that are increasing. So wherever there are use cases where the cost is not adding up, the ROI components might not be the ones that would take off for sure.

Speaker B: No, I'm fully aligned. And I think honestly where it's getting the most traction is in mundane tasks that are easily repeatable. That you would generally hire a junior team to do, like you said, first line bug tree support or investigating a latency SP or something to that tune. Talk to me a bit about how you think it's going to be important for brands like Target or other brands to open up their catalog to third party AI agents to scrape. It's really interesting because I think like I will never buy a luxury handbag for my wife through AI, but like I broke a light bulb in my house and it is a very strangely shaped light bulb and I took a picture of it and it was like I need to buy one, like figure out how I can get one in two days. And it was like do do do

Speaker A: do do do do.

Speaker B: So I feel like there's going to be some use cases for sure for AI like agent pick shopping. So maybe talk to me a bit about how important it is for the catalog and is there some security risks in that?

Speaker A: Sure, definitely. I think that's how I see it too. Even if we do not buy these kind of fancy purchases. If you want to discover what is a fancier purchase or what would make a good handbag or what color options, what is trending out there, especially when you do not have knowledge for the discover, still go to these platforms to get a sense, right? So for context, Target also has developed with Google in partnership with Google on the Universe Commerce Protocol ucp. That has been the backbone of how these kind of retailers are integrating with these apps. And it has been one of the. Target has been one of the first companies to raise and to integrate with ChatGPT. Microsoft Copilot along with Google as well developed this protocol and has pioneered this. I think it is coming from a sense of being discoverable as they're calling it. Instead of search engine optimization, now they're calling it Agent engine optimization. So Google was a search where you click, you might not complete the purchase itself, you might come to app and search more. But again you want to be discoverable on Google, right? Similarly, now instead of people are not spending as much time purely on Google as before and browsing through catalogs, but asking these AI agents to kind of figure out what they should be buying. For an example, I am a new mom and walking into motherhood, I had no clue what to buy for my baby. Would I complete the purchase on ChatGPT? No. But hey, what kind of bottles are safe? How do I ensure which plastic versus glass feeding bottles for the baby. I had all these questions and all of the discovery, browsing and informed perspective and maybe just seeing through some options to kind of paint the picture in my head on which of these make sense for me versus not that discovery has been happening on these platforms. So even though conversions are not happening, definitely that building a brand image and kind of giving more exposure to your catalog through these agents is definitely need of the hour for all the retailers and everyone is competing for that space as well. On how previously they also competed on the top slots on the Google search. Now similarly, all the retailers are competing for that. And now with the concept of sponsored ads, uh, as well of these platforms, I am not sure how it's all going to shape up, but eventually retailers might catch up and do that as well.

Speaker B: It's funny you say that one, Congratulations. My wife and I just had a baby as well and very similar. My wife every single day is asking a different question. Is this normal? Is this not normal? Like, should I be concerned? What do I do? Are these chunks too big? Do I need to blend it more? You know how it is. So no. Very cool and I appreciate the time. This was a phenomenal conversation, obviously. Huge congratulations on all your success at targeting your previous businesses and I learned a lot about enterprise AI deployment. So I really, really appreciate you taking the time today.

Speaker A: Yeah, likewise. Thank you so much. It's a pleasure to join the conversation with you and share some of what I'm seeing here to the broader audience.

Speaker B: Awesome. Thank you so much.

Speaker A: Thank you. Goodbye.

Speaker C: The E Commerce Toolbox AI in Retail is brought to you by noibu. To find out more about noibu and how we unify error monitoring, site performance and experience analytics to uncover growth opportunities and skyrocket your revenue, visit www.noibu.com. that's N O I B U.com and then make sure to search for the EE Commerce Toolbox AI and retail on Apple Podcasts, Spotify or anywhere else. Podcasts are found and click subscribe so you don't miss out on any future episodes. On behalf of the team here at noibu, thanks for listening.