Why AI Metrics Are Misleading Leaders Every Day with Elaine Barsoom

Speaker A: AI is fundamentally a leadership issue, not a technology one. There are decisions about an organization, about who owns the outcomes, what judgment remains human, how are people brought through this transition? Who needs to be reskilled? Those are all leadership choices that are really, really difficult ones. The organizations that, uh, struggle aren't the ones that are missing budget for the best tools, the great models. They're missing the willingness to change incentives, to change the workloads, to change decision, those tools, and to restructure their organizations. That is not a technology test. It's actually a leadership one.

Speaker B: Welcome to from the Top with Chad Hesters, the podcast for CEOs, founders and decision makers looking for a straightforward perspective on issues facing global leaders. No fluff, no jargon, just real conversations with people who've made tough calls and are here to share what they've learned. Here's our host, Chad Hesters. Hello, everyone. Welcome to from the Top. I'm your host, Chad Hesters, back here again with another exciting guest that I think is going to be very interesting conversation. We're here with Elaine Varsoom, who's going to tell us a little bit about her background. But I just want to tee this conversation up before she jumps in and say that after this podcast, you really got to go check out her LinkedIn profile. She's had a very, very eclectic and interesting path that has put her at the epicenter of a lot of technology and different types of industries and trying to solve the hard problem of how do companies actually do what's next? And hopefully I'm not setting the expectation too high, but I just think you've got a fantastic and interesting career. So welcome. And would you be willing maybe just to take us through the thumbnail snitch of how you got to where you are today and what you're working on now?

Speaker A: Yeah, thanks, Chad. I appreciate that the key point you said is trying to solve the hard problems. I don't know, I've actually solved them. But early on in my career, as I would say, young college student, I was really drawn to international affairs and got my degree and worked at the UN and worked on some really difficult problems that was affecting emerging markets and the economy. And I realized after graduating that I'm drawn to what motivates people, what drives systems thinking, and how do you translate that into execution? And my frustration there was always on the execution path is that can never really effectuate a lot of the change that was happening. Particularly I'm first generation immigrant born of Egyptian, uh, parents. And so I pivoted and Actually, advice from a, uh, mentor of mine at the World bank was like, go into the private sector. That's where you can really effectuate change. And so I did. And fast forward, I always took on the hard problems at a lot of these large companies. Postgraduate school, my MBA at Wharton. I was at Amex for a long time and I always gravitated towards where are we trying to build, how can we expand, or how can we really go through this digital transformation? What is that needed from a company perspective? So I would say I take the 20,000 foot layer and then really bring it down to the execution layer and then Airbnb M and trying to start a company from scratch, getting that, uh, to about 20 million and then failing. And then most recently five years at Nike, helping them build an emerging technology division, innovation, and then moving into really building out their AI center of Excellence and looking at how to bring an outside in perspective into Nike to really transform the company and stuff. So I've really seen it all from really small to really, to much larger companies and always trying to ask the hard questions and translate that into how does that translate into an execution layer? So it's a little bit about me.

Speaker B: Yeah, that's a lot. And I think the hard part about preparing for this podcast today was limited time. And there's so much we could talk about with your background. But I'd like to start with something that really stood out to me in your profile. I think it's hard with some of your background to actually encapsulate in a paragraph or lesson, like, what do you do right? And like, how do you do it? And there was this comment you make about being an architect of the execution layer. And it really stood out to me because in our line of work here at Boyden and just in my own career, I've been in the room when a lot of really good ideas have been on the table, or the sense of urgency around a topic such as AI or digital transformation or whatever the topic du jour was of the day, and everybody can sit around the room and nod and say, yes, we need to be more AI. Um, right, whatever that means. And then they sort of look at the CEO or CFO or coo, whoever is responsible, responsible for executing, and they like, okay, go do AI. So that really stood out to me. And I would love for you to maybe just tell us a little bit about what have you learned about being an architect with execution layer when you're taking existential theory and trying to apply it to real world of places like Amex and Nike and Airbnb and whatnot.

Speaker A: Yeah. You know, I was having this conversation with a friend of mine recently and it's people, it's processes, it's culture. And unless you have all three, it's really hard to drive a transformation, particularly in large companies. So you can have uh, a grandiose strategy envisioned by the senior level leadership team or by the CEO, but how does that actually boil down from strategy into action? What are the priorities? So I've always had to be a, uh, translator into initiatives and owners to what does it mean for the engineering team, what does it mean for marketing team, what does it mean for the finance team? And then you have to design it and you have to design an operating model around those people, around the processes, around the culture. And there's some work to do on the culture side of potentially upskilling and change management and aligning people so that everybody feels like they're part of the same team, even though they may not be. And then create a repeatable growth engine. That's so important. If it's not repeatable and if you can't convert priorities into something that continue to grow in connecting all of these different teams and client development and in a coordinated and collaborative fashion, it won't scale and it won't be commercially focused. So it all comes down to people, processes and culture. It's so important. Particularly in this age of AI, we're seeing infiltration of AI tools. And what do you learn and how do you reskill your people? And how do you really get the tacit knowledge that every single person in an organization has to effectuate change? That's so tough. I believe it's a complete human and organizational change that you have to go to in order to do this.

Speaker B: Well, so you just said something in there that has always been, it's hard as a leader if I'm just being honest and vulnerable. And that is the idea of, like you said, that you can't scale something if it's not repeatable. What are some. Do you have any sort of tips or observations about how you make something repeatable without necessarily having to just have like dogmatic rules and regulations? And I mean, I guess that sometimes that's necessary, but you gotta keep people agile, like uh, today's world, and they gotta be able to be flexible. But at the same time, to your point, it can't be the Wild West. Like there has to be some consistency and some framework. So how do you balance framework and repeatability with flexibility? And am I making sense Here.

Speaker A: Totally. And I love frameworks, so I'm smiling because I'm all about frameworks. So I'll give you just an example. Even when we were looking early days at AI, it's start big and then narrow on. But we had many, many different use cases and then we had frameworks for how do you even evaluate these use cases? And then once you evaluate the use cases, if there's ones across marketing, ones across engineering, ones across hr, how do you even approach the problem? So it's asking the right questions from the outset and really getting to what is the problem we're trying to solve. I always say that's the best and biggest question that you've got to ask from the outset. And it doesn't matter what type of problem. And then really mapping out, okay, this is the problem that we're trying to solve here. On the marketing, we've got. We have to scale. We can't scale. We don't have enough people. We need to do. We need a repeatable engine to produce content. So math out like, where are the areas that you've got barriers that you need to fix? And then what is that process? Mapping out the workflow design from end to end and all of the people that effectuate this entire engine. So you could approach all of the different problems the same way and ask the same questions. The outcomes are different, obviously, but having that process and that repeatable process and ensuring that you're mapping that out is just a way to systematize and use the frameworks to approach. And if you have a clear priority, then not everybody is going out in the wild, wild west and running their own. AI pilots, you know, having guardrails. Here's what we're trying to solve for and maniacally reprioritizing. Does this really make sense? Does this drive value? Sometimes the things that have the highest ROI are not necessarily the problems that you should go after first for many different reasons. Strategic, uh, priorities. It's too hard, it's expensive. So those types of questions and actually mapping that out and balancing against these different priorities, and if you have that framework from the outset, it will govern everything that you do. So it's not all going to get right.

Speaker B: Yeah. And I guess one of the things that I've learned in my leadership journey is that, you know, it's the whole, like, you know, perfect's the enemy of good.

Speaker A: Right.

Speaker B: Like, the framework is not going to solve 100% of the circumstances that are going to come across somebody's desk. But it might solve 85% or 90, so that you're sort of limiting the amount of noise that's in the system. And I think that in today's environment, like the world's changing faster than humans have ever experienced. And your comment just then about like, the framework matters because at a minimum, it's sort of like it gives you the guidance and the direction, but it's not, it doesn't limit flexibility. I think that is absolutely essential in today's environment.

Speaker A: And just to add to that, I call it the three Cs. I was just talking about this last week. Leaders now have to be curious, have to ask the better questions and declare, what's the real business? What's the human value, what's the roi? What is the front line need? Then clarity, what are the values, boundaries, ownership, what outcomes have to move, who owns this? And then the courage to actually make the organizational decisions. Because it's really a lot of this is around org design and layers. And so to actually move that into actions, so that is so important is to be human in the lead and from a leadership perspective, to be able to use those three Cs to really drive the transformation. And that's hard. It's hard in large organizations. I'm sure you see that every day in a lot of the companies that you're working with and stuff.

Speaker B: Yeah, it's interesting, the return on investment, like ROI of curiosity. It's almost impossible to measure. I mean, you can measure like innovation to some degree, but I think that's the nature. Humans have all kinds of faults, right? I mean, we're definitely an imperfect species, but curiosity is something that, I don't know if AI ever replaces that. The innate human curiosity of like, what is this and why would we walk this way? But maybe there's a question in there to you is about the ROI of curiosity. Like, is it different today than when we started out our careers? Is like, is there a higher ROI now on, um, curiosity? Or is curiosity just always been the key to business innovation and evolution?

Speaker A: I think it has, but I think we're in a different era and we're at a different inflection point because curiosity keeps urgency from just turning into theater. I always call it innovation theater. And we are just at a where there's. The technology is just moving faster than any human can actually move, than any organizational change can actually move. So if you're just responding to the urgency and you're not asking the right question, you'll end up with maybe a whole lot of what we're seeing. In the news today, lots of pilots, not a lot of roi, not a lot of value. Everybody's trying to understand what type of strategy and moving at a record pace. So I always say go slow to go fast. So curiosity is part of that. Ask the questions, what are we learning? How do we keep the urgency from just turning into theater? It's just massively important at the outset.

Speaker B: Yeah, theater is expensive in resources, people's time and shareholder money. And like I was talking to a CEO recently in Europe about AI adoption and this person was sort of lamenting the idea that every board meeting they had to show up and talk to the board about their AI strategy. And it felt a bit performative. Even though they were doing work, they were trying to be innovative and apply. He had to uh, apply some kind of theater or performative presentation in the board meeting. Like satiate the gods as it is on the topic.

Speaker A: Oh, uh, I've heard so many stories of this. Talking with lots of CEOs of how employees are just logging on or doing things on Claude or ChatGPT to make sure that they show the adoption. They're coming with these numbers, but they don't know what they're doing. So it's all performative and because there's fear and people are worried about their jobs and stuff. So is that the right metric that we should be really looking at these days of just adoption? What does adoption really mean? Can you peel back the layers a little bit and say, have you actually really made the change, the organizational changes? Have you actually redesigned your workflows? Where were you learning? What have we moved in terms of outcome? Who owns the results? Have we reskilled our people? Are there new apprenticeship models? So yes, adoption has become the word of the day, but I'm not really sure we're measuring the right things with adoption. Not saying it's not important, but I'm saying that we, I think have over indexed on this one particular measure.

Speaker B: Yeah. And you get what you measure typically. So you better be careful what you're measuring. Right. And it's the whole like spend a lot of time getting the right KPIs in place because they end up mattering. And there's a whole lot of brilliant, funny and humorous and less than humorous examples in the world of the incorrect KPIs.

Speaker A: I mean the KLARNA example is such a great one of how they invested heavily into AI and their customer service. And it worked. I mean they saw it, but they were over indexed. I mean the CEO came out later and said I think we over indexed on cost being the major driver of efficiency. And we need to bring back these customer service agents because they forgot all of the trust and the human element of these humans dealing with people and making those exceptions calls or the actual one to one interaction. And I think a lot of companies are coming to that realization that actually started their journey early on is that they're forgetting the human element of uh, where the decisions that are not on paper, that are not in the systems are being made every day. And how do you account for that? That's not what AI can do.

Speaker B: Yeah, it's hard to put human psyche on a process flowchart.

Speaker A: Very hard.

Speaker B: You know my early career, um, I worked as intelligence officer the Navy and you know those sort of very formative years for me and sort of yeah, I did a whole lot of PowerPoint but I also got to do some interesting work on terrorism and counterarcotics and things that they were kind of hard problems to solve. And there was always this element of like people's gut feel and psyche and curiosity were involved in sort of decisions. And I can't tell you how many times that somebody was like we're not going to do that. And somebody go why? Like bad things will happen if we do that and we'll explain like I can't, but we should not go do that thing that we were just talking about. And sure enough they were right, you know, and like, and it always reminds me is like in my sort of civilian kind of corporate world that I've been operating for the past 20 some years, like we really got to find a way to build in people's spidey sense, that sense where you're like you're sitting in an interview and the person's resume is awesome and the answers to the questions are spot on. There's just something there that could also be bias. Like it can be lots of bad things can come out of that sixth sense, but also a lot of good things. And I worry that to your example about Klarna, like how do you build the human back in the system and try to eliminate the tendencies and the bias that aren't helpful, but keep the human side of the curiosity and the concern and the drive that humans have when they interact with each other.

Speaker A: The reason why I do what I do and why I've been working with small and medium sized companies now and their leadership team is because I believe that AI is fundamentally a leadership issue, not a technology one. You know, there are decisions about an organization about who owns the outcomes, what judgment remains human? How are people brought through this transition? Who needs to be reskilled? What, what's appropriate for your customers? What you can't decide whether recovered capacity should become margin, should become growth. Better service, like what is or more meaningful work. Those are all leadership choices that are really really difficult ones. And so you have to really get that right. And the organizations that uh, struggle aren't the ones that are missing or don't have the budget for the best tools, the great models. They're missing the willingness to change incentives, to change the workflows, to change decision rights around those tools and to restructure their organizations. And that is not a technology test, it's actually a leadership one. And it's so important today to take the time to really do that and map that. I know it's hard work. We've gone through many technological evolutions in the past and this is no different, just kind of a different technology.

Speaker B: You've worked for really big companies and you've done a lot of work in the VC space and as you mentioned in your short bio there, you've sort of seen the whole spectrum. I feel like part of this conversation we've been having feels maybe more directed at kind of large companies with a lot more resources, human capital resources, but budget. And what about the VC or the small family owned company or even mid sized company that doesn't have those resources? Is it different for them on their AI adoption or innovation and curiosity cycle? How could they be more effective in your opinion with limited resources but still got to keep up with the world we're in?

Speaker A: Yes, absolutely. I'm um, working with. If you're right now they almost have the easiest and the hardest because the easiest may be they may not have all the systems and tools and way to our technology debt to like actually transform. But the hardest is they want to get it right right away and they don't have the resources or the technology of working with an organization. They have no technology team at all. And so they need to bring in someone from the outside to really assess. So I would approach it. I uh, think there's a real opportunity there for these organizations to actually unlock um, ah, growth and margin if they ask the right questions to bring their teams along and involve their employees. And so as a small organization a tad bit easier to involve your organization, ask those right questions and just it's hard with the Wild Wild west is bringing in people that you trust first not just going to execution. I know there's a lot of organizations or implementation agencies out there that are going straight to execution again, it's like ask the same questions that you would at a large organization, a small organization, and then bring in, do the work of the strategy or really documenting all of the knowledge of the employees to actually go through the change. It will benefit you much more and you'll get a much higher ROI if you invest in that first. Go slow to go fast. So difficult. But also you see solopreneurs now that can become billion dollar companies. So there's a lot of opportunity there and I don't want to discount that the smaller companies can grow much faster. Not bound by a lot of agencies or other outside firms that they had in the past. And things that they would normally take 10 people can now maybe take one person or two people. And so reskilling your own team to work on different things is a huge opportunity, huge unlock for these companies.

Speaker B: I think that the solopreneur is an interesting concept that I'm relatively just becoming familiar with. I've seen a few articles recently, ah. In some pretty prominent business journals talking about how whether it's economic or social or whatever those influences are, there are more and more people that have started their own independent businesses and have done pretty well, but they're doing it with high degrees of efficiency utilizing AI and you know, whether it's back on billing tools or it's, you know, productivity tools like answering emails and things like that, or it's in design tools or whatever. And I think it's pretty fascinating. One of the things we worked on at Boyden with my team has been how do you create the opportunity for people to be solopreneurial? Is that a word?

Speaker A: Solopreneurial? Yeah.

Speaker B: I don't know. That feels awkward, but maybe it's right. How do you break the opportunity for people to be entrepreneurial in their roles, but still creating the team environment and you know, I feel like the adapting to the world around us and giving people permission to try new things and fail when they try new things and almost celebrate like, hey, Barbara tried this new AI tool and it was a complete dumpster fire, but hooray, Barbara. You know, like we sort of have some fun with that on our team and try new stuff. But I mean, can you scale that? Can you scale that kind of mindset at a big company?

Speaker A: Yes, 100% you can. And I've seen it, we've done it. And there's elements of it's all about culture, training the trainer. How do you create communities around celebrating folks Trying these new tools or people that may be early adopters that are very excited. And I've seen this inside of Nike of training other people, hey, have you thought about how you use it for this? And then when you have employees training other employees, getting excited, all of a sudden you're creating this culture of uh, like a growth mindset or of an AI mindset rather than it be just something on paper that a CEO is dictating from the top down. So creating that culture and that community we had webinars for that were not exact, we didn't call them webinars but best practices. We had hackathons. We created all day events around different use cases or had people pitch their own ideas and stuff. So you can really create this fun startup innovation mindset even within large companies. You can to that also within small companies. Like who are the folks within your organization that are excited and are using it, uh, even for use cases at home that they can possibly teach other people. So that is important within an organization regardless of what size. And that's where you start to diffuse the element of fear that some people are of change, that some people might be drawn to and just create a more virality around. How do we really get to changing an organization and changing the mindset, having that organizational. Because you always have to bring your employees on the journey when you're going through this. It's so important.

Speaker B: You were talking about the inflection point where we're at. I think that's the inflection point on the technology adoption curve that we're at right now, is that everybody gets the idea that AI can be helpful. Well, we're sort of at this phase now where people are like, all right, well it turns out that the human element is super important too in the practical applications or in the architecture of the delivery. And that's a good point.

Speaker A: And in the architecture of delivery of Eden, I'll just make a point. You don't have to be the most technically fluid to be the person that's putting together like the AI strategy. You need to be curious, but you need to be able to connect technical capability to the business consequences. And so like a systems thinker, you be able to operate across strategy, across hr, across talent, across risk and be able to look at that cross company value across culture and those are going to be the strongest leaders or the strongest companies that really rise to the top in this era.

Speaker B: Yeah, well I think that's a great point to uh, end our mainline conversation, but I can't let you Go without asking. Sort of the one question I ask all my guests that have been leaders around the world a bunch, and that's a scar tissue question. And that is like, if you're willing to, I'd love for you to maybe share a, uh, war story or what's the story behind one of the scars you've got as a leader throughout your career? I mean, was there something that, whether it was early on or it was yesterday, where you're like, I tried it and it didn't work out and I learned a lesson that others might benefit from that scar. Lane, is there anything you do on the show?

Speaker A: I'll give you an example of even from my American Express days, and that's I get excited about technology, for sure, emerging technology. And as we were going through a digital transformation, there was a martech technology that I just thought would really change the way we did marketing and startup that you were trying to integrate within a large company. And it actually became super challenging as we tried to sell scale, what was a pilot scaling it to a few hundred people. And the learning and the adoption was just not there. And we ended up abandoning it. But the lesson that I learned was, ah, this goes back to what we were talking about initially in this podcast, is that, uh, just because something looks good on paper or strategically it makes sense, doesn't translate into execution and everything from strategic fit and alignment and doing the culture and the organizational change is so important in this day and age. And that really set the stage for the rest of my career to avoid making those same mistakes.

Speaker B: Well, thanks for sharing that. I think that's a great lesson. And I'm trying to figure out how to put this doesn't get me in trouble. Like, I'll use a personal example. Sometimes being older and having more gray hair, when you can explain where that gray hair came from and where that one came from, when you're talking to somebody, you're like, I'm not busting your chops here because it's fun for me. I'm telling you, because that one there is almost the exact experience 20 years ago. And I really have enjoyed that element of this podcast as we had a lot of interesting leaders on that are willing to kind of share a little bit of that one gray hair they got from a while back with others. So thank you for being here. I think it's been awesome conversation. Lorraine. Now, if somebody wanted to get in touch with you or wanted to see more about what you're doing out in the world, what would be the best way for them to follow you or keep track.

Speaker A: Feel free on, um, LinkedIn or I have my own website, Waveco AI. You can message me there or it's just ElaineWaveCo AI.

Speaker B: Great. Well thanks Elaine and wish you a good rest of the year and all the best.

Speaker A: Thanks so much.

Speaker B: If you're ready and make better leadership decisions and avoid the costly ones, subscribe to from the Top with Chad Hesters on Apple, Spotify or wherever you get your podcasts. Until next time, thanks for listening.