What Customers DO Tells You More Than What They SAY

Monisha Saldanha: Welcome to Colorado Tech People, where we talk with the leaders shaping how technology improves the way we work and live. Most companies think they're customer centric, but they're often making decision based on feedback from only 5 to 10 % of their customers. In this episode, I'm joined by Jason Ballier, VP of Product at Medallia, to explore how companies can go beyond surveys and use AI behavior and experience data to truly understand their customers. We'll talk about the future of customer experience, the hard decisions behind building enterprise products, and what it really takes to turn insight into action.

Jason, welcome. ⁓ what drew you to Medallia and what product challenges excited you most when you stepped into your role? Jason Balliet: Well, in actuality, I was acquired. So ⁓ didn't choose Medalia.

Medalia chose us, I guess you could say. You know, at the time, customer experience really was about VOC, voice of the customer. So really focused on surveys. So the exciting part for us through the acquisition was that we were a journey management orchestration platform.

And it seemed like Monisha Saldanha: Yeah. Jason Balliet: journey management, journey orchestration was a great evolution and differentiator for that space. Cause that space was still stuck into the small amounts of feedback they were getting and not in the holistic view of what we now know as customer experience, right? Which is more than just the feedback piece.

So once we were acquired and became aware of that potential was really untapped, right? Specifically in areas like automation within the CX landscape. There was a lot of manual action being taken, but no levels or very small levels of automation. So there was a huge white space and being able to take the ideas of orchestration, journey orchestration, personalization, and start to play with it in terms of what does it mean in a holistic experience program, right?

And not too long after that, along came our friend, ⁓ chat GPT, which everyone was just kind of playing with. If you remember the beginnings of write me a poem in the voice of these different people, which have now really reshaped the idea of CX in general with all the agency that's come along, game changing evolutions these types of legacy solutions that are there. Monisha Saldanha: And what was the name of the company that was acquired that you were working for? Jason Balliet: It was called Thunderhead.

was based out of the UK and it was one of the premier platforms that first introduced the idea of journey orchestrations and the value of the customer journey. So they were very much a pioneer in that space. Monisha Saldanha: And you were Chief Solution Officer at VisualCube. What lessons do you have from that company and how does it shape how you think about product leadership today?

Jason Balliet: Well, first, as you said, we started that company, few of us that had known each other for a while. And as everyone knows from starting a business, it's hard, right? It's really hard, but it's worth it, right? It's very much worth it.

As we all know, what, 20 % fail in the first year and 90 over the first five years or something like that. So it was very difficult, but very informing. And the way that it's informed my leadership specifically around product is, you really have to be clear about the vision and what you want to become and ruthlessly stick to that. We wanted to be a product company when we started and we had a huge influx of requests to do services because of all of our background and we thought, hey, wouldn't it be great if we can fund this organically using services versus.

staying strictly with the product side. And what we ended up being was a services organization long-term and not really the product piece of it. So that ruthless prioritization, that ruthless focus really came to bear for me when it comes to other leadership there. And that really disrupted our vision.

So we were successful and ran that company for 12 years. But if you really look at the intent of what we wanted to start, We never really got to it because we let that vision become clouded with other things that was one. The other sort of the old adage of surround yourself with great people. mean, we had the to surround ourselves with some really good talent.

We didn't hire all of it. Some of it was consulted and we had networks of people that we had. We built some great presence remotely in India for a while and actually started to dive into new, at the time, Amazon technologies for virtualizing services and other things. And we started to sort of look at how that could play a role in future product leadership that was there.

And I think the biggest thing was, and ⁓ I ⁓ Steve did it the best, which was, the idea that we would anticipate customer need, not just ask for it. So in product leadership today, there's a lot of organizations that weigh on what customers are asking them to do and getting customer insight as to what their next thing is. And I think the customer feedback is important, but sometimes as product people, have to look beyond that, anticipate what the next things are and move that.

And so in product leadership, it's sort of informed the way we were looking at as an organization at Visual Cube which was what's the next thing we should do. So a lot of that informed kind of how we looked at it going ⁓ Monisha Saldanha: Yeah, some really good lessons there. What product decision has had the biggest impact on Medalia's success? Jason Balliet: Yes.

Well, I think like everyone today, admin and integration of AI is really what is leading to the biggest impact, Medallia really has taken a different lens or different way of positioning the use of AI. They trademarked an idea called frontline ready AI. ⁓ we're there to make the ⁓ users or frontline users ⁓ more productive. more automated, not replace them Monisha Saldanha: is there a rapid evolution in CX?

Have you seen a lot of changes in how people are thinking of CX? Jason Balliet: very much so. It's no longer just a ⁓ thought that it's about the feedback and survey programs. It's started to envelop all of the interactions customers could do and trying to figure out how to, promote or provide customers with the most optimal experience, regardless of how they're doing it.

That's no longer siloed. We think that's coming together in a CX piece. There are still technologies like your marketing technology, your service technology, which remain separate, but they now all work together in more of a ⁓ CX ⁓ ecosystem or mindset where have to understand all of those things, including the feedback to know how you can improve experiences for those customers and ensure that you have relevant experiences going forward. Monisha Saldanha: And shifting gears a bit, how do you prioritize features when different stakeholders, executives, analysts, and frontline employees all have competing needs?

Jason Balliet: And it was completely unbelievable. First, you have to have alignment, The needs of what those different stakeholders are asking for have to be aligned completely with the business outcomes that you're trying to drive, Obviously, people will have needs of doing something that don't. quite aligned or they're not the prioritized list of what we're trying to achieve as a business. So I think first and foremost, you have to look at how do those requests or competing needs align with what we're trying to achieve as a product.

Next, you have to look at the standard factors that all look at. Can I ⁓ monetize this? ⁓ much is going to grow adoption? Does this increase stickiness within the base that I have or allow me to expand the base in a way that is going to help the business grow in those directions.

think those then if you start to compare those, you're going to start to see some differentiation in those needs and where they fall out. And then obviously as you get towards the remainder, we some bits of scoring rigor. So we use RICE so that we look at the business impact, the reach as well as everything else. but any sort of scoring algorithm that gives you that you're comfortable with to see how those similar features that have passed those first few gates align to where the investment is going to drive the greatest impact, Monisha Saldanha: How does Medallia turn massive volumes of customer feedback into insights that companies can act on?

Jason Balliet: Yeah, well first, like we said before, it's more than just about feedback in today's world, ⁓ That's a small portion. If you look at, I know you're familiar with feedback response rates, but they're somewhere ⁓ in low single digit percentages, like 5%. Even the best companies get like 10%. So really that's expanded beyond feedback.

now to your point about the massive data set is, it's increasing that data set first and foremost to be more comprehensive and reflective of what customers are telling us. So what are they doing in their digital sessions? What kind of conversational do we have attached to clients? The speech, the chats, the things of that nature and the in-store interactions and all the different types of interactions they have.

I the key to that, to turning volumes of that into insights that can be acted on is really looking at few things. One is understanding that user's role and the scope of data that they have access to. So in other words, even though we have a massive set, if I am someone who is in operations that's dealing with a certain location, a store, things like that, the actual scope of data that is insightful me is what's happening in my store, for one. But two, I can then use the insights of my store to baseline and compare against others to see how I might balance it that's there.

So I think the key to that is understanding the role of the people who are using it that can convert those insights into action or that you can take action on versus assuming that we have to analyze the entire data set from end to end. And that's how you drive away from that massive raw data into relevant insights that are really meaningful to the people who are sitting in front of it or to the processes that sit on top of it because it's not always about taking everything from all aspects of the data set.

It's more of a focused domain based expertise that you take to then drive insights forward. Monisha Saldanha: And can you give us some examples? It's a larger data set than just customer feedback. So what are the other customer insights or customer data points that are being collected?

Jason Balliet: If you're in digital sessions, for example, like what parts of my website have you gone to, Have you interact with most? Does that interaction tell me areas of interest that you have within my product or brand, certain categories of products, if it's a more product-based company, certain types of blogs or articles or things that might be there. I can also look at your digital scoring. or engagement that's there?

Are you highly engaged by the content you're reading? Are you finding frustration that you can't do or get to the things you're exhibiting? Are you exiting quickly that's there? Are you doing some comparison shopping by copying and pasting and do other things?

So all of that digital side comes into play. And then when you get to the conversational side, speeches and chats, you think of... what am I learning from the conversation that was happened, that was there? Am I using text analytics or natural language processing to actually distill down those conversations into, ⁓ a of emotions?

What were you feeling at the time? ⁓ How were you referencing different product services that we have as a positive negative ⁓ feeling about those things? I can interpret all of that and use all of that data without any feedback to it to actually understand are there trends in my business that need to handle at my store level or something of that nature, If you're going back to the role-based view of the customers that are there. And so once you bring all that together, you get a more holistic view of what customers are telling you.

You get more relevant and insightful views as to how you might need to apply action to solve different types of customer problems. Monisha Saldanha: Very good, thank you. What role is AI playing in experience management today? And where is it creating the most real value versus hype?

Jason Balliet: ⁓ The reality is, ⁓ that it's creating value If you think CX again, just as we talked about this holistic view of customers are interacting and getting insights on it. So if you back to those digital channels, Customers are using chat types of interfaces. chat GPT, Claude, whatever it might be versus standardized web browsing. They're doing all of their that ⁓ had recently done online your website, sort of looking around, searching, and they're actually going right to those tools to figure out where they should spend their time and effort, That's there.

So that's actually changing the way customers interact digitally because they're starting in a and a non-first party location. They're not starting on my website, they're starting somewhere else. They're getting more understanding from those engines than they did from standard search, and they're starting to do more and more there, So AI is already impacting the way we look at digital experience. And then once they do get to me, digital agents are alive and well already on digital sites, assisting with shopping, help people resolve issues.

You know, a lot of those applications tend to say that a lot of the agentic bots can actually solve 80 % of some of your service problems and only 20 % need to go to a human in the loop. Those are already in play in a lot of organizations and therefore they become part of influencing CX and how CX responds to that, In the contact center, you've got voice agents which are activated, and models as we talked about that are able to extract and summarize conversations. So where you may have had very specialized tech in the past, AI can now look at the transcript of a conversation and summarize it for you, where you may have had to break it apart and done other things in the background.

CX organizations or software is now starting to implement model-based approaches to understanding thematic things about the conversations that are happening and everything else. And then inside of our tools, you're really getting to chat GPT style interface. So conversational interfaces for getting insights. You're no longer looking at dashboards.

You're starting to ask questions about the data like you would in those style interfaces. You're able to summarize those interactions and feedback like we talked about versus having to read through them. and groups of them that are there, you can perform things like root cause analysis those as to why, what's to this negative feedback and do those things if you're looking at it from a feedback lens. Agentic workflows are coming more and more relevant where you can turn those insights immediately into some form of action without a lot of prescriptive rules and things of that nature, And then lastly, in our partner networks, partnering with companies like Adobe and Salesforce and other things.

They already have agentic technologies that are there. So what's coming to the surface is less of the, ⁓ integrate with that through API, traditional API integration? It's agent to agent, so A2A integration, MCP, how can I expose my data or perform tasks? If you think of agent-based technologies across the ⁓ ecosystem.

⁓ ⁓ it, know, in CX, because of the breadth of it, I feel like it's got a lot that are starting to see real value worse, you know, kind of staying away from ⁓ the general hype errors. And these are very domain specific sort of features that AI is helping people solve. Monisha Saldanha: how do you balance automation with the human touch when designing tools meant to improve the customer experience? ⁓ Jason Balliet: I think first and foremost, coming from the spaces I came from, like journey orchestration, always the key was in any form of automation or orchestration is to try to replicate that human interaction as much as possible.

Again, you can't be completely humanized or well, we're getting closer to it. I guess, if you think of it from now using agent based technologies, right. That are there, but really what it starts with is the profiling of a customer. Understanding that customer from everything, from what you know, historically to more real time, recent interactions.

The of Thunderhead used to draw this analogy of having coffee with friends, When you go have coffee with friends, you meet them, you know, have a conversation. You know, you are empathetic during the time that you're there or you're providing guidance or whatever it might be. But the next time you come back, you're not starting over again. You're not saying, well, you know, hi, how are you?

What would you like to talk about? It's like you're picking up where you left off. And it's that engagement that actioning or the automation builds on, If you understand the person or the group of people and their entire history of the organization, can make that automation. closer to a human-like touch in a sense that you can personalize it in a way that makes sense to relevant to those individuals that's there.

The other side to automation is understanding intent. It's no longer automation for automation's sake, I'm just not doing things because someone told me to do it. ⁓ trying to figure out what is this person or group of people trying to solve. And in that, I'm trying to automate in a way that's, again, highly relevant to those.

⁓ so, in today's world, like we said, agentic technologies, other things can handle like 80 % of the simple and repetitive tasks and leaving the 20 % for the human touch sort of pieces are there. So now the human touch, the things that really require, deep interaction with a customer, longer in-depth understanding of a conversational piece and other things that, aren't quite repetitive now get sort of human touch. But I think. The automation piece is really, about bringing more of a customer understanding into it and intent-based to really still do some level of personalized and one-to-one automation for those versus just leaving it as a cold digital thing.

You can actually apply some humanity ⁓ and empathy to it. Monisha Saldanha: what are the biggest mistakes organizations make when trying to become customer centric? Jason Balliet: Well, I think first everyone starts with the idea that you need to create a central version or source of truth, One database that all the knowledge you think of it from data warehousing, CDPs. The challenge there is that you always have a new source of data.

So taking that approach that it has to get into that location first to become part of your customer centric model. I think is a flaw. CDPs did it, like I said, data warehouses. You have to actually be able to ⁓ data virtually in some ways.

So I think the first flaw is that people think like all the data lives in one place versus I have to be able to elegantly collect it ⁓ grab it virtually if I don't have it in that one place that's there. ⁓ what that leads to sometimes is bias selection of the data. So people will make decisions about what data to include, not include, and that doesn't give them the complete picture. So you're going to bias your understanding of what a customer is simply by choosing data that you think is relevant or not relevant versus choosing it and deciding whether it's relevant and you continue with it going forward.

I think also when customers think of customer ⁓ contact, customer centricity, they think reactively in like service modes. Customer has a problem, want to react to that customer problem and understand the customer that's there. If you truly want to be customer centric, you have to start thinking about proactivity If I understand what customers are trying to do and what they're about, so their I can preemptively decide. how to fix problems before they happen.

Because I know the flaws and impediments in their journeys and other things based on past reports. I can preemptive about those things being reactive. So being customer centric is about more being proactive and preemptive than it is reactive. I also think people leave out the influencing events, things like flight delays, They seem ⁓ like, you include them in customer centricity?

Because they're not. ⁓ If I'm a hospitality organization and there's going to be bad weather and I want to be preemptive, I've got to understand that the customers I'm dealing with want to do outside activities and they won't be able to, so I can offer them something more. I have to think more than just the attributes I know about an individual. And then lastly, really it to be customer centric, stop measuring individual metrics and look at the full experience because An individual metrics might give you a of one thing in one area.

But if you look at really managing the full experience, you're going to become more customer centric in that way. And you'll get out of the pitfalls of just saying, ⁓ well, this metric is really good. So I must be doing great with my customers. When your customer's view of it might be, yeah, that one thing is great, but these other nine things are not.

Because I'm not taking into account the whole thing. Monisha Saldanha: you Yeah, the journey orchestration is really important. Yeah. And what has been the most surprising insight you've learned from customer experience data at scale?

Jason Balliet: I think the first one to me was, well, I think we talked about it before, was the very small percentage of people willing to give feedback, I mean, I knew it was never 100%, but I was very surprised becoming onto this side of it that percentages were so low in that five to 10 % layer, even if you're good at it, That's there. I also noticed the surprises that behavior, and I guess this wasn't as much as a surprise to me as it may have been to colleagues who weren't used to it, that behavior and engagement tell you more about how people are feeling than just what they are saying.

Like they could actually give you feedback that you did pretty well and everything else, but their behavior after that or before, know, ongoing tells you how they feel about the brand. Are they as highly engaged? Are they going through long periods of disengagement? Things of that nature, right?

So it's sometimes you refer to it as the silent majority. People who aren't giving you feedback, but they're telling you through their body language, through their engagement with the brand, how they really feel, right? That's there. And then things like likelihood to recommend is actually useless, In my Telling me you're likely to recommend is great.

You gave me a 10. Did you recommend anybody? Have you ever? Have you brought family members in?

Those things are what mean more. The fact that you told me you are likely to do it used to be seen as one of these things like, we're doing really well because they're likely to recommend me. But did they ever? Do people ever recommend you?

So those kind of spot scores, again, this goes back to measuring the entire experience and how people are feeling with behavior and everything else. They're not as influential. NPS is not one of the major metrics anymore Are they talking about you on social? Is there positive sentiment coming out of it?

It's sort of a self-fulfilling score. It's like, my customers say I'm doing well, so I must be doing well. But it's not the end all be all, There is more out there than just those traditional CX ⁓ insights are well-known. Monisha Saldanha: So looking ahead, how do you see experience management evolving over the next decade and what role will AI play in shaping it?

Jason Balliet: This is a completely different. ⁓ well, I think, you know, it's not going to be about a single solution and it will be heavily in the future agentic driven in my opinion, like you will see sort of a network of agents sort of interacting in different across different vendors and, and ecosystems across the client ecosystems that sort of round out all customer experience, It'll never, it, it won't be about a single solution that's there. ⁓ And as we've been talking about, experience management historically has been a reactive environment, Customers gave me negative feedback or I've seen this kind of in conversations, let me go try to address it after the fact.

And there's some reactivity that will always be in things you can't see going forward. But it's going to move extremely more towards proactivity and preemptive sort of behaviors where it will constantly looking for the conflicts in customer in different experiences before customers ever give you feedback and try to preemptively resolve those things long term or proactively change your experience going forward to sort of guarantee you more of a positive experience than a negative one.

So I think it's really going to move towards that direction. And AI is the huge contributor to that because you know, there's there's been this real ⁓ understanding or goal since the nineties to be really have one-to-one personalization. If you've heard people use that, I think with AI technology, that's become more of a realization. The fact that AI can look at it, digital twins, other things that could represent those individuals or mimic what they're doing, understanding them, deal with large, vast amount of data will get us to a place where we can truly have more personalized one-to-one experiences.

in experience management or in CX, And the use of synthetic data will actually be huge because today you look at trying to understand customer behavior and you only know up to the point where you've gathered the last interaction. Well, what if I wanted to experiment on changes going forward? If I can use synthetic data from AI to actually generate, know, mock interactions based on post past customer behavior and the reality of how customers interact with my brand. Now I can predict where to make my investments.

I can put my money in the right place because the impact of those changes will be a lot higher, If I can predict where to spend my money using synthetic data. So I think it's going to have a a huge impact on it, Monisha Saldanha: Yeah, sure. As a product leader, how do you build teams that stay close to customers while operating inside a large complex organization? How do you stay close to the customer?

Jason Balliet: ⁓ Yeah, well, mean, you're the point of your question, meeting customers is a must. For us, it's one thing that we look at very heavily within our product teams is to see how often they're meeting with customers and stakeholders and things of that nature to really get perspective. And it's difficult as a product team because there's so much to do. on a daily basis to get things done.

I think the one thing that's, if you lean into it, AI again is changing the PM workload landscape, right? Today, you could, with the right tools in place, generate new brainstorming and ideation. You could generate PRDs. You can build prototypes and other things all from...

all in areas which may have taken a PM out of the mix from meeting with customers and managers can now meet more directly with customers while using these tools to do some of the workload things that they would normally do, So really it's about instilling ⁓ in sort our objectives and results for every quarter is how often are we with customers, Have we done... enough customer meetings to understand have we created you know our own advisory board and leveraging that critical feedback because you might not be able to meet with every customer.

But if you have great users of your product that are highly open to ideation ⁓ new things and they're willing to give you feedback on it having advisory boards at different levels you can have them at product you can have the feature level advisory boards I think are critical to gaining customer feedback that's there ⁓ meeting with those customers, maybe not always one-on-one, but at least getting soliciting that insight from them as they go forward. Monisha Saldanha: Final question, what is one book every builder should read and why?

Jason Balliet: ⁓ there's probably so many, right? For crossing the chasm to all these worlds. I would say this. I really enjoyed a book called Talk Like Ted, if you haven't read it.

It's about the sort of idea behind Ted Talks, right? And the thing I think you get out of it, especially as product leaders who have to tell sometimes very elaborate stories or make storytelling in is ⁓ it you two things, right? How do they get complex messages across in those TED Talks. You'll notice that they don't use any slides with words.

They use visuals. They have a good talk track. There's a cadence to how they talk at a speed that's there. It helps the storytelling aspects of it.

And two, biggest learning for me out of that was there's a reason why TED Talks are 18 minutes or less is because that's the level at which humans can consume new information, right? So when I think about that from a product perspective, it's like, how can I tell really concise stories, use great storytelling with great visuals, and how do I keep it to a level that is in that timeframe that customers, prospects can actually consume that information, understand the value and relevance?

So that's why I like that book. It's not directly about technology, but it's application to how to Storytell in our business, I think is invaluable. Monisha Saldanha: Cool, great advice, thank you. And thanks so much for joining us today, Jason.

Thanks for listening to this episode of Colorado Tech People. If Jason's insights on customer experience, AI and product leadership spark new ideas, consider sharing this episode with a colleague or a leader in your network. Be sure to subscribe so you don't miss future conversations with the innovators shaping Colorado's tech ecosystem. You can find show notes and links to Medallia in the episode description.

Until next time, keep listening, keep learning, and keep building better experiences. Thank you again, Jason. Thanks for joining us. Jason Balliet: Thanks, bye, bye everybody.