Why Most QBRs Fail And the 15-Minute QBR That Fixes Them

Speaker A: It's the first framework that I have really implemented that is a total game changer that executives do show up to the conversation, even if they're just curious, because they're like, none of my other vendors can do this in 15 minutes. How do you do it? It actually works. And it's all based on, like, my own experience declining QBRs, um, with vendors, as well as working on them with my team.

Speaker B: This sounds like a story, like the start of a future story where you will probably write a book. I'm reading the future here.

Speaker A: It's going to be a 15 minute read. Okay, I'll give you that. Like, it's going to be short. And this is the thing is, like, it's always harder to be concise than it is to be long winded, right? So I promise you, if it's a book, It'll be a 15 minute read.

Speaker B: Hello everyone. Welcome to another episode of Customer Success Talks. Real Challenges, Experts Advice. The podcast dedicated to helping those transitioning into customer success and early career customer success managers. I'm, um, your host, Byron Toruno. And just like you, I'm still learning about this amazing world of customer success. Also a customer success manager. Super curious to see where our role is moving forward in the future and also learning from amazing, um, individuals, amazing professionals, amazing people who sit here with us, invest their time, their energy, their insights to share with everyone here, including myself. Today is not the exception. And today we will be talking about a framework that has been viral recently. And it's a framework that is actually covering one of the most holy rituals that we have in cs, which is the qbr. And there has been a lot of debate. Should we not do qbr? Should we actually change the name of it? Should we change the cadence of it? There's so much out there, but today it's going to be a really interesting one that we touch on a different way of seeing QBRs, I will say, even more effective and impactful. And when I hang up a call with a customer, I always ask myself, how was that impactful? Was it worth my time? Was it worth the customer's time? Were they multitasking or were they actually hyper focused on what I was saying? And that's part of the questions that we're going to be answering today. Today we have the pleasure of speaking with Andrea Bumstedt. He's the founder and CEO of CS Impact, a consultancy providing fractional leadership and strategic advisory services to B2B SaaS companies with 50 million to $150 million in ARR. Known for turning customer success into a revenue engine, Andrea has helped grow stage companies generate millions in incremental revenue through outcomes driven retention and expansion strategies. So let me just repeat that part. I think super important outcome driven retention and expansion strategies and award winner, thought leader in CS as well. Andrea is the creator of the 15 minutes QBR framework. And that's what I was talking about. 15 minutes QBR. Not one hour 40 minutes, not even 30 minutes. 15 minutes QBR. Uh, a framework that is really practical and is ready for, um, is ready for us to use with our customers and align with them. Andrea, welcome to customer success talks.

Speaker A: Thank you so much for having me.

Speaker B: Andrea, um, you are tackling one of those topics in cs, so I would love to know from you, like, what are we actually fundamentally getting wrong in the QBRs?

Speaker A: So QBRs, I mean, we talk about them a lot in customer success, right? Um, in fact might even say we're kind of tired of talking about them. And I think the reason why we're tired of talking about them is because they really don't work. Right? And so I know as a vice president of customer success, um, I declined every QBR that a vendor wanted to have with me. So on my team, right? Like we were using various solutions, right? Um, with my customer success managers and those vendors would want to have a QBR with me. And I was always, always declined the qbr. And the reason why I declined it is because I felt that they really low value, that these vendors truly didn't understand the problems that I was looking to solve and the time commitments and the stress that I was under. And so I always declined them. And yet I was such a hypocrite because then I would turn to my team and I would say, how many QBRs are you running this quarter? And you hit your goal, right? And so, um, I felt like such, such a hypocrite, um, in that case, um, because QBRs are a core measure of my team. Um, a leading indicator of customer health, if you will. Um, and yet I declined them all.

Speaker B: It's interesting how you say low value. It's like everything is surrounding value basically for the stakeholders.

Speaker A: Yes. Yeah, yeah. I just, I always found them to be the biggest waste of time. Um, um, and yet I always asked my team to do them.

Speaker B: Is that is. How did you start with this 15 minute QBR? How did everything started?

Speaker A: So after, after I finished my vice president of customer success roles, in my last, um, VP role, I actually was training some customer success team. So I was training on them on Things like how to run commercial negotiations, how to have a consultative approach, um, et cetera. And in one of the Q and A sessions, at the end of one of those trainings, a CSM put up their hand and they said, how do I get executives to attend my QBRs? Um, and I sort of paused and I thought for a few minutes, because this is a big problem, right? Like, just like me declining QBRs, executives decline them all the time or they commit to them and then they don't show up. Or they do show up and then they only stay for 15 minutes and then they're out. Um, and I know my own teams were spending, you know, I can think of upwards of five hours to prepare for a qbr, right? Pulling metrics data and usage data and looking at support tickets and trying to verify, like, the value that the customer was getting, right? So spending all this time preparing for a meeting that then executives don't show up to, right? And that's what this CSM was asking me is like, how do I get people to show up? And so I sort of paused and I thought, you know, um, why don't you run the meeting in 15 minutes? And she looked at me like I was crazy. And I said, just hear me out. And I said, every executive has time for a 15 minute meeting. In fact, it's the easiest way to get time within an executive is just to tell them, all I need is 15 minutes. Um, I know as an executive myself I could make time for a 15 minute meeting, but in that meeting, you want it to be very high value. And so I said, there's really only two questions that a customer ever wants to answer. Um, regarding a vendor solution. And those two questions are this. One is, am I getting value from this solution? So that is always, number one, am I getting value from this solution? And two, should I keep spending money on it? Right? And so I was like, so in those 15 minutes, you better answer those two questions, right? AM I getting value? And, um, should I keep spending money on this? And so I said, in those 15 minutes, why don't you cover these three things? And we said, cover the value that they're getting. So the value to date, talk about how they can get more value. So with your experience working with other, like, customers, with their competitors, right? You are the expert. How can they get more value from your product or solution? And then make three recommendations. M. That's it. That's all right. So then an executive leaves that meeting and they're like, I know the value I'm getting. I know how I can get more value and I know three recommendations for how to move that forward. Right? And so that was actually the birth of the 15 minute QBR, um, which I then, um, on a whim, decided to write a post about it on LinkedIn. I was like, okay, well QBRs are broken. We can do this in 15 minutes. Here's what you would cover. And then what I did was I just created a very simple framework. It was just a one page slide of like, here are the three things that you want to cover. And on that LinkedIn post I said, um, if you'd like me to send you the framework, just comment 15 in the comments. I left it at that. And the next day there was like a thousand people that had put 15 in the comments. And I was like, oh, wow, okay. So I start sending the framework. By the end of the week there was over 5,000 people, um, that had put um, 15 in the comments. And what was surprising is it wasn't just CSMs that put 15 in the comments and it wasn't just CS leaders that put 15 in the comments. It was CROs, CEOs, CMOs, um, and also executives in organizations that are not SaaS, companies that are just like, we need to talk with executives. We're having a really hard time getting an executive audience. I want to see the framework. Um, and so it clearly like struck a nerve, um, in the community, which is just that like, executives don't have time for these meetings. They don't show up, or if they do, sometimes they say they only have minutes, right? And yet it's one of the most high impact meetings that you can actually have with a customer. So you better not mess it up. Right? And so, um, and so that's how the 15 minute QBR ended up going viral. Um, in fact, I've spoken on, I think at least five podcasts about the 15 minute QBR, um, and continue to share the framework. In fact, I'm implementing the 15 minute QBR, um, with one of my clients right now, um, who has enterprise level Fortune 500 companies, um, and executives that they are trying to get time with, um, and it's extremely difficult because they are one of a hundred vendors, right, um, trying to get time with an executive. And so I've actually trained those teams on how to deliver A, ah, 15 minute QBR.

Speaker B: What a good skill to have. I mean, nice for your team that they were able to, that they were able to have you to train them on having this big topic that we keep talking about in cs. And it seems like you found the fix of how to first bring the attention. Because it's like, what do we do before the meeting, during the meeting, and after the meeting? And then you already cover the before the meeting into calling the attention. When someone says 15 minutes, they might even enjoy just to be curious into how that's going to look like in fact, they do.

Speaker A: So it's a hook to get executives to show up. And, um, the team will say, I will deliver the QBR in 15 minutes. And the executive is thinking, I, uh, have never seen that ever. Right? All of our other vendors, because what typically happens in a qbr, and I know this because I've sat in on so many and I've developed them for my teams, right? Is that the whole first 15 minutes is like, here's, let's do introductions, and here's the who's who on the customer side, and here's who we are on our side. Um, here's a little bit about the vision of us as a company and where we're going, right? And that's like the whole 15 minutes. But if you have an executive that is really strapped for time, they don't have time for that. Again, they just want to answer the question, like, am I getting value from this solution and should I keep spending money on it? Right? And so I was like, just scrap all that. Right? Let's get right to the value, which is actually really hard to do. It's. It is easier to create a deck that's got an intro to the team that goes through, like, you know, their. Their usage stats, the vision of the company, how we're performing against their goals. Right. Even I've seen QBRs that have dug into, like, support tickets. Right? Um, and then, okay, let's have an open discussion about your strategic priorities and capture some next steps. Like, I've seen this. I've also seen the product roadmap thrown on the back half of the qbr, right? Where it's like, let me tell you about, like, the newest features coming in our product and what that feels like to an executive. Because I've been on the other side, but on both sides, it feels like this QBR is all about you. It is all about the vendor. It's all about your chance or your opportunity to tell me.

Speaker B: Tell me yourself.

Speaker A: Yeah, your company vision, your products, how we should be using your product. Like, it's not about me. And I know as an executive, I was trying to solve very big problems at a strategic level. And so I didn't have Time, Right. For a vendor to tell me all about them. Um, selfishly I was like, I just need to talk about my problems and how you are helping me solve my problems. And if your solution is not helping me solve my problems, then I'm not interested. Right.

Speaker B: Yeah, yeah. I mean, it also, we're giving so much information to a human being that can just capture X amount of information per day. Probably already had one QBR already before us, or we are the first one and then another one comes. So you remove all of the slides that you know that there was not generating value, but in 15 minutes it was just hyper focused on the QBR or is there also time for discussion? Is there also time for questions like how's the dynamic during those 15 minutes looking like.

Speaker A: Yeah, so what I recommend and how I train teams to run this, which works very well, is, um, they basically. So let me back up a little bit because there are some really key sort of things that you need to have in place to run a QBR in 15 minutes. Um, um, so one is you need a deep understanding of the problem that you are actually solving for your customer. And that comes through conversation. It comes from knowing their business. Right. It comes from discovery. It comes from knowing the industry that they're in and the challenges that they're facing. Right. And it comes from knowing who you're talking to, the Persona within the organization, and what matters to them. So all of that, you need all of that to have a fundamental understanding of what value means to them, because value is solving their problems. So one, you need to know who they are and what problems do they have. Right. Um, and then you need to connect the dots and you need to know how your product or service solves those problems. Right? And then the only way in my experience to truly track all of that is to document it. Right. And so I recommend documenting that in a success plan with the account we have a good understanding of. Okay, what is the problem we're looking to solve? What are the objectives they're looking to achieve? Right. Um, how would we measure that? Like, what are the key benefits if we hit those objectives? But more importantly, how will we know if we're successful? How will we measure it? As much as you can quantify that value. And then the success plan, um, is also your sort of blueprint of action in your account. So like who's going to do what by what dates? Right. So you want to have those components of the success plan and be actively meeting with the customer before the QBR to make sure that you are actually solving their problems, that they are actually getting the value, that things are actually moving forward, right? So that's the foundation. And then there's another piece that I talk to a lot of organizations about because I really struggle with it. Uh, is the data piece of like, okay, we know that our customers are using our products and services. We can see it, we can see the usage, right? But we don't truly understand what they're doing in here and how that translates to value. And we don't really know how to measure it in terms of the customer's business outcomes and the impact that we're making. Meaning we can see that they're in the product, they're using the product, they've activated these features, right? They've used this many licenses or this many tokens or whatever. But we don't truly know that our product has then helped them save this amount of money or help them generate this amount of money or help them get to this level of efficiency. Right? And so that's the gap is a lot of companies, a lot of B2B SaaS, companies fundamentally know the usage statistics around their products. They can see that if they're tracking that right in the product, all that what we call telemetry, right? They can see that. But what I often hear is they have no idea how that actually translates to value. They have no idea how that actually solved a customer's problem, how that's actually saving their customers money or making their customers money. And that's why you need a success plan to have those conversations. It's not just about usage. In fact, I've seen phenomenal usage in customers that churn, right? Because that usage didn't translate into value.

Speaker B: It's like moving from being a product expert to being the strategic advisor for our customers in 15 minutes by collecting the information and working together with the customer. So this is the success plan. These are the next steps. This is the value that we have brought, and this is where we want to be with you together.

Speaker A: Exactly, yeah. And if I were to take that one step further, one thing I'm experimenting with right now is what's called a value map. So if the success plan is like the execution piece of, like, how are we working together? Who's doing what? How are we measuring it? The value map is like that executive facing piece, right? That truly answers the question of, like, what is the value you're getting from this product? How does that translate into business outcomes, which all of that is really hard to do. It does involve that consultative Approach of like this isn't just about usage, it's about a CSM fundamentally understanding a customer's business. Right. The problems they're looking to solve, um, what is meaningful to them, um, and what produces meaningful results in that business. That's, that's very different. And in fact this becomes more and more challenging to do as you move down market. I'd say at the enterprise level this is a little bit, I don't say, I'd say it's easier, it's not easier, but um, it's a little bit easier to get at, to capture this information, to run the discovery. Right. Um, as you move into like mid market and even SMB and long tail customers. Right. Because I often get asked this question of like, okay, well I can see how we can run a 15 minute QBR in an enterprise executive level with Fortune 500 companies.

Speaker B: You read my mind. Yeah.

Speaker A: Right. But then how do you also run that down market? Because that would be great if we could also run a 15 minute QBR with 200 customers or 2,000 customers or 20,000 customers. Right. And, and I often get asked, well, how do you do that? And in that case my answer is you're not going to be able to talk to every customer to understand exactly the challenges of their business, um, exactly why they bought the product, exactly the outcomes that they're getting. However, what I love about um, success plans, what I love about QBRs is that they're not infinite. Meaning there's a finite number of reasons or use cases why someone would buy your product. And there's a finite number of Personas that are using your product. There's a finite number of industries that are using your product. So let me explain. So, um, for instance, you may have a, um, group of customers buying your product, right? This ICP that fits into accounting firms with 10 people who are looking to streamline their operations. Okay, so that's like one bucket and then you might have another bucket, um, which is something different and they're using your software in a slightly different way and this is their Persona and how they're using it. Right. And then you might have another one. Um, so what you want to do is truly understand each of those buckets then. So like, okay, Let me take 2,000 customers and let me just kind of segment them into some buckets. Right? This is the use case over here. And this is typically why they want to buy the product and what they're looking to solve. Here's another use case here. Um, this group of customers are typically buying it for this reason. And this is the problem they're looking to solve. When you segment like that and you're like, okay, these are the Personas that we're selling into, These are the problems that we're solving, then you can infer a lot of the qbr, right? Again, this is not infinite, it's finite. So like, so based on your business, I know that you are likely trying to solve these challenges in these ways. And our product offers you a return on your investment this way, right? And you probably have that about 90 to 95% correct, okay. Especially if you know your customers, right? You do a bit of market research, like you've talked to enough of them, you can segment them, right? And then your qbr, based on looking at their usage, you can say, okay, here's the value you're getting, here's how you get more value and here's three recommendations based on my experience working with customers like you. Does this resonate? That's what you're doing. So you can absolutely do this at scale, but what it requires is understanding who you're selling to, right? Which solutions, what problems are you solving, what value you're getting, right? And if you're selling to different groups of customers, you just segment them, right? And your qbr, I'm going to tell you right now, is going to be 90 to 95% correct, right? And then you can automate that. You can do that digitally. You can say based on what we see of your usage, right. In our platform, based on your interactions with us, here's the value you're getting, here's how you get more value. Here are three recommendations. Does this land with you? You do want to do a bit of a check here of like, does this land with you? Um, and you can also open it up to, would you like to have a conversation right around this? But you can absolutely automate that. You can do that at scale. Um, but you just really need to understand who your customers are and be able to clearly connect the dots between what they have purchased from you and the value that they are very likely getting. And you can do that because the features within your product would have been developed for a particular use case to solve a particular problem. Like you just need to connect those dots and you can absolutely automate this. You can send that 15 minute QBR as an email, right? With an option for a follow up meeting. Um, you could even do some cool in app stuff with that, right? That like, oh, I see that you're using these features in this way here's three recommendations we have for how you can get more value. So this is. I know I went on a very long tangent there, but I get asked this question all the time of like 15 minute QBR. This sounds great. I think we could do this with our mid market and enterprise customers. How the heck do we do this with our digital customers? You can absolutely do this.

Speaker B: It's definitely obvious that you have been asked many questions about the QBR framework. You went ahead into the questions I had. So it's time to finish the podcast. Yeah, I'm just kidding. This is amazing. Sounds really impactful. And I was literally going to ask you about how to scale that to smaller group of customers. And you touched on the segmentation and so you touch on the segmentation which help us on the discovery side with the benchmark and then how to deliver that as well. So it doesn't necessarily mean that we need to have, um, a meeting with the stakeholders in the same space and time. It could also be delivered via email. Right?

Speaker A: Exactly. Yep. Or even portions of it in app. Right. What you're really doing is you're constantly answering those two questions I said earlier, which is what is the value I'm getting from this product? Right. And should I keep spending money on it? So you want to just keep coming back to your customers and saying, this is the value you're getting. Here's how you get more value and you're affirming in them that, yes, you should absolutely keep spending money on this. You are getting a return on your investment and it needs to be that clear. Right. Um, because if I asked you, I don't know what, and I'll just name some random CS tools but like, what is the value that you're getting from Gainsight? What is the value that you're getting from Tatango? Or vitally, what is the value that you're getting from Gong? What is the value that you're getting from Salesforce? You probably need to think about this for a second. Like, okay, I use this tool every day. What value am I actually getting? Right? And so it's up to a vendor if you want to secure your renewal and you don't leave it up to your customer to connect those dots right at ah, renewal where they're like, oh, am I getting value from this? Is it actually saving me time? Is it creating more efficiency? You want to just keep connecting those dots all the time. You are saving time. You are getting more efficiency. Um, this is helping you make money. This is helping you do whatever those big Outcomes. Right. They're looking to achieve. Um, like, I think of like, you know, customer success platforms and the tools that we use. It's like, okay, here's how you're getting more efficiency across your team, right? Here's how you're saving time. Here's how you are improving your retention rates with this product. Right? Here's. Here's a direct link of, like, how we help you improve your retention. Here's how we help serve up expansion opportunities for you. Right. Which helps your nrr. Those are the things that I care about, right? Where I'm like, okay, am I getting value from this? Meaning, does it solve my biggest challenges, which, as a CS leader are I need to protect my customers. So I want grr. I want to grow my customers, I want nrr, and I want greater efficiency across my team. Yeah.

Speaker B: Um, I'm thinking as well, if I want to implement this, Andrea, and when you started implementing this yourself or your team, what were they doing wrong at the beginning? Because we need to sometimes, or we don't need to, but sometimes we learn from the mistakes. What were those first mistakes that then you adjust here and there to make it really, um, really impactful?

Speaker A: Yeah. So. Well, there's a lot. There's a lot of mistakes. Right. But the biggest ones I've seen are those decks that open with, like, here's an, you know, here's an introduction to our team and like, let's. Let's make sure everyone knows each other and like, let's talk about our company vision. And like, I don't care about any of that stuff. Right?

Speaker B: Yeah.

Speaker A: Ah, Like, I mean, on this side of the table, like, as a vendor, right. We get very caught up in. In ourselves, right. And thinking like, oh, our customers care about our vision, uh, as a company and where we're heading. Like, they don't really. They're like, are you integrating AI? Right? Like, do you have AI in your platform? Yes or no? Right. That's kind of the question that we're answering in the market right now. Like, are you innovating? Yes or no. Right. Um, so I've seen a lot of sort of mistakes, if you will, around how these QBRs are opened, right. With we're just wasting the first few minutes rather than getting to the true value right in the solution. And then I've also, I mean, when I said earlier, I've seen QBRs take five hours prep time in those instances where it took five hours of prep time. This was my own team, by the way, what we were doing for Those five hours is we were trying to connect their usage that we could see in the platform, right? With the conversations that we'd had with them, with the reasons we thought they bought our solution. And we were trying to prove value, right? We were trying to, like, bring these three things together and say you're getting a return on your investment. And the reason it took five hours is because we couldn't definitively do that. We couldn't actually say that, like, oh, this usage means that you're getting this value. We were like, we were trying to draw sort of connections where we didn't know that they actually existed. And so that's why I recommend anchoring this in a success plan. Um, right. Which comes ahead of the qbr. So you actually know what are the problems you're solving. You can actually measure them. You're keeping track and actually measuring it. Um, because then QBR gets really easy where you're like, okay, here's exactly the value you've received because we have measured it with your team. There's no surprises. We did this together, right? Um, here's how you get more value. That's really easy because you anchor that in your success plan. Of like, okay, here are the things that we have not covered off yet, right? We ran a pilot in this region, but we can roll out to all these other regions, right? We implemented with this one group, but we could implement with all these other groups. We launched this product or feature, but you have other products that are not implemented yet, right? So, like, that value connection becomes really easy, and then your recommendations become easy because you've had deep conversations with your customer and you're like, based on what I know about your business and your strategic priorities, here are the things that I recommend we do next. You can't do any of that if you have not had the conversation with your customer in advance, and if you haven't tracked it in some sort of a success plan or. Or a value map, right? And so that's why QBRs take five hours is because you didn't have the conversation, right? You don't know enough about your customer's business. You don't know how they're measuring value in your solution, and you're trying to connect dots that fundamentally you can't connect. And so then you get into the QBR and you're trying to present something that's more fluff than factual. Right? Like it's just conspiracy. Exactly.

Speaker B: Andrea, um, are these principles apply for that first discovery call? Because it's about understanding the problem. Understand uh, who we're talking to. And I do want to also talk about the value map that you mentioned. But is it the same the first time that we meet with the customer? Ideally all of that information is going to be in the CRM. So we will have some um, a lot of knowledge beforehand. But maybe not everyone has the luxury. So should we have like a different session a bit longer to uh, answer all of these questions or are they already how, like how have you deal in your teams deal with this part?

Speaker A: Yeah. So the short answer is yes, you do want to be asking those discovery questions. You want a good understanding of why the customer bought your product in the first place. The problem they're looking to solve, how are you going to measure it? And it's all documented somewhere, ideally somewhere that it can be automated. So, um, Salesforce, your CRM or another tool. Right. And as fields, um, that you could actually pull from. Right. So this is where things get interesting with AI, right? As long as you have captured this information, you can layer on an AI tool to basically say, summarize this for me. Right? As long as you've asked the questions and captured the information and AI can spit out, right. Your sort of 15 minute QBR. That's not hard. It can also spit out like here are the problems that we're solving for this customer, here's how we're going to measure it can spit out your success plan, right? So you can get great efficiency there. Again, as long as you've asked the questions and you've documented it, an AI tool just layers on top and can pull that information into whatever format. Now let me back up. Let's say you're running a high velocity business with a very short sales cycle, right? Meaning you're bringing in a lot of customers quickly with a very short sales cycle of under 30 days. And there's not a lot of discovery, right. That has happened. And then the CSM gets them and it's high velocity. So you just want to get them fast, time to value, maybe they onboard themselves, right. And so you haven't had a lot of actual conversations with them. That's okay in those cases. It goes back to what I said about digital customers where it's like understand who you're selling to and what are their common problems.

Speaker B: Right?

Speaker A: Common things that you are solving. Right. Um, you can collect data, right. Do survey. What were your top reasons for purchasing this product? You can look at like which features do they activate first in the product? You can connect all these dots without actually even having to Have a conversation, right? If you truly understand your customer, you understand that Persona. Right. Um, and then can layer on some data of how they're using the product. You don't need to necessarily have a conversation every time to be able to fully understand the likely problems you're solving. Right. The likely pain points, how you are likely demonstrating ROI and create A QBR that's 95% correct.

Speaker B: This is amazing. Andrea. You created a framework that can be literally implemented even before talking to the customer. Like to be at least ready with data, uh, to be proactive and then confirm that if that's the first conversation. So this is amazing how you were able to, to cover one of the biggest challenges that we have as customer success managers in the field is like how to bring the, our stakeholders to the call, how to have them focus and also, well, if there are some follow up questions, they can always be answered after via email as well. If they need the roadmap, we can send them to them.

Speaker A: Yeah.

Speaker B: Do you have any AI prompt that you are, that that is your favorite one to use right now when it comes to preparing for QBRs or during QBRs or after?

Speaker A: Um, so I do use AI, I use it every day. Um, and I have experimented with a lot of different tools in terms of like, can it automatically like crawl different conversations, touch points, um, emails, et cetera. Can it just create, you know, a success plan automatically? Can it create a QBR automatically? Um, there's a couple of tools I'm actually exploring right now just to see, see, you know how they can do that. Because I absolutely see a world in which this is possible. Right. Again, as long as you have had the conversation and it's documented, you really just need an AI tool to layer over top of it and pull out the relevant information. Where I have had a challenge is depending on the AI tool that you use. You just have to be careful because of AI's bias towards answering your question and producing something that I have experimented with some AI tools that will infer how we are going to measure success with a customer, but it's not actually pulling what was agreed upon. And so that's the only challenge that I've run into is like you need a more sophisticated AI tool that can um, basically tell the difference between something that was directly said. Right. Versus what might have been implied in a conversation but never directly said. So let me give you an example. If I spit out a, um, or I gave you a 15 minute QBR that talked about how we um, had decreased um, tickets for your IT team by 15%. And that was a measurable goal that we were working towards. Okay, so let's say it's like an IT or a software or whatever solution. And I said, okay, we have decreased, um, your support tickets into your IT team by 15%. And then you came back to me and you said, we never set 15% as a benchmark. Like, that's great. I'm glad that we did that. But it's not meaningful to you because we never actually said that that's what we were trying to do. That, like, 15% would be good. Right.

Speaker B: It's a bad decision to be in.

Speaker A: Right. So it's not. It's still good. It's still like, oh, 15%. But because it was never actually said and agreed upon or documented, it doesn't carry quite the same value weight. Right. Because maybe I actually wanted to decrease them by 50%. Right. Um, and so this is where I've had to be careful with AI tools is like telling them, like, I don't want you to infer what good looks like or infer how we're going to measure this or infer a metric. That metric needs to be explicitly said and agreed upon with the customer, with the exception of those digital customers. Right. Where you're looking for patterns across cohorts where, like, typically 15% reduction here is good. Does that make sense?

Speaker B: Completely, yeah. Um, I have an AI system right now where I work at that is AI and it connects with my documentation, with my email and my Slack, which is amazing. But it also means that has a lot of information that has to go through, and it tends to have that issue that you're, um, referring to. So that filtering part, it's such important to not trust AI with our eyes closed, basically.

Speaker A: Yeah. This is where I worry. Right. Because customers are real people and these are real businesses. And so I, um, think, you know, any AI tool that you are using, you need to be careful that it is actually pulling out real insights. Um, so that if you go back to that customer and you're like, okay, um, here's a success plan. Here's what we agreed upon. What you don't want them to say is, we never agreed on that. Right. And they will know. So, um, I. This is, I mean, I think AI tools are great, but this is where you need to be careful because they are real customers and they're real businesses. Like, the accuracy, um, even, like, I'll go so far as to say, like, I've seen call summaries. I don't know how how much you leveraged AI for call summaries. Right. But like there. If I took a call summary today as fact, right. And did all the things that the call summary said that I was going to do, I'm probably 90 to 95% correct. Right. But there's that 5% that's probably incorrect, right. That if I actually executed, um, and did, the customer would be like, we never talked about that, or that wasn't exactly what we agreed upon. You have to be careful, right. Because at the end of the day, these are relationships.

Speaker B: That's the part of AI, uh, that sometimes is like there are two parts to this. Is it making us efficient and is it making us effective? So that's a part.

Speaker A: Yeah. I mean, again, like grain of salt. Right. And all these tools are relatively new. They're still a work in progress. Um, and, and they don't, they don't replace a human. Right. I, um, know I experimented with using AI to analyze CSM calls, um, for one of my clients. So I, after listening to about a hundred CSM calls, like, like, literally like listening to them, you know, on two times speed, but listening to a whole call after listening to about a hundred, I. I got the bright idea, or brilliant idea to create like a custom chatgpt that would analyze the calls for me. Right. And spit out the coaching that I wanted it to coach on. Um, and. And right. So I, I tried this out and it was somewhat correct, I'll give it that, but it was not fully correct. And um, one of the biggest things that the AI couldn't pick up on in the conversation is what was not said. Right. And how the customer showed up, how they said something.

Speaker B: The sentiment.

Speaker A: The sentiment, right. It couldn't pick up on that. And so it would give coaching in certain areas that I was like, no, that's not actually really a big deal. Or like, that's. That is okay, given the sentiment of the call. Right? What wasn't said? Like, I wouldn't give coaching like that. And so I, I definitely found it. You know, AI can get you. There's lots of efficiencies with AI. Um, however, M. It's. It's not the thing that you want to put all your eggs in that one basket. Meaning, like, it's just, it's not a hundred percent. Um, so let me give you. I'm going to tell you a story if that's okay. I'm going to go off the cuff. I would tell you a totally different story as an example. So, so it was a personal story. Um, so I. I wanted to cross the border, um, a few months ago, right, to see one of my clients. Um, and I got held up at the border. Um, now I have crossed the border. So I'm in Canada. I've crossed from Canada to the US Multiple times. Okay, Maybe more than multiple. Like, lots and lots of times, right? Um, as an employee of different companies, personal travel, whatever, so I'm crossing the border, but this is the first time that I'm crossing the border as, um, as a consultant owning my own business, okay? Very first time. So I get held up at the border, right? Because they want a work visa, and I didn't have a work visa, okay? Um, which was brand new to me. I posted on LinkedIn, um, about this. Um, but, you know, this was like the very first time, and it was just, you know, lesson learned, didn't know that this was a thing. Um, and so I decided to, you know, go back home, not cross the border. Um, I worked virtually with the client. Fine, right? So, so then I wanted to cross the border again, okay? And I was like, um, again, this time I wanted to go to, um, sales kickoff for my client. Um, and I was like, I. I think I can go to this because I'm a participant. Like, I'm an observer of this. I'm not being paid on US Soil. I don't think I need, um, a work visa, right? I was like, okay, I'm going as a participant. I'm observing only. And so I hopped into, like, ChatGPT, and I was like, here's the circumstances. I want to cross the border. I got held up last time. Can I cross the border? Do I need a work visa?

Speaker B: And ChatGPT sounds like something that a lot of people have done in the past, too, right?

Speaker A: And ChatGPT was like, no, you don't need a work visa. Like, you're. You're not being, you know, you're. You're observing only at, uh, the sales kickoff, like, you should be good, blah, blah, blah. And then I hop over because I'm like, well, let me do a bit of due diligence. I'm not going to just take ChatGPT's word for this, right? So, so I hop over to, um, US Immigration websites, right, and look up, like, the criteria for, like, a B1 visa versus a TN1 visa, right? And I do my due diligence, and I'm like, I still don't think that I need a work visa. I think that I'll just get, like, a letter from the company, right? Um, and have all my documentation, and I think I'll be able to cross the border. Okay. And then I was like, you know, I should actually be sure. Like, I'm 99% sure right now, but I'd like to be 100% sure.

Speaker B: The way that you're telling this, Andrea, is making me, like, anxiety to hear the end of this. Sounds like I don't know.

Speaker A: Well, so I was like, I want to be 100% sure, because if I show up at the border and they decline me again. Right. Um, then that would probably be bad. I'd probably have a really hard time getting into the US Again. And so. So I got an immigration lawyer. Okay. And I was like, wow. Right? I was like, right. Like, I was like, here's. Here's. The circumstances. Here's my business. Here's what I'm looking to do. I want to go to the sko. I got held up last time. Can I cross the border? Okay. And according to ChatGPT, I was like, I'm 99% sure. I did my research on, like, US immigration. Right. So I'm, like, 99% sure, but I'm going to pay you to tell me if I'm actually right or not. Okay?

Speaker B: Yeah, yeah, yeah, yeah, yeah.

Speaker A: You know what he said?

Speaker B: I'm, um. Uh, my experience says that he said the opposite of ChatGPT.

Speaker A: Yes. He said, you cannot cross the border.

Speaker B: Okay. That's a good feeling to hear that before crossing the border. That's true.

Speaker A: Right. So, um, that is a very. Like, a very personal story, but a long way of saying that, like, you can do your due diligence on ChatGPT, and you can leverage AI and, you know, you can do all the right things. But at the end of the day, I needed a human who was an expert in immigration law to tell me that, like, no, you can't actually cross the border under these circumstances. And here's why. And that's. And what I realized after that is that I failed to give ChatGPT a very important piece of information that probably would have changed how it responded, but, uh, because I didn't know to give it that very important piece of information, it was like, yeah, you can cross the border. And the immigration lawyer, it took him all of one minute to tell me, uh, no, you actually can't, because you're still being paid while you're at the SKO by a US Company. So even if you're not on stage, you're technically working.

Speaker B: Thank you so much for sharing that. That's not A better way of letting us know that we do have to pay attention and actually read, not be lazy, like actually read and think that the question is always about is it AI or is it us how we ask stuff, you know, but like we know that AI is improving every day. Three years, four years ago, we were just literally talking about AI, how it was going to take customer success. And it actually has helped me personally many things, but sometimes it actually takes me a bit more than if I actually take decision by my own. So it's like that about of, uh, love and hate still going on in my end. Andrea, it's time to start wrapping up today's episode. Super, um, happy, super grateful. And I have learned a lot of stuff from you into how to be more, how to actually call the attention of the stakeholders and how to even implement. You literally gave us the recipe of how to implement and what to look for. So the last question I have for you, actually I have two. One is, what are we missing? Is there anything that you feel that we're missing that we should touch regarding the 15 minutes QBR framework for people transitioning into CS or early career customer success managers?

Speaker A: Oh, I feel like we pretty much covered it and I did more talking than I intended to. But I'm just so passionate about this topic. I mean, I have been in customer success a long time. Um, before that I was in sales. Before that I was in marketing. Um, and um, I've seen these QBRs go just crazy, very badly. But just, they have not been good. Right. They have not been efficient. They've been a big waste of time, both for the companies and the teams that are running them and for the companies and teams that attend them. Right. And so I know that there's, you know, we love to pick the QBR and talk about it on LinkedIn and talk about how it's broken and whatever. But this, um, framework actually works. Um, it's the first framework that I have really implemented that is a total game changer that executives do show up to the conversation, even if they're just curious because they're like, none of my other vendors can do this in 15 minutes. How do you do, um, actually works. And it's all based on like my own experience, you know, declining QBRs, um, with vendors as well as working on them with my team.

Speaker B: This sounds like a story, like the start of a future story where you will probably write a book. I'm reading the future here.

Speaker A: It's going to be a 15 minute read. Okay, I'll give you that like it's going to be short. And this is the thing is like it's always harder to be concise than it is to be long winded. Right. Um, so, uh, I promise you, if it's a book, It'll be a 15 minute read.

Speaker B: Andrea M. Where can people find the 15 minutes QPR framework? Where can they, where can we learn more about it and where can we know more about what is going to happen for you and your career in the future? Uh, where can we find all of those news?

Speaker A: Sure, yeah. So, um, there's two places to find me. Actually there's three. Um, but there's um, so one you can follow me on LinkedIn. So I post, um, pretty much every day. Um, all from my own experience in customer success. Um, I have I think over 12,000 followers now and growing. Um, so I try and post things that are very relevant, that speak to, um, CS operators, people in the position. And it's things that I've either learned in my own career or I'm learning now with my clients. I'm always learning, so I'm always posting. The other place you can find me is on Substack. So I have a real talk with CS Impact on Substack. The framework is also posted there. Um, and I'm committed to posting weekly. Right now I'm working on a how to series. Um, so I covered things like how to forecast how to grow in your career when your company or role feels stagnant. I've covered, um, oh my gosh, what else did I cover? I mean a number of topics on, I mean how to scale, how to turn customer success into a revenue engine. But the last one I posted just last week, um, was a sarcastic, cheeky sort of post about how to get yourself fired. Um, so how easy would that be? Yeah, maybe easier than you think. Um, but, um, so that's, that's all on my substack. If you like Instagram, you can also find me on Instagram. Um, customer success impact. Um, so those are the three places to find me.

Speaker B: Perfect. So everyone, in the description of today's episode you will find all the links to find Andrea, to find the QBR, the 15 minutes QBR framework and what is common in her career. That is so exciting to hear everything that you can do in 24 hours and sleep. So it's super interesting to hear that what it stays, uh, for me today it's the word value and also to answer two questions as a customer, am I getting the value from the product? And also should I keep spending more money and then you also cover the steps to actually do that in a qbr. So Andrea, I don't have. I just have one more thank you to give you and um, I wish you all the best for whatever project is going to come in your career.

Speaker A: Thank you. Thank you for having me today and everyone.

Speaker B: Remember, keep learning, keep growing, and let's keep improving the world of customer success. Until next time.

Speaker A: Sam mhm.