Episode 268: Rethinking What Done Means in Product Ops

Speaker A: Creating great products isn't just about features or roadmaps. It's about how organizations think, decide and operate around products. Product Thinking explores the systems, leadership and culture behind successful product organizations. We're bringing together insights from multiple product leaders pulled from past conversations to explore one shared topic offering different perspectives and lessons from real world experience. I'm Melissa Perry and you're listening to the Product Thinking podcast by by Product Institute. Today we're looking at what it actually means for something to be done in product. Not code shipped to production, but customers buying, using and loving what you built and the systems that make that happen consistently. We'll start with Tricia Price, at the time chief Product Officer at Pendo, who shares how her team shifted the definition of done from code complete to whole product complete and what that means for how a product ops, product marketing and every go to market function have to work together from the start, not just at the end. After that, we'll hear from Kate Touzzi, an independent Research Ops advisor and educator who started Research Ops at the BBC and was Research Operations Manager at Atlassian. She makes the case that knowledge is the most valuable asset in any product organization and that most companies are letting it leak away without even knowing it. And we'll wrap up with Jessica Cirocchi, senior Director of Product Operations at Pendo at the time, who shows what whole product launch and strong team cadences look like when they're actually running. Well, let's start with Tricia. Can you tell us a little bit about, you know, what product operations looks like now at Pendo? Where you've come from, what changes you've made.

Speaker B: It's really transformed quite a bit over time. And you know, I think a lot of what they did in the beginning for us at Pendo was take some of that manual work off of product managers, help with some of the operational bringing data, telling them where to pay attention. But a lot of that is automated by Pendo and AI now. And so when we think about product operations now, we take a step back and we say, what is the outcome we're trying to achieve? Let's take for example, a new product launch. We're trying to launch Session replay. What's the goal? Well, the goal is that our customers are buying, using and loving that product. And so what are all the things that the product operations team can help us do to achieve this? And one piece of the puzzle is program management. We used to, in product and engineering often think about scrum masters and just getting code into production. But if people aren't using it, if there's no Salesforce CPQ skew to sell it, if it hasn't gone through enablement, if product marketing hasn't helped us with messaging and positioning, it's not done. I can tell you there was a couple of times early in my time here at Pendo where something was marked in our systems as complete. And then I went into our product and I couldn't find it and said, oh well it's done but it's not turned on. What do you mean it's not turned on? Well it's not turned on because we haven't finished with all the go to market activities. And so we changed our concept of engineering or code complete to whole product complete. And whole product is customers using it and loving it. And so product operations helps us with program management across that whole thing. Setting the goals. How many people do we want signed up for beta? Do we want our lists over subscribed? What's happening? How do you coordinate with customer success to run these betas? Well right. How do we. And then on the other side is systems, systems like Pendo we believe and at ah, Pendo we do this. Our product managers themselves can set up their dashboards, run their reports, have access to the data. It doesn't need to be funneled through an operations team, but the operations team does need to make sure that the integrations to Salesforce are working, that the metadata is clean. Right. So that every team, when they go in and they set up these dashboards are getting consistent data and correct data.

Speaker A: Uh, product marketing which you've been mentioning as well in this, uh, it's just like a hot button issue I think for people out there. Um, I believe it's because there's a lot of companies out there with really bad product marketing and the, the product management teams are picking it up or it's falling on them and they're, they're more concerned let's say with just launch and right. Creating materials for launch but they're not doing a whole full gamut of what product marketing typically would do. Can you tell me a little bit about like how do you see the product marketing team at Pendo and what are their responsibilities and how does that compare, I guess to both product management and then product operations?

Speaker B: You know at, at Pendo product, um, product marketing reports um, in to me and they're a part of our product team. I think if you ask most of them they equally feel a part of our product, of our marketing team as well. Like it's a very shared team between both. You know, I think that often people think about product marketing as a bridge from product over to go to market. And I always joke and say, we don't need a bridge and we need a tsunami, right? We don't need, you know, someone to take the ticket, for lack of a better word, and bring it over to the other desk, um, to connect the two and change the words and the messaging from, like the product speak to, to the market speak. I mean, of course that's a part of the role, but it's so much more strategic than that. When you think about how are people going to buy, how are you going to improve your win rate against your competition, how's the industry changing in terms of ROI and value and how. What is their business case? And then you bring that back to pricing and packaging that might change what features you prioritize or build. When you have a good understanding of the competitive landscape and when you think about how people are buying, when you think about packaging, right, you might need to build, let's say you're doing a good, better, best model, right? If you build this feature, this feature, this feature, and you haven't already thought about what's in pro versus base or good and better and best, or where your lines are between your modules that you're selling, you know, you might come out with sort of a Frankenstein set of features that solve problems and jobs to be done, but aren't actually aligned with how you're going to sell in the value story. You need to think about that up front. So I think about product marketing as such a strategic, important voice at the product table. And I also think if you're a highly innovative company who's launching new products often, who has an AI story, who's doing a rolling thunder of features, coming out regularly, having product marketing in those product conversations all the time is really important, rather than trying to look at things at the end and be that bridge.

Speaker A: Yeah, a lot of people don't love to talk about process. I think they got, you know, stuck on scrum. And uh, you say that word and people think it's a, it's a bad word. Uh, but to me, I've found that in a lot of companies that look like very successful companies as well, uh, sometimes it is just a process problem, right? Like culture is good, this is good, but there is no process to do X, Y and Z. And you put that in and it solves it. How do you think about operationalizing your process and what do you and your product leaders do to make sure that all of this works together to keep delivering value.

Speaker B: So to me, process is not fill out this report. If that's what you're doing, then I can understand why it is frustrating and it feels like a waste of time. And I'm not saying you never have to write a document. We write tons of documents. But it's not about a report, it's not about a template. To me, it's about the cadence and the questions that you ask in the preparation. So you know, you should have a set of quarterly cadences, a set of monthly cadences, and a set of weekly cadences for different levels, for different teams, for different efforts. And so what do you do in those cadences? Right. You either write down on a back or you're looking at a dashboard. Sometimes a combination of both. What are your goals? What did you achieve since the last one and what are your goals for next time? And sometimes that could be we're going to run experiments, right? You talked about experimentation. Hey, we're trying to solve free to pay conversion. We don't exactly know what is going to improve free to paid conversion. We're going to run these seven experiments in the next quarter and come back and figure it out. But, but we know this is our goal is to improve this and that's what we're holding ourselves accountable to. And so when you come to that cadence, you know, you're gonna review what worked, what didn't work, and then once we double down on it, we're watching the metric and now we're talking less about the experiments and what we're gonna do and we're just watching the metric. And then if it stalls or changes or starts trending in the wrong direction, we rethink the strategy. We might try different experiments. That is one useful process. But it depends what you're trying to solve. That works very well for second week retention or free to paid conversion. That doesn't work as well. Typically when you're trying to launch a brand new product and get product market

Speaker A: fit, what Tricia laid out is a reframe. A lot of organizations resist. Done isn't code in production. Done is customers using and loving the product. And getting there requires product ops, product marketing, and every go to market function. Working in sync from the beginning, not just at the end, but whole product. Thinking also depends on something most teams never treat as a system. The knowledge you have about your customers and what actually happens to it. Let's hear from Kate. I don't recommend tools unless I actually use them. So when I Tell you, Granola has become a, ah, daily essential for me and most of my team. That means something. We're all in a lot of meetings. Granola is an AI notepad that quietly enhances your notes in the background. No bots joining your call, no awkward recordings, just cleaner thinking after every meeting. It's the rare tool that gives you time back instead of asking for more of it. You can try it yourself with 3 months free on any paid plan. At Granola AI ProductInstitute, the entire point

Speaker C: of research is to be able to gather knowledge and provide knowledge, and yet we don't manage it at all. Very few people have knowledge managers in their team. But that is becoming more and more, um, an impetus for getting research operations in place is how do we, how do we manage the knowledge that we're creating? It's not just about creating knowledge. How do we show value to the organization because we have managed our uh, knowledge well.

Speaker A: That is one of those critical pieces in product management that when I saw research ops be implemented well, I went, oh my God, why do we not all have this? You know, and it just makes it so much easier to like, be able to read interviews that have already been done with customers. I might answer my question. Or be able to understand that if a team like all the way across the company did something already, they may have learned something that I need to learn too. And it amazes me that at scale, so many companies just don't have that. It's just a free for all. You're like, oh no, you have several hundred teams. Like they could all just be on an island by themselves.

Speaker C: It's like saying that water is your most valuable asset. Which most organizations will say knowledge is um, our most valuable asset, but they're not building a dam for it and thinking of tributaries and understanding where it's going. It's just leaking and you don't even know where it's going and then you're all sitting there thirsty all the time.

Speaker A: It's so true. And where I've seen it struggle too, is that uh, especially for product managers. Like, because the two parts that you're talking about, right, because it's hard to get in touch with people. They just won't do the customer interviews. They're like, oh, nobody's answering me. It's like, well, how many people have you contacted? Like five, six? Like, I can't get the opt ins right. There's so much time that goes into actually recruiting people for a, you know, for any kind of research that they just Give up a lot of times. And then we'll just be like, oh, send out a survey, I'll get five responses. And you're like, that's not the type of research we need to do. So they won't do the research or somebody did the research or, and they don't share it. So in the absence of all that data, what happens in product management is they just like start making guesses.

Speaker C: Because research is knowledge, which is like this ethereal thing and the real value you want to deliver is that someone has all of the knowledge within them, whether they got it from one report or multiple reports or many experiences have that knowledge to make smart decisions. So really for me, you cannot monopolize knowledge. You cannot say, well, I'm the only person that should be spending time with customers and I'm the only person with the skills to be able to do that. You can say that about specific ways of generating really reliable knowledge because in that instance, in a strategic instance, and when you're making the big decisions, you really need robust, reliable knowledge being well tested. And there are definite like highly skilled people who know how to get that out of the right people and blah, blah, blah and that sort of amazing stuff. But what we need to do as research is look down the line and go, there are lots of different forms of learning and forms of gaining knowledge. We cannot monopolize the relationship with the customer. What we need to do is operationalize it.

Speaker A: When you think about research ops and product ops, right? So I wrote in the book a little bit about the research ops function and you know, what we did at athenahealth and what Jen Cardello implemented there and then at Fidelity, how do you think these things work together? I write in the book, if you have a good research ops teams, please do not reinvent the wheel. What can research ops do to work better with product ops and how do you see the divides between these things?

Speaker C: Yeah, I wish there wasn't a divide at all. I think there is currently a divide because they're both pretty, um, emerging spaces and people still getting to grips with what they're doing, nevermind with what others are doing. But I am seeing more and more research ops folks asking about product ops and trying to understand, I see that products. Have you got such a strong, um, muscle around business acumen, MBA level business acumen. We don't necessarily have that. We need that. So that's a great exchange. We've got a huge amount of muscle around data privacy and around the system set up to be able to recruit participants Manage knowledge, move knowledge around the organization, socialize things, set up cultures around knowledge exchange that I, um, think could be very useful to you. So really I, it's a yin and a yang. And once we all get to see that, we can come together and work in harmony. I know in your book Product Operations, one of your pillars is research and insights and that's where we sit. So you know, one of the entire pillars is research operations. So I foresee, and this is happening already, it's not like this is like looking way down the line that more and more um, teams are created that have product operations, research operations and then design operations working like absolutely, like dovetailed. There's just no distinction between them as one team that enables an entire developed product organization to have the knowledge that they need, have the business intel that they need to be able to make good decisions. And all the design, you know, the design operations folk, they do such interesting stuff too with in terms of getting the designers figma and all the practices and structures and design patterns and design systems that they need to be able to do their work really effectively. It's an amazing trio. Why we don't see this more and more often, I don't know.

Speaker A: Yeah, it's kind of similar to the trio that we think of with the uh, user researchers, product managers and you know, or designers. I mean product managers and engineers, but user researchers in there are two. I feel like we all have to work together as a product team. We'd say too, in the product ops world, go work with sales ops people too. I think you pull them into the tree like you pull them into the ops group. Maybe we need like an ops guild.

Speaker C: I think they do because there's also like marketing tech, you know, like Martech

Speaker A: and then there's the, the market. So the, the customer and market insights piece. Like there's the market research piece and people always ask me, what about product marketing? And I'm like, if they're doing that piece, let them do it. Like, I, I, I don't, I don't have a full ownership over like who does what as long as it's done in a way that actually helps product management. Sometimes when those things get siloed like we were talking about with user research and becomes like a us versus them thing, they're not thinking about designing it or doing in a way that can be leveraged by product management. And sometimes there's a m product marketing team that's like, no, we do all the market research and you can't, you can't come to us and ask for it. Like, we're just out there researching. You're like, well, that's pointless. Right? Like, we have to do this together and then we can form a hypothesis and then we could both go look at it and you can give me insights and that might spark me to actually start to think about other trends I've been seeing. And then we form a hypothesis together and look at it and, um, I think we need to get away from those divides. So many companies, I feel like, just get so functionally aligned instead. M of remembering if you, especially if you're a SaaS company, like, you build a product and you sell it. So, like, we're all not just working towards that product or trying to make that product better or any of that. Like, trying to make our customers happy with that. Like, what are we doing? Why are we all. There's no. Should be no competition.

Speaker C: Yeah. I mean, I might get shot for saying this, but, you know, uh, I say things anyway. I think there's also need a need to accept that product managers are the top of the pile in the sense that product managers are making the decision, they're managing the product. And it really is about giving the product. Man, this is my understanding, at least. Correct me if I'm wrong. It really is about giving the product manager the ability to make the decisions along with C suite level and founders and people like that. I'm not saying that they're the end of the road, but everything really should be pointing to in terms of research, pointing to enable. Yes, sure. Designers, researchers are, uh, not researching to make decisions. They're researching to understand, to enable others to make decisions. Designers need to make design decisions. Product managers need to make decisions about the direction of the product. That's really, uh, for me, I'm like, why are we not all working towards that? And then marketing, of course, marketing and product need to align. And oftentimes you hear that they're not aligning very well on the direction of the product versus the marketing that's going on. Why? Company organizations are big and we all get stuck in our little space and it's easy. Like for someone. It's easier for someone like me who doesn't. I don't work in an organization anymore. And so I get to kind of hover above the breadth of companies and have a look down and kind of go, why are we not all working together? It's because I'm hanging out in outer space. It's the same as people going to outer space and then they say, how you just see that the Earth is one and we don't have all these boundaries, but then you m come down to Earth and you travel back to your little home and then you're just back into your little world again. So it's maybe, you know, like Melissa, it's like we do have that guild or something, which is something of stepping away, coming to outer space just for like two days to basically see that really big picture again of how we can all be working together to achieve what we all want to achieve, which is ultimately success and more sales and like abundance for all. If you work in a uh, for profit company and if you work in civic spaces or in charities, even then you want to be delivering more value for your sponsors, for your donors.

Speaker A: I think that would be a beautiful world that I would like to live in and I want us all to get there. Kate put her finger on something that's easy to dismiss until you felt the cost of it. When knowledge isn't managed, it doesn't just sit idle, it leaks. Teams repeat the same research, miss what others have already learned, and end up making decisions in the dark, building the systems to stop that is as much a product ops problem as a Research Ops 1. For our final perspective, here's Jessica on what building and running those systems actually looked like at Pendo. What shifts have you seen since implementing program managers?

Speaker D: Way better communication across the company. We call it whole product launch. And what we really mean by that is it's whole company product launch, meaning we drastically decreased the number of surprises. So I mentioned earlier, okay, you never want to have a customer call you and tell you what feature your company just released. So we've almost entirely. I cannot say with certainty that we've completely erased it, but I think we've gotten as close to erasing it as we possibly can. And I think we've had better product launches with it. Like given that we have now 1. Legal and Security get way more involved much earlier in our PDLC now, which minimizes waste and rework and long term issues there. Our support, both customer success and technical success are typically more enabled and I think we're being able to serve our customers more effectively. And ironically, weirdly, I think we're actually more agile as a company because we are seeing such consistency through that whole process. We can adjust and adapt a lot easier than we were in the past

Speaker A: with the team as well. What's like your. Do you have cadences that you follow or you oversee? What's the daily life look like for a product Operations Manager.

Speaker D: Yeah, we just went through a shift in our cpo. So one of our co founders, uh, stepped into our CPO roles for redefining our operating cadences as a product organization. Which has been mildly terrifying if I'm honest with you, but also very exciting because it gives us a chance to reevaluate what's working and what's not working. So our current, our new cadence is that we do every six weeks what we're calling a product impact meeting, which is really cool and it's cross functional. Um, from the standpoint of all those partners I was just talking about, they also come and talk and present to product and product comes and talks and presents to them. And it gives us this more holistic view of what is truly going on with our customers with their usage of our tool. How do we keep the folks making decisions about the strategy of our product as in tune with our customers and their true feelings? Even with such a strong analytics tool, it's easy to get lost in the noise. So having this every six week cadence I think lets us really hyper focus and center and come up with a common theme or themes that hits on all of these different areas and we can march together towards a solution. So that's a big, big difference for us. It's been, been really good. And then we also do monthly roadmap reviews. We're trying to get into a cadence of a rolling planning versus like a point in time planning. And so we're moving into those where all of our products, our product areas come in and walk through what they just built, how it's adopting, how it's being engaged with, what are the key metrics and then based on that information, what are they building next?

Speaker A: This kind of like heartbeat of the company and cadence is, is a really hot topic for people out there right now. So it sounds like you've got the product impact review, you have the roadmap review. What else do you suggest companies look at to make sure that they're monitoring their strategy and figuring out if what they're shipping is working?

Speaker D: Yeah, I think it's different, right. Based on every company structure. We're obviously a company that I don't know that we put any organization higher on the pedestal than our product organization just given, um, what we build and what we do. And so I think our product leadership team is also in a really good cadence. We meet every single week for two hours, which sounds really heavy, but it is a meaty full meeting every single time, which to me means that we're Doing something right. And we start that meeting every week with going through multiple dashboards in Pendo about what is working in our product and what's not working and what have we learned and what are the customers saying. And it drives the rest of the agenda of that meeting, which has been a really big and I think improvement. It's really forced us to just look at things a little bit differently. Again, like, I think it's really easy to get out of those metrics and think about them as, again, just noise versus like how do I really utilize this to make better decisions. Our cpo, Rahul, uh, he takes the conversations we have in those weekly meetings up to his C level meetings on a bi weekly basis as well. He's very informed about our product, which seems to be really helpful. So if you have the ability to drive that, I think that is great. We also push and encourage that our product area leadership team. So you're talking product manager, pgm, uh, designer, so on and so forth that they're also constantly looking at Pindo and the data that it's being used. And then of course we do the thing at the end of each quarter where we summarize it all and send it to our board and get their input and what they think and their feedback.

Speaker A: That's all for today. I hope you took away something useful. Whether you're rethinking what done means on your team, figuring out how to stop valuable knowledge from disappearing, or trying to build the cadences that keep your product organization genuinely aligned. If you want to hear the full conversations with Tricia, Kate and Jessica, check out episodes 184, 208 and 217. If you want to build stronger product management skills and learn practical approaches you can use day to day, head over to Product Institute to learn more. And one more thing. I want to recommend a tool I use every day called Granola. It's an AI powered notepad for meetings that helps capture notes and decisions automatically without interrupting your flow. You can get three months free on any paid plan at, uh, Granola. AI productinstitute. Thank you so much for listening to the Product Thinking podcast. We'll be back with another episode bringing you practical perspectives from across the product community. We'll see you then.