The Mike Belsito Hypothesis: Evidence-Based Product Decisions Require Customer Conversations and Experiments

Speaker A: M foreign.

Speaker B: In today's episode, I talk with Mike Belcito about teaching product management and running conferences for product managers. We answer some of your biggest questions about his product management journey. Like, how does he think AI should impact product's discovery?

Speaker A: Maybe we could use AI to start to get quantitative, uh, data and analytics that we wouldn't have been able to access otherwise. But it's not a panacea. We can be only relying on that. It doesn't mean that whatever AI tells us, you know, that's the end all be all. Like, we need to hear it from our customers directly. So I'm hoping that the rise in AI doesn't push people further away from qualitative. If anything, hopefully it gives them more dedicated time to focus on getting qualitative data. But I guess we'll have to see about that.

Speaker B: What did Mike learn from Bob Mwesta, who's famous for his work with jobs to be done?

Speaker A: And Bob go, oh, okay, hold on a minute. Level up your skills. What does level up mean? You know, so it'd be things like that, like just going deep, really drilling down and asking and understanding the why, you know, why did that person answer the way that he did? And it turns out again, in this particular case, leveling up your skills. And that was actually something that. Those were words that his boss used.

Speaker B: And how does Mike implement evidence based product management in his work at Product Collective?

Speaker A: For us, what it comes down to is doing things in small bites first and learning from those things and growing or sunsetting it off, right? And sometimes you're doing both, you grow from it and then you get to a point where, okay, it's not doing the things that we thought it would. So let's either pivot or sunset it off. And that's a hard decision to make.

Speaker B: Hi and welcome to the Product Science podcast where we're helping startup founders and product leaders build high growth products, teams and companies with people who have been there and aren't afraid to share their lessons learned from their failures along the way. I'm your host, Holly Hester Riley, founder of the Product Science Group. This week's episode is the Mike Belcido hypothesis. Evidence Based Product Decisions requires customer conversations and experiments. Mike Belcito is a startup entrepreneur and product management influencer, co founder at Product Collective and organizer of Industry the Product Conference. Mike is also co host of Rocketship FM um and is an adjunct professor at Case Western Reserve University. Last year Pendo bought Product Collective and now Mike is senior Director of Mind the Product at Pendo with that, let's go to the conversation with Mike. Welcome, Mike.

Speaker A: Yeah, thanks so much for having me on the show. Holly.

Speaker B: I'm so excited to finally have you on the show. I was thinking one of the things that we actually have in common is that we're both teaching and uh, at universities. And I'd love to hear a little more about some of your experiences with students and like how, how does product management get taught these days? Now that it's starting to make its way into more universities, it's been a

Speaker A: lot of fun for me. So I started teaching as an adjunct professor at Case Western Reserve University, really when I first started Product Collective. So this goes back to probably 2015 or so and um, 20, 2016, right around that time. And at first I was asked to teach entrepreneurship classes, um, that they had a professor that sort of last minute was leaving and they had a need. But when I took that on in my mind, I'm like, all right, my secret plan is eventually I'm going to see if they'll let me start a product management class. Because there were none. And you know, coming into product, when I came into product, when most people today that are working in product came into it, you couldn't go to school for product management. That was not a thing. So I thought it'd be really fun to start a product management class. And so sure enough, a couple years later, I think it was 2017, uh, there was an opportunity to create an undergraduate product management class. It's just a one credit class. It's a small class, you know, as far as the rest of these, you know, college credit classes are. But, um, it's been a lot of fun for me because first of all, yes, I've been able to introduce this now to students. And so those that were never exposed to product management, now they're getting exposed. I'm also a startup person at heart. Right. So starting a class from M scratch was a lot of fun for me, but it makes it a little harder, right. Because there's not anything to really go on. So going back to your question, like how is product management taught? Well, again, before my class it wasn't taught, at least not at Case Western Reserve. And so for me, I wanted to make sure I could design a class that, yeah, went over some of the basics. I mean most of the students that are taking the class, it's, they're not even taking it because they are deeply passionate about product management. That's what, uh, that was sort of a, uh, thought on my end. I Thought, oh, people would be taking this class because they're excited about products. Well, I remember now what it's like to be a college student. Sometimes you just need a class to fit your schedule for, you know, schedule Tetris, and you need to get the credit to graduate. So many of the students, that's the case. Right. It's not like they have this plan to get into product management or anything like that. It just fits their schedule. So to design a course where it's interesting enough even for the people that aren't necessarily, um, super interested in product, but exposes them to product leaders from all around the world, that. That's, you know, I wanted to create a class that sort of did both of those things. So my class, yeah, there's, you know, there's lecture and discussion, but every other class we have a guest speaker from really all over the world. I mean, we have. Most of our guest speakers are coming in virtually. So the benefit of that is I might have a guest speaker from the Bay Area one week, then from Europe the next week. And, you know, we're getting to learn from the best of the best. I have the benefit of sort of drawing on industry alumni, you know, industry. The product conference, the conference I organize. Some of the best speakers we ever had are now helping teach my students. So it's been a lot of fun to put that together.

Speaker B: Yeah, that sounds amazing. And your students must be getting a real treat with that many guest speakers from all around the world.

Speaker A: Yeah, absolutely. I mean, again, I sort of thought about it, like, if I was a student, how would I benefit most from a class like this? And to me, it's learning from people that are deep in it today. And so I just wanted to expose them to as many of those people as possible.

Speaker B: Yeah, that's awesome. I remember I heard you speak earlier in. What year is it now? 2023. Um, about what you learned by teaching product. And I remember some interesting pictures that you shared. But what are some of the top things that you've learned from teaching?

Speaker A: Yeah, well, that's been the interesting thing for me is that, you know, I come into this class as an instructor, as a professor, but really, I'm getting a lot out of it, too. I mean, I, uh, hope the students are getting a lot out of it, but for me, I mean, I'm learning a lot as I'm teaching it, even about product management, which is kind of a thing that was a surprise to me. And I think one of the things for me is just that the job that People hire your product for. You know, if you want to think of it in jobs to be done context, it may not be what you think it is. You know, I remember that first time I taught the class again. I was pumped to teach it because I thought, you know, these students, they never had access to a product management course. They're hungry to learn about product management. They want it. You know, they're. They're here because they want to be here. They want to learn about what it takes to be a great product manager and product leader. And so how I structured, even the first introduction, uh, of the course was a certain way I went about the class, you know, the very first day, and went around and said, so why are, you know, why are you here? You know, be honest. You know, what. What is it that you want out of this class? And sure enough, maybe three quarters of the class, it just fit their schedule. You know, the job they were hiring it for was not to get to their, you know, insatiable craving for product management knowledge. Really wasn't that. It was. They needed to graduate, and they needed to fill a requirement to graduate. So that's a big. I mean, when you learn that, it means that you need to step back and sort of reimagine how you even approach this course, right? Like, there you're starting at the very beginning. Um, you are trying to excite them about the topics. But I love that. I mean, so for me, it was sort of a reminder of, okay, jobs to be done. It exists not just in the product world, but it exists even when you're teaching a class like this. You need to think about what are the jobs that they're actually, actually hiring for, and how do you. How do you actually, you know, help them with those jobs? So that was one of the big learnings. I mean, there. Uh, I will say another thing for me is that. And this. This actually kind of relates a lot to what I, um. Just when I think of customer discovery. But sometimes it's the people who are often silent. They are the ones that can add the most value. I'll share a story from class, but then I'll kind of bring it back to putting my product hat on. I remember the. The first year I taught the class, you know, I. I got to experience what probably a lot of professors experience every single time, which is that there are students that are very active, very engaged, you, uh, know, asking great questions. And that's awesome, you know, helps make my job a little more fun when there's conversations and people are active but there were also students that literally never said a thing the entire semester. And, you, uh, know, at first I'm like, how do I get these students to talk? How do I get them engaged? And it just wasn't working. And there are, you know, a couple students that first year, now that I taught several years, I realized every year there's a couple students where that's the case. So we get to the end of the class, the end of the semester, and I go around and ask the students, well, what is it that you took out of this class? You know, like, what. What are your big takeaways?

Speaker B: And.

Speaker A: And you know, some of those students that would raise their hand right away, they're really active. They raised their hand right away and shared their learnings. It was fine. But I remember at one point, and so I'm about to sort of end things and move on. But then somebody that never spoke up the entire semester, I. I thought this person was. Wasn't even paying attention the entire semester, just sort of doing the bare minimum. He raised his hand and he said, you know, in the beginning of the semester, I assumed product management was project management, but now I know that it can't be, because the end of a project is when a product is finally launched. But if all goes well with product, products forever. And it was like, it sort of blew me away because, of course, all of us in product, we know that to be true. But I got this amazing insight that I think probably was the biggest insight from any student that year that really showed me, like, hey, I've been paying attention. You know, I've been consuming all this knowledge. I'm a. I now know more about product than when I first started the class. And it was a reminder to me that this person. I almost counted them out. I almost assumed they didn't really have many insights to share because I haven't shared them all all semester. And here they are with maybe the best insight in the entire semester. To sum up the class and bringing it back to my role as a product person. It's a good reminder, you know, oftentimes we turn to those people that are maybe like, I remember we used to call them the friendlies. You know, when I was. When I was an active product manager, there were customers we could turn to that they were always willing to give us feedback. And. And it wasn't just telling us what we wanted to hear. It was, you know, they might give us critical feedback. But I know if I sent them an email and I wanted to get feedback, I could Turn to those people. And that's great. It's great to have those types of people to turn to. But who are we missing out on? What insights are we missing out on when we only turn to the friendlies, you know, so this interaction I had with the student was just a reminder for me that I need to not only, you know, go to the friendlies to get feedback from, and even though I know it's going to be easy feedback, but I need to put in the work to do whatever I can to reach those people that I think they don't have much to say and they haven't been, you know, they're really hard to get a hold of. They're. They're not the people that volunteer often to give feedback. But who are those people out there and how can I put in that work to, to get that feedback from them? So those are a couple things that come to mind. But again, for me, just sort of a reminder that even teaching something, you could actually learn more about that subject.

Speaker B: I love that story. I love how you learned something so valuable from a student who'd been so quiet. And what you're saying about Friendly is. I mean, it makes me think. So many of the companies that I work with struggle to reach outside of their customer advisory board or, um, their power users. They're people who are really eager to give them feedback. But a lot of times those people are kind of unique because by virtue of the fact that they're so eager to share feedback, it means that they have a particular way of interacting with the software in the company. And it's really important to get feedback from people outside of that group as well. And it takes more work and it's harder. And so a lot of times we just don't do it because we can check the box that we talk to a customer, but it's so much more valuable when you get that wide view.

Speaker A: 100%. Yeah. And I get it. We're busy as product people and product leaders, and in our minds, we're not saying, oh, I've checked the box, let me move on. We're, we're saying we've gotten feedback. You know, we wanted to go get feedback from customers, now we're getting feedback. But I agree we need to push ourselves even more. I mean, that was a lesson for me that it's like there are people out there that have insights that you're just not going to get if you don't get to them. And so how can we get to them? Like, it's only going to benefit us most if we will actually get those people that aren't usually raising their hand to give us feedback.

Speaker B: So maybe we could turn this a little bit from that to. I'm wondering if you've learned similar lessons when it comes to running the conferences that you run, um, you know, around how you get feedback and how you make the conference stronger.

Speaker A: We are product people at heart, right? So I used to, you know, be a product manager of software products. Um, actually, before that I was an entrepreneur and I, I was a product manager too. You know, essentially was the acting product manager for, for that team. That's when I felt like I was really sort of learning as I was, as I was going, right. Like I was doing the job, but I was also learning the job at the same time. Fast forwarding to now, you know, back in 2015, you know, that's when we co founded Product Collective. Product Collective, uh, you know, it's a community for product managers and product leaders and, and many people know us for the conferences that we put on. We organize Industry, the Product Conference, a European edition of Industry, the Product Conference, which takes place in Dublin, Ireland, and also the New York Product Conference, as well as a virtual edition of industry. So I guess technically there's four conferences now. Um, but we don't approach it as if we're conference organizers. I mean, like, if you were to ask, hey, you know, define yourself, what are you? I'm not a conference organizer at heart. I'm a product person at heart. Right. So we approach the way that we run our conferences as product people that, yeah, it's not software anymore, but it sort of doesn't matter. So when it comes to getting feedback from customers, I mean, we're doing it in multiple ways. Um, as an example, you know, when we go through an addition of industry, we put a survey out. You know, of course we want, we, we like having a baseline so things like catering and things like the level of qu. You know, the quality level of the speakers this year and which individual speakers. Like, yes, we'd, we'd love to know ratings that, that attendees want to share with us, but we can't stop there. I mean, for me, running a survey, it's, uh, one piece of feedback and it's nice, but you really can't extract too much from that at all. You need to go deeper. And that's where interviewing our customers, I mean, that's been invaluable to us. And I'll say that's changed over the years in terms of the way that We've interviewed customers. I remember in the early days, I might have an interview with a customer and I might get 15 minutes of their time or 20 minutes of their time. Uh, that would be all I'd ask for because I'm like, like we're all busy. I'm not going to be able to get too much of their time. So, you know, you have a 20 minute conversation, you jot down some notes and that's, it's nice, but it's really, really hard to get actionable insights out of a 20 minute conversation. And so there's a conversation I remember we had with Bob Mesta at, uh, one of our conferences. Bob, of course, he's one of the early pioneers of the jobs to be done movement and today runs a company called the Rewired Group. And Bob was speaking at our conference and he came back to the green room afterwards and we were talking about interviewing customers. And he said, yeah, you, you do jobs to be done interviews with customers. I said, well, we do interviews. You know, I wouldn't call them jobs be done interviews. You know, we don't really know what we're doing, you know. And he goes, well, what if we were to help you with that? You know, what if I were to join you on some of those customer interviews and just, you know, help help you along if you feel like, you know, you, that you'd get value out of that. And when you have somebody like Bob Mesta offering to literally interview your customers on behalf of your company, you know, do it with you so you could learn from him. Like, you jump at the opportunity, right? So that's what we did. I'm like, of course, Bob, I would love for you to interview our customers with us. I remember what Bob had recommended to us was we need to go deep with people, though. He's like, mike, we're no longer product people on these interviews. We need to go into this as if we're documentary filmmakers. We need to really uncover the story here. So, you know, how much time do you ask of these people? Well, you know, again, 20 minutes. He's like, no, no, we need to go a lot deeper than that. We're gonna ask for 60 minutes. I'm like, 60? Yeah. I don't, I don't know. I don't know if anybody's gonna say yes to that. He's like, no, they'll say yes. The right people that we need to get feedback from will say yes. You're not going to offer them an incentive or anything. The incentive is they're sharing this feedback with you, I'm like, oh, okay, you know, you. You know best, so we'll do that. And sure enough, I got some interviews and I asked for 60 minutes. People were willing to give me 60 minutes. You. Maybe not everybody, but, um, I was able to get enough of these interviews where we had. They were full 60 minutes. And again, Bob said, like, again, we're going to be in documentary filmmakers here. We're going to go so deep that it might even get a little uncomfortable. And so I didn't really know what he meant by that. So we get on the first interview that we had, and there was an attendee. I remember to this day his name was Matt. He was an onboarding lead. So he actually wasn't even a product manager, but he identified as a product manager, identified as a product person that was transitioning into product management. And I remember starting the interview with them and saying, all right, well, you know what. What brought you to industry? And Matt would say, well, you know, I wanted to sort of level up my skills as a product person. Okay, great. Level up your skills. Uh, okay, next question. And Bob go, oh, okay, hold on a minute. Level up your skills. What does level up mean? You know, so it'd be things. Things like that, like just going deep, really drilling down and asking and understanding the why, you know, why did that person answer the way that he did? And it turns out again, in this particular case, leveling up your skills. That was actually something that. Those were words that his boss used to him. He said, hey, Matt, if you're an onboarding lead and you want to be a product manager here, you want to be a product person, you need to own what it means to be a product person. You need to level up. You need to, you know, take on more responsibility. You need to go out and learn the things that product managers are doing. Even though that's not your title now, you basically need to start doing the job of a product manager, and then you could become that product manager. But you really need to level up in order to get there by understanding where those words level up even came from. Came from his boss, the boss. Putting the pressure on. That's. That tells us something right there, right? Like that. That is a key takeaway to us. Uh, in fact, like, later on, we introduced this whole, like, here's a letter to your boss. You know, you can edit this, and this helps you get in an industry. It's a small example of one piece of what we introduced, but that from this conversation and other conversations, it Helped us, um, yeah, just identify things that would actually help. I wouldn't have known that if it wasn't for going so deep. So taking it back to your original question. What are the ways you get value from, you know, learning from your customers? Today it's like, yes, we'll do, we'll do the quantitative things like running surveys and things like that, but the qualitative piece, uh, that's what's so important to us. And so that is just an ongoing effort that we have. And it's hard because again, product people are busy, we're busy, we're a lean bootstrap company. We've got a lot of different hats that we're wearing. But honestly, like, we wouldn't be able to have grown the way that we have if it wasn't for continuing to learn from our customers and responding to that.

Speaker B: Yeah, I love that. Um, because I'm curious if you see this because I think you also have a lot of insight into sort of what is the pulse of product management. But I think that in the years I've been doing product, it's sort of gone back and forth as a pendulum between, um, the hot topic or the hot skill being quantitative discovery versus the hot skill being qualitative discovery. Um, and I feel like right now we're in a place where it kind of went over to the quantitative side. And, and a lot of companies that I go into, people are like, oh, yeah, I'm all about making evidence based decisions, but they make it with only quantitative data and they're not getting those deep, meaningful conversations with customers. Is that something that you see as well?

Speaker A: For sure. I mean, I think people forget the fact that being data driven, qualitative data is data. Right? So, uh, being data driven doesn't mean that we're relying only on the reports and the analytics. And that's all great, but it's not an either or. It's not quantitative versus qualitative. It's using them both together. And we, we should never get to a point where we're swinging one way or the other. Like, we should really always be using both quantitative and qualitative. And so, yeah, like we, even in my class that I teach, like, I asked the question, you know what, what is better quantitative versus qualitative? And start this whole debate and then it gets to the end of the conversation and the students want to know my answer and I'm like, the answer is yes. It's yes to everything. And of course, you know, nobody likes a professor that gives you a Trick. A trick question with a trick answer. But, um, I really believe that we got to use all of those things. And yeah, I mean, you know, like AI. Everybody's talking about AI right now. It's definitely the hot thing and I'm grateful for it. Right. Like when you see technologies bubble up that are pretty transformative. Right. Um, it's an exciting thing. We should be using it to sort of fast track how we're continuing to get qualitative feedback too. Um, and in fact, like, for those of us that maybe aren't so technical, this is a beautiful thing. Maybe we could use AI to start to get quantitative, uh, data and analytics that we wouldn't have been able to access otherwise. But it's not a panacea. We can't be only relying on that. It doesn't mean that whatever AI tells us, you know, that's the end all be all. Like we need to hear it from our customers directly. So I'm hoping that the rise in AI doesn't push people further away from qualitative. If anything, hopefully it gives them more dedicated time to focus on getting qualitative data. But I guess we'll have to see about that.

Speaker B: Yeah, I think some of the interesting potential applications have to do more with pre qualitative conversation and post qualitative conversation. You know, like, what are you doing with the data that you've gathered? And maybe AI, uh, can help us be smarter about how we access that information after it's sort of faded from our brains. There's definitely a place for it, but it can't, it can't replace that, that deep empathetic conversation between two humans.

Speaker A: Not at all. But again, I think it could be another tool in our toolkit that we're using while we're having these conversations that can help us analyze those conversations. It can help us, help us do a lot of things. So I'm, I'm excited about it for sure. I just, again, I hope people don't think that AI replaces the need for those qualitative conversations.

Speaker B: So one of the things that I'm curious about is, uh, another of our product science principles is evidence based product strategy. And I'm curious to hear sort of how evidence based product strategy might play out for you with Product Collective. Whether it's for the conferences or the other work that Product Collective does, how do you make decisions about essentially finding new problems to solve or ways to grow?

Speaker A: It does make an impact for us both with conferences and even on the non conference side. For us, what it comes down to is doing things in small bites first and learning from those things and growing or sunsetting it off. Right? And sometimes you're doing both, you grow from it, and then you get to a point where, okay, it's not doing the things that we thought it would. So let's either pivot or sunset it off. And that's a hard decision to make because a lot of times you're really excited about doing that thing. But there's lots of examples I could give. One that comes to mind is not on the conference side of the business, but again, um, more on the community side of the business. So we sort of saw is this growth in people turning to product coaches, and we saw the benefit from that. A lot of our speakers are product coaches. Just a lot of value you get out of working with. And I think it comes back to the fact that again, most of us didn't go to school for product. So it's hard to necessarily turn to certain people that haven't been there before. But if you have a coach that you could turn to, that you trust, that knows you really well, that could be really, really powerful. And so we did a couple of things. First, we started a leadership development cohort. Um, we call it Rising Leaders. And basically it was our way to offer small group coaching to the Product Collective community. We partnered with Ken Norton on that. Ken is, uh, an amazing product coach. Usually traditionally he's done one on one coaching. But with Product Collective, you know, this was his way of being able to offer small group coaching. And so a, uh, part of it was coaching, a part of it was other elements like meetups with your cohort. Often there's resources I would share. Um, so we put together this cohort and it went really well, you know, so we took the feedback off of that and iterated it on a second time and a third time we've had three sold out courts and, and they've gone really, really well. But a year ago we thought, okay, if things are going well here, you know, we're proving out that small group coaching is definitely something that it's adding value at least to the cohorts that we've had. What does this mean for us? Can we, can we scale up with this? And so we started another experiment, sort of an offshoot of that called collective coaching. And it was a little different. It gave people access to these small group coaches. But it was a, you know, you, you pay one price every month, so it's a subscription. And you can opt into whichever small group coaching session you might want to have. So we had several coaches. These are awesome coaches, you know, great caliber coaches. Um, but it'd be a different coach potentially. You know, you would pick whichever one you wanted and the people that were in your small group session, they'd be different too. So we went through with those and um, started growing up a little bit of a good business from that too and had recurring revenue and that's always a nice thing. But after two or three months it became clear that we were missing the mark in terms of the value that we were offering. Our revenue was growing up, you know, we were able to make more and more money every single month. But we started to question the value of what it was that we were providing really for two things. One, that three month experiment, we, what we noticed is that the, the coach, the, the coach that you have like that person, it's a relationship that they're developing with you. So a part of our um, offering was that you're opting into whichever session you want. But if you're just choosing different coaches each time, what it means you're almost starting from scratch. And so now you're sharing your, you're, you're trying to get feedback from a, uh, from a stranger essentially instead of this person that you're turning to all the time, they have a built up relationship with. The other thing we learned is the relationship goes both ways. It's not just with that coach, but it's with the other people that are part of your small group coaching session too. And what worked well with our uh, Rising Leaders program is it was the same coach and the same participants. And so you're developing real relationships with everybody. So by the end of the three month period, six month period, it's not just what you've learned, but it's these relationships that are a real value to you. And you didn't have that with our collective coaching. And we felt it. And again, uh, we could have kept kind of pivoting to find a right model, but we decided as, you know what, let's put this on pause and just continue on with Rising Leaders, which isn't necessarily all about like, hey, how do we scale up? It's like, no, we know we're going to have one cohort at a time. Maybe in the future we'll do multiple, but it's not a matter of like, let's do as many as we can. It's like, let's if for the people that we're going to help, let's add as much value as we can to them and let's make sure it's a really positive experience. But again it, it goes back to, at least our approach is doing things in very small bites first, doing things in experiments first and learning from that. Um, I think there might have been some companies to just see the money that we would have made from collective coaching and just keep running with it even if it wasn't providing as much value as they thought. It's like, well, the money's coming in and you can, you know, have one person churn out and more people joining to try it out. And even if it wasn't adding much value, no, the money's coming in. But that didn't feel right to us and for us it's just, it's not in line with what we wanted to do as a, as a community too, which is, you know, how can we make an impact on product managers and product leaders lives, um, for the positive, not just take their money. So anyway, for us that's how we use evidence based product management. Product strategy is let's test things out and we'll use, you know, information and hypotheses that we have for, for those experiments. But then it's like once those experiments are live, we have to learn from them and do something about it. Like it's all that data isn't really worth anything unless it's actionable, unless you're actually taking action on it.

Speaker B: I love that you made a hard decision with that information as well that you know, as you were trying out the new thing that in the story you were sharing about collective group coaching. I love that in the end you decided that, you know, even if the business numbers were going up, if the value wasn't there, it wasn't worth it to you. I think that's really well aligned. Like that's keeping yourself well aligned with what your core is and making sure that you're delivering on that outcome for your customers, which is awesome.

Speaker A: Yeah. Well, thank you. Well, we want to feel good about what we're doing, right. Like when we have our conference industry, the product conference, like I, um, I'm biased, right. But I wholeheartedly believe like it is the best product conference experience that an attendee could have. So like we put that on. I'm confident about the value that we're bringing. Like there's no doubt about that. But when it gets to be like again in that example, if we have those doubts, we either have to do something to change it or again, it's like sunsetting and you're Right. It is a hard, it's a hard decision because you put a lot of time into building even these experiments. Like it's a lot of time to build them out. Um, even by getting people to start to pay you for something early, like you're, you're asking for their trust, you know, with these early products too. So it is a hard decision for sure, but it's the right decision. So like, when you feel like it's the right decision makes it not so hard.

Speaker B: I love that again, so much. And ah, you know, one of the things that's really important to me as a product leader is still holding true to some values and principles, not just the product science principles, but like my own principles as a human being. And I love seeing other people do that as well. So with that, I would love to hear from you about where people can follow you and, or the product collective if they want more of this.

Speaker A: Yeah, well, definitely. Visit productcollective.com youm, um, could join our member hub. It's free to do that. It's@productcollective.com hub. That's where, you know, people can connect with each other, access all sorts of resources that we have there. I think it's like 100 plus hours of videos from past editions of all of our conferences. Um, so that's one place you could join us at our conferences coming up@industrial conference.com. so in 2024 we have the New York Product Conference on April 18th, um, Industry Europe in Dublin, Ireland, June 17th to the 19th. And then our flagship edition of Industry, the Product Conference, that's in September of 2024 as well. So you could find all that info out@industrial conference.com and yeah, I mean I'm pretty active on LinkedIn so you could look me up on LinkedIn at Mike Belcito. Feel free to, uh, jump into the conversation there. I'm really happy to connect with people there too.

Speaker B: Wonderful. Well, thank you so much for your time today, Mike. It's been a pleasure to talk to you.

Speaker A: Thank you, Holly. I really appreciate it.

Speaker B: What great insights with Mike Belcito. Next up, our new Lab Notes segment where I and my team at the Product Science Group discuss our key thoughts and takeaways from the conversation. In our first lab note, Adam Thomas shares his thoughts on the power of diving deep in customer research.

Speaker C: Depth is where you find differentiation. And when you do any type of research, finding out what words mean and getting deeper, uh, is I think very key to finding that differentiation. So something I like to say is Verbs are questions and adjectives are questions, right? Whenever someone says, you know, I'm going to, uh, I need to be online, well, what does that mean? Or in the podcast he mentioned, I need to level up, that was a great time to ask, what does that mean? Because as we communicate, we tend to expect people to get our context when the truth is, well, we really don't. One of the most beautiful things about talking to our customers is the fact that we get to really understand their language. And we do that through depth.

Speaker B: So lab note 607.1 is depth is where you find differentiation. In our next lab note, Adam shares how he likes to get started when using jobs to be done.

Speaker C: So when it comes to jobs to be done, the, uh, place I like to start is observation, uh, going to see how the product is used. And through that observation, it gives me a chance to check in on someone's story. Folks have an ideal way of doing their work, and then they have an actual way. One boondoggle, one place where you'll get trapped with jobs to be done is to start thinking about the ideal way that folks like to do jobs instead of thinking about, uh, what is actually happening. So as I start to understand what jobs to be done, I think the most important place to start is to start in the place of observation. What do we see? And then, of course, you follow up with conversation, uh, about those particular things and then go deep with the language to see if it verifies what you're seeing. With the combination of both of those things, uh, you get a really clear picture of what the job is, and that is going to allow you to build off of that and construct a story.

Speaker B: That brings us to lab note 607.2. Start with observation of the jobs to be done. In our final lab note, Adam talks with Dina Levitan about different ways of engaging with audiences.

Speaker D: One of the topics that, uh, Mike brought up was the students in his class who were really silent, um, and kind of struggling to hear from them their thoughts, their feedback on the course, kind of how they were feeling when he was talking about that. I was really thinking about the product science principle of empowered teams and the importance of making sure everyone on the team feels heard, but then also the stakeholders are feeling heard. Both the friendly stakeholders and maybe the stakeholders who are not making their opinions known and might, uh, not agree with your approach. So I was curious to hear a little bit about your take of the concept of engaging with stakeholders, empowering teams, empowering people with different communication styles to make sure that those voices are heard.

Speaker C: Dina I love that example too, because I think the example of class is very similar to the example of what happens in real life. I'm lucky enough to be a teacher as well. At Micah University. I teach product management. And the thing that I've learned in the last two years that I've been doing that is back, uh, to communication styles, opening things up in different ways to allow students to communicate in their way. What happens when that happens is I tend to get a richer level of feedback from those students and the more often that I do it, the better the feedback becomes. They become more comfortable sharing and they know that even though they're not the person that raises their hand, their feedback will be heard. So in my current class, uh, this is playing out as I have a student that can't talk. And you know, in other classes he's been, he's talked about how it's been really difficult for him to engage. And the thing that I've made a point to do for him and for anybody that may not be in that state but aren't comfortable with raising, um, their hands and speaking up is to provide text options everywhere as well as using Canvas learning management system to put, uh, exercises for students to do during class. So there's always a written component as well as a spoken component when it comes to speaking up and engaging. And the thing that I've noticed with him is that he tends to be one of the first people to comment on other people's work, which is amazing. I think that mirrors in real life, when you give people the opportunity to speak up in an office setting, not only are you going to get a lot of feedback, sometimes those folks become your biggest champions because other folks may have marginalized them. So there's an opportunity to, not just to get better feedback, but to get a tighter team as a result.

Speaker D: Totally. I love how you talk about the different modalities, the different mechanisms to get feedback and to get that in the context of a product getting continuous feedback, Is it a user interview, is it a survey? And the importance of reaching people beyond just those friendlies, the ones who are eager to share their feedback and to go beyond the power users, the customer advisory board, whoever it is that is itching to share, um, the people who might be more reluctant or who might not be comfortable giving their feedback, um, initially in kind of the most straightforward way and finding a way to reach them and hear their voices and include their thoughts in the product development process.

Speaker B: So lab Note 607.3 is use different modalities to get feedback from a range of people. Well, that concludes our lab notes section. Each of these lab notes will be in the show notes as well. I had a great time talking with Mike Belcido about how his journey through product management has evolved. I hope you enjoyed this episode. The Product Science Podcast is brought to you by the Product Science Group. We teach startup founders and product leaders how to use the Product Science method to discover the strongest product opportunities and lay the foundations for high growth products, teams and companies. We do this through community coaching, training and consulting. Learn more@productsciencegroup.com Enjoying this episode? Don't forget to subscribe so you don't miss our next episode. Or visit us uh, @productssciencepodcast.com to find show notes or sign up for more information from me and our guests. If you like the show, please leave us a review. It goes a long way towards helping others find the show too. Now go out there and experiment. M.

Speaker A: Sam mhm.