Making AI Work In Product Teams: Roundtable Insights

1 00:00:00,160 - > 00:00:02,319 SPEAKER_02: We had some really great discussions, roundtables, 2 00:00:02,479 - > 00:00:04,320 specifically focused around AI. 3 00:00:04,559 - > 00:00:09,279 The first was AI is making teams faster, but not necessarily 4 00:00:09,279 - > 00:00:09,759 smarter. 5 00:00:10,000 - > 00:00:11,279 This concept of AI sloth. 6 00:00:11,599 - > 00:00:14,880 A lot of leaders even brought that up as the term that they're 7 00:00:14,880 - > 00:00:15,279 using. 8 00:00:15,439 - > 00:00:19,760 We saw frustrated leaders developing things like I built 9 00:00:19,760 - > 00:00:23,600 my own persona to hand off to my team so that they could evaluate 10 00:00:23,600 - > 00:00:25,839 their documents against my perspective. 11 00:00:26,000 - > 00:00:26,320 Wow. 12 00:00:26,480 - > 00:00:30,079 Because they were just getting junk from their teams. 13 00:00:33,039 - > 00:00:35,600 SPEAKER_00: Hey Product Rebels, I'm Vidya Denamani. 14 00:00:35,840 - > 00:00:37,280 And I'm Heather Samarin. 15 00:00:37,359 - > 00:00:39,920 And you're listening to the Product Rebels podcast. 16 00:00:40,079 - > 00:00:42,399 Today we're going to be doing something a little different. 17 00:00:42,640 - > 00:00:47,439 There's no guest, it's just us over the past couple of weeks. 18 00:00:47,600 - > 00:00:48,719 Woo! I know. 19 00:00:49,119 - > 00:00:53,039 We have been running these AI roundtables with senior product 20 00:00:53,039 - > 00:00:53,520 leaders. 21 00:00:53,679 - > 00:00:56,719 There have been CPOs, there's VPs, there's directors of 22 00:00:56,719 - > 00:01:00,560 product, people in the thick of it, all working on AI. 23 00:01:00,799 - > 00:01:04,480 We ran four sessions and we came out with a lot to talk about. 24 00:01:04,719 - > 00:01:07,840 Today we're going to be sharing what we heard, what surprised 25 00:01:07,840 - > 00:01:10,400 us, and what it means for product teams right now. 26 00:01:10,640 - > 00:01:12,079 Wanna kick it off, Heather? 27 00:01:18,000 - > 00:01:20,799 SPEAKER_02: Yeah, shoot, we had some really great discussions. 28 00:01:20,959 - > 00:01:25,040 We had three roundtables specifically focused around AI. 29 00:01:25,359 - > 00:01:31,120 Two that were more leaning in on building AI products, and then 30 00:01:31,120 - > 00:01:34,799 another one that was really leaning in on utilizing AI to 31 00:01:34,799 - > 00:01:36,719 drive efficiency in the organization. 32 00:01:36,959 - > 00:01:40,239 And I think there were some really interesting themes that 33 00:01:40,239 - > 00:01:41,280 came out of it. 34 00:01:41,519 - > 00:01:47,359 The first was I think across all roundtables was really AI is 35 00:01:47,359 - > 00:01:50,959 making teams faster, but not necessarily smarter. 36 00:01:51,120 - > 00:01:53,760 And we'll talk a little bit more about this, but it's this 37 00:01:53,760 - > 00:01:54,879 concept of AI sloth. 38 00:01:55,200 - > 00:01:58,560 A lot of leaders even brought that up as the term that they're 39 00:01:58,560 - > 00:01:58,959 using. 40 00:01:59,120 - > 00:02:05,439 We saw some really frustrated leaders developing things like I 41 00:02:05,439 - > 00:02:09,039 built my own persona to hand off to my team so that they could 42 00:02:09,039 - > 00:02:11,919 evaluate their documents against my perspective. 43 00:02:12,159 - > 00:02:12,400 Wow. 44 00:02:12,639 - > 00:02:16,080 Because they were just getting junk from their teams. 45 00:02:16,319 - > 00:02:19,280 And so things like that really, there's a lot of manual effort 46 00:02:19,280 - > 00:02:22,319 going on to help in address the AI slot. 47 00:02:22,639 - > 00:02:26,240 Another one that I thought was super interesting is the 48 00:02:26,240 - > 00:02:31,039 inability to see the true ROI in the investments they were 49 00:02:31,039 - > 00:02:35,360 making, whether that was AI for the sake of building efficiency 50 00:02:35,360 - > 00:02:39,439 in the product development lifecycle or in developing AI 51 00:02:39,439 - > 00:02:43,520 products, connecting those dots to the outcomes, whether that be 52 00:02:43,520 - > 00:02:46,400 the outcomes for the business and the customer, or just the 53 00:02:46,400 - > 00:02:48,319 outcomes in sort of efficiency. 54 00:02:48,479 - > 00:02:51,439 Everyone could measure efficiency, which obviously is a 55 00:02:51,439 - > 00:02:52,639 pretty simple thing, right? 56 00:02:52,800 - > 00:02:57,439 We got 25 points done in this sprint versus our normal 50, 57 00:02:57,520 - > 00:02:58,159 whatever, right? 58 00:02:58,240 - > 00:02:59,360 So that's an improvement. 59 00:02:59,439 - > 00:03:02,240 But did that really drive ROI to the business? 60 00:03:02,400 - > 00:03:06,719 That was one big question that came up across all roundtables. 61 00:03:06,960 - > 00:03:09,439 The third, which was really interesting, I thought we were 62 00:03:09,439 - > 00:03:15,120 going to get a ton of different ideas or inspiration on where AI 63 00:03:15,120 - > 00:03:18,080 was making a huge difference in efficiency. 64 00:03:18,159 - > 00:03:24,240 And what we saw was prototyping was the number one sort of task 65 00:03:24,240 - > 00:03:29,360 in the PDLC that was across the board where people were 66 00:03:29,360 - > 00:03:31,599 standardizing on AI, but nowhere else. 67 00:03:31,759 - > 00:03:35,199 Everything was just still experimental, trying different 68 00:03:35,199 - > 00:03:39,840 things, failing at different things, and certainly not seeing 69 00:03:39,840 - > 00:03:42,960 huge benefits from AI other than in prototyping. 70 00:03:43,120 - > 00:03:46,719 They're using prototyping now, these AI prototypes as stage 71 00:03:46,719 - > 00:03:49,840 gates in their PDLC, which I thought was super interesting. 72 00:03:50,080 - > 00:03:52,800 And then I think we'll talk about this more today, which is 73 00:03:52,800 - > 00:03:54,319 just role anxiety, right? 74 00:03:54,400 - > 00:03:56,960 I think the definition of product management specifically 75 00:03:56,960 - > 00:03:58,319 is being blurred. 76 00:03:58,560 - > 00:04:00,719 I think everyone is now a builder. 77 00:04:00,960 - > 00:04:05,520 And so being able to figure out as a product leader, what is my 78 00:04:05,520 - > 00:04:05,759 role? 79 00:04:05,919 - > 00:04:07,120 What is my team's role now? 80 00:04:07,280 - > 00:04:11,520 How do I ensure that we're harnessing the power of AI, but 81 00:04:11,520 - > 00:04:16,160 still yielding the outcomes in customer delight and business 82 00:04:16,160 - > 00:04:20,000 growth from this acceleration and from this new technology, 83 00:04:20,079 - > 00:04:20,319 right? 84 00:04:20,399 - > 00:04:21,680 How do I do that as a leader? 85 00:04:21,839 - > 00:04:22,560 What's my role? 86 00:04:22,639 - > 00:04:25,920 And how do I define the definition of success of product 87 00:04:25,920 - > 00:04:26,240 management? 88 00:04:26,399 - > 00:04:28,319 So I think those were the really big themes. 89 00:04:28,560 - > 00:04:29,680 SPEAKER_00: You know, it's so interesting. 90 00:04:29,759 - > 00:04:33,120 Just the consistency of those roundtables and what we've been 91 00:04:33,120 - > 00:04:35,519 hearing when we've been talking to product teams and product 92 00:04:35,519 - > 00:04:36,000 leaders. 93 00:04:36,240 - > 00:04:37,920 We're going to dive into all of these. 94 00:04:38,079 - > 00:04:39,519 I just want to give a quick recap. 95 00:04:39,680 - > 00:04:42,639 I did just do one round table because I wanted to focus on 96 00:04:42,639 - > 00:04:45,439 what's the future of product management now. 97 00:04:45,680 - > 00:04:48,160 It was really interesting because everyone came in with 98 00:04:48,240 - > 00:04:49,680 like, where are we going? 99 00:04:49,839 - > 00:04:50,879 What is our industry? 100 00:04:50,959 - > 00:04:52,800 And again, this is a leadership group. 101 00:04:52,879 - > 00:04:55,439 So interesting are the themes that came out here. 102 00:04:55,680 - > 00:04:59,439 The first, there's a real concern, I think, as more senior 103 00:04:59,439 - > 00:05:00,000 leaders. 104 00:05:00,160 - > 00:05:02,000 It's my way of saying we're old, Heather. 105 00:05:02,399 - > 00:05:03,279 I already knew that video. 106 00:05:05,360 - > 00:05:08,639 I think we kind of grew up where there was a lot of mentoring. 107 00:05:08,800 - > 00:05:12,160 We grew up coaching our product teams, and gosh, we're product 108 00:05:12,160 - > 00:05:14,560 coaches now because we take this so seriously. 109 00:05:14,800 - > 00:05:18,879 And we're used to folks coming in, and maybe product managers 110 00:05:18,879 - > 00:05:19,920 weren't straight out of school. 111 00:05:20,079 - > 00:05:22,800 They came out of MBA or they came out of a different part of 112 00:05:22,800 - > 00:05:24,959 the industry, but they had experience. 113 00:05:25,120 - > 00:05:27,920 And there was this idea that you're starting from scratch. 114 00:05:28,079 - > 00:05:31,439 And so, where is that talent pipeline going now? 115 00:05:31,839 - > 00:05:35,279 And across the board, across everyone in the roundtable, they 116 00:05:35,600 - > 00:05:39,360 said they are cutting intern programs completely. 117 00:05:39,600 - > 00:05:42,800 There's very few junior PMs coming in. 118 00:05:42,959 - > 00:05:46,720 There's an expectation that, you know, that is going to be done 119 00:05:46,720 - > 00:05:47,519 elsewhere. 120 00:05:47,759 - > 00:05:51,680 And so as we think about this, and I think we're all so proud 121 00:05:51,680 - > 00:05:55,120 of people that were on our teams, that are now in VP roles 122 00:05:55,120 - > 00:05:56,800 and chief product officer roles. 123 00:05:56,959 - > 00:06:01,120 Well, who's training the CPOs in 2030? 124 00:06:01,279 - > 00:06:03,439 So that was one of the things that we spent a lot of time 125 00:06:03,439 - > 00:06:04,079 talking about. 126 00:06:04,319 - > 00:06:07,199 As we were talking about this, another theme came up, which is 127 00:06:07,279 - > 00:06:10,240 we've always talked about product managers, is we're 128 00:06:10,240 - > 00:06:12,879 connected, we're the connective tissue. 129 00:06:13,120 - > 00:06:17,040 And this term was coined called the cardiovascular PM. 130 00:06:17,120 - > 00:06:20,720 We talked about PM now being the heartbeat, which I think is so 131 00:06:20,720 - > 00:06:21,680 interesting, right? 132 00:06:21,839 - > 00:06:25,279 Because it's like directing the flow, it's setting the heartbeat 133 00:06:25,279 - > 00:06:25,680 of the product. 134 00:06:25,759 - > 00:06:27,360 And I love it for a couple of reasons. 135 00:06:27,600 - > 00:06:32,160 One, because it shows that it's so much more pervasive in terms 136 00:06:32,160 - > 00:06:36,319 of like where you are, but also just in terms of heart, you are 137 00:06:36,319 - > 00:06:38,399 the heart of the product in more ways than one. 138 00:06:38,560 - > 00:06:42,079 So that was a fun term that came up spontaneously, which was 139 00:06:42,079 - > 00:06:42,639 really interesting. 140 00:06:42,720 - > 00:06:44,160 So the future of product management. 141 00:06:44,240 - > 00:06:46,800 So the heartbeat is absolutely necessary. 142 00:06:47,040 - > 00:06:49,600 And then we got it a little more on the downside. 143 00:06:49,759 - > 00:06:53,839 The first is around, gosh, AI is getting us to, and I think this 144 00:06:53,839 - > 00:06:56,160 is starting to cross into the themes that you heard, Heather. 145 00:06:56,319 - > 00:06:58,079 But AI is getting us to 80%. 146 00:06:58,720 - > 00:07:00,639 But what 80% is it? 147 00:07:00,800 - > 00:07:04,079 Because at 20%, it's not really the finish line. 148 00:07:04,240 - > 00:07:07,040 It's happening and it's missing at the beginning. 149 00:07:07,199 - > 00:07:10,879 It's missing in the foundations, it's missing in setting the 150 00:07:10,879 - > 00:07:11,519 right directions. 151 00:07:11,600 - > 00:07:12,879 So we're moving really fast. 152 00:07:13,040 - > 00:07:17,040 AI can get us to 80%, but don't think about this as the first 153 00:07:17,040 - > 00:07:17,519 80%. 154 00:07:18,000 - > 00:07:20,720 And so that role of product management, the future of 155 00:07:20,720 - > 00:07:24,399 product management, understanding how to leverage 156 00:07:24,399 - > 00:07:29,759 that 20%, where we're absolutely vital in that, again, connective 157 00:07:29,759 - > 00:07:30,319 tissue. 158 00:07:30,560 - > 00:07:33,519 And then the last one, and I think this again speaks to the 159 00:07:33,519 - > 00:07:35,839 future of product management, which I'll give you the big 160 00:07:35,839 - > 00:07:39,759 reveal, is like we all absolutely believe it's strong, 161 00:07:39,920 - > 00:07:42,160 it's as important, if not more important. 162 00:07:42,480 - > 00:07:45,120 And I've started to see over the last couple of weeks some 163 00:07:45,120 - > 00:07:49,439 articles around product is the career of the future, which I 164 00:07:49,439 - > 00:07:53,360 think is maybe it's just someone making us all feel better, but I 165 00:07:53,360 - > 00:07:53,920 believe it. 166 00:07:54,079 - > 00:07:57,600 And this idea that you know prompting like is table stakes, 167 00:07:57,759 - > 00:07:58,000 right? 168 00:07:58,240 - > 00:08:02,079 But judgment and discernment is really the skill. 169 00:08:02,319 - > 00:08:06,160 And that requires a human, it requires someone and product, it 170 00:08:06,160 - > 00:08:08,480 understands someone thinking about the end-to-end. 171 00:08:08,959 - > 00:08:13,120 It's understanding the cost structure, it's understanding 172 00:08:13,120 - > 00:08:16,319 all the different pieces of it, knowing where the human needs to 173 00:08:16,319 - > 00:08:19,759 be, understanding the customer at the beginning, at the end, 174 00:08:19,920 - > 00:08:20,800 all the way through. 175 00:08:20,959 - > 00:08:23,839 So those are some of the themes that came up for the future of 176 00:08:23,839 - > 00:08:24,720 product management. 177 00:08:24,879 - > 00:08:27,439 But I think I want to start diving into some of the things 178 00:08:27,439 - > 00:08:30,079 that Heather first started talking about, which is really 179 00:08:30,240 - > 00:08:33,120 this, you know, I love this term, and I slop. 180 00:08:33,360 - > 00:08:34,639 It's such a so sad. 181 00:08:34,799 - > 00:08:36,240 It's so sad, but it's so real. 182 00:08:36,960 - > 00:08:38,799 And it's also so descriptive. 183 00:08:38,960 - > 00:08:41,440 When Heather talked about her sessions and the conversations 184 00:08:41,440 - > 00:08:45,279 that we had afterwards, so many leaders described PMs producing 185 00:08:45,279 - > 00:08:50,320 these huge documents that nobody read, these PRDs full of jargon 186 00:08:50,320 - > 00:08:51,279 that nobody really explained. 187 00:08:51,519 - > 00:08:54,399 I don't know who's reading these, and prototypes that 188 00:08:54,399 - > 00:08:57,200 didn't necessarily have real customer grounding. 189 00:08:57,360 - > 00:09:00,480 I remember you saying that one leader actually said, So what do 190 00:09:00,480 - > 00:09:01,679 you think we should do? 191 00:09:01,840 - > 00:09:03,759 And they got nothing back. 192 00:09:04,159 - > 00:09:07,759 SPEAKER_02: So literally the product manager was like, you 193 00:09:07,759 - > 00:09:11,120 know, no answer because it was like, well, AI produced this and 194 00:09:11,120 - > 00:09:11,679 it looked good. 195 00:09:11,759 - > 00:09:12,159 And yeah. 196 00:09:12,399 - > 00:09:13,759 SPEAKER_00: Yeah, nothing to do. 197 00:09:14,080 - > 00:09:14,879 So here we are. 198 00:09:15,039 - > 00:09:18,159 And with leaders, when they're building their own persona, 199 00:09:18,320 - > 00:09:22,080 their prompt libraries, their bots individually because 200 00:09:22,080 - > 00:09:23,279 they're frustrated. 201 00:09:23,519 - > 00:09:24,960 This is a pretty scary system. 202 00:09:25,120 - > 00:09:28,159 So I want you to talk a little bit about this idea about AI 203 00:09:28,159 - > 00:09:29,679 slop across your sessions. 204 00:09:29,919 - > 00:09:31,679 Do you think it's really a skills gap? 205 00:09:31,759 - > 00:09:34,159 Is that what came out in individual PMs? 206 00:09:34,320 - > 00:09:37,519 Or is it something more structural about how teams are 207 00:09:37,519 - > 00:09:38,720 being set up right now? 208 00:09:39,039 - > 00:09:40,320 SPEAKER_02: Gosh, it's a good question. 209 00:09:40,480 - > 00:09:41,360 I think it's both. 210 00:09:41,600 - > 00:09:45,360 If I think about some of the challenges we hear, you've got 211 00:09:45,360 - > 00:09:46,159 this tension. 212 00:09:46,320 - > 00:09:50,320 You've got senior leadership saying, How are you AI'sing? 213 00:09:50,799 - > 00:09:53,360 It's a technical term, your organization. 214 00:09:53,519 - > 00:09:55,919 What are you doing to harness AI? 215 00:09:56,080 - > 00:10:00,399 And so you have all this experimentation and utilizing AI 216 00:10:00,399 - > 00:10:03,759 for absolutely everything you possibly can, in some cases, 217 00:10:03,840 - > 00:10:07,279 just to say, hey, this was AI driven and producing as fast as 218 00:10:07,279 - > 00:10:08,480 you possibly can, right? 219 00:10:08,720 - > 00:10:12,960 And that tension then doesn't necessarily mean that we're 220 00:10:12,960 - > 00:10:16,720 building the skills to utilize AI in a way that produces the 221 00:10:16,720 - > 00:10:17,440 best outcomes. 222 00:10:17,600 - > 00:10:22,399 And so you see a lot of product managers experimenting, quickly 223 00:10:22,399 - > 00:10:26,639 throwing in prompts and getting responses and saying, hey, it 224 00:10:26,639 - > 00:10:27,600 sounds good, right? 225 00:10:27,679 - > 00:10:31,919 We've even seen research that the more polished the output 226 00:10:31,919 - > 00:10:36,480 sounds, the more confident you are in the output, even when 227 00:10:36,480 - > 00:10:37,360 it's not right. 228 00:10:37,519 - > 00:10:40,000 And so I think you're seeing this need for speed. 229 00:10:40,159 - > 00:10:41,200 I feel this need for speed. 230 00:10:41,279 - > 00:10:42,159 What movie is that from? 231 00:10:42,320 - > 00:10:45,600 Anyway, so they want speed as fast as they can. 232 00:10:45,840 - > 00:10:49,600 They're trying all these tools as fast as they can and not 233 00:10:49,600 - > 00:10:55,200 necessarily seeing the output that is representative of the 234 00:10:55,200 - > 00:10:58,320 context that all of these product managers have, right? 235 00:10:58,399 - > 00:11:01,039 They have the customer context, they're closest to the customer, 236 00:11:01,120 - > 00:11:03,279 they're closest to the business, they understand the 237 00:11:03,279 - > 00:11:06,000 idiosyncrasies of the market and the like. 238 00:11:06,159 - > 00:11:09,919 Whereas LLMs are all just pattern recognizers and pattern 239 00:11:09,919 - > 00:11:10,480 finishers. 240 00:11:10,639 - > 00:11:14,639 And so if you treat the LLM as the answer man, you're gonna get 241 00:11:14,639 - > 00:11:19,840 the slop without the skill sets to best prompt it and review 242 00:11:19,840 - > 00:11:20,480 what you get. 243 00:11:20,639 - > 00:11:23,200 And so I think it's a combination of that tension from 244 00:11:23,200 - > 00:11:27,120 leadership and then this overzealous need to try these 245 00:11:27,120 - > 00:11:30,480 new tools as fast as you possibly can and show the 246 00:11:30,480 - > 00:11:34,240 production of what you've done without really taking the time 247 00:11:34,240 - > 00:11:36,639 to learn how to use them to get the best outcomes. 248 00:11:37,200 - > 00:11:38,240 That's great. 249 00:11:38,559 - > 00:11:43,039 So I think you surfaced a quality trap too in your 250 00:11:43,039 - > 00:11:44,080 roundtables. 251 00:11:44,240 - > 00:11:48,080 And so everyone wants to be a builder. 252 00:11:48,399 - > 00:11:52,960 PMs are building, they're designing, they're writing code 253 00:11:52,960 - > 00:11:54,559 with the different tools. 254 00:11:54,799 - > 00:11:59,279 Talk to me a little bit about this quality trap that you 255 00:11:59,279 - > 00:11:59,840 discovered. 256 00:11:59,919 - > 00:12:02,000 And is that different from what we just talked about? 257 00:12:02,159 - > 00:12:06,320 This AI slop and this 80% versus 20% is right on. 258 00:12:06,399 - > 00:12:09,440 But how did you hear it come out and how is that manifested? 259 00:12:09,600 - > 00:12:11,919 What the leaders are doing, and how do you prevent it? 260 00:12:12,080 - > 00:12:13,440 SPEAKER_00: Yeah, it's such a great question. 261 00:12:13,519 - > 00:12:16,480 And I think it sort of builds, I think, on that AI slop 262 00:12:16,480 - > 00:12:17,840 conversation we just had. 263 00:12:18,080 - > 00:12:20,240 I think that one of the things which is going to warm your 264 00:12:20,240 - > 00:12:23,039 heart, Heather, because it certainly warmed mine, was 265 00:12:23,279 - > 00:12:29,279 really reinforcing this idea about understanding why, having 266 00:12:29,279 - > 00:12:34,240 PMs be in a position that they could really describe the why 267 00:12:34,240 - > 00:12:36,240 behind some of these artifacts. 268 00:12:36,480 - > 00:12:39,600 So moving, I think, okay, we don't necessarily need to author 269 00:12:39,600 - > 00:12:40,639 all of these documents. 270 00:12:40,799 - > 00:12:44,639 I think we constantly teach, start with AI, that's fine, 271 00:12:44,799 - > 00:12:48,559 input the material, and a first draft can be AI. 272 00:12:48,720 - > 00:12:52,320 We don't have to be the authors, but this idea of us turning into 273 00:12:52,320 - > 00:12:54,399 something like an editor-in-chief, right? 274 00:12:54,559 - > 00:13:00,720 And so this ability to question, to be able to really defend why, 275 00:13:01,039 - > 00:13:03,360 why are you making the statement? 276 00:13:03,519 - > 00:13:07,759 What's the sentiment, the insight, the customer learning 277 00:13:07,759 - > 00:13:08,480 behind it? 278 00:13:08,720 - > 00:13:12,399 This difference between this 8020, those are the places. 279 00:13:12,480 - > 00:13:16,960 And you can start to see how we started to say, where are those 280 00:13:16,960 - > 00:13:17,840 intersections? 281 00:13:18,000 - > 00:13:20,159 Where are you, the editor-in-chief, that you're 282 00:13:20,159 - > 00:13:20,799 stepping in? 283 00:13:20,960 - > 00:13:23,600 You're not making the first draft, but you're sure as heck 284 00:13:23,679 - > 00:13:27,360 aren't sending that to your CPO who's like, now what do I do 285 00:13:27,360 - > 00:13:27,919 with this? 286 00:13:28,240 - > 00:13:32,080 Without actually understanding and being able to explain the 287 00:13:32,080 - > 00:13:32,799 why behind it. 288 00:13:32,960 - > 00:13:37,039 We spent a lot of time talking about that role and that ability 289 00:13:37,039 - > 00:13:38,480 to be able to question. 290 00:13:38,720 - > 00:13:42,480 The other thing that I think was interesting around this AD20 was 291 00:13:42,480 - > 00:13:46,559 being able to double down on something that's, I think maybe 292 00:13:46,559 - > 00:13:50,480 the best way to describe it is like mucky customer evidence, 293 00:13:50,720 - > 00:13:51,039 right? 294 00:13:51,279 - > 00:13:56,960 When you have unreal or synthetic personas, when you 295 00:13:56,960 - > 00:14:00,960 are, I don't want to say making up the insight, but you're not 296 00:14:00,960 - > 00:14:03,600 really using primary research. 297 00:14:03,919 - > 00:14:07,200 When you can then mandate, we talked a lot about mandating 298 00:14:07,200 - > 00:14:08,320 primary research. 299 00:14:08,480 - > 00:14:10,720 And this is a non-negotiable stake. 300 00:14:10,879 - > 00:14:14,080 You've still got to be close to your customer, you've still got 301 00:14:14,080 - > 00:14:17,120 to be as in your learning plans, understanding. 302 00:14:17,279 - > 00:14:20,000 And that again, use AI by all means. 303 00:14:20,080 - > 00:14:24,799 This is the 80% to summarize, to give you synthesis, to help you 304 00:14:24,799 - > 00:14:25,600 with framing. 305 00:14:25,840 - > 00:14:29,440 However, you've got to be able to defend and answer the why. 306 00:14:29,600 - > 00:14:32,159 So those are the couple of the things that we really talked 307 00:14:32,159 - > 00:14:32,320 about. 308 00:14:32,480 - > 00:14:35,519 And then I think the last one I probably quickly mentioned, and 309 00:14:35,519 - > 00:14:39,039 I think it speaks to that product leader who said, I'm 310 00:14:39,039 - > 00:14:41,200 going to create this persona. 311 00:14:41,360 - > 00:14:42,480 We've heard this before. 312 00:14:42,639 - > 00:14:46,320 We've heard about product leaders saying, these are the 313 00:14:46,320 - > 00:14:49,360 templates, these are the standardized frameworks. 314 00:14:49,600 - > 00:14:53,600 We're not going to go and all recreate the wheel or all do 315 00:14:53,600 - > 00:14:54,320 different things. 316 00:14:54,480 - > 00:14:55,519 What's that structure? 317 00:14:55,679 - > 00:14:58,799 If you don't have that structure, go find one, go work 318 00:14:58,799 - > 00:15:02,320 with people like us who can give you a great one, but don't let 319 00:15:02,320 - > 00:15:04,559 everyone kind of like do it on their own. 320 00:15:04,720 - > 00:15:06,399 This is still a team sport. 321 00:15:06,559 - > 00:15:09,440 The leaders in the room are like, even though things are 322 00:15:09,440 - > 00:15:12,480 changing fast, those foundations, and again, I think 323 00:15:12,480 - > 00:15:15,840 it was just so rewarding to talk about sort of those foundations 324 00:15:15,840 - > 00:15:18,639 and the customer being so critical for us to be able to be 325 00:15:18,639 - > 00:15:19,120 effective. 326 00:15:19,279 - > 00:15:21,200 So those are a couple of themes there. 327 00:15:23,360 - > 00:15:28,799 I want to move now to this idea about value and outputs and 328 00:15:28,799 - > 00:15:29,200 value. 329 00:15:29,360 - > 00:15:32,480 So one of the things that I remember you saying was this 330 00:15:32,480 - > 00:15:36,559 idea about AI connecting to business outcomes and you know, 331 00:15:36,720 - > 00:15:39,679 your your note about sort of the story points. 332 00:15:39,919 - > 00:15:45,120 And I've heard several product leaders say to me, we have made 333 00:15:45,120 - > 00:15:48,559 this investment and it's ending up costing us more. 334 00:15:48,799 - > 00:15:52,879 They really didn't think through sort of that end-to-end cost 335 00:15:52,879 - > 00:15:56,559 output and what really was the business outcome, to the point 336 00:15:56,559 - > 00:15:59,200 where I think you had someone say they would never have 337 00:15:59,200 - > 00:16:01,039 launched if they'd understood this. 338 00:16:01,279 - > 00:16:04,080 So when you think about this, like when leaders go to their 339 00:16:04,080 - > 00:16:07,919 boards, because there's so much pressure for going fast and 340 00:16:07,919 - > 00:16:11,440 building more effectively and getting these prototypes out and 341 00:16:11,440 - > 00:16:12,480 cutting people. 342 00:16:12,799 - > 00:16:15,120 Like, what do you think they're actually showing? 343 00:16:15,360 - > 00:16:18,559 And where's that gap between what they're showing and what 344 00:16:18,559 - > 00:16:19,679 the board wants to see? 345 00:16:19,840 - > 00:16:21,600 What was the conversation around that like? 346 00:16:21,919 - > 00:16:23,840 SPEAKER_02: Gosh, I think there's two separate 347 00:16:23,840 - > 00:16:24,879 conversations, right? 348 00:16:25,039 - > 00:16:27,360 One is really around the internal efficiencies. 349 00:16:27,440 - > 00:16:33,840 How do we leverage AI and how do we define or measure the ROI of 350 00:16:33,840 - > 00:16:38,639 infusing AI into the system beyond just tackling more story 351 00:16:38,639 - > 00:16:39,279 points, right? 352 00:16:39,519 - > 00:16:46,399 And the second conversation is how do we measure success of the 353 00:16:46,399 - > 00:16:48,559 AI products that we build? 354 00:16:48,639 - > 00:16:51,840 And how do we course correct as quickly as we possibly can to 355 00:16:51,840 - > 00:16:54,879 ensure that we're achieving the outcomes that we think we're 356 00:16:54,879 - > 00:16:56,240 trying to solve for, right? 357 00:16:56,559 - > 00:17:00,879 I would say the latter discussion is an age-old problem 358 00:17:00,879 - > 00:17:03,759 that we have had since day one in product management. 359 00:17:03,919 - > 00:17:07,759 It's how do you get aligned on the definition or measures of 360 00:17:07,759 - > 00:17:10,000 success before you launch, right? 361 00:17:10,160 - > 00:17:13,440 One of the examples that we talked about in a round table 362 00:17:13,440 - > 00:17:15,680 was launching this chatbot for support. 363 00:17:15,839 - > 00:17:18,079 It was augmenting their technical support. 364 00:17:18,319 - > 00:17:21,920 And because they hadn't articulated what the definition 365 00:17:21,920 - > 00:17:26,480 of success was that first contact resolution, was that the 366 00:17:26,480 - > 00:17:29,680 number of times a customer needed to be handed off to a 367 00:17:29,680 - > 00:17:30,480 live person? 368 00:17:30,799 - > 00:17:35,359 What are the measurements that really do tell us that the ROI 369 00:17:35,359 - > 00:17:36,319 is there, right? 370 00:17:36,720 - > 00:17:41,119 Because that wasn't set up front, then everything after 371 00:17:41,119 - > 00:17:43,759 launch, there was no measurements, operating 372 00:17:43,759 - > 00:17:48,880 mechanisms, or understanding of the implications of that tool, 373 00:17:49,039 - > 00:17:52,640 the chatbot, and whether or not it was actually generating ROI 374 00:17:52,640 - > 00:17:53,279 for the company. 375 00:17:53,440 - > 00:17:55,440 And so to me, this is the age-old problem. 376 00:17:55,599 - > 00:17:56,720 It hasn't changed. 377 00:17:56,960 - > 00:18:00,799 It may change in terms of the criticality of defining it 378 00:18:00,799 - > 00:18:05,200 before you launch, because your learning and pivoting with an AI 379 00:18:05,200 - > 00:18:07,519 product happens usually right after launch. 380 00:18:07,599 - > 00:18:10,640 It's not going to happen prior through concept testing or the 381 00:18:10,640 - > 00:18:10,880 like. 382 00:18:11,039 - > 00:18:13,839 It's going to happen when it's in reality and people are 383 00:18:13,839 - > 00:18:17,119 actually utilizing it for the tasks that you're asking it to 384 00:18:17,119 - > 00:18:17,359 use. 385 00:18:17,519 - > 00:18:18,799 And how accurate are you? 386 00:18:18,880 - > 00:18:21,440 And how much trust are you building and the like. 387 00:18:21,759 - > 00:18:26,640 So the dynamic of how we do this and how we measure ROI has 388 00:18:26,640 - > 00:18:31,119 changed a bit with AI, but it hasn't changed the need to get 389 00:18:31,279 - > 00:18:33,920 alignment up front, which is what you and I do all the time, 390 00:18:34,079 - > 00:18:35,599 Vidya, in product rebels, right? 391 00:18:35,680 - > 00:18:39,279 We really focus on establishing the measurements that matter 392 00:18:39,440 - > 00:18:43,759 early and make sure you're instrumented before you even go 393 00:18:43,759 - > 00:18:47,359 live, even if it's beta, alpha, whatever that may be, right? 394 00:18:47,440 - > 00:18:51,519 So I feel like that is still the age-old problem that I don't 395 00:18:51,519 - > 00:18:54,640 think we realize the implications of not doing it are 396 00:18:54,880 - > 00:18:57,519 even more painful once you launch AI. 397 00:18:57,599 - > 00:19:02,000 Because this particular leader struggled because trust went 398 00:19:02,000 - > 00:19:02,240 down. 399 00:19:02,480 - > 00:19:08,079 The brand identity and value went down because they didn't 400 00:19:08,079 - > 00:19:10,240 have those measures of success early on. 401 00:19:10,400 - > 00:19:14,400 They didn't know how the AI was solving the problem that they 402 00:19:14,400 - > 00:19:17,359 set out to solve, how well it was being solved, and then the 403 00:19:17,359 - > 00:19:21,200 collateral damage of when you lose trust and the like through 404 00:19:21,200 - > 00:19:22,160 an AI product. 405 00:19:22,559 - > 00:19:27,279 So I feel like that one, that RLI discussion, unfortunately, 406 00:19:27,440 - > 00:19:30,400 is an age-old problem that I think it's just more critical 407 00:19:30,400 - > 00:19:33,359 now and in a different dynamic that we're trying to solve for. 408 00:19:33,680 - > 00:19:36,880 Internal efficiency, I think, is we're still not connecting the 409 00:19:36,880 - > 00:19:39,440 dots to the quality of what's being produced, right? 410 00:19:39,519 - > 00:19:43,440 Yes, the speed is there, but is this AI slop, has that taken 411 00:19:43,440 - > 00:19:43,680 over? 412 00:19:43,839 - > 00:19:47,759 And what we release, is that still really the quality? 413 00:19:47,920 - > 00:19:48,880 Is the delight? 414 00:19:49,039 - > 00:19:50,400 Is it achieving the outcomes? 415 00:19:50,559 - > 00:19:53,119 Do we still have the early indicators of achieving the 416 00:19:53,119 - > 00:19:54,480 outcomes that we need to? 417 00:19:54,720 - > 00:19:58,240 I just think that one is just connecting the dots to more of 418 00:19:58,480 - > 00:19:59,680 beyond efficiency. 419 00:19:59,759 - > 00:20:02,799 It's really about making sure you're balancing the efficiency 420 00:20:02,799 - > 00:20:07,440 aspect of infusing AI and the customer outcomes that you're 421 00:20:07,440 - > 00:20:10,480 trying to solve for through the products that you're still 422 00:20:10,480 - > 00:20:10,880 building. 423 00:20:11,039 - > 00:20:12,640 So yeah, it's tough. 424 00:20:12,799 - > 00:20:16,720 And it's just more critical now because we move at lightning 425 00:20:16,720 - > 00:20:21,599 speed and the cost of launching something and failing that much 426 00:20:21,599 - > 00:20:24,559 faster with that much more collateral damage, I think, is 427 00:20:24,559 - > 00:20:25,279 just more painful. 428 00:20:25,680 - > 00:20:27,359 So, how did this come up in your discussion? 429 00:20:27,519 - > 00:20:30,799 I feel like it came up in every discussion, this concept of ROI 430 00:20:30,799 - > 00:20:34,319 and how we present this to our boards of here's the investments 431 00:20:34,319 - > 00:20:36,640 we've made, here's the implications it's had on the 432 00:20:36,640 - > 00:20:38,240 business beyond story points. 433 00:20:38,400 - > 00:20:41,519 How did it come up in this future of product management for 434 00:20:41,519 - > 00:20:41,680 you? 435 00:20:41,920 - > 00:20:42,799 SPEAKER_00: Yeah, it's interesting. 436 00:20:42,960 - > 00:20:46,319 The way that I think we talked a little bit more about it was 437 00:20:46,640 - > 00:20:48,960 when and how to use AI. 438 00:20:49,119 - > 00:20:51,839 And because we're really talking about the future of product 439 00:20:51,839 - > 00:20:55,599 management, the conversation, rather than very specific 440 00:20:55,599 - > 00:20:59,920 examples such as you gave, was more around sort of decision 441 00:20:59,920 - > 00:21:04,799 making and this idea that how you both show up never to 442 00:21:04,799 - > 00:21:09,119 delegate fully, but also how you are coaching your teens never to 443 00:21:09,119 - > 00:21:10,160 delegate fully. 444 00:21:10,400 - > 00:21:13,440 So when you're thinking about outcomes, you have to be 445 00:21:13,440 - > 00:21:17,759 constantly thinking about, you have to understand what you're 446 00:21:17,759 - > 00:21:18,559 solving for. 447 00:21:18,799 - > 00:21:22,319 You have to understand what insights came into prioritizing 448 00:21:22,319 - > 00:21:23,759 this specific area. 449 00:21:23,839 - > 00:21:26,720 So when you go into the boards and you're justifying it, you're 450 00:21:26,720 - > 00:21:28,559 still so important that you're rooted. 451 00:21:28,799 - > 00:21:30,880 You can see that we took a little bit of a different tone 452 00:21:31,039 - > 00:21:34,799 because I think it's almost a bit of a defensiveness of there 453 00:21:34,799 - > 00:21:36,480 is a future of product management. 454 00:21:36,640 - > 00:21:39,279 We all came in a little bit like, well, everyone else thinks 455 00:21:39,279 - > 00:21:40,240 so too, right? 456 00:21:40,400 - > 00:21:43,359 And so rather than specifically talk about necessarily ROI 457 00:21:43,359 - > 00:21:48,799 decisions, it very much focused on our role in ensuring that 458 00:21:48,799 - > 00:21:51,680 those business outcomes, and as you said, it's a problem we've 459 00:21:51,680 - > 00:21:52,559 always had. 460 00:21:52,799 - > 00:21:57,440 How do we maintain our ability to influence those types of 461 00:21:57,440 - > 00:21:58,480 decisions going forward? 462 00:21:58,640 - > 00:22:01,440 So a little bit of a different Spin on that question. 463 00:22:01,680 - > 00:22:03,359 SPEAKER_02: Yeah, but I think it's the right one, right? 464 00:22:03,440 - > 00:22:06,559 We as product leaders and product managers should always 465 00:22:06,559 - > 00:22:08,240 be the connective tissue. 466 00:22:08,400 - > 00:22:10,960 And I know I want to talk a little bit about this sort of 467 00:22:10,960 - > 00:22:14,559 switch from connective tissue to cardiovascular PM. 468 00:22:14,720 - > 00:22:16,880 I think that's a really interesting shift. 469 00:22:17,119 - > 00:22:21,920 But I feel like that's our job is to maintain the connection 470 00:22:21,920 - > 00:22:24,880 between what outcomes are we solving for, what problem are we 471 00:22:24,880 - > 00:22:25,279 solving for? 472 00:22:25,359 - > 00:22:28,079 Who are we solving that for, and making sure that whatever we do, 473 00:22:28,319 - > 00:22:32,720 whatever speed at which we do it in is connected back to those 474 00:22:32,720 - > 00:22:33,279 foundations. 475 00:22:33,519 - > 00:22:36,720 But let's talk about this idea of cardiovascular PM. 476 00:22:36,880 - > 00:22:41,599 I want to hear about that shift because there's still this 477 00:22:41,599 - > 00:22:44,000 concern that where is PM going? 478 00:22:44,160 - > 00:22:44,960 Do I have a role? 479 00:22:45,119 - > 00:22:48,319 Even some of the leaders in my roundtables had said, do I have 480 00:22:48,319 - > 00:22:49,039 a role here? 481 00:22:49,279 - > 00:22:53,519 How do I become a great product leader in this new era of AI? 482 00:22:53,759 - > 00:22:55,920 And your round table really talked about this idea of 483 00:22:55,920 - > 00:22:57,200 cardiovascular PM. 484 00:22:57,279 - > 00:22:58,720 How do we rationalize those two? 485 00:22:58,960 - > 00:23:00,480 SPEAKER_00: Yeah, no, it's interesting, isn't it? 486 00:23:00,559 - > 00:23:03,119 Like I think this is a lot of the dilemma that we as an 487 00:23:03,119 - > 00:23:04,319 industry are going through. 488 00:23:04,559 - > 00:23:06,000 You've got to be all of these things. 489 00:23:06,079 - > 00:23:08,480 You've got to be a builder, you've got to be everywhere, 490 00:23:08,640 - > 00:23:11,359 you've got to do everything yourself. 491 00:23:11,839 - > 00:23:13,680 And you are the connective tissue. 492 00:23:13,839 - > 00:23:17,839 Okay, in some ways, you could argue that we've always been 493 00:23:17,839 - > 00:23:20,400 that connective tissue now, the cardiovascular system. 494 00:23:20,640 - > 00:23:23,519 But now there's this question that you have to do it hands-on 495 00:23:23,680 - > 00:23:24,319 yourself. 496 00:23:24,559 - > 00:23:28,240 And so, you know, boy, I think we're all struggling with this. 497 00:23:28,319 - > 00:23:31,440 And I think the first thing for us to acknowledge, and for 498 00:23:31,440 - > 00:23:33,680 everyone listening, we're all in this together. 499 00:23:34,000 - > 00:23:39,119 There is not one product leader or product team that I would say 500 00:23:39,359 - > 00:23:41,440 has nailed this or understand this. 501 00:23:41,599 - > 00:23:44,079 We're all figuring this out at the same time. 502 00:23:44,240 - > 00:23:47,359 One of the things that you're going to hear from us and as 503 00:23:47,359 - > 00:23:50,799 product rebels is how do you hold on to those things that 504 00:23:50,799 - > 00:23:54,160 really matter as a core discipline of product? 505 00:23:54,400 - > 00:23:59,039 How do you ensure that you don't delegate those pieces, how you 506 00:23:59,039 - > 00:24:03,920 understand how to work in this ecosystem, how you use AI as a 507 00:24:03,920 - > 00:24:06,720 thought partner, not that decision maker. 508 00:24:06,880 - > 00:24:11,119 And then how do you increase the confidence in your decisions by 509 00:24:11,119 - > 00:24:12,400 the way that you interact? 510 00:24:12,559 - > 00:24:15,759 I think there's a lot of experimentation right now, which 511 00:24:15,759 - > 00:24:16,319 is great. 512 00:24:16,480 - > 00:24:17,839 Heather and I have both talked. 513 00:24:18,000 - > 00:24:21,039 We've gone back to building because you need to understand 514 00:24:21,039 - > 00:24:21,279 this. 515 00:24:21,440 - > 00:24:27,200 But this ability to really get fluent, to be able to move into 516 00:24:27,519 - > 00:24:30,240 this new role and this new way of being. 517 00:24:30,480 - > 00:24:34,720 While we don't actually have a clear vision of that yet, I 518 00:24:34,720 - > 00:24:37,920 think the the traits that really matter to us, that curiosity, 519 00:24:38,000 - > 00:24:41,519 and that is right now showing up in the way that we interact with 520 00:24:41,519 - > 00:24:44,799 tools, the way we interact, the way that we create prompts, the 521 00:24:44,799 - > 00:24:48,240 way that we're using AI as a strategic thought partner. 522 00:24:48,559 - > 00:24:52,160 Those are the things we need to lean into, not create 50-page 523 00:24:52,160 - > 00:24:54,240 PRDs that nobody ever reads, right? 524 00:24:54,480 - > 00:24:57,440 And we're seeing, I think one of the things that came up is 525 00:24:57,440 - > 00:25:00,559 there's definitely a fluency gap in a lot of organizations. 526 00:25:00,799 - > 00:25:05,200 There's a lot of surface level sort of interaction and playing 527 00:25:05,200 - > 00:25:09,440 in writing prompts, but they're not actually, there's a depth of 528 00:25:09,440 - > 00:25:10,960 understanding that is lacking. 529 00:25:11,200 - > 00:25:14,160 And I think that in order to truly feel like you're the 530 00:25:14,160 - > 00:25:17,119 heartbeat, like you're flowing through all the systems. 531 00:25:17,200 - > 00:25:19,680 You understand, maybe I'm kind of taking this analogy a little 532 00:25:19,680 - > 00:25:23,359 too far, but you're really having to understand what that 533 00:25:23,359 - > 00:25:28,720 interaction looks like and get hands on and then choose 534 00:25:28,720 - > 00:25:32,720 deliberately and strategically like where it is that you are 535 00:25:32,720 - > 00:25:34,079 going to intervene. 536 00:25:34,319 - > 00:25:36,559 Where are those places that you need to be? 537 00:25:36,720 - > 00:25:39,200 And so this, gosh, we could probably spend another hour 538 00:25:39,200 - > 00:25:42,000 talking about tool stack and teams and how to work on all of 539 00:25:42,000 - > 00:25:45,599 this, but really that conversation is we're all in 540 00:25:45,599 - > 00:25:48,880 trial mode right now and just know that you're experimenting. 541 00:25:49,039 - > 00:25:53,200 There's no specific, this is the new PM, whether it's a builder, 542 00:25:53,279 - > 00:25:54,880 with whatever that could look like. 543 00:25:55,039 - > 00:25:57,440 I think we all understand that we're in this and we're 544 00:25:57,440 - > 00:25:58,640 experimenting together. 545 00:25:58,880 - > 00:26:01,920 And I think learning from each other in these round tables, 546 00:26:02,079 - > 00:26:03,039 please do that. 547 00:26:03,200 - > 00:26:06,400 If you don't have a group to talk to, get in there because 548 00:26:06,400 - > 00:26:11,599 you can feel like either you are ahead of the curve and maybe 549 00:26:11,599 - > 00:26:14,400 you're not because you're not necessarily using the prompts 550 00:26:14,400 - > 00:26:17,519 directly, which is some of the conversations that we've had, or 551 00:26:17,519 - > 00:26:20,319 you feel like you're really behind, but you're not. 552 00:26:20,480 - > 00:26:22,400 The experimentation is just fine. 553 00:26:22,559 - > 00:26:25,359 So this turned into a little bit of a therapy session, but that's 554 00:26:25,359 - > 00:26:27,359 really kind of like a little bit where the conversation went, 555 00:26:27,440 - > 00:26:28,720 honestly, in these round tables. 556 00:26:28,880 - > 00:26:29,039 SPEAKER_02: Okay. 557 00:26:29,440 - > 00:26:32,000 And I love the idea of the cardiovascular PM. 558 00:26:32,240 - > 00:26:33,039 Here's one reason why. 559 00:26:33,200 - > 00:26:35,440 One of the things that we're seeing a lot among our clients, 560 00:26:35,519 - > 00:26:36,000 even, right? 561 00:26:36,160 - > 00:26:40,559 You know this, we've talked about this, is the PM being the 562 00:26:40,559 - > 00:26:44,400 facilitator of the best context for AI, right? 563 00:26:44,480 - > 00:26:47,119 And making sure their team members, whoever's building, 564 00:26:47,279 - > 00:26:50,000 right, whether it's the engineer, the designer, or even 565 00:26:50,000 - > 00:26:55,759 the PM, have we seeded our AI tools in a way that yields the 566 00:26:55,759 - > 00:26:59,599 best outcomes that are tied to the context, that are tied to 567 00:26:59,599 - > 00:27:01,279 the customer insight and the like. 568 00:27:01,440 - > 00:27:03,039 There's nobody really doing that right now. 569 00:27:03,200 - > 00:27:07,119 And so this experimentation yields empty output. 570 00:27:07,279 - > 00:27:10,960 And so that concept of, you know, the responsibility of the 571 00:27:10,960 - > 00:27:15,119 PM is to ensure that our tools are seated with the right 572 00:27:15,119 - > 00:27:19,519 insights, the right data, the right business context is 573 00:27:19,519 - > 00:27:20,319 actually huge. 574 00:27:20,480 - > 00:27:23,359 I feel like that's the blood flow throughout all of the AI 575 00:27:23,359 - > 00:27:24,240 tool stack, right? 576 00:27:24,319 - > 00:27:25,359 Anyway, I know I love that. 577 00:27:25,759 - > 00:27:28,240 SPEAKER_00: No, I know this is why I'm so glad that you picked 578 00:27:28,240 - > 00:27:30,480 up on that because I was like, gosh, I'm overusing this, but I 579 00:27:30,480 - > 00:27:31,599 also love that analogy. 580 00:27:32,000 - > 00:27:34,480 I want to talk about a little bit about building because I 581 00:27:34,480 - > 00:27:38,400 think one of the people in your session said, if we can build 582 00:27:38,400 - > 00:27:42,480 anything great, how about we don't build everything? 583 00:27:43,440 - > 00:27:44,240 So true. 584 00:27:44,480 - > 00:27:48,480 And along with that, I think if you vibe coded it, you maintain 585 00:27:48,480 - > 00:27:50,079 it, which boy, can you imagine that? 586 00:27:50,400 - > 00:27:50,960 I loved that. 587 00:27:51,119 - > 00:27:52,079 Oh my God, I love that. 588 00:27:52,160 - > 00:27:53,119 Isn't that so cool? 589 00:27:53,599 - > 00:27:54,720 SPEAKER_02: It's a big issue. 590 00:27:55,119 - > 00:27:57,599 And I will say that leaders are split. 591 00:27:58,079 - > 00:28:02,720 I heard some leaders talking about hey, building is so cheap 592 00:28:02,720 - > 00:28:02,880 now. 593 00:28:03,039 - > 00:28:04,480 Who cares if you make a mistake? 594 00:28:04,640 - > 00:28:05,920 Making mistakes are cheap. 595 00:28:06,079 - > 00:28:09,920 And I heard then another half of the room going, Are you kidding 596 00:28:09,920 - > 00:28:10,160 me? 597 00:28:10,400 - > 00:28:14,160 The second we build something and we take it to our legacy 598 00:28:14,160 - > 00:28:17,759 systems and we start investing in that, we are sunk. 599 00:28:17,920 - > 00:28:20,880 It's hard to unravel what we're doing, right? 600 00:28:21,039 - > 00:28:23,279 And so I think it depends on the business, right? 601 00:28:23,440 - > 00:28:27,119 Startups, a lot less to unravel, a lot cheaper to make mistakes. 602 00:28:27,279 - > 00:28:27,519 Great. 603 00:28:27,680 - > 00:28:28,319 I get that. 604 00:28:28,559 - > 00:28:31,440 But most companies out there that have been established that 605 00:28:31,440 - > 00:28:34,799 have some legacy system, even if it's two years old, three years 606 00:28:34,799 - > 00:28:38,960 old, four years old, mistakes can still be very expensive. 607 00:28:39,200 - > 00:28:42,799 And if you're building at light speed, all these different 608 00:28:42,799 - > 00:28:48,400 prototypes and really cool stuff, you're still gonna churn. 609 00:28:48,559 - > 00:28:53,119 And it's still an expensive proposition if you aren't taking 610 00:28:53,119 - > 00:28:57,839 the time to understand what's the right thing to do, what's 611 00:28:57,839 - > 00:29:01,759 the right problem to solve, and what is the thing that we should 612 00:29:01,759 - > 00:29:04,160 be doing to make the biggest impact in the market. 613 00:29:04,400 - > 00:29:07,039 So I think it's a balance and I think it's still mixed. 614 00:29:07,119 - > 00:29:08,480 I'm still a little worried about this. 615 00:29:08,640 - > 00:29:10,160 Well, making mistakes is cheap. 616 00:29:10,319 - > 00:29:14,640 I get it, but I don't think that's the case in many 617 00:29:14,640 - > 00:29:15,599 organizations. 618 00:29:15,839 - > 00:29:18,079 And so that critical thinking, that judgment, that 619 00:29:18,079 - > 00:29:20,799 prioritization is even more critical now because we're 620 00:29:20,799 - > 00:29:21,759 moving at light speed. 621 00:29:21,920 - > 00:29:24,480 SPEAKER_00: You know, I'm gonna go on a limb and I'm gonna say 622 00:29:24,640 - > 00:29:29,200 that's gonna bite people in that yes, A S S. 623 00:29:29,599 - > 00:29:30,640 You can say ass. 624 00:29:30,880 - > 00:29:31,680 Okay, thank you. 625 00:29:31,839 - > 00:29:35,599 In about three to six months because yeah, it's just like the 626 00:29:35,599 - > 00:29:36,240 prototypes. 627 00:29:36,319 - > 00:29:39,039 This isn't really anything more than sort of the A-B experiments 628 00:29:39,200 - > 00:29:39,759 we used to run. 629 00:29:39,839 - > 00:29:43,200 It's like until you said stop the madness, stop counting this 630 00:29:43,359 - > 00:29:46,079 back to outcomes, what's important, stop counting the 631 00:29:46,079 - > 00:29:47,440 number of experiments you're doing. 632 00:29:47,599 - > 00:29:50,640 Start figuring out are you actually achieving the goal that 633 00:29:50,640 - > 00:29:51,519 you're looking for? 634 00:29:51,839 - > 00:29:56,240 Here we're in this place where it's cheap to build, but boy, 635 00:29:56,400 - > 00:29:59,440 you've got your customer reputation, you've got 636 00:29:59,440 - > 00:30:02,880 experience, you've got loyalty, you've got expectations, you've 637 00:30:02,880 - > 00:30:04,960 got brand, you've got all of these things. 638 00:30:05,200 - > 00:30:07,920 You've got operations that have to operationalize what you're 639 00:30:07,920 - > 00:30:08,160 building. 640 00:30:08,240 - > 00:30:08,480 Yeah. 641 00:30:08,960 - > 00:30:09,920 Oh, you've got costs. 642 00:30:10,000 - > 00:30:11,599 I mean, we could go on and on. 643 00:30:11,759 - > 00:30:13,759 I mean, there are real repercussions. 644 00:30:13,920 - > 00:30:16,079 And I laugh too when they said you maintain it. 645 00:30:16,240 - > 00:30:18,640 One of the product leaders, um, this is not at the round table, 646 00:30:18,720 - > 00:30:20,079 but I think this is a really interesting story. 647 00:30:20,240 - > 00:30:22,960 One of a product leader I was talking to recently essentially 648 00:30:22,960 - > 00:30:25,599 said they couldn't get their CTO to build. 649 00:30:25,680 - > 00:30:27,359 And so they did it themselves. 650 00:30:27,599 - > 00:30:32,160 And I'm like, I get it from a let's show you how it's done, 651 00:30:32,319 - > 00:30:35,279 let's show you what I'm thinking about, let's get this on board. 652 00:30:35,519 - > 00:30:38,400 But at some point, this partnership, I think, between 653 00:30:38,400 - > 00:30:41,599 technology and product right now is a little fractured. 654 00:30:41,839 - > 00:30:46,319 And this ability to really make sure that again, you are 655 00:30:46,319 - > 00:30:48,799 partnering, you understand the repercussions, you understand 656 00:30:48,799 - > 00:30:49,519 the support. 657 00:30:49,759 - > 00:30:51,519 This is product 101. 658 00:30:51,680 - > 00:30:54,799 We never would have released something that couldn't be 659 00:30:54,799 - > 00:30:59,039 supported by our company, the infrastructure, but yet somehow 660 00:30:59,119 - > 00:31:01,519 we believe that we can just go build something and throw it 661 00:31:01,519 - > 00:31:02,480 over the wall right now. 662 00:31:02,720 - > 00:31:06,079 To me, luckily we didn't hear much of that in the round table, 663 00:31:06,160 - > 00:31:10,160 but I am hearing that outside of the group of product leaders 664 00:31:10,160 - > 00:31:11,440 that came into these sessions. 665 00:31:11,519 - > 00:31:12,720 Are you hearing something similar? 666 00:31:12,960 - > 00:31:16,000 SPEAKER_02: Yeah, what I'm hearing is even the product 667 00:31:16,000 - > 00:31:17,680 leaders are building their own stuff. 668 00:31:17,839 - > 00:31:21,279 And there was one like epiphany moment that I think we all had 669 00:31:21,279 - > 00:31:24,079 in one of the roundtables where the CEO came to the product 670 00:31:24,079 - > 00:31:26,000 leader and said, Hey, I think this is really cool. 671 00:31:26,079 - > 00:31:26,640 Let's build this. 672 00:31:26,799 - > 00:31:28,079 Can you ask routine bevel? 673 00:31:28,319 - > 00:31:30,240 He's like, better yet, I'll do it over the weekend. 674 00:31:30,480 - > 00:31:31,920 So he builds it over the weekend. 675 00:31:32,000 - > 00:31:33,599 He said it took him a couple days. 676 00:31:33,759 - > 00:31:35,839 He came back and he goes, Let me show you this. 677 00:31:36,000 - > 00:31:36,880 It's not a product. 678 00:31:37,039 - > 00:31:39,119 This is not gonna make big impact in the market. 679 00:31:39,279 - > 00:31:40,160 This is a cool idea. 680 00:31:40,319 - > 00:31:43,680 Yeah, I did it in two days, but I wouldn't put any resources 681 00:31:43,680 - > 00:31:44,240 against it. 682 00:31:44,400 - > 00:31:44,720 Right. 683 00:31:44,880 - > 00:31:49,920 So I see that sort of thinking and that, yes, we can build it, 684 00:31:50,079 - > 00:31:50,720 but should we? 685 00:31:50,880 - > 00:31:54,960 And so it's coming, but I still feel like there's this, I hate 686 00:31:54,960 - > 00:31:58,480 even saying this novelty of, oh my God, I can build it. 687 00:31:58,559 - > 00:31:59,839 So let's build everything. 688 00:32:00,240 - > 00:32:01,920 And the mistakes are cheap. 689 00:32:02,160 - > 00:32:05,200 I just feel like, like you said, I think it's gonna start biting 690 00:32:05,200 - > 00:32:07,839 people on the butt if they're not operating, like the product 691 00:32:07,839 - > 00:32:10,240 leader that I talked about, which he built it over the 692 00:32:10,240 - > 00:32:12,799 weekend, but realized this isn't actually a big impact. 693 00:32:12,960 - > 00:32:16,400 It's basically technology, which is cool, but it's not really a 694 00:32:16,400 - > 00:32:17,039 problem solver. 695 00:32:17,200 - > 00:32:17,279 Right. 696 00:32:17,599 - > 00:32:20,319 Get it up in the system, get it in your system. 697 00:32:20,559 - > 00:32:24,640 But the thing that I love about the speed and AI is you can 698 00:32:24,640 - > 00:32:28,720 build 20 different lo-fi concepts in minutes to solve a 699 00:32:28,720 - > 00:32:28,960 problem. 700 00:32:29,119 - > 00:32:30,000 Then go out and test it. 701 00:32:30,160 - > 00:32:30,799 Test it. 702 00:32:31,039 - > 00:32:34,720 But don't build the code and start trying to infuse it into 703 00:32:34,720 - > 00:32:36,960 your legacy systems and operations. 704 00:32:37,359 - > 00:32:37,920 Test it. 705 00:32:38,079 - > 00:32:41,039 So leverage AI for the speed to discovery. 706 00:32:41,200 - > 00:32:41,920 Awesome. 707 00:32:42,240 - > 00:32:46,640 But I'm just not there yet on, hey, let's just build 25 708 00:32:46,640 - > 00:32:50,000 features in a day and go launch them because it's low risk. 709 00:32:50,079 - > 00:32:50,720 I'm not there yet. 710 00:32:50,880 - > 00:32:53,200 SPEAKER_00: So let's carry on that theme and talk about when 711 00:32:53,200 - > 00:32:54,160 the rubber hits the road. 712 00:32:54,480 - > 00:32:57,839 Like actually generating new revenue from AI, right? 713 00:32:58,000 - > 00:33:01,759 Customers aren't going to pay extra for agents to use a 714 00:33:01,759 - > 00:33:03,680 product that already works for them. 715 00:33:04,000 - > 00:33:04,400 Yep. 716 00:33:04,559 - > 00:33:06,960 And this idea about building, like we're just going to 717 00:33:06,960 - > 00:33:10,640 continue this theme of why on earth would you just throw more 718 00:33:10,640 - > 00:33:12,720 agents onto existing SaaS platforms? 719 00:33:12,799 - > 00:33:15,119 And in fact, you're just saving your own cost structure. 720 00:33:15,359 - > 00:33:16,319 You're not generating more value. 721 00:33:16,480 - > 00:33:18,559 You're not generating value, you're not generating revenue. 722 00:33:18,720 - > 00:33:21,519 And in fact, if you don't do it properly, you can actually add 723 00:33:21,519 - > 00:33:22,799 cost to your bottom line, right? 724 00:33:22,960 - > 00:33:23,279 That's right. 725 00:33:23,599 - > 00:33:26,559 And it's this idea about, I'm not going to go as far as to say 726 00:33:26,559 - > 00:33:27,839 AI discovery doesn't work. 727 00:33:28,000 - > 00:33:30,240 I think there's a lot of really interesting things pulling 728 00:33:30,240 - > 00:33:33,759 customer information in, about being able to, as you said, 729 00:33:34,000 - > 00:33:36,640 generate a whole bunch of different ideas that you can 730 00:33:36,640 - > 00:33:38,240 test with and understand. 731 00:33:38,400 - > 00:33:40,079 That I think is really good. 732 00:33:40,319 - > 00:33:44,559 But this whole revenue problem feels like in your round tables, 733 00:33:44,640 - > 00:33:46,640 has anyone got an answer for this? 734 00:33:46,799 - > 00:33:48,240 How are they thinking about it? 735 00:33:48,480 - > 00:33:50,880 SPEAKER_02: No, I think there's a couple things. 736 00:33:51,119 - > 00:33:54,880 I think leaders have been faced with what you said. 737 00:33:55,119 - > 00:33:58,880 Customers are unwilling to pay more for the same service, even 738 00:33:58,880 - > 00:34:00,960 if the technology has changed, right? 739 00:34:01,200 - > 00:34:06,880 And so what I'm hearing is we have to understand the problem 740 00:34:07,519 - > 00:34:13,280 and find big problems we can solve uniquely with agentix 741 00:34:13,280 - > 00:34:13,920 systems, right? 742 00:34:14,079 - > 00:34:18,320 It's stuff that you uniquely have particular data that can 743 00:34:18,320 - > 00:34:22,480 only solve this problem that creates a moat for you or your 744 00:34:22,480 - > 00:34:23,119 company, right? 745 00:34:23,280 - > 00:34:27,519 You've got the data, you might have some unique or patentable 746 00:34:27,519 - > 00:34:32,000 business logic that then lends itself to a unique offering, a 747 00:34:32,000 - > 00:34:35,840 unique big problem to be solved that you couldn't otherwise 748 00:34:35,840 - > 00:34:36,400 solve it. 749 00:34:36,559 - > 00:34:41,280 But I think it comes down to the discovery and the pressure 750 00:34:41,280 - > 00:34:46,639 testing of problems among your customer base that can be solved 751 00:34:46,639 - > 00:34:53,199 better through agentic systems and AI than just automation and 752 00:34:53,199 - > 00:34:53,920 the like, right? 753 00:34:54,079 - > 00:34:58,239 And I don't think we've unlocked that recipe, right? 754 00:34:58,320 - > 00:34:59,679 Of how do you find it? 755 00:34:59,920 - > 00:35:04,559 I think it comes down to the data, the unique business logic, 756 00:35:04,880 - > 00:35:08,480 and no one else can do this, which is getting pretty hard now 757 00:35:08,480 - > 00:35:10,159 in this day and age with AI. 758 00:35:10,320 - > 00:35:14,320 And so I didn't hear anyone having unlocked that challenge 759 00:35:14,320 - > 00:35:14,639 yet. 760 00:35:14,880 - > 00:35:17,199 SPEAKER_00: But I can say, I want to end sort of on a hopeful 761 00:35:17,199 - > 00:35:20,239 note, but I feel like the teams that we've been coaching outside 762 00:35:20,239 - > 00:35:24,239 of these roundtables, one of the things and really understanding, 763 00:35:24,400 - > 00:35:27,599 we always start with truly and deeply understanding the data 764 00:35:27,599 - > 00:35:30,000 that you have, the mode that Heather just talked about. 765 00:35:30,159 - > 00:35:34,079 Like this deep understanding and then a structure of being able 766 00:35:34,079 - > 00:35:37,519 to start with the data, then be able to apply. 767 00:35:37,599 - > 00:35:39,519 I'm gonna call them more effective prompts. 768 00:35:39,599 - > 00:35:41,760 I don't know if they're the right ones, but we've tested our 769 00:35:41,760 - > 00:35:43,920 way into things that we believe are very effective. 770 00:35:44,159 - > 00:35:47,119 And then using AI as a thought partner to be able to 771 00:35:47,119 - > 00:35:50,480 interrogate and strategically think through this and then that 772 00:35:50,719 - > 00:35:52,880 foundation around customer. 773 00:35:53,119 - > 00:35:55,920 So much of this is gonna be recognizable to our listeners. 774 00:35:56,000 - > 00:36:00,559 It's core product, but it's core product, like leveraging AI in 775 00:36:00,559 - > 00:36:03,440 ways that really we believe make a difference. 776 00:36:03,599 - > 00:36:06,400 And there, I think we've seen some pretty like great 777 00:36:06,400 - > 00:36:07,519 breakthrough ideas. 778 00:36:07,599 - > 00:36:11,679 Now, we have to yet see if they'll be revenue positive, but 779 00:36:11,679 - > 00:36:15,199 indications are that they're on a really good path. 780 00:36:15,360 - > 00:36:18,800 So again, I think this probably felt a little bit like therapy 781 00:36:18,800 - > 00:36:19,760 and the roundtables. 782 00:36:19,840 - > 00:36:22,159 It's great to know we're all figuring this out together. 783 00:36:22,320 - > 00:36:24,079 Let's please keep sharing. 784 00:36:24,239 - > 00:36:27,440 And if any of this resonated, we would love to hear from you. 785 00:36:27,599 - > 00:36:30,079 We're going to keep running these roundtables because the 786 00:36:30,079 - > 00:36:34,400 conversations are just so good and they're too honest to stop 787 00:36:34,400 - > 00:36:34,880 doing it. 788 00:36:35,039 - > 00:36:35,760 We love them. 789 00:36:35,920 - > 00:36:38,320 So if you're a product leader who wants in on a future 790 00:36:38,320 - > 00:36:39,760 session, just reach out to us. 791 00:36:39,920 - > 00:36:42,639 You can find us at productrebels.com or you can 792 00:36:42,639 - > 00:36:46,480 just DM either me or Heather on LinkedIn and we'll hook you up. 793 00:36:46,639 - > 00:36:49,679 And if your team is navigating any of these challenges we 794 00:36:49,679 - > 00:36:53,440 talked about today, whether it's a judgment gap, it's the ROI 795 00:36:53,440 - > 00:36:57,360 question, or figuring out how your product team should operate 796 00:36:57,360 - > 00:37:00,239 in an AI world, we would love to talk. 797 00:37:00,400 - > 00:37:03,599 We can give you some more specific examples when it's not 798 00:37:03,599 - > 00:37:04,800 like a public forum. 799 00:37:05,039 - > 00:37:06,880 So just schedule some time with us. 800 00:37:07,039 - > 00:37:08,960 It's in the link in the show notes. 801 00:37:09,199 - > 00:37:10,639 Thank you for listening, Rebels. 802 00:37:10,719 - > 00:37:11,760 We'll see you next time. 803 00:37:12,000 - > 00:37:13,119 Good luck. 804 00:37:14,400 - > 00:37:17,440 Thanks for listening to this episode of the Product Rebels 805 00:37:17,440 - > 00:37:17,920 Podcast. 806 00:37:18,000 - > 00:37:20,000 SPEAKER_01: If you enjoyed this conversation and want to learn 807 00:37:20,000 - > 00:37:22,719 more from Product Rebels from companies like Netflix, 808 00:37:22,880 - > 00:37:26,239 Amplitude, and beyond, please follow us wherever you listen to 809 00:37:26,239 - > 00:37:29,599 podcasts and join us for another impactful interview in about two 810 00:37:29,599 - > 00:37:30,239 weeks.