EP 159 - Human Judgement and Empathy with Rebecca Douglas

Speaker A: M hello product people. Welcome to the Product for Product podcast hosted by Matt Green, data advocate and product manager, and Musha Mikanovsky, product leader and author. Our goal is to serve the product community by helping you find products that can help make your work in product management easier. Thanks for joining us on another episode of the Product for Product podcast.

Speaker B: Welcome back everyone. On today's episode, Mishae and I continue our series on PMs in the age of AI. Today we are excited to speak with product leader Rebecca Douglas about human judgment and empathy. So let's dive in. Welcome to the show, Rebecca.

Speaker C: Thank you for having me. It's great to be here.

Speaker B: It is a pleasure. Hey Mishae.

Speaker D: Hey Matt. This time we have someone from your neck of the wood.

Speaker B: Absolutely. It's uh, great to have a fellow, uh, atlanten product person in our neck of the woods on the show with us.

Speaker D: Rebecca, uh, let's first introduce you. Tell us about yourself.

Speaker C: Yes. So I am a product leader. I've been in product management for over 14 years. I have specialized in employer services and what we call human capital management. And that runs the spectrum from hr, uh, benefits, payroll tax and payments, uh, for clients in uh, that need those services. And I've specialized in this because I find it a really interesting industry. It has a lot of complex regulatory and operational and customer requirements and there's never uh, a dull day. I actually got into product management from being on the customer service and client relationship side. And I was asked when ACA, um, many years ago, 15 years ago, um, was written into law to um, become a product manager and had started learning the craft ever since then. And I think it's really important as a product manager that you're constantly um, enhancing your skills and working on improving your frameworks and how you make decisions around product. So it's been a really interesting transition from M that customer service and client relationship background, uh, but I haven't regretted it at all. And I would encourage anyone who's interested in product management to look for a pathway into the industry.

Speaker B: That's a common thread.

Speaker D: Some of the best, uh, product managers I've worked with came from customer service and customer support. They had amazing empathy to the users and they knew the product in and out.

Speaker B: Exactly. Yeah.

Speaker C: Yeah. The domain experience is certainly helpful although, and certainly you can learn it over time, but it, it is helpful to have that specialty.

Speaker D: Mhm.

Speaker B: Mm.

Speaker A: Yeah.

Speaker D: So we are talking these days about a lot about AIs. And in this series we really are focusing on what is our position still and where Are we in that um, journey? And maybe what everybody is asking, can AI replace me? Uh, and we believe that AI cannot replace us completely. It can definitely help us, but that's where we're looking for those first principles, et cetera. We had this um, poll, both of us, Matt and I, on LinkedIn, and we asked a few, uh, we gave a few options over there. And uh, you like actually most of the winning uh, item on the poll for what is still uh, important for us to have, uh, was human judgment and empathy. Uh, so tell us first, how do you define human judgment and empathy?

Speaker C: It's really around understanding the user's experience where they have pain points, where they're having friction and having trouble completing a particular task that they need to do with your product. In my industry it may be hiring a new employee and that there's a lot of complexity. Some people think it's a very simple action, HR action, but it involves um, maybe local, state and federal regulations around how you hire that employee, where you hire, um, what taxes may apply to that employee, what employment laws, like whether you have to offer them pto and certainly your AI can use generally or publicly available information and regulatory information, but it can't understand, um, things like culture. It can't understand, at least not yet. Right. Um, where the user needs to see that information and absorb it in order to make the right decisions as they're completing what seems like a really simple action. But it has a lot of complexity to it. And that is where we see that the human can't be replaced at this point in time, when we have a user who may not have full information, they may not understand a particular law applies into the action that they're taking. And that's where AI can help us. AI can help service that information so the user has full information when they're making that employment decision and employment action. But it doesn't replace that human judgment, it doesn't replace that empathy for the user understanding the, that it may be a low information user versus a high information, um, knowledge user making that action.

Speaker B: You just spoke to what I was thinking. I've worked in payments before, highly, uh, regulated industry, and people just need the human touch. And so I think when, when users are working in these, in these environments, like, yes, to your point, AI can do onboarding. They can make things, oh, I checked this. You can be comfortable with my answer. But like I still think people need reassurance that they're making the best decision because they could, they themselves could be impacted through regulations and like hey, you checked the wrong box. You're not actually compliant here. You didn't, you didn't meet government standards. Oh, the A.I. told me I did. Well, the I, the A.I. was wrong. Uh, you didn't do it right. And so I think having the human touch there and having that reassurance is like key. And that's where product managers and customer success come into it.

Speaker C: Exactly. And AI can't do the prioritization of what you build first. It requires human judgment to understand, based upon the user research that you've done, what is the most impactful thing that you can build first and enhance your product over time. The sequencing of that is important. And I think where a lot of human judgment comes in and empathy for the user, what do you solve first?

Speaker D: You really touched on two areas connected. But let me just break it down a bit more. One area is human judgment in the product that we put in front of our users, whether there is, needs to be a place for them or for any users in there. So sometimes it's not the end user itself, but it could be an admin or an operations person, uh, to make decisions. Um, the second one, and this is more towards what I want to talk about in our episode that you touched on, is our job, our job as product managers and where, uh, human judgment and empathy, uh, by us is important in how we build the products, what do we build, where do we make decisions, etc. The first one is definitely really important, but, um, I really want to focus on the second one because that's really the problem we're trying to latch on here or to see whether in the future. And you did edit over there, this disclaimer. Not yet. So, uh, none of us know what the future is going to be, but we kind of have the tendency to believe that probably it's going to be m much worse than what it is right now. When I say worse, I mean AI will be much better and it will do m much more than what it does today. Um, maybe we are wrong with everything we're saying, but for now we are still believing that there is some part of our job that cannot be replaced. Let's talk a bit more about that. You talked about judgment of prioritization and what needs to be built and in what order. What other areas do you see in product management, in our job, that cannot be replaced by AI still needs to have human judgment?

Speaker C: One of the things that I think is really important when you're looking at AI is don't just throw AI at a problem Maybe it's not the right solution, maybe it's more simplistic, maybe it's more um, guiding the user versus making the decision on behalf of the user. Um, in employment law there is a lot of gray areas and can AI really help us get to that point where we are making that employment decision on behalf of the user? There's still a lot of um, feelings and emotion around uh, whether AI will help us do that. Uh, but for right now we still have people who want to make decisions for themselves. They're not ready to release that to a tool like AI. And so I think that it's really important to think about how you can simplify things for a user and reduce friction, uh, and um, better the M user experience. But not necessarily using AI. AI is just a tool that can help you progress through things. So could AI make a workflow better where they have to make multi step decisions? Obviously there's an opportunity there, but that's not the end game in terms of our solution. Our end game should be does it better the user experience? Does it help remove any manual tasks that are painful or time consuming for the user? And can um, it help with that? Because who can't use some extra time in their day? You know, using it to improve your productivity and take off the burden of those tasks that are burdensome that you don't really like to do. And so in product manager language, that's reducing your pain points, right? Reducing the friction and helping the product help that end user. And it's no different from, you know, five years ago when we were still trying to do that same thing. But now we have new tools through AI to do that.

Speaker B: Mhm, mhm. And I think it's important to remember, like product managers still define the why. Why are we doing this? I can do really almost anything when it comes to building something with AI, but like why am I doing this? And I think about what you mentioned with onboarding, you know, why are we onboarding people? Why do we need to hire these roles? If AI is doing greening of resumes these days, what are we looking for in somebody? What kind of candidate are we looking for? Is it capturing all the information that you need? Because you have to tell it, this is exactly what I'm looking for. It's simply lacking your culture. It's simply lacking the human element of uh, this person would probably be a good fit because I know Rebecca, I know Moshe, and I know how they work. So I still think the human in the loop is critical in the beginning of Things to define the why we're building this, then prioritizing like what we're going to build.

Speaker C: And look at AI and all the AI specifically in your example recruiting, it has gotten so many complaints, right? It is definitely user friction on both the recruiter side and the hiring manager and the candidate. And does it really help having a thousand applications for every job or to have so many applications that a recruiter can't possibly move through them? And obviously the candidate can do AI to match to keyword search and that kind of thing. But does it help us get to the user goal which is to hire the right candidate? And there are a lot of great. Let's look at product manager hiring. There's some, a really great amount of talent out there and because of layoffs they looking for jobs. But can you get to that right talent, the right person for the role without human intervention. Like a referral.

Speaker B: Exactly right. So you have a product manager. Yeah.

Speaker C: Yes, exactly. And so that's a great example of how AI was supposed to solve the solution for recruiters and the hiring managers and parsing through, you know, hundreds of resumes. Now we've got thousands of resumes and we still are relying on more referrals because we can't get through that volume and the keywords match. And so how can you tell between candidates what's the right fit?

Speaker D: It actually make me think that there should be um, an experiment where AI agents are trying to apply for jobs and see if the hiring managers are actually getting to m. Fight any of them for interviews. That will be really, really interesting to see. But uh, this is really off topic now.

Speaker A: Uh,

Speaker D: uh, but it actually is interesting because part of our human judgment, um, and maybe that's where the world will go eventually is also we are doing user testing and we have to talk with users and we have to create that empathy to users. And when we think about users, we think all the time about real people. But it might happen that we'll have to start thinking about agents and um, understanding are we talking with an agent or are we talking with a real person? So that example of uh, an agent trying to apply for a job could be a very uh, interesting uh, thought, uh, process there to see where are we as product managers applying our human judgment to see, um, you know, and we are getting this all the time when someone is trying to scam us, we have to have this judgment to say is this real or is that not real? Um, and um, I, you know I, I can tell you I almost got into a scam last week, last minute, I was like thinking, oh, no, this is, this is not real. This is a scam, right? So there is a lot to say there about. And I don't know where my thoughts are going here exactly. There is a lot to say there, I think, about we sharpening our human judgment and empathy to really work in this world where our users are not necessarily going to be human.

Speaker C: And is there going to be a person behind the AI checking, uh, that things are successful? Many people have built evaluation loops into their AI coding and their testing and making sure that they are having a human in the loop to check the work of the AI and make sure that it's not coming up with false positives and the accuracy is continuing to improve over time. And I think that that is all part of having a human in the process. And many AIs started out without the evaluation loop or with some more simplistic version of it and realized that AI can hallucinate and AI can provide false answers and AI can, um, sound confident when they're making a statement that seems to be true. Like you mentioned the fraudster. It seemed to be true and it seemed plausible. Um, but on further examination, you're like, is it too good to be true? Is it accurate? Is it, um, trustworthy? And that's where I think we as product managers need to talk about, um, checking the work of the AI and making sure it's not delivering AI slop that degrades the trust that our product users and our clients have put in us, uh, because we're still gatekeepers around that. And once you lose trust because you haven't checked the work because you're relying on AI too much, you lose that trust. And when you lose the trust of users, it's really hard to gain that back doing.

Speaker B: While I was thinking about this episode, I was looking into empathy debt over relying on AI to build things, because you can obviously build things so quickly now with prototyping, and it's happened so quickly and so fast. I don't think the human mind is like, caught up with it, uh, and I don't think product management has quite caught up with it, but you can over engineer and overbuild things because it's just so easy to do now. And so you can build up this debt of empathy or lack thereof, into what really users want. I mean, maybe they just want something minimal, like they just want to have a better experience, add a button, add, you know, add a little bit of workflow, uh, flavor to something. But like, hey, I could do all kinds of other cool things.

Speaker D: And so it's always a problem and it was mainly developers problem in my

Speaker B: perspective and I was a developer, former developer. So you could, I can blame myself,

Speaker D: I can blame myself for that. But you're right, you're absolutely right. Now it becomes easier to include. Let's just do this and let's just do that. Right?

Speaker B: Yep, yep.

Speaker C: So, but think about the positive side too. AI and its speed has helped us get to maybe bugs that, that we couldn't, didn't have the time to research and resolve before. AI has helped uh, us tackle projects that never would have seen the light of day because they were too hard. You know, we always knew we needed to do that one feature, fix that one feature, but it was too complicated and it would have taken a year to do it and now we can do it in much shorter period of time. So there's good and bad in every aspect of AI. Uh, but that to me is where I'm um, I'm really positive on AI is we're going to fix those bugs. You know, yes, it needs to get a code review before it gets pushed uh, out the door. But that is where I think that product managers who's really struggled with that problem that we knew we needed to fix but we never had the time, no one really had the time to dig into it and figure out how to fix it and make um, the user experience better. Now we do. But, but you're right, the user experience prototyping and getting in front of the user is great. But remember that's qualitative research, that's not quantitative research. Um, 10 people may tell you it's a great design but that's not your, you know, 100,000 users that are using the product and, and so are you, you have to question and this is where I think human judgment and um, understanding the feedback you're getting from the user, not just what they're saying because some people want to, as user researcher, some people just want to say everything's great but you know, it's not right. Um, so sometimes you have those, that feedback. AI is going to look at that and say oh, let's move forward. But as a um, product manager you understand that that user is being overly optimistic or positive about that review and um, that all users won't think that way. Um, and so it is important that we review the user research, not just what they said from a transcript as AI would do and that you participate in the user research. So you understand and you can ask follow up question when it sounds too positive and then you can dig into what might um, be some of their concerns that they aren't voicing because they've just been asked a very simplistic question.

Speaker B: Yeah, uh, that's a great point. And as you're walking through that, I'm thinking also about product managers in general. How do we level up? Like AI enables us to level up as PMs to be more leadership and be more, hey, where do we draw the line between what is possible and what is needed? And that's where that human judgment comes into it. Like even sitting on user calls, like you're saying they're like, oh, this looks great. This, this is, this is exactly what we're looking for. But is that truly what, uh, you can read people's emotions and sense the emotion in the call? That's just something I don't think I can probably get relatively close to it at some point. But like, I still think that's where like PMs come into it and say like I have human judgment. And this is where PMs can ultimately level up their skills to be most valuable to a.

Speaker C: And I don't know if you're like me, but AI has saved me a lot of time on some of those tasks that have been time consuming in the past so that I can spend time on user research. Right? I can attend all the user research calls versus only 50% maybe watching the video, you know, later on I can be present, I can be participative and I can ask those follow up questions where I see that the user research is, is being too positive too, um, you know, accepting of um, something that maybe I had a question about and I'm not getting the right feedback back from the user research. And, and you know, once that user research session is over, it's, it's gone.

Speaker A: Right?

Speaker C: You can't, yeah, you can't do that. So um, that's where I think that AI is great in terms of data synthesis, you know, going through the transcripts, highlighting things, uh, maybe highlighting things that I may have missed in the, the interview, doing that marketing intelligence, you know, making sure that you complete the trust circle is what I would call it by checking the links for the market, intelligent research. Because I've certainly found false links when I've had AI help me with marketing competitive intelligence. But uh, I can't read every news article that comes out about a competitor and how they're doing. So it's important to do that. And then I certainly think that it's helpful in synthesizing requirements, writing PRDs. I know uh, there's a question about whether PRDs are helpful or not, but you still need a requirements list to build your prototype. Even in claude design you still need that requirements list. And then um, you know, user stories and test cases, having it do that so you're not spending a lot of time on that and that you can focus. Um, let's talk about test cases on the edge use cases that maybe AI is not identifying but you still need to test. Um, so those are all things that to me was, were really time consuming in the product manager day to day. And AI has been enormously helpful in doing that so that I can be more present with people versus documents or recordings or something like that. And that's where I think that it not only helps you but it allows you to do the things about the product manager role that are really exciting and connecting with that user and designing to be user centric and helping that user.

Speaker D: I think you already answered my next question which was examples at work where you implemented AI, uh, for you and your team and how you balance this with uh, human judgment and uh, empathy. You definitely answered that. I don't know if you have any other examples you wanted to add.

Speaker C: Yeah, I still think that it's important to, to give people the opportunity to make decisions on their own. That an AI can make a recommendation. Maybe there's a uh, people like me recommendation or a company like me a ah, recommendation. A lot of people do it in benefits where they're making benefit comparison and they're saying you know, people like you choose high deductible health plan versus a PPO plan. Let's just say that it's that um, but the, in the end it's the user making the decision, not AI making the decision for you. And so I think that that's an additional example where um, we as users understand that the human is still in the decision process, that they need to be in the decision process. We don't want AI to make the that decision, make that selection for user.

Speaker D: Mhm. And I think this is especially true for individual contributors in product management because when they report up uh, uh, or they say why we made those decisions, they're probably not going to say AI told me to do that. And their managers and management will probably not going to say okay, that's fine, uh, they're probably going to be held responsible for those decisions. So if they're held responsible for those decisions, they need to make those decisions and they need to understand the trade offs and the whys and all that stuff.

Speaker C: Right, exactly, exactly.

Speaker B: Yeah.

Speaker C: In human decision making too, there's the need for exceptions. AI doesn't allow for exceptions. Right. Um, it's more rigid, it may offer recommendations, but uh, there's still that need to make decisions and have that control over decisions. And what you're talking about is the product manager making the decision to trust the AI or to verify. And we always had that saying, um, with people trust but verify. And the same thing should be true of AI where um, we trust to a certain point the output that the AI is giving us and then we go back and we validate it. Because like you said, we're making the decision to trust what AI said and giving that output to our leaders. And AI doesn't care that you've broken that trust with a leader. Uh, you care. And so you're responsible in the end for that decision to rely solely on the AI without trust or validation.

Speaker D: But let's say we are hypothetically taking that individual contributor out of the loop, letting them go. Now it's only the leader with the AI. Um, so now if we do that, that means the leader will have to trust to AI to make those decisions or they will become individual contributors.

Speaker C: Well look at the example of ford and their AI. They fired um, over 300 engineers and they relied on the AI and their product became worse, not better. And so did they save anything. By that you then lost the trust of your buyer who is purchasing your product. And that may have multi year implications, long term implications for whether people will buy your product in the future.

Speaker B: Yeah.

Speaker C: And that is that to me is where a uh, great example of how AI uh was supposed to fix everything and save money and move us into the future. But it didn't. It had multi year implications, long term implications for, for.

Speaker D: I wanted to touch on another point that you made related to individual contributors in product management. And this is writing stories. And I know that you're absolutely right that it does save a lot of time, um, taking away from that, not hard work but it was uh, you know, um, junior product managers had to learn how to write improperly and how to think about them properly and then write them. And it took a lot of time and all of that stuff. Now AI does it very easily and very nicely. But I, I can tell you from my experience, I started recently, um, Vibe coding and I really enjoy it. And um, so I'm developing this um, app for mental health. It's an app for uh, mobile. If Anyone? Quick shout for anyone. If you're interested, let me know. Um, and I'm, I tried different applications for that. Some of them we talked about on this show like lovable and base 44. I actually moved away from those and moved into more um, developer based vibe coding. I landed on Kiro, which is a great tool. We might have an episode about that in the future. I don't know uh, by AWS. Now the thing there that I love is that it actually has functionality for both the product manager and engineer because I can write uh, PRDs and specs and requirements and stories and acceptance criteria before I start developing anything. The problem is that it's developed for the. I believe it's developed for the perspective of developers and developers don't. Because I see it myself that when I write with the AI, the specs, I will go through the specs, I will read the acceptance criteria and I will find problems in the acceptance criteria that it created. When you write the uh, design and the tasks to develop which is developer head, I will not care about that because although I've been a developer, it's been many years since I developed and I think the AI ah is smarter than me on that. So I will not go through those details and I will trust it. I don't trust it yet on the acceptance criteria of the stories. What I think is developers that will use it will not read those acceptance criteria. They will trust it on that because they don't wear the product manager hat. But they may not trust it on the full execution of their task that they understand and they know about. So my worry is that too many, correct me if I'm wrong, but I guess I don't have empirical data for that. Too many product managers let AI write the stories and acceptance criteria for them. Don't read them thoroughly, uh, because I saw myself actually skipping some of them and then what happened, don't read them thoroughly and then it doesn't really develop what you expected it will develop because there are always there are some things that are not exactly how you imagine there will be. They're not exactly what you as a human would do. So I think there is still an area of human judgment there, even in the stories and the PRDs that it writes for you. And I fear that too many people skip them.

Speaker C: Yeah, well I think that that was also true in traditional development and if you didn't have the story right, you're covering it with the engineering team and there's misunderstandings that occur uh, in interpretation of those user and you get to the end of the sprint and you're looking to accept the story and you're like, oh, that's not what I meant as part of that user, um, criteria. So I think that it's going to be true for AI as well is that maybe it turns out that whoever trained it had a certain way of looking at it. Part of that is supposed to be that the AI learns over time the way you work and is better at interpreting the instructions and therefore producing better outcomes. Right. But whether you're a product manager or a developer, what you care about is the outcome like what it produced. And so if the outcome is not what you envisioned, is it any different from traditional software or old time, I'll call it old time, uh, software development now where you have that misunderstanding, misinterpretation of what the user requirements were. And to me it's a little bit easier now. You just go back in and say, claude, that wasn't what I meant.

Speaker D: You're right.

Speaker C: Let me be more specific. And that's where I think all of us, um, need to rely more on specificity, um, of our requirements, like telling it exactly what outcome and then showing them what good the AI tool, um, what good looks like. Right. Um, so bad is this good? Is this right? Um, and training them on the differences between good and bad. Um, you know, whether you're using it as a design skill and you want a button versus something else there. Those are all things that AI can learn, just like a development team could learn in the past, like the, the specs on build and, and that's where I think you just have to train yourself now instead of, um, having a story review with the development team, you are giving explicit instructions to your AI tool. Um, this is what I want and this is what good and bad looks like. And then correcting, um, where you got off track in terms of the misinterpretation.

Speaker B: Yeah.

Speaker D: And I think what you're saying is more, um, you know, strengthening my theory that software developers will be replaced sooner than product managers by AI. But, uh, because, you know, but let's put that aside because I don't know any software developer like, uh, you know, hate, uh, messages from them or anything like that. But, um, you're absolutely, I think you're right. And when I work with Kiro, um, that's exactly what they do over time. And what I love also about that specific product is that it has those steering documents, so it knows what I like, what I don't like and stuff like that. And also when it create um, stories and requirement and specs. It asks me for things it doesn't understand. So it basically give me options to choose from. Very um, similar to how a developer might ask me, you know when we review. So it does, it does behave there as uh, to me as a developer um, asking for clarification, seeing uh, uh, things that uh, might conflict or things like that. I love that interaction with it actually.

Speaker B: Yeah, that's the key as PMs need to be more involved in the details I think in guardrails of making sure what you're building truly is a value to the user. These days PM can write anything for you. We're like writing less, reading less, like doing a lot of these things less because we can do other things. It's so easy again that if you don't look at the details and AI is not asking you for feedback, you run the risk of building things that like nobody wants.

Speaker C: Yeah and I do the same in Claude Cloud's my primary um, and I build skills, you know, this is my design pattern. Um, this is what my business is about. This is the voice I want you to use. And then I have a skill that access my developer partner and it, it creates diagrams for me and, and shows me pictorial workflows and, and, and builds those for me so I don't have to build myself in Figma or whatever it is. And so as it's learning that about you, it's training itself on how better to respond and, and I think the AI tools have gotten better at asking for clarifications and asking for permissions. I do try to be explicit that don't start working on anything until we have the plan down and um, tell me what the plan is and let's cover it together and I'll give you permission um, to move forward when we're ready and, and so it doesn't go off and build something and I don't use a bunch of tokens. Yeah, I'm not to, I'm working. I'm not worried about token maxing. I'm worried about making sure that uh,

Speaker B: value maxing, you know, I have it right.

Speaker C: Yeah. Value maxing.

Speaker B: I like that. I like that. Yeah, yeah, yeah.

Speaker D: Great. I have one more question for you. I don't know Matt.

Speaker B: Yeah, I got, I got, yeah I got another one too.

Speaker D: Um, on the poll you voted human judgment and empathy, which is the topic of our. The show today is the uh, most important thing, uh, that product management cannot replace in the age of AI. Why did you vote that as One, rather than strategy and vision or stakeholder alignment or anything else, it still comes

Speaker C: down to the user. Any design has to be user centric. Otherwise why build the product if you're not building it? Uh, for the end user, right? There are many great ideas out there, um, but if they don't solve the user's problem, they don't understand the user's pain points, the friction, what the user truly wants to do, the why of everything, then, you know, why do you, why do you even have a product? And to me what will be really interesting in the next phase is all these startups that don't have engineering support initially, right? But they're, they've got a great idea, they think they understand the user and how they uh, evolve over time and how they may beat multimillion dollar or billion dollar company with that idea because they understand the user. And that billion dollar company is so focused on AI and integrating AI that they forget about the user. So I think that that is a dangerous trap that a lot of organizations will get themselves into and not think about the long term value that they're delivering to the user and how they really work against their brand by trusting AI too much without considering the human element.

Speaker B: Mhm.

Speaker D: Okay, great. Uh, uh, just you know, that you know, and also for our audience on my poll it was 81% of people that voted this great. So most of us take that. And I uh, think also on yours,

Speaker B: Matt, it was, I had stakeholder alignment and that kind of leads me into the uh, the next question I had around non users. So as PMs, we, you know, build product, but a lot of the time we have to deal with stakeholders. So sales, marketing, customer success, legal. So when it comes to building empathy and human judgment, how do you go about dealing with non uh, users as a product manager? Because I think that's still critical and not something AI is tackling right now.

Speaker C: Yeah, certainly legal always has concerns about AI. And when you work in a regulatory environment where your answer needs to be 100% instead of maybe 70% which is acceptable in some areas you have a higher bar that you have to meet. And that uh, maybe leads to a hybrid approach, right, where you're calling an API for the exact answer instead of providing more um, context in general, say in a chat, which may not understand the intent. Right. And you work in a lot of gray areas where it requires human judgment. So I start with AI fluency, right. Where is their background on AI? Do they even understand more than just the scary headlines around uh, AI and then, uh, I tried to work with them in terms of what products and services are. Are they seeing AI, do they trust it in their daily life, Are they using it for, for certain things. And, um, within my last company, uh, Trinet, we had trainings, organizational trainings on AI and how you could use it and what is acceptable use as well, because you have to worry about that. And we had an AI governance committee. And so we were very focused on making sure you had the right privacy and security and legal compliance around that. Um, you know, are you giving the agents access to perform actions and what actions are acceptable? All of those things are steps along the AI maturity of an organization. And, and there's recent AI maturity assessments that are out there publicly available or per charge, if you want to use like the McKinsey model or the BCG model and other models that are out there. And so that is what I would recommend, is that you look at an AI maturity assessment for your organization and then you work on that AI fluency and you assess where different parts of your organizations are in terms of their knowledge about AI, their use of AI, getting them to use it in their daily life. If they're using it for themselves, they're going to build that trust themselves versus the scary headlines. And then showing the accuracy of the models that you're using and making sure that you have a benchmark that you're agreeing to for that AI governance and also your legal group. Um, so if you're using it, for example, uh, to answer service questions, maybe your accuracy is better than your call center, uh, reported accuracy, and legal feels more confident around that. Because if a human can make a mistake and have a wrong answer, then do you have flexibility for the AI to perhaps misinterpret a question and provide the wrong answer? So those are all things that can be used within an organization to help help bring them to a certain maturity level and, and continuing to accelerate that use of AI and acceptance of AI and understanding where we still need to have that human in the loop. And I would argue that that human in the loop still needs to be in every process and you still need to validate outputs that you're getting from the AI, because again, it all comes to that trust. If you don't check the work and you don't have some type of evaluation and correction and learning model where you're getting better over time, then, you know, why build it in the first place? Because you're just going to erode the trust that you've built from your human relationships. And your human, uh, work. And that to me is where, um, it's hard to rebuild that trust over time if you don't do that.

Speaker B: Absolutely, yeah. Anytime I talk to a new PM or a pm, wondering where they fit in the age of AI, I always focus on the business and focus on the other stakeholders you have. You can contribute a lot of value by evangelizing AI, but also ensuring that you're doing things correctly because all these other stakeholders have other roles. They do legal, they do sales. I mean, they're focused on other areas of the business, like finance. They're not focused on like what's the latest, greatest thing in AI and how it's going to make our product better. You know, so many PMs are great at greater product and are great at talking to users, but not so much their muscles aren't as strong when it comes to talking internally, uh, with certain people. So I always say, hey, that's an area you can lean into as a pm. If AI is going to do a lot of the dirty work on, like making documents, you can lean into the strategic side of things and make sure everybody's aligned on, hey, we're going to do AI not just for the sake of doing AI, but because it's going to deliver value to the user. And as a it, uh, director who wants to make sure like you're using the best in class and you're capturing the best data and not over, over capturing it in a highly regulated industry, you can help ensure that like, hey, we're doing the best we can and you know, we don't need to capture all this because that's not what the user's looking for. So um, I think, I think it's great, great to be mindful of that, uh, for all PMC days, right.

Speaker C: You're trying to, to maintain those relationships and even improve upon them. And if you let AI or the, the lack of monitoring of AI impact that it's just like any other tool that you, that you may use. And I look at it no different, it's still the human relationship that as a partner you need to have because those are also gatekeepers. If your operational folks and your legal folks don't uh, want the feature to roll out, it can. They could kill it.

Speaker B: Yes. Yeah, absolutely. Yep.

Speaker C: They could kill it. And um, they are important partners. It's important to bring them up with you as you're working on AI, so that they have that confidence, everybody's on board that you want them to have when we roll it out.

Speaker B: Absolutely. One last question. What's One thing for PMs to think about when it comes to good judgment? What's uh, like one good technique to have that you might lean into?

Speaker C: Wow, that is a great question. I would say that it helps to slow down. Things are moving so fast. But we all know that when you're moving too fast you make mistakes and then you have to recover. And it makes more sense, I believe, to slow down and prove accuracy, prove the training, prove the outputs that you're getting from AI so that you're continuing with that uh, trust. You're building that trust and you're keeping that stakeholder alignment, um, where it needs to be from a product manager perspective because that's the ultimate goal, um, to have that stakeholder alignment and then your user trust being user centric and understanding that your user wants to trust your product as well.

Speaker B: I love that answer. We all need to slow down. AI has made us very quick and made us very efficient, but if we're not slowing down and smelling the roses, we could wind up building something nobody wants and you could wind up making poor decisions that really risk the security of your business and your company, uh, when it comes to how you handle data and AI. So I love that point.

Speaker D: Uh, my version of slowing down was my late, uh, adoption of AI. Matt.

Speaker B: There you go. Yeah, I, I hung on the word. You said you loved vibe coding or something like that earlier, so that rang in my head. That rang in my head.

Speaker D: Yeah, I know, I'm so, I'm so different from last year ago.

Speaker B: Uh, you adopted AI probably slower than other people, but that's only because you have a foot in mindfully.

Speaker C: Yeah, yeah, but think about the entry barriers to business, new business startups that are being broken down by the ability to use AI and conversational AI. Like you mentioned, you don't need a developer to develop an idea anymore and you don't need a developer to test your idea. You don't need a developer to, uh, you know, a designer to design your idea. You can, you can do it all if you want to do it all. And, and that to me is the most exciting thing about product management now is that anybody can, can become a product manager really. And that's where I'm about to say that is where, you know, having the right frameworks and continuing to be user centric and understanding all of uh, the, the many years of experience, you know, we gained over time, all the mistakes we've made, you know, but it's really an exciting time I think for People with a great idea and trying to figure out how to get it out in the market. Just the entry barriers and, uh, there's a lot you can do with a $20 subscription to an AI tool.

Speaker B: Absolutely.

Speaker D: And that's. We talked a little about it in our starting, um, episode for this series, um, about, um, Product Sense and what exactly is Product Sense? And maybe that's the one that we didn't put on our poll that we should have put on, um, our poll, uh, that might have won if we had put it in there. But no one in the other, uh, said Product Sense. So, uh, that's an interesting one.

Speaker C: Um, well, it would be interesting to see how it changes over time. So maybe in three months you put it back out with Product Sense and.

Speaker D: Absolutely.

Speaker C: It will have a different result.

Speaker D: Absolutely. Amazing. I don't know, Matt, if you have any other questions.

Speaker B: No, that covers it. Rebecca, where. Where's the best place for people to reach you?

Speaker C: I'm on LinkedIn and they can connect with me on my website, workintellabs.com. uh, so happy to connect with people and, uh, in any way possible, get into product management to understand what they need to learn and to help them in their AI journey if they want that help. So thank you for having me. It's been a great conversation. I appreciate it.

Speaker B: Yeah, it's been amazing. Thank you so much. Thank you, Misha, as always. Thank, uh, you to all the listeners and we'll talk to everybody next time.

Speaker D: Thank you, Rebecca. Thank you, Matt. Thank you, everyone.

Speaker B: Take care. Bye.

Speaker D: Bye.

Speaker B: Thank you to all the listeners. We really appreciate the feedback and support. Please leave us a review to help others find the show on Apple or Spotify or anywhere else you're listening to the show.

Speaker C: Sam.