Combating Claudefishing: The Solution to AI Slop Isn’t Measuring AI Involvement

Speaker A: Since ChatGPT launched at the end of 2022, I personally don't know that I've encountered a single person who has said. One of the things I am most grateful for and most excited about with the proliferation of generative AI is the massive onslaught of AI slop people are creating with it. Now, you may feel differently, you may love it, but I personally haven't encountered anyone who does. And this week's podcast was inspired by what many organizations and institutions are trying to do, which is combat it. Specifically, if you aren't aware, Substack last week rolled out a partnership with Pangram Labs to assign an AI generated score to any post over a hundred characters in an effort to combat what they're now labeling Claude Fishing. And it follows a very predictable pattern that many people and organizations follow where whenever they face quality issues or encounter something that's unpopular or they don't like, which is they jump to compliance, policing and labeling. It's a natural response, but I would argue it's an incomplete response that doesn't get what you're really going for. And it creates a lot of unintended problems that in my opinion, I think can actually outweigh any benefits you get from it. Which is why I wanted to talk about it this week. So for those of you who don't know, I am Christopher Lind. This is future focused. If you enjoy my content, it would mean the world to me. If you would like, subscribe, share, do all those things. And by the way, I completely overhauled my website just this week, last week. Uh, so go check out Christopher Lynn co and you can see what I do, how I help people and all that other good stuff. But with that, let's get into this because this AI generated score, which is what these AI detectors are trying to do, they're trying to assign a score to something and this is happening all over in academic institutions are doing it. Ah, companies are now starting to do it as they're frustrated we're seeing it on social. It sounds like a cold, hard scientific measurement. And I don't want to dismiss the fact that there's, you know, that it's completely unscientific or there's not weights and measures. There are. It's actually analyzing things. And I will give credit to the fact that it does detect, you know, poor AI usage fairly well. Um, and it'll do a pretty good job of detecting if it is truly 100% human. Pretty good, mind you. But there's a whole lot in between there that it is a lot more gray and causes a lot more problems. And this is the other thing. It isn't a scientific measurement. It's a statistical guess wrapped in this false sense of mathematical security that often plays to our psychological response to this idea of AI generated. Talk to 10 people and ask them what AI generated means. You will get 10 different answers. And I think why I want to talk about that is because when we start pretending a label like this percentage AI generated, we say, AI generated is this label. We start to create this illusion of control that we can now assign something to something and it means something when it really doesn't. And what happens in response is, and I'm seeing this happen all over, I don't know how many messages I've gotten from people asking for input, feedback, advice on how to respond to this, and I've had this in organizations and academic institutions as well, is it ends up breeding paranoia. It starts punishing structured thinkers or people who may really genuinely be using AI as effectively and intelligently and honestly. My biggest frustration with it is it misses the conversation entirely because when you really think about saying, was this AI generated? The only thing you're measuring is, was AI involved in this? We have completely abandoned the conversation of right, but what is the merit of what it's trying to accomplish? We are becoming 100% focused on on what is in it. It'd be like if you looked at somebody's shopping list and then judged whether they were a good cook or not because of an ingredient on the list. It just, it misses the conversation. Now, before you just turn off this podcast and say, you know what? I hate AI slop and it sounds like you're about to go to defend that we shouldn't do anything about it. Hold your horses. Okay? Because I do want to say I am extremely empathetic and I too get very frustrated with what I would consider slop or work slop, which for clear on definitions, when I say AI slop, I am talking about this hyper, generic, low effort, unvetted AI output. Somebody simply just took a brainless prompt, threw it in gen AI or asked it for some basic thing and just straight up copy and pasted it into something with zero thought. I do not like that I get frustrated with it. It drives me nuts how much when I scroll through my news feeds or I read emails or things come to me where I'm like, oh my goodness. Because nobody wants to wade through 10 pages of AI generated garbage or scroll a news feed of nothing but AI just blah. And I get it. Especially organizationally leaders right now. So if you're a leader and you're frustrated with this, whether personally or you're seeing it within your own teams, your teams are sending PowerPoints generated by AI that they didn't really put the time into. The emails coming through. I saw an article just a little while ago where like a CEO blew his top and said, I'm going to fire the next person who sends me an AI generated email. I understand it. Leaders are burned out right now. Things are crazy. There is immense macro pressure to keep teams running, to do more, to do it faster while trying to understand what the work is happening and then try to understand how does AI fit into the mix. So I understand in that environment it is already our default response to just want to shut it down or compliance it, then you stack on top of it. The current environment we're in and the just polarithy the amount we'll just. I'm not going to try and say words I am struggling with right now, but the amount of this happening as people are pressured and just experimenting and learning with this stuff. I get why grabbing on to a tool that seems to make you feel better by slapping on a label, I get why that seems like a lifesaver. It feels like an easy button rather than doing the harder work that we're going to get into later on here of what it actually takes to fight against this. But I want to just say I understand why it is so tempting to lean on this and to lean into this as a solution and to believe the promises that tools like this can magically make work slop or AI slop go away. But I promise you it doesn't. Because when you're reaching for like al algorithmic surveillance tools, which is basically what these AI validators are, they're just surveillance tools. It's looking at what everybody's doing and it's trying to stamp it with a rubber stamp of approval based on whatever standards these tools are using. All you're doing is you're driving the bad behavior underground while breeding an environment that just torpedoes performance. And just on the substack front, I will tell you, I have talked to multiple people who are doing fantastic work and are using AI to help them do that, but they're doing it very surgically and they're now debating, am, um, I just going to quit altogether because I just don't want to deal with the AI mob of people who now, regardless of whether they liked my content before they found value in it, now they're seeing this label assigned to it and making an arbitrary decision to cast it out or keep it based on a label that isn't actually even related to the quality of the work. And so I think this is a really important thing. I understand where it's coming from, but we got to fight against this. Because here's the thing we have to talk about. The fact that whether something is AI generated or not or somewhere in between is rarely actually the thing we care about the most. We do not. And as leaders, especially in an organization, this is something I do. In a lot of the coaching work I do with folks as they continue to move up the ranks, we should be spending less time worrying about how someone is doing their job and be focusing on is it achieving the thing it is supposed to be achieving. Is the quality of what it's producing, what it should be. Because when we miss that, we think we're preserving human agency, which is the big argument behind this whole thing. And a lot of these pushes against AI slop is we want this to be human, we want humans involved in this. And I do too. But what actually happens is when you go to this compliance route you label, you make people feel shamed or guilty over a measurement that actually is in no way related to the product or the outcome you're driving towards. You actually rob people of their human agency because you start to dictate and mandate every step of the process. By installing these digital hall monitors, you're actually stripping away the human agency from this stuff. So I, you know, as I talk to some of these people, they're either thinking of like, how do I game the system? Or how do I work around it. Now, like before I was actually using it in a really productive way, but now apparently my productive way is, is being assessed based on these measures that don't actually measure the quality. So now I've got to figure out how to game the system and now have to accommodate whatever this new system says is acceptable. And this is one of the most agency robbing things I've ever seen. Because you can't actually hold people accountable for the outcome if you truly just strip away all the agency they have over their ability to think and execute what needs to be done. Because while you may say I don't think you should use AI this way based on that, that is based on your personal experience, your personal capability, your personal comfort level, and your knowledge and experience with the tool that you're talking about, you're now trying to place that on someone else. And largely that's what's happening. When you assign these scores is you say, we've decided. We, as in the AI Validator company, have decided what acceptable AI use is. Now we're placing that on top of you. Now go do your thing. But you have to do your thing in accordance to our standards, otherwise you're going to get penalized. And I get. There will be the people that push back and say, well, uh, it technically doesn't penalize, it just puts a score on it. Well, in this instance, you look at that human is in like all green bar. AI is in like this gray thing. So even psychologically, it is visually indicating to someone that someone with a bigger green bar is better, superior, you know, producing a higher quality piece of content than somebody that has this ew gray bar. That is communicating something. And that is forcing people to go, well, now I need to cater to, to make sure I get the green bar, not the gray bar, which completely takes them off of what they're producing and strips them of their agency. And really, when we think about agency, it's not about auditing what's in the kitchen. Yes, you should know what's going on, how people are doing things, what they're producing, the way they're going about. It's good for you to know that and to coach and guide and do all this. But what you really should be focusing on is saying, hey, here are the standards we have and here's what we're shooting towards. You have agency and autonomy to figure out what the best way to get to those things is. And granted, you will be held accountable to that, which that's still what's missing from this. We think we're now getting accountability because we're seeing a witch hunt for people who are producing AI generated content. And yes, to be fair, some of them probably have it coming because of how they're doing it. So is there some accountability coming? Yes, but it's dragging everybody down with it. And I think that's the issue. And so when I talk to people about this, they say, so you're opposed to something like this? I'm like, I'm not saying I'm opposed to something like this. I'm saying we have to be so careful with how we do it. Now, as an example, just because you might go to my website now and go, well, Christopher, don't you have an AI effectiveness rating? I do. So do I have a way of measuring and assessing how effectively someone is using AI? I do. And is there a score associated with it? Yes. However, there is a full rubric and like system that is communicated out to people of how that's being done. Now, I will be the first to tell you that one of the things in any time I do the work that I do when somebody's planning on using the AI effectiveness work, most of my work is making sure that the way this is established, the way it's rolled out and the way it's managed does not do what I'm seeing happen on Substack. Or somebody gets a score assigned to them and that score suddenly becomes a label, a red letter they have to wear around with them. It's designed to be something that works them towards something. So in my case, the AI, ah, effectiveness rating is designed to say, hey, here is where you are. Here in these six categories are your greatest opportunities. And here's the things you can do to move from point A to point B. That is completely missing from these AI scores that get assigned to things like on substack, where it just says here's who they are and what the quality of this is. Even though it's not based on quality, you get no insight into how it came to that conclusion. You get no insight. It's not for development purposes. It literally is to just assign a value to something based purely on what ingredients are in it and those ingredients. It's not even really great at telling you what the ingredients are. And like I said before, I'm not saying that they don't do it scientifically or that there's not a way it does it, but it's still a guessing machine and you're assigning a permanent label to something that isn't even related to what they're trying to accomplish. That then completely hijacks the whole thing. Now you may be listening to all this and go, okay, maybe I hear you out, Christopher. Maybe assigning an AI score to everything that isn't actually related to the desired outcome isn't the most helpful and some of the side effects. You don't believe me, go over to Substack and look at the arguments. People are just blowing up left and right over this. And I've seen this in higher institute at institutions as well. I get it. But hopefully at this point I've at least have you convinced there might be a better way. Maybe there's a better way. Maybe this isn't the best way to do it. And maybe just assigning a label isn't there. I hope you're there. If you're not, well, go listen to this again or stick with me because next week I'm going to publish an article explaining a little more and digging into some of the philosophical side. But if I've at least got you there, I don't just want to leave you and go, okay, great, we don't do that. So what, just leave it? We just let this AI slop continue? No, that's not what we do. Now, the first thing I will say is I do think in some cases we do have to stop trying to worry about things that are outside of our control. You know, if you're on social media and somebody's creating AI slop, you do have the option to not follow them. And if a, uh, publication puts out a bunch of AI slop, you do have the freedom to not subscribe to it. If your team is sending you a whole lot of AI slop or another department is doing it, well, if another department's doing it, mind that if your team is or people you're working with are, uh, have a conversation with them. But let's move to the now. What? Because the people I know who listen to this and who follow my content, many of you are in a position where you're like, okay, but I, I'm responsible for leading or doing something related to this to try and influence this either within my own teams, within the circles that I run in, things like that. So I've got four things that I think all of us can work together on that I think are going to be more effective at combating this than slapping labels on people and saying you're a bot, you know, when really you don't know anything about them. The first is we've got to get so much better at anchoring on observable, measurable outcomes. Because like I've said through this whole thing, AI generated is not an outcome unless the question you're assessing is how much human involvement was in this. Which doesn't really tell you much of everything. Now, if you're in a workplace, then maybe you're frustrated by the AI generated PowerPoints and emails you're getting. Well, you have to ask the question, have we been clear on what we're targeting and what we want? Maybe the reason there's so much fluff is the people sending this stuff aren't really clear on what they're supposed to be doing or working towards. So maybe they're just dropping the very unclear direction. They're getting the very unclear generic crap they've been shared into AI and saying, can you help me pull together some stuff because I can't make sense of it. And AI's guessing. I can tell you that AI is very good at distilling things down and being specific when you actually anchor it to something observable and measurable. Uh, as a perfect example for my company, I have taken the time to outline what I call my single sources of truth. They are basically constitutions for everything in my organ, my company, the way I've structured it, things like this, and it is very detailed documents that says, these are the things, this is the voice, this is how we handle things. These are like. And I, and I personally have taken the time to define all these things. When I put that into generative AI, it does a very good job of refining things around those categories. Now, do I still need to be involved? Yes. But focusing on those anchor points of what are we really shooting towards? And this is a huge problem. This is not a problem that is new to the AI generation. Um, but to all of you, I would encourage you to actually take the time to ask yourself, have I been clear on what is the desired outcome? And if there are parameters, if there are things that you say, hey, this is not acceptable, then make those clear. Because if you have not made those clear, it's not just AI who's struggling to understand what you want, it's your people too. AI is just in accelerating and amplifying what your people are already dealing with. You just probably don't notice it as much with your people because they're people and you like them or you don't like them or whatever, and that influences how you see them. But they're at least a person. When AI comes through, you're a little bit quicker to snap to judgment on it. And I think this is one of the biggest things. So right now, more than anything, if you're struggling with AI slop in anything you do, you need to be clear on, uh, what it is you're actually going for. And like I said, this doesn't just apply in your organization. You should be thinking about that in the content that you consume. If you don't like all the AI slop you see, then you need to be taking the time to get clear on what you want to see and then hide the stuff. You don't give feedback to the system, you will see less AI generated slop. If you're on a social site and you say, hide this from me and then explain why every single time you do it, pretty soon the AI slop disappears. I know because I have almost none in my feedback. The second one is, we've got to get away from this labeling and picking sides. I called it out, uh, two weeks ago in my podcast in my mid year update. I am very discouraged and disheartened by how divisive and tribalistic things are getting as it relates to the pro AI and anti AI side. Um, I understand we want to have more human agency and critical thinking and all this discernment brought back into the, the work that's being done, but slapping a label on someone as you're human, you're a bot, it is just breeding this tribalism and gaming of the system because it's just leading people to go, well, whatever side I want to be known as, I'm just going to find a way to game the system so that I'm viewed that way and then pick a fight with somebody who doesn't see it that way. And it completely, not only does it miss the mark that I just described in terms of the measurable outcome, but it actually results in everybody fighting each other instead of actually moving forward together and learning from one another. I can't tell you how many times I've been in groups who have very different opinions, but when we actually start talking about the way we do things on a shared outcome and how we're trying to get there, we actually learn from one another and go, hey, I can appreciate that. Like, that might not be super helpful to me because I don't actually struggle with that or, uh, that's not an area that I need help with. But I can see for you that'd be a great copilot. Not in a condescending way, but just in we're all different. Like, let's appreciate the diversity of what people are doing. You can't do that when people are picking sides and seeing, oh, you're the, you know, you're the AI group, you're the non AI group. One's looking at one as the Luddites, the other is looking at the others as bots. That doesn't go anywhere. And as a leader, you have to be really mindful because this shows up in even your own preferences and how you show up to things. And uh, it's something I've had to be very careful of over the years because I tend to be very tech forward and lean into some of the stuff. And so I've had to be very careful to make sure my teams have known, hey, I respect and appreciate that not everyone shares the same perspective. I do, I do want you to be able to, you know, speak, to articulate. Why, be able to, you know, dig into those things. But I don't expect you to be an extension of me. And that can't happen when we go to labels or pixi. Uh, we need to actually talk about what I said before, which is the observable, measurable things. If you see something that's AI Slop, don't just call it AI Slop. And I know I'm saying that as here I've used that term throughout, but what about it makes it feel AI sloppy? Can you point to it? Because if the only thing you can point to is I don't like it, that doesn't make it AI slop. AI slop has just become an excuse to not actually have to do the hard thinking of. I'm hm, critically evaluating this. Here are the things I like, here are the things I don't. Here are the points that I agree with or don't agree with. Here's where I'm concerned about the person putting this out doesn't seem to be qualified or consistent in their ability to answer questions related to that. Let's focus on those things that we can have discussions around rather than you're a bot or this is AI slot. Well, that doesn't get us anywhere. Now the third one, I think this is a huge one that will naturally help with the previous two, which is we have to get much better at leading with curiosity and asking questions throughout the process. I am not suggesting that we just say, hey, let's do away with, with any sort of assessment or qualitative score for how AI is being used. I would be a hypocrite as I have an AI effectiveness rating as part of my portfolio. What I'm saying is that should lead us to have deeper discussions and understand what's really going on under the hood with being able to actually have dialogue around this kind of stuff. Because when we just say, hey, and here's something I tell leaders all the time, if you get something that's AI Slop, or even if you encounter something that you think is AI Slop, we don't ask enough good faith follow up questions on this. So, you know, if you're, if you've got a team member who is notorious for passing along things that feel half baked or like they're just, you know, put through, have you actually done the work to go, hey, can you walk me through your thinking? Like, help me understand what you were thinking here? Uh, you'll see real quick if they've actually just like passed the buck or if they've actually been working with AI to come up with it. It just happened to be output by AI, which is why it might have some of the typical indicators of AI generated. Ask the person like, how did you arrive at this conclusion? You said this is the conclusion you arrived to. How did you come there? You can do the same thing on social media. You can do this when you're engaging with people where you feel like, boy, I really feel like I'm engaging with just a robot. Well then ask some follow up questions and actually see how they respond. I think when you have those clear outcomes, uh, uh, and this is where this is hard. When you don't have those observable measurable outcomes, it's very hard to ask follow up questions and be curious and dig into things when you aren't even really sure what you're asking. And that's why I ordered these the way I did. Because if you get clear on the observable measurable outcomes, then you resist the urge to say, okay, well when those outcomes aren't met or whatever, I'm going to resist the urge to just pick a side, pick a binary label on this. And instead I'm going to lean in with curiosity. I'm going to ask like, hey, help me understand what you were thinking here. Why did you structure it like this? Where did you come up with these ideas? How did you arrive at these conclusions? When you start to do this, you can start to move away from getting bogged down in these silly details and you can focus your conversations on outcome driven results and actually saying like, hey, okay, that's helpful. That's not super clear. Like maybe they did do what you wanted. It just didn't come through because the way they went through it, it was so natural to them. Uh, they didn't think to include it in there. That's not going to come out. If you just see, oh, AI generated. That's the problem. They didn't think. Maybe they did think, maybe they just didn't think you didn't. That wasn't as intuitive to you. And that'll come up when you actually ask those questions and seek to understand. Now the fourth one I think is really important is we really have to have, you know, because again going back to this curiosity in question, this is why they're in the order they are. We have to go into those conversations celebrating different perspectives and having and welcoming open dialogue, recognizing and this is, I think one of the things that I've seen with substack, um, is I'm fine and I have conversations like this. I'm fine. If we don't agree. There are certain lots of people who don't agree with my takes on AI or feel the same way about it as I do or maybe think differently, you know, as it comes to worldview things. I am, um, totally fine with that. So long as we can have a respectful dialogue and embrace and celebrate the different perspectives that we have and the different outcomes we're driving towards. Not everybody is focused on, on the same things. Not everybody's going to come to the same conclusion you do. Not everybody. Even if you have the same destination you're going to, is going to choose the same path. And just because somebody chose a different path, that doesn't mean it's bad. And I think we have to throw this mindset out that oh, you went that route. Well, that's the bad route and that's what's happening with Substack. Oh, this article is AI generated. That's a bad way to do it. How do you know? How do you. Just because you wouldn't have done that doesn't necessarily mean it was the wrong way to do it. If there's specific observable things, we'll ask follow up questions, dig into it, see what things actually are in their process. And I think if we can actually move towards some of these things, I think we're going to see a big difference not only in the outcome because I think if every single person were to take, I don't even think, you know what, we could go herd, we could go herd mentality on here. If we could even get most people to do these four things consistently moving forward. I actually think you would naturally see a lot of the things that are frustrating. You start to fall off the map. And the reason for it is, and I can tell you this, people who are doing AI slop, they don't actually like they're knowingly, like they're knowingly doing no thought, just churning out crap for likes, views, engagement, whatever. They don't actually want to be asked questions. They don't actually want people to engage. You know, they don't actually want to have to engage with their audience. They just want to put things out there for clout and reputation and you know, clicks and views. So if you start asking questions, if you start digging in one, you're going to quickly start to expose the fact they aren't interested in that, which is going to disincentivize that. But also you're going to start hiding that stuff. They're going to get less engagement in the first place. So I think if we actually could all commit to this, we actually wouldn't need to have these AI validation tools and then be arguing over, should substack have an AI score on content or not? Because people m like, my perspective on this is, who cares? Why do we need this? If we did these things, we wouldn't need this kind of stuff. And I get there are times when you may need stuff like this as a starting point. That's exactly what my AI effectiveness rating is designed to be. It's not supposed to be a. A final grade on how someone is. It's supposed to be a starting point of, here's where the snapshot is of right now. Let's figure out where we want to be based on that, and let's build a plan to get there. And unfortunately, AI verification or AI validation tools, they don't do any of that. They just assign a score to somebody and say, here is how valuable you are. And that value is interpreted based on whoever reads it and judges it based on their own personal interpretation of what that value is. So all that to say, you know, leading in the digital age, this AI age, we gotta get past, you know, trying to build a better digital prison for people. We've just got to move to higher standards and rebuild and focus on trust. So my encouragement to you this week, if you're in this, if you're frustrated with AI slope, um, you know, if you're frustrated with some of these things, is take a look at what's like in your own wheelhouse. Personally, take a look at your news feed. You know, dig into some of these things, hide the stuff that's been frustrating. You don't engage with it. You're not obligated to read it. If it's happening within your teams, have some conversations, invite people to dialogue about this. But just whatever you do, please don't. Don't create a new label or new arbitrary score that you slap on people. And then some people walk around with a red letter and. And you end up with the infighting. We just don't need more of that. There's enough of that going on right now, and we can do so much better. So with that, I'm Christopher Lynde. This has been future focused, and we will see you on the other side.