108: Using AI Detection Tools to Fight AI Slop and Preserve Authenticity Online with Max Spero

Speaker A: What we see is consumers check out when they're reading AI generated content.

Speaker B: What do you think Max, that the Internet looks like in two to three years if we don't do anything about detection or verification or distinguishing synthetic from authentic content.

Speaker A: A couple years ago the Internet was 20, 30% AI generated and by mid-2025 it's already 50% AI generated. And I think we're really, wow, we're going to be approaching like 80 or 90%.

Speaker B: So where is this line between smart assistants, let's say, and outsourcing your thinking? Too much?

Speaker A: So I actually think best work to outsource to AI is the work that you understand deeply and you're very knowledgeable about because then you can tell when AI is bullshitting you.

Speaker B: So with this manipulation, this flood of slop that's happening, where is it showing up first?

Speaker A: I think the biggest one I'm seeing is SEO. There are some companies that let you fully automate your blog writing process with AI. At first glance it's like, oh, you know, I can put out as many blog posts as I want. And then what we see in reality is once you do that, you start ranking really well. You start getting a whole bunch of page views and then at some point Google notices and it drops to zero.

Speaker C: Max is the co founder and CEO of Pangram, an AI detection startup fighting to protect human authenticity in a world

Speaker B: flooded with AI slop.

Speaker C: A Stanford trained machine learning expert from Google and Neuro, he's helping people tell what's real from what's machine made. Welcome to using AI at Work.

Speaker B: I'm um, your host Chris Staigle. Each week we'll be learning how today's

Speaker C: business owners, entrepreneurs and ambitious professionals are getting more done with smart use of tomorrow's tech.

Speaker B: Let's get started.

Speaker C: Right now, every business leader is asking the same question. What are we going to do about AI? If this is you, chiefaiofficer.com has the answer. We give you a simple path forward where we provide executive and team training so your people know exactly how to safely use generative AI in their day to day. We also manage the deployment and implementation to make sure tools actually get adopted and deliver results. And we'll also guide company wide transformation so AI becomes part of your operating system, not just another shiny object. The companies that act now will increase productivity, cut costs and grow faster than their competitors. Those that wait will get left behind. So if you want to make AI work in your business, visit chiefaiofficer.com and see how we're Helping companies of all sizes finally get results from AI.

Speaker B: Uh, hi everybody, and welcome back to another episode of Using AI at Work. My name is Chris Daigle and I am the host of the show. Our guest today is Max Spare. Max is the co founder of Pangram, which is an AI text detection company. They're based in Brooklyn, um, and a growing team. At the time that we last spoke in, in February, I guess you guys had around 10 people working in person, which is, uh, not the, the standard. I think in the, the, the AI space there's a lot of remote teams here. But one of the things that Pangram is doing is, um, I guess one of your claims to fame is that you guys uniquely detected an unreleased creative writing model output that everybody else just assumed was human or authentic output. And, um, not according to the Pan Gram algo. So one of the things that I think is important for our listeners today, um, is that leaders are drowning in AI slop, whether it's on LinkedIn, whether it's in, you know, what's being sent to you via email from your clients or your vendors or anything like that. And it's important for you to be able to truly judge what's authentic, what's real, what's human output and what's synthetic, especially to be able to do this at scale. And that's what Max's company does. So, Max, welcome to the show. Um, we'll dig in in a minute, but is there anything that you think that the audience needs to know about why they should listen to this episode?

Speaker A: Oh, thanks so much for having me. I mean, I think this is really important, especially for people who use AI in their daily lives like I do. I think there is a lot of, there's a wide spectrum of what people believe is acceptable AI use or not. And I think, you know, I think that's going to probably be a lot of what we cover is like, what are, how are we, how do we use AI in ways that empower us to, uh, be better and more effective without just flooding the people around us with AI slope?

Speaker B: So, uh, I mean, especially now that I talk to executives regularly and even if they don't even have an AI use policy, even if they've never trained anybody in their organization, the first thing they're saying is, hey, this openclaw thing, how can we set that up? Right? They all want agents.

Speaker A: They heard about Open claw somehow from LinkedIn.

Speaker B: Yep. You know, I think what's happening is that every, like, business peer group, there's one or two enthusiasts in There for sure, that are, like, way ahead of the pack of everybody else and, you know, executives and CEOs and stuff, that they like the shiny stuff for sure. Um, I mean, they appreciate substance as well, but when they see somebody else, you know, saying, oh, my agent's doing this for me, my agent is writing all my content, my agent is handling all of my, you know, my thought leadership posts, or sending the updates to our team, or whatever the executive is thinking, ooh, I want that. But they're not necessarily, um, seeing the output and going, oh, that's the kind of quality output I would want to be associated with my brand or my name. Right. They're, uh, they're assuming AI output. AI slop is something's better than nothing. And I don't know that that's necessarily the position that Pangram takes. What's your, what's your thoughts on where these executives are heading with this, like, enthusiasm for AI generated output, even though it may not necessarily, be, uh, in the long or short term, supporting their brand or supporting their narrative?

Speaker A: Look, yeah, I think it's easy to get Shiny object syndrome around AI. It's new and it enables things that truly were not possible before. Um, and I think this is, yeah, the efficiency gains. Very exciting. But I do think quality, uh, if you're letting your bar for quality drop because you're using AI, then I think this could come back to bite you. Especially because you're not the only one. I think, like, everybody out there who's, like tons of people are using AI and, um, kind of like if you're ever on LinkedIn, you can tell, like, things like the feed is flooded with AI generated content. And like, I think people are pretty discerning like, like they can tell. And it's not like there's a shortage of things for me to read. I'm already picking and choosing the best of the best to read or summarize. And so if there's twice as much content, but it's half as good, then, or 70% as good, I don't care. I'm not reading anything but the highest quality content already.

Speaker B: So we're going to talk about some things that people should be looking out for, for sure. But I think, uh, listeners, yes, you can produce more content, but so can the bad actors. Right. AI has become this force multiplier for those folks too. So the, the, uh, flood of synthetic content and these autonomous agents that can manipulate platforms at scale is something that you, as an executive, you need to be concerned about. This not just so that you're not wasting your time reading, you know, AI generated content. But you could face brand or reputation risk and definitely operational noise like fake reviews, fake content, fake engagement that is going to distort the decisions that you could be making based on the data points that you're receiving from web traffic or responses to certain marketing pieces or whatever those things are. So, um, what do you think Max, that the Internet looks like in two to three years if we don't do anything about detection or verification or distinguishing, uh, synthetic from authentic content?

Speaker A: Oh yeah, we've seen the trends. Uh, it's gone from, uh, a couple years ago the Internet was 20, 30% AI generated and by mid-2025 it's already 50% AI generated. And I think we're really, we're going to be approaching like 80 or 90%. And the really amazing thing, I guess not amazing, but the thing that's not that great about this is that I think what we see is consumers check out when they're reading AI generated content. Uh, we ran a study, um, and people just, once they clock something that they read as AI generated, they start just caring less. They go to platforms less with AI generated content. And they have this pretty high distrust around misinformation and hallucinations in content that they can tell us AI generated.

Speaker B: So, okay, in that case, what would be the business cost of, um, certainly you as a business owner or something like that, not knowing what's real. Um, but I guess you kind of touched on that for the consumer or for your clients. Um, but what's the cost for you as a business owner, not you specifically, but a business owner or somebody who's a decision maker in a business for them to not necessarily have this discernment to be able to tell what's real and what's not.

Speaker A: So I think where I'm seeing this is these untrusted layers of communication where you don't necessarily know that the person on the other side is a good actor. So one example of this is RFPs. I see people all the time, like in government elsewhere they're putting out, uh, a request for rfp. I want to hear from several different businesses. But what we're seeing now is just more and more just fake AI generated RFPs. There's no actual substance behind it. There's not even often a real business or individual behind it. There's people who are trying to completely fake it till they make it like fill out fake RFPs. And then, you know, if once they get a meeting then, then they're scrambling

Speaker B: to put something together, but.

Speaker A: But it's just a waste of everybody's time. And, and this is just one example. I see this. I get AI generated, uh, emails in my inbox. This cost me time. Yeah, that's like, very quantifiable in terms of the cost of the business. And it's starting to just be like a flood.

Speaker B: You know, I'm noticing it. Uh, I don't spend a lot of time on content and thought leadership. We're too busy doing the thing, I guess that's good in, in LinkedIn in particular, man. Uh, like, you know, I was encouraged by some, I was like, all of a sudden some of my posts are getting responsive, like, oh, this is great. It's good to see I'm engaging with the audience. But it turns out, I mean, I've got a. I kind of talk to business owners. It's almost like a wine drinker. If I put, if I'm, if I don't have a palate, if I haven't spent the time, like, really, um, understanding the nuance of wine, somebody puts a glass of red wine in front of me, it's just red wine compared to the individual who spent, you know, uh, went through the windows of the world, sommelier course or whatever. Like, they can tell you the, the terroir, where it was grown, what vintage, what grape, all that stuff. Same thing with AI. If you're, if you're not paying attention, it's just all red wine. You don't really have a palette. Right. You can't discern that, uh, that synthetic from something that I actually wrote. Now I've got a palette that's developed and what I'm seeing in platforms like LinkedIn, I'm not on Instagram or Facebook or anything like that. Is that, that, um, that cadence of your reply, the, the words you chose in your reply, that seems a little. I don't trust it. And that's what I'm seeing. I don't know, is anybody even putting out real content on these platforms anymore? Or do you have data points that say, like, how much of what I'm reading on something like LinkedIn actually came from the mind of the person that's posting it?

Speaker A: So I guess two things to pull out here. So one is, yes, there have been papers that show that people who use AI regularly, they're pretty robust at detecting AI themselves. They can tell. Uh, and then when we've run studies, we've seen it's about 30 to 50% of people can tell, uh, about 30% of the older generations can tell. And by gen Z it's 50 to 60% can tell. Um, with that said, how much of the Internet places like LinkedIn is AI generated? Um, I actually don't have a number for you on, on LinkedIn, but I have ballpark it.

Speaker B: Ooh, just to, based on your understanding

Speaker A: of like the 30 to 40%. I, I, I don't think it's the majority yet, but I think it's, it has grown pretty rapidly and I actually think that like LinkedIn is starting to realize this and backpedaling a little bit. It used to be I, I try to compose a post on LinkedIn, I write 10 words and there's a little like right with AI button that lights up that's like hey yo. Well you don't have to finish writing your, don't finish your thought. AI can finish it for you.

Speaker B: Yeah.

Speaker A: And now they actually removed that. Like I don't see it anymore. And I think it's because they realized like it was hurting the quality of content on their platform.

Speaker B: Very good. I hope that's the case now.

Speaker A: Yeah, yeah. The replies are actually the worst for me. The replies, when I write a LinkedIn post and then I read it sounds like somebody's engaging with my content. They're like, oh, that's really interesting. They mention uh, something from my actual post and then they ask a follow up question and then I like squint for a second. I'm like, oh, this is just like AI reply slop. Like I, they're, they're literal time wasters. They're wasting everybody's time by uh, having AI auto reply to every post that they see.

Speaker B: So for the listeners, if you're on these platforms, especially LinkedIn, please be aware that this is happening. Right. Like, and don't be, don't be one of those people.

Speaker A: So yeah, hey, don't be one of those people. Like, yeah, the number of people I've talked to that said, I want to get my OpenClaw to go like run my LinkedIn or Twitter account for me, I'm just, I understand it's exciting technology but it's not great. Uh, and then I can also tell what we're doing to try and combat this and help you not spend time on your, these time waster things. So Pangram, we're a platform to detect AI generated content. You can go to pangram.com, paste in a piece of text and we'll tell you if it's AI or not. We have a 1 in 10,000 false positive rate. So it's pretty.

Speaker B: Wow.

Speaker A: So if we say something's AI, we're pretty sure that it is. Uh, but obviously you're not going on LinkedIn and copy pasting everything you see into another website. That's absurd. So what we did instead, we put out a Chrome extension that as you scroll on LinkedIn, Substack, Twitter, Reddit, it will automatically tag the, uh, comments and posts and replies that you see as AI, if they're AI. And so you see that little badge next to reply, and I'm just like, okay, I'm just not going to engage. It's not worth my time.

Speaker B: I'm on the extensions, the Chrome web Store and I'm adding it to comment now. I didn't realize you had a, um, download. So for my team, let's add in the show notes for sure. A link to this, um, in the Chrome store. This is, I mean, that's a lot like, I would love to have it as a copilot rather than copy and paste it. So it's good to know.

Speaker A: That's awesome. Yeah, it's been supremely useful for the team and we just put it out there, uh, on the, on the Chrome extension store last month, so.

Speaker B: Oh, nice.

Speaker A: I hope people find it useful.

Speaker B: I'll, uh, let you know, I'll be using it, leaving a review for sure.

Speaker A: Yeah.

Speaker B: So with this, with this manipulation, this, this uh, flood of slop that's happening, where, where is it showing up first? Is it in reviews? Is it in, you know, obviously we talked about social. Is it in customer support? Is it like, where are you seeing that it's showing up?

Speaker A: I think the biggest one I'm seeing is SEO. So, so people want their website search engine optimized. And so to do that, you want blog posts that are relevant to the topic and keywords that your target users are searching for. So what people did in the past, they'd hire blog writers, uh, they'll, they'll get some, some blogs up and then hopefully some of the, some of these start ranking for the keywords they want. But then they start coming around these. Some companies like Air Ops, if you've heard of it, uh, there are some companies that let you fully automate your blog writing process with AI. Like, sounds like a dream. And like, uh, at first glance it's like, oh, I can put out as many blog posts as I want. And then what we see in reality is once you do that, you start ranking really well. You start getting a whole bunch of page views and then at Some point Google notices and it drops to zero. So I saw this on Twitter just the other day, ClickUp. You see this exact trend with their search results where their search traffic was going up and up and up and up and then it dropped off a cliff and then their like head of SEO was on Twitter commenting. Yeah, we just like, we relied too hard on air ops, we flew too close to the sun and we got burned.

Speaker B: Now one of the things that we talked about um, on our kind of our pre call was this idea about ah, the impact of AI in education, this concept of skill formation. Right. Um, and one of the takeaways for me was that you were talking about how AI can offload the cognitive work that students need to develop. That kind of widens this gap between motivated M and self driven learners and those who are just looking to get through and get the class right.

Speaker A: Absolutely.

Speaker B: Where is this line between smart assistance, let's say and outsourcing your thinking too much?

Speaker A: Um, I mean I think this line, I don't think it's just for students, I think it's actually for everybody. There's oftentimes skills that I'm trying to learn and I would much rather learn the skill myself and put in some reps so I fully understand uh, uh, exactly what I'm doing. And then I start to outsource some of this work to AI. So I actually think the best work to outsource to AI is the work that you understand deeply and you're very knowledgeable about because then you can tell when AI is bullshitting you if you're going into something and being like, I have no idea where to even start. I think AI is also great for getting you up to kind of average. But you're never going to know if there's a hallucination, uh, if it's not doing it as well as an expert would. You just have no sense of that. So I think, I think it's really important to in my opinion what we're doing, we continue to work with experts. I'm always trying to hire experts in different fields like SEO experts and like design experts. Because what they, the people I hire, I hire them because they can do what they do way better than ch Claude or ChatGPT can. So, so yeah, uh, I still think there's a ton of value in having experts and then letting them use AI to make themselves more productive. Rather than like me as the CEO being like we're getting rid of all designers and we're replacing them all with ChatGPT. I think that's, that's just a disaster waiting to happen?

Speaker B: No, I like this perspective a lot. That encourage them to use AI on the things that they're already good at because the human in the loop will be able to detect, you know, slop or hallucination or whatever. Right. You know, I saw something. It was probably on X or whatever, but it was indicating, uh, some measurement of uh. Is your cognitive skill declining from using AI? They said, can you recall the last five requests you made to an LLM? I was like, man, I don't even remember the last one I did.

Speaker A: I don't know if that is that relevant. Do you remember the last five Google results you did? Remember the last five times you called your mom? I don't know. No, it's like, I think that that's fine. Like, like that's not necessarily a problem. I do think it's a problem if like I am putting together like a. Yeah. A proposal, a document, something, putting it, I'm sending it to somebody else, but I don't fully understand what's in it. And I couldn't tell you what's in it by heart. I think that's where you, you get a problem.

Speaker B: Mhm. So, ah, I don't know. Do you have any, any tips or how are you making sure that you're staying, ah, cognitively challenged and you're not relying too much on the models?

Speaker A: Um, you do anything in particular? Well, so what I do, my background is in software engineering. So I, I actually take, I block off my calendar every Tuesday. I take no meetings and I just use this time to code. Some of the coding is with AI, some of it is without. But I'm trying to still stay close to the product and make uh, sure I'm still really deeply involved. I think it's great to take a step back and be in manager mode. But I also think staying close to the core product is really, really important. Uh, and as well as I think grounding in experts, like I mentioned before.

Speaker B: Yeah, yeah, yeah, you know, I guess that's one of the quotes. I don't remember who it was, but it was, you know, um, a scion of industry. He was being interviewed and the interviewer, um, was kind of taking a swipe at the person. It wasn't Thomas Edison, somebody of that stature talking about how, um, you know, how can you lead this company when you don't even understand the, you know, the dynamics of the product that you're making or the engineering behind it.

Speaker A: Exactly.

Speaker B: And he said, said oh, all I have to do is pick up a phone. This, this line rings. The expert that, you know, the PhD on that, this one. So it was almost like the same concept, but just with actual humans as compared to what we are able to do almost with, with models. But um, so Pangram certainly, like you said, copying and pasting into something to determine if this is like am I reading synthetic or authentic or is it predicted to be authentic information, not viable. Using the um, plugin certainly makes a lot more sense. It's on demand. It's right there with me as I'm going through stuff. But where else in like maybe a more sophisticated stack would Pangram fit? And for what? To what end? For, uh, different industry, I guess.

Speaker A: Yeah. So we have a pretty cool stack in recruiting. I think this is the one I'm most proud of. Uh, so if anybody uses Ashby, we have this integration. Uh, and the reason I think it's valuable. Again, it's again with the bad actors. There's a million, uh, there's these websites that will auto apply for you. They'll mass apply everywhere. And why would I hand apply to 10 jobs myself when I could use this auto applier that applies to ten or a thousand jobs and then, you know, whatever interviews I get, then I can decide if I like really even want to work there in the first place. Uh, so the problem with that though is I think there's a lot of candidates that are actually like low interest or low signal. So what I've been seeing people do is they just add a little text box at the end to like upload your resume, you know, give your information, but also just like, hey, please write a few sentences about why you're interested in this company. Like don't use AI. And if they use AI, uh, when the instructions say don't use AI, you can just actually automatically disqualify them, which I think is really great because then you're only getting the high intent candidates who actually wanted to work there.

Speaker B: Yeah. How did it come about that that was one of the areas in the spectrum of business that Pangram started getting real traction.

Speaker A: I think this is just like something that customers had been asking for for a while. Like, like we were seeing recruiting teams that were actually just like buying Pangram subscriptions and doing the copy paste themselves. And they're like, hey, can we like, can I instead like upload a CSV and have you like, ah. Or yeah, can I upload a spreadsheet and then have Pangram give you give me a result for every candidate and I'M like, oh, well, we could just like work inside your, um, inside Ashby instead. We'll work directly with you on this. And so that's how this came to be. Uh, we work pretty closely with a lot of our early customers still. And I think this is really like, this product shape has hit out of the ballpark. Um, uh, we're still, I think, looking for more like this. I think something that I really want to do but just haven't gotten around to doing yet is building this, uh, email integration, which is the same thing. Wow. I just want AI generated emails to go to a different folder. I'm not going to delete them. Not sending them to spam, just putting them in a different folder so I can take my time and see if they're relevant.

Speaker B: You know, that's like my, uh, like a lot of others. My inbox is scary, right. And I would assume that a lot of it is certainly augmented, if not outright synthetic content.

Speaker A: Totally. Especially there's these cold outreach pipelines that of course, it just uses an LLM. Write a quick paragraph about like, why Product X is relevant to person Y. Yeah.

Speaker B: And for those, uh, of you listening who have a business account, like, I don't think that you can't get in trouble for spamming business accounts. Not quote, unquote, not like, I know that there are different rules for me as a business sending to a B2C a consumer, as compared to me as a business sending to another business.

Speaker A: Oh, interesting.

Speaker B: Right. So I can, I can be a lot more aggressive or I don't have to worry. I can just get a list, scrape a list from Apollo and send to everybody on there and not worry about consequence other than, you know, brand damage or something like that. So if you're a business, like, please understand that, uh, as this AI slop, um, increases, you are going to be seeing a lot more of those, what Max M was talking about, those cold email outreaches that are what seems human. But again, if you have that palette, you can be like, nope, this was AI generated. Um, so what should a full trust stack look like for a business? If I want to make sure that, I mean, hell, I'm in the AI space, right. And I want to make sure that, um, our clients understand that we're behaving in an authentic manner, that we're not just outsourcing everything to AI, which I guess would kind of be eating the dog food, but not necessarily the, the, the brand experience. I would.

Speaker A: Yeah, yeah. I don't know. It's, it's like if, if, if outsourcing to AI is even cheaper than outsourcing to the Philippines, then I think that's sort of what it's going to indicate to customers.

Speaker B: Yeah, um, well, what would a full trust tech stack look like for a business then?

Speaker A: Mhm. Yeah. So I think getting the Chrome extension for the socials and just like the general Internet is really good. Getting the um, recruiting stack, that's what we use ourselves. I um, guess hit me up if you're interested in the email thing. Yeah, I'll probably code it up using AI, uh, some weekend and then put it out there. Uh, and then beyond that I think we just see uh, I see some people who use the Pangram API to automatically check some things that come in. So like publishers, anybody who has submission queues, I think that's a great way to kind of just validate that what you're getting inbound is a real person and you want to take your time and treat them as such.

Speaker B: So uh, what are your thoughts on when would detection be enough versus maybe more robust approach such as watermarking or identity verification or internal policy on usage, stuff like that.

Speaker A: I love watermarking. I think watermarking is one of the most interesting technologies of this time. I think it's really cool how you could look at an image and it looks just like real life but then it has this invisible Gemini watermark and it comes back as AI. Super cool. One of the problems is watermarking on text is a lot less robust just because of how text is often a lot shorter. There's no pixels that hide uh, this invisible watermark in the same way there is for images.

Speaker B: Yeah, yeah.

Speaker A: Uh, the other issue with watermarks is it's an opt in system. So Google Gemini, like Google Images are all watermarked and, and I think only recently like OpenAI said, they're going to start watermarking images too. So OpenAI images also watermarked. Great. Uh, if I want to create some like Iran disinformation, I'm just going to use a model that's not Google and not OpenAI. I'm going to use one of the unwatermarked um, models. So I think it's the trust issue is you can't trust that a bad actor just happened to use a watermarked model because there's so many unwatermarked things out there. Um, I think it's really great for uh, giving additional validation and I think it's cool technology but I think it's not the only thing that you can rely on.

Speaker B: So now let's kind of put a little twist on this. Uh, what if I as a company want to understand how my knowledge workers are using AI generated output to. For both internal and external communications? Um, how do you recommend companies deploy detection internally?

Speaker A: That's a great question. I've had some people come to me and ask about this. I, uh, we don't really have a product solution for this, like just. But I think, yeah, I, Yeah, we, we've had a couple companies who have just gotten Pangram for all their employees and they talk about they want to fight against work slop. They don't want people sending other people 20 page documents that are fully AI generated that they haven't even read themselves. They're very anti workslop. And so what they do is they instead just empower all of their employees to have Pangram and to kind of use it liberally on internal docs and make sure, you know, if something's AI, like there's no shame in it, but you should kind of just disclose it as such. Like, hey, this was uh, like I generated this with Claude. Uh, it might have some hallucinations, but like, you know, I double checked it for hallucinations. So like we should be good versus like this came straight out of Claude. I haven't really looked too deep into it. Uh, you know, you should approach it a bit differently if that's the case.

Speaker B: So this is interesting because I hadn't really thought much about it prior to asking you that question, but this is something that maybe should be in like an AI use policy for a company. Because one of the things we do is we help companies because a lot of them, they don't have any guardrails, they're just like, go use AI. Right. And this would be one that I think by default a, uh, leader that was concerned about risk is concerned about a breach or IP leakage from somebody, you know, putting something into the models and whatever. But they're not necessarily thinking about a use policy about how do I want internal communication that's generated by AI to be uh, indicated or how do I. What's our stance on that? And I think that that's something that probably should be a part of a use policy. Like if you're going to use AI, um, it's not a copy paste situation. We still want quality output from you.

Speaker A: Yeah, and I also. Look, I think transparency is so great because it works. It helps in both directions. One, it encourages people to not just um, pretend that AI content is their own, uh, Encourages people to be transparent, uh, and put out quality work. And then on the other side, it also shows other people what they can be capable of if they're using AI properly being like, not only if you disclose, hey, I use some AI, but this is how I use AI. This is how I still produce, uh, a quality document, even though I leverage AI. Yeah, great, do it.

Speaker B: That's something that we need to think about internally at Chief AI Officer. So I know you guys have some big clients. Quora is a client, right?

Speaker A: Yeah, ah, Quora is a client and

Speaker B: they're processing obviously large volumes of content with you guys.

Speaker A: Millions and millions. Yeah.

Speaker B: So I guess. Were they using detection prior to their relationship with you guys?

Speaker A: They had some stuff internally. Uh, it just wasn't as accurate. Yeah, they were one of our earliest customers. So, uh, they've got a pretty good deal, uh, at high volume. But I think we worked with a couple other social media platforms. Um, we work with some publishers, uh, we obviously work with a whole bunch of schools and universities,

Speaker B: you know, and that kind of. We'll get into that education question, but I have a question about, uh, cora's experience. What does a major platform like that see once it turns detection on? Like, how does that, like how does that change their economics and the way that they operate?

Speaker A: Um, I mean, I think it's, as always, it's a quality issue. Right? If, if I suddenly like, I'm able to turn on a ultraviolet light and see like, uh, oh, actually my, my users, 10% of what they see on my platform is spam or, or is inauthentic, then that's like, suddenly that's a huge problem. That's like, you know, we also, it's just such a huge win if we say we're going to take this inauthentic content and just like silently remove it from their feeds. Suddenly Everybody's experience gets 10% better. People, uh, are happier and they're going to see higher retention. So I think that was sort of, uh, like 10% is not the number, but like that was kind of the process that we went through where we did a full audit where they ran a month's worth of content, ah, through Pangram. They had actual data labelers going and looking at the content and trying to see does the Pangram prediction match what we think? Um, is this actually inauthentic content? And they went super deep because they are obsessed with quality. That's why they're still one of the websites with the most traffic. I think there's still like top 25 trafficked um, websites in the world. Um, so many years after they launched, they still have a lot of users and I think it's because they care a lot about quality.

Speaker B: I like to know that they do care that much about quality. They'll be in it for the long haul. Um, so I don't know if it was you and I when we were on the original conversation, but, but this, an AI agent had tried to contribute code to an open source project and then got rejected because it was humans only. And then. Are you familiar with this?

Speaker A: I think I saw that, yeah.

Speaker B: Yeah. And then the agent generated a long blog post takedown of the person that had rejected him. So how do we like. And if you think about this like somebody a, ah, malicious actor in your industry. I mean content warfare could become cheap and scalable. If this, you know, this is a small preview, but what happens when that's 10,000 posts.

Speaker A: Exactly. Yeah, yeah. What's stopping them from stopping at one blog post? Let's make 10,000. Let's reply everywhere on the Internet where this individual has a presence or where they're named and badmouth them. Oh, you could just run that with an agent for a few bucks. And so I think that's where I'm really worried about the future of the Internet. A lot of people talk about dead Internet theory where it's just bots talking to bots. And I think we've never actually been as close to this being true as we are today.

Speaker B: All right, now let's talk about the education side. A lot of the people that we work with, typically they're executives, they got kids and they ask me, so what should I, what like what should have my kids studying? Or what, what is this? What's what jobs should my kids be focused on? And I was just listening to a podcast today about how in 2023 it was like, you're gonna, you're, you'll fail if you use like, we're gonna give you a failing grade if you use AI, Right. And, but to me and to you, it's like, do you want to discourage or, or plant the seed in these students minds that AI Bad, you shouldn't use AI. I mean, what happens when they get into the uh, environment, like the arena of commerce, and they're like, oh no, no, no, I don't use AI. I mean like, well then, oh no, no, we don't use you kind of thing. So what is the, uh, appropriate approach that education should be taking so that people aren't doing the full Outsourcing of my thinking copy paste from Claude. Um, but they are still able to develop the skills of thinking in AI and being able to use the models effectively to enhance whatever job they're going to be having.

Speaker A: So my opinion, the number one most important skill to develop is still critical thinking. You have to be intelligent to be a good user of AI tools. If you're not there, then, then you're kind of just like a meat body for AI to tell you what to do. I don't think that's a great way to interact with AI. So I think number one is ensuring critical thinking skills. And I do think part of this is learning the skills, learning to do it without AI and then learning how you can use AI to be a force multiplier. The best way I see, um, schools and universities, uh, use this is by having no AI policies early on in the coursework and then letting students do, for example, a really ambitious capstone with AI. And the idea is if you have AI, if you're using AI, you should be able to take on something that's three or five times as ambitious as you would have done without AI. It's not about doing the same amount of work, but you're less involved because you're using AI. You have to be able to do more with AI.

Speaker B: Are you guys being approached by, um, schools and universities and things like that?

Speaker A: Um, yeah, all the time. I think this is, um, one of our biggest sectors of business. I, uh, think there's a really big need for this especially. Look, I think there's a lot of places where it's reasonable to use AI, but if we're talking about a composition class or an English literature class, it's like, don't use AI to just write your essay for you. It's so obvious to a lot of people. But I think still this is something that educators, uh, need tools to help them enforce because otherwise a student choosing between no homework or a lot of homework is going to choose no homework.

Speaker B: Yeah, you know, there's, there's the, the devil on the shoulder that says, but if they're using it to, to creatively game the system.

Speaker A: Yeah, the thing is, it's like not that creative. If you just like copy paste in the assignment and then copy paste the output into a word doc. Uh, uh, yeah, I think there's, there's, there's ways to be. Yeah, Oops, sorry, I almost closed out on my tab. There's ways to be creative about it, but I think overall, I think the best way to, um, make sure students learn is to give instructors a lot of control over what their exact AI policy is. And I think it's really great that there's some instructors that say, hey, don't use AI at all for this assignment or this class. And some instructors that are like, here is like, we're going to learn AI skills and like, we're going to learn how to use it. And getting, getting a sense of both, I think is really the best of both worlds.

Speaker B: Okay, so if, uh, I'm listening to this podcast and I run a company of 200 to 2000 people, what are the first three moves that my leadership team should make this quarter? Immediately to protect that reputation of trust that we have with both our internal stakeholders, our employees, but also our customers.

Speaker A: Sure. So I think the two things I talked a little bit about this untrusted, um, layer of communication. So anything that you're getting inbound that you don't know the source, I think you want to make sure that it's human written or not AI generated, or if it is AI generated, you want to know about it. So I think getting Pangram and using it there is really helpful. On the flip side, we have, um, things that are going outbound to clients and customers. I think if things are AI generated, we don't want it to look like total slop. So I think it's important to disclose our AI generated outputs. And especially when we're showing them to customers and people we want to impress, we want to make sure that, yeah, we're not sending out slop.

Speaker B: So, you know, I guess I should have probably brought this up a lot earlier in this episode. I've just been assuming that the detection is for text content.

Speaker A: Yes.

Speaker B: Are you guys capable of detecting image

Speaker A: and video, uh, today, just text.

Speaker B: Okay. Is there, like, is there a whole, um, industry that is about identifying, um, deep fakes on video or images?

Speaker A: Yeah, there's a bunch of companies. I think one of the bigger ones is Reality Defender. They do image and video. Uh, and they're pretty big on deepfakes. Uh, they can, like, their product is more like, we're going to help you make sure you're not, like, hopping on a zoom call with the CEO. But it's actually just like a deep faked, uh, actor. And then they tell you to wire money somewhere else and money. So I think that's the deep fake angle that they're helping solve, which I think is also, like, that's a very important and financially motivated.

Speaker B: Sure. Now, so if I'm, you know, all Things aren't created equal. If I'm getting a high success rate of detection with Pangram, I shouldn't assume that that same high detection rate is going to run with image or video detection. Is that correct or is it as good?

Speaker A: I don't think there's anything in image or video that has a 1 in 10,000 false positive rate like Pangram. Uh, I may be wrong. We've done some uh, benchmarking, but we haven't seen anything really come close. I think Pangram is really, really powerful technology and uh, yeah, yeah, I think it's great. But uh, I think it's always worth testing every tool for yourself as well. Don't even rely on the benchmarks. Just, you know, run a test yourself.

Speaker B: Good point. Okay, let me ask, where do you think detection in general will be in 12 or 24 months? Like what will it catch? Well, and what do you think? It'll just never catch reliably.

Speaker A: Look. So I mean I think a lot of people said like AI, like detecting AI, uh, content is impossible in 2023. They were actually right. Like there was nothing good months after ChatGPT came out. And so I think a lot of people assumed that technology wouldn't get better, which I think is a really interesting assumption to make given that every technology I know is always getting better. Um, detection is interesting because it is a bit more adversarial. But I think what we've seen is that um, we've so far been able to keep up with and even outpace a little bit the rapid development of LLMs. So the future that I'm looking at, I'm seeing that major, uh, model providers are getting a bit more responsible with how they deploy, especially um, image and video. They're all now watermarking their image and video except grok, uh, and I think the text as well. I think this is where Pangram will continue to be relevant is just as agents scale up and are producing more text content, uh, we are going to need to find uh, ways to differentiate between uh, these AI agents and real people. And so the direction I'm seeing the next couple of years go is we're going to have to keep up with these agents that have a bunch of context and they're synthesizing a bunch of other information. And we're going to find a way to not only catch, uh, these high context agents, but also to kind of do better at differentiating between AI assisted content and content that's fully like de novo AI generated.

Speaker B: Yeah, yeah, interesting. Uh, you know, this is certainly every time I do one of these episodes, my intention is that I expand my worldview of how people are using AI and that sort of. And do the same obviously for the listener. This has certainly been that. Because the last time that we had a guest on to talk about text, uh, detection, it was one of your competitors based in Canada. Um.

Speaker A: Oh, hey, yeah.

Speaker B: Uh, it's been over a year. Right. And you and I and the listeners at this point hopefully know that a year in AI like ability, I mean, Pan Gram was a completely different company a year ago than they are today as far as what they're able to do. Right. So it's good that, um, because I hadn't really considered the, the, um, a number of things that you brought up, especially the internal angle. I always looked at it as like, oh, how do I make sure that I don't get that I'm not ingesting, uh, a bunch of spammy AI slop. But the reality is I also really need to consider what our internal policy is for internal usage. I want my people, we're an AI company, man. I want my people using the tools, but I also don't want them doing copy paste ship. I want them.

Speaker A: Yeah, yeah, you want them to be effective without, uh, like inundating everyone around them with sloth. Yeah, right. There's like a great, um, line in the sand. And personally I think like, disclosure and honest disclosure is like a really important part of that.

Speaker B: Yeah. Um, now I got to figure that out for our use policy, but I'm glad we covered that. So.

Speaker A: Yeah, have fun with that.

Speaker B: Yeah, I'll let you know how it goes. Um, for the listeners who like, they're like, we don't have a solution for this. We haven't even considered this. We have no filtering. We have no mechanisms in place to make sure that we're not wasting our time. Because if you think about it, a competitor could flood you with a bunch of crap and have your customer service team, like all of a sudden your Net promoter score goes down because AI

Speaker A: generated one star reviews on Yelp and Google. Yeah, could happen to, could happen to you.

Speaker B: So if, if the, the uh, audience wants to know more about how they can protect themselves from it, what you guys do, and that sort of thing. And again, I said we're going to put a link to that, um, to the, uh, Pangram Chrome extension and the show notes. But where else should they go to get some more information about how you guys are protecting them?

Speaker A: Yeah, you could find us online@pangram.com Twitter @Pangram Labs, LinkedIn, Pengram Labs, uh, where we're all over but come find us, get in contact with us. We're really happy to talk with you. We're a mid sized team. I know you said we were 10, uh, in February. We're 17 now as of Nice. So, you know, growing. Uh, but yeah, we're happy to talk with you.

Speaker B: Very cool. Yeah. And if you're a listener, I wouldn't encourage you. Oh, we'll get around to it. This is something that if you're not prepared for it, it's only going to like, start paying more attention to your LinkedIn, uh, thread and you'll, I think you'll start noticing. Like this is.

Speaker A: Once you start noticing, you can't stop noticing.

Speaker B: Yeah, yeah. So I need to go back through my stuff and see who's, who's real and who's not so. Well, Max, thank you so much for taking the time and sharing. Obviously your expertise on the subject of learned a lot and I've got a lot to think about on again, how we're going to do it and how we're going to advise our clients to prepare for this future.

Speaker A: So, um, very interesting.

Speaker B: Awesome. Yeah, thanks man. And for all the listeners, uh, if you got something out of this episode or any of our episodes and you've got peers or teammates or co workers that you think, um, they're not quite as far along on the journey as they need to be, we'd love it if you would share, uh, what we're doing here at, uh, using AI at Work with them so that they can hear more interesting guests like Max and all of our guests and start to get a better perspective of how this is going to apply to them using AI at Work. So thanks everybody. We'll see you next week for another amazing episode of the show.

Speaker C: Thanks for tuning in to Using AI at Work. Don't forget to subscribe for more conversations about how to use AI at work. And a special thank you to our sponsor, Chief AI Officer for Empowering Businesses with AI Education and Training. Visit their website for a free AI Readiness Assessment and AI Strategy Guide to help you get started using AI at work. That's www.chiefaiofficer.com. follow us on Twitter at the handle using AIAwork and visit www.usingaiatwork.com for free resources to help you harness AI in your role.