Reddit Is Training The Robots

Speaker A: Foreign m. Welcome to Ad Exchanger Talks, the podcast devoted to examining the issues and trends in advertising and marketing technology that matter most to you.

Speaker B: This episode is sponsored by Basis, the leading intelligent operating system for autonomous advertising. Its enterprise AI solution transforms campaign briefs into strategies and media plans that integrate directly into omnichannel activation.

Speaker C: I'm Allison Schiff and you're listening to Ad Exchanger Talks, or rather let Ad Exchanger Talks, because I was just in Cannes. My guest this week is Jen Wang, the chief operating officer of Reddit, and we recorded this episode in the podcast studio Reddit built inside its two story space on the beach. We'll dig into how Reddit ended up training a lot of the LLMs people use every day, what those big data licensing deals with Google and OpenAI actually mean, and how Reddit is trying to protect the human signal in its corpus. At the same time, others are trying to use Reddit to influence AI search. But first, quick little plug for programmatic IO New York. Taking place on September 28th and 29th at the New York Marriott Marquee, the open web is being reborn and will stuff your mind full of innovations, insights and practical information. Come mentally hungry podcast listeners. You get 10% off the price of your ticket when you use the promo code POD10. So, whatcha waiting for?

Speaker D: See you there.

Speaker A: M hi.

Speaker D: Hey. I usually say welcome to the podcast, and yes, this is my podcast, I guess, but this is your studio space at Cannes lion, which is very cool and people can't see, but on the other side there's a glass window and people are milling about in the Reddit space. How's can been for you so far?

Speaker E: Been great.

Speaker A: It's been great. I love our space. This year it got a little bigger, um, maybe our third or fourth year here. So we have the Reddit deli. Um, we've had just a banner number of great meetings, um, with a more diversified group of clients and partners than ever. And I think the level of conversation, because there's so much happening, there's so much change happening, is really interesting. Like what is on people's minds? That's sort of my favorite part of it.

Speaker D: What's on people's minds? Can you give me a little taste?

Speaker A: Well, I'm curious what you, what you've heard too. But, uh, you know, from, from my side, I think one is last year I think people talked about AI in a very theoretical, maybe anxious way.

Speaker D: There was a, uh, nervousness and attention and a sort of trailing off in people's voices, like they were worried about their futures.

Speaker A: Yes, like a dot, dot, dot. Right. And that still, still remains. But I think that this year people have come back and are really ready to talk about MTP and opportunities to not just improve workflow, but just improve the whole way that things are done. And I, um, think that's exciting. So it's gotten a little bit more practical and moved forward. Um, and I think there is maybe a little less anxiety and more uh, knowledge. I'd say that's one. I think the second is, uh, just the changes in the Internet, uh, the dramatic changes in the front face of the Internet, um, with LLMs, I think is on people's minds, um, a combination of organic search going away to what is the new path of discovery and what is the new consumer decision journey in this world where the products and the experiences have changed and how does it all come together? I think that's very much on people's minds a lot. Like how do I remain, you know, how do I get visible in this new world? Um, and it's different from different, you know, apertures. So that's interesting I think on people's minds and from a marketer standpoint, like what does that mean for search and all the work that I've done in search for so long.

Speaker D: I do want to talk about AI, obviously. Um, so, but from a zoom out, right, because Reddit has been doing data licensing deals with AI companies. So it's 60 million a year with Google, right. And 70 million with OpenAI. Um, and that gives the LLMs access to your guys content to train their models. But now Reddit is kind of going back to the table and pushing for something a little bit different. Dynamic pricing, which I think is really interesting. So instead of a flat fee, you get paid when your content is used to generate an AI answer and maybe more if the answer is central or the content is central to the answer. So uh, I'm just curious how the LLM makers are reacting to that. Are they open to it? Like how are the negotiations going?

Speaker A: So we, we've not publicly, I think, or committed to any model or model change. I mean that's something that just, there's nothing really to share on that. And we will share at the time that there is, but there isn't. Um, I think we've learned a lot. I mean the environment is dramatically different than when we did these partnerships two and a half years ago. Right. We didn't know that, um, Reddit would be such a high portion of citations

Speaker D: and influence because of what it is.

Speaker A: It does it's so different when it, you know, happens.

Speaker D: And it's so interesting too, because it's so much of humans. So humans truly beating the LLMs with their humanity, with their anxieties, their fears, their advice.

Speaker A: Yeah, it's such a funny thing. Like we stand in awe of the technology of AI. And of course that was all created by humans and is taught by everything we know as society and humanity. Um, so sometimes we forget that. But yes, I mean, we continue to believe in the power of humans and we're building, you know, we have the biggest trove of human intelligence, I think, and continue to, continues to get wider and deeper, I think, you know, so for us, that's what matters to us, right? We need and want humans on Reddit. It's human experiences in the real world that generate the posts and ideas on Reddit. The ranking of what's good, the commentary. Right. The point, counterpoint, the perspective that you can't get anywhere else. And that is what's important to us because that's aligned with our mission. I mean, I mean, we just turned 21, believe it or not.

Speaker D: Oh, uh, you can drink now. Congratulations.

Speaker A: Yeah, we can drink now. And you know, we still feel so young as a company in our journey in accomplishing that mission. But what's important to us is that we have more people on the platform and that's, you know, we think about feeding that flywheel, making sure that more people use Reddit as a destination. That's kind of our North Star above, uh, all else. And when people come and they enjoy Reddit and they contribute to Reddit, honestly, from there, we have a great business that flows from that.

Speaker D: But how do you make sure that continues? Because humans are obviously integral to Reddit's mission, which is to bring people together so they can share information. And AI slop is a problem just on the open web, but you have to protect the sanctity of your corpus because that's what you are. It's the most important thing. It goes right to the central heart of why Reddit exists. So how do you do that? How do you guard against the, the deluge of crap which is coming and kind of already here?

Speaker A: Well, I think we've been inoculating ourselves to spam for 21 years, right? Spam is just bad content. And AI generated bad content is just bad content.

Speaker D: But it's kind of a new creature.

Speaker A: It's, it's a, there's a new frontier in potential abuses. I think that is, you know, that is true, but we've Also been doing this for 21 years and we have multiple layers of ways to inoculate against that. Um, if you think about our voting system, that's humans judging the content. And what's important to us is that there's a human behind the idea. And that's, you know, the work that we've announced regarding human verification to make sure our human is close to the idea when it's posted, obviously, commentary, etc. That's really important for the sanctity and the what people expect from Reddit, what they want from Reddit. Now, we've been fighting spam for a long time. Will most people have content that is AI touched? Maybe. Maybe it's the next incarnation of spell check. And maybe translation is something that you need because your English is not perfect. We can't stop that.

Speaker D: And that feels different though, right? That feels. But there's.

Speaker A: But there's gradations in the spectrum.

Speaker D: Yeah.

Speaker A: And I think it comes down to, is the idea good and helpful? It all comes down to the same thing. Is it good? And humans are discerning in being able to say, was this good and was this helpful? And that system, I think, has helped us for many years fight spam and not good content. And I think it's a system that will help us in the future too. In addition to AI tools and safety tools and all the things that we have that AI also unlocks for us, but at the heart of it, the judgment by humans is still, I think, really quite discerning.

Speaker D: And by that metric, some AI generated content is fine because you're asking whether something is useful as opposed to designating something as problematic just because it was generated by AI.

Speaker A: That's right. I mean, look, most m. A lot of our, um, communities, they have rules against AI content. Um, and that's fine. And that's fine. They've made rules about other communities might be open to, like a creative community. That's fine. These things are AI generated. When we're comparing them, that's fine too.

Speaker D: So we're going to take a quick break and when we're back, I have some questions about a study that you guys recently did, uh, into behavior on the platform. So stick with us. Foreign.

Speaker F: I'm Sarah Sluice, editorial director at Ad Exchanger, and I'm with Katie McAdams, the chief marketing and commercial officer at Basis. Welcome, Katie.

Speaker E: Thanks for having me.

Speaker F: So Basis has found that media teams today are juggling an average of nine different platforms to run a standard digital campaign, which makes my head spin Because I know when I like switch browser tabs or Switch products, I'm like, wait, what was I here for again? Um, so how does impact their ability to be successful, um, as an advertiser?

Speaker E: Sure. So it's a great question. What we find in our research is that our industry is losing anywhere from 80 to $100 billion annually in value leakage from errors, inefficiency and siloed campaigns sitting in all of those different platforms that you're talking about. And advertising is just becoming more and more fragmented, whether it's across teams, channels, tools, finance systems, and now different AI solutions. So that's a lot of context swishing for one in one day.

Speaker F: Wow. Um, so what would a connected advertising system look like as an alternative to those nine platforms?

Speaker E: What we find in talking to agencies and brands is that uh, that journey really needs to start with consolidation. And by consolidation I mean getting all of your media contracts, your campaign plans, your invoices and your client communications into one place so that you have a single source of truth. And once you have that foundation in place, then something important really starts to happen. You actually have data that is clean and reliable so that your AI can function with it more meaningfully and more predictably. So, uh, the brands and agencies that get to this state fastest are not going to be the ones who are bolting on the most AI tools. They're going to be the ones who are able to build that operational foundation first.

Speaker F: I like this point that centralization isn't just about me as the media planner, but also about having more unified data that will then help me with AI, which I'm glad you brought up AI. So tell us a little bit more about how AI is being added on to this connected advertising system.

Speaker E: Sure. So this is where having that solid foundation in place is going to actually help AI become more of a multiplier for your organization and your teams. As an example, basis has Compass, which is our agentic AI planning tool. Uh, it lives right inside our platform and it solves the problem of media teams spending hours and even sometimes days synthesizing media briefs, building frameworks, building media plans, and then creating client ready presentations before a campaign even launches. So Compass actually takes that brief and generates a complete omnichannel strategy across programmatic, search, social and direct in minutes instead of weeks. That strategy then becomes connected and pushed into their media plan, which can then be activated on through the basis platform, across programmatic, search, social and direct media buys. And what we find is that agencies and brands who are using basis overall are seeing 30 to 40% operational efficiency gains when they operationalize all of those workflows into one place. And that really creates an expansion of capabilities with teams being able to gain back time to focus on strategy, creativity and growth.

Speaker F: So we have more efficiency through centralization, which then enables more use of AI, which is even more efficient. So, really interesting to talk to you, Katie. Um, and thank you to basis for supporting our podcast.

Speaker E: Thank you.

Speaker D: All right, we're back. And before we talk about the research that Reddit just conducted, I have one more AI related question, and it's to do with integrity. So Reddit is a, uh, primary input for how AI answers a lot of questions. For all the reasons we were talking about during the first half, that makes it useful for brands, but I think it's also making Reddit into an incredibly attractive target for bad actors that want to try and shape what AI says. So how do you defend against that? How do you inoculate, I guess, to use your word against that?

Speaker A: I mean, we have a way of addressing manipulation. You're not allowed to manipulate Reddit. That's actually one of our, um, core rules. And our communities also, um, really, you know, they also inoculate themselves against that through their moderation and their rules. So it's not something new. Obviously, this is like another level, you know, in the arms race. But I think our systems work really well, um, in that there's policies, there's technology and tools that allow for monitoring and for moderation, and then there's the communities themselves that have their own rules and moderators to be able to actually see whether the content's good and whether there's manipulation happening, a coordinated manipulation, you know, let's say, uh, you know, across multiple communities or across groups of users. So this is not something new. This is something that we spend, we spend a lot of time on. I think we've done a really good job. If you look at so many moments, you know, over the last, let's say two decades where you might think that there might be, you, uh, know, people who are interested in manipulation, I think we've done a really good job in handling that. So I always tell people like, you know, Reddit is set up so that it cannot be manipulated. It's not perfect, but it has systems unlike any other to really prevent that. And the biggest being that our users and our moderators are in it with us.

Speaker D: It's the sort of thing that Wikipedia also has to do except guard against astroturfing. Yeah, um, okay, so, um, you're Hearing hand dropping. This research, actually, um, it's a report called the Mission Mindset. You did it with WPP Media, and it looks at the purchase journey, specifically the role of validation, which is the idea that people don't just research products, they seek out communities to build confidence before they buy. And just to rattle off a few stats, apparently 79% of Redditors say they do more research before making a purchase now than they did a year ago, which is a noteworthy shift in behavior or just an acceleration of a behavior that was already there. What's driving that? And should marketers be doing anything different as a result, to kind of lean into it?

Speaker A: Uh, yeah, I mean, I'm not surprised by that because I think the. It's very noisy. Getting clear information is very noisy, and a lot of sources of information are, I'd say, kind of gray. Right? Like, uh, you know, how credible is this? And there's different gradations of that.

Speaker D: Um, gradations.

Speaker A: Yeah, yeah, sad pun. Right, so, so there's no doubt that people, for example, have been increasing their LLM use. Um, but if you look at how you get an answer in an LLM, that is usually a very efficient, highly synthesized answer with little texture for additional research. Contrast that with what you get on Reddit, either Reddit search or Reddit, where you get point, counterpoint, you get to see all the, yeah, two people thought this. Like, you get volumes. You get, you know, the nuance of words and how people describe things and how people experience things. And that gives you the ability to build your confidence because you need that texture and that perspective in order to form your own opinion. And it's, you know, it's kind of interesting. There's this new kind of behavior which is, okay, get, maybe get an LLM answer, and then I might cut and paste that. It's almost like a new post. Type into Reddit and say, well, what do you think? Or here's what, like this LLM told me, um, what do you think? What do you actually think? Right. Uh, like that actual validation or I searched here and then now we're seeing, you know, Reddit, uh, Agentix search grow. Um, type in the same query of Reddit and look at the compare contrast of the, uh, of the two answers. So I expect that to happen more and more because I think people want, will always want to hear, especially on big decisions, like, what is somebody's experience with this product and especially with new products where there's very little corpus of information anywhere, you know, they might want the formal information and then they'll want the, okay, what's the rule? Scoop. And I think that's just happening more and more. It's becoming. Everything that we see is becoming a more ingrained behavior in the decision journey and a more influential factor in the ultimate decision.

Speaker D: What a world, though. So Reddit is feeding LLM answers, and then people are taking those answers and putting them back into Reddit and cycling them through to see whether they're valid or not. And then that feeds the LLM, and I guess that just happens over and over again.

Speaker A: Well, what's interesting, it's true. And it almost stands in contrast.

Speaker D: Yeah.

Speaker A: So it's kind of a funny thing.

Speaker F: Right.

Speaker A: So the. What I would say is the product matters.

Speaker F: Right.

Speaker A: So we think a lot about what is the best way to enjoy and experience Reddit. So in that sense, the product and the corpus and the human intelligence, like, are hand in hand and come together in harmony to have the best experience. And that's why we think what we're developing is the best search experience on top of Reddit. Uh, the best way to find communities in Reddit, which is different than, you know, maybe a way like how Reddit adds value to, like, an LLM's perspective. The other thing is, on Reddit, you're getting everyone's opinion. It's very different than the opinion or perspective of the LLM. And what's interesting is, like, the LLM based on your personalization has that perspective. Um, but it's one perspective.

Speaker D: It's a synthesis, usually. Right.

Speaker A: It's a synthesis sometimes articulated as a perspective.

Speaker D: Yes.

Speaker A: With an objective function of your retention and happiness.

Speaker D: So it's well said. Uh, well, one other stat that stood out to me. So, uh, 62% of Redditors say that a thread debating the pros and cons of something gives them more confidence in that thing, in that product, more than reviews, more than influencers, even though those things are important. So what does that say to you about how brands should be showing up on Reddit? They should be honest about, about their products and not just rah, rah, rah all the time. People respect that.

Speaker A: I guess what I would say is don't fear different opinions. It's okay to have people who are your fans. You always want to talk to them. You always want to create more of them. That's great. That's what the marketing platform's for. Um, and you should talk to Redditors because they're influential and they have important conversations that are valued. But don't fear the fact that there are people who might disagree. That's just, you know, what they really think. And it's better to know what people really think, because that creates an opportunity for discourse. You may disagree, but the best thing is to, like, let's talk it out. Because if we disagree, it's usually we have different facts, we might have different values, we might see the facts differently. Like, all these things might be a part of it. And so it's better to know where you stand. It gives you the opportunity to engage in discussion.

Speaker D: And having comments on ads, I think is also cool. That's been there for a while, and probably at the beginning it was a little bit of a challenge to get brands to do that, but it fits right into what you were just saying. I mean, open yourself up to feedback. Feedback can only be good, even if it's, uh, criticism. Because criticism, maybe it doesn't always feel constructive, but constructive if you take it in that way.

Speaker A: Yeah, I mean, I agree. I think that's the right, um, perspective, which is like, how can we make our product better? How can we take the insights on Reddit make our product better, our customer service better, our experience better, our messaging better, all these things on ads. Interestingly, the majority of campaigns have comments off, actually.

Speaker D: Okay, they do. We haven't convinced people.

Speaker A: Yeah, well, it's not that we need to convince people to have comments on, it's that every, um, space on Reddit needs to be moderated. And so, ah, it's just a lot of work. It's. Yeah, exactly. It's a commitment to, you know, is the purpose of this campaign to have engagement such that I'm going to moderate it and have a conversation? Or is it this more about messaging or is this about, like, selling, you know, items, uh, or app install? So those are very different objectives. So you can turn comments on when your objective is engagement or asking a question, collecting information, polling, like these kinds of things. Absolutely. But in many cases that's not the objective. And so, you know, you go comments on and don't have to moderate. So.

Speaker D: And so, uh, we're in Cannes. Um, the studio is so nice because I'm not sweating bullets. It's so hot outside. Um, but when you're having meetings with CMOs, I imagine all manner of marketing and agency person. What are some misconceptions people still have about Reddit? Do you find yourself having to correct the record, like, still, even though you guys can drink now?

Speaker A: Yeah, it's a great question. Um, you know, I think most of the things, you know, when I started eight years ago. We talked a lot about moderation and how the ADS platform worked fundamentally. Like why could it convert intent to outcomes and all the basics, like the fundamentals of Reddit, I think those are, you know, brand safety, all of those things are well understood now such that, you know, those are not the conversations. We're talking a lot about business outcomes frankly. How do we help someone grow? How do we help someone get discovered? Um, that's a big part of the agenda, like just growing their business, which is what we want, um, to be a great partner on. We are there. We are also talking about um, and I think there's this kind of discussion about GEO out there. Like yeah, it's kind of a funny geo.

Speaker D: Ae.

Speaker C: Aeo.

Speaker A: This is kind of like the idea how do I, how do I get into like LLMs and the kind of, you know, stuff that I did with SEO. And you know, I have to be honest, I, I am skeptical that there's like a program where you do X and get Y and have a formal, formal as much as like I, you know, I sympathize with like I think what are people, what people the problem people are trying to solve. And, and I think it's for all the reasons that we discuss like SEO and traditional search was based on relevancy and you know, as unknown as it was, it was also known in that it was, you know, universally shared and was ranked on relevancy. And now you're an environment that's hyper personalized and different for everybody and that's not controlled. I mean nobody knows what comes out until the time that the model actually gives its you know, opinion or synthesis. And it's so, it's much more complicated than that. And so I have a lot of empathy for brands are trying to, to, to, to you know, work through that. And I uh, you know, I often say for Reddit we had a conversation about manipulation. Like Reddit is valued because it is not controlled because it is free of um, influencers because it is free of commercial interest in the community. Table set conversations. If you want a table set, a conversation, you can, you do it through the marketing profit. That's how communities expect you to show up and when you do and you talk to people across multiple communities and you have you know, good sentiment and well ranked, um, you know, conversation, ah, that is started by the community. You know, that's when you have you know, really like great, I think community health. But that's not something that you can control because that's the whole point um, and so I think, you know, a lot of people are having this conversation about geo, which I find really interesting, but I guess I'm kind of skeptical. Maybe I'll be wronged.

Speaker D: I mean, I think it's a. A healthy form of skepticism now for the reasons you said, and for another reason. You could prompt an LLM with the exact same question within minutes and get slightly different answers, or it's phrased differently. So it's not a, uh, static thing that makes it even harder.

Speaker A: And the models are changing. The UI is changing. The view that those, uh, m experiences have of me or you as an individual is changing.

Speaker D: So we're pretty much out of time. And I want to give you time for, like, a coffee or two seconds to stare at a wall quietly before your next meeting. But my last question is, I don't know where I heard this. Maybe I read it somewhere that you follow something like a hundred or more than a hundred subreddits. Is there one that would surprise people, like, oh, not only is there a subreddit for that, but you're into it.

Speaker A: I mean, there really is a subreddit for everything. I have, admittedly, like, seven or eight alts that are tuned for all of my different interests. Um. Uh, you know, I, um, I. I'll tell you, I think that, uh, it's funny. So I have some subreddits that are not in English, too. Um, part of that is my effort to learn other languages and kind of test out Reddit in other languages. And I've had, uh, the pleasure of understanding, like, learning German memes.

Speaker D: Any different?

Speaker A: They. They kind of are different.

Speaker D: All right. Yeah.

Speaker A: Um, it's.

Speaker E: It's.

Speaker A: It's a really.

Speaker D: Are they serious?

Speaker A: No. No. Okay. No, they're, they're, they're, they're. They're funny, but very different.

Speaker D: Unhinged.

Speaker A: Um, no, no, just. Just entertainingly different. But, um, yeah, so that's been kind of entertaining for me recently. Um, I. So I spent a lot of time, you know, I spent some time there. I spent time on my hobbies, like gardening and cameras. And I love, like, Find the Sniper, which is not what you think it is, which is just basically kind of a game of finding, you know, a drop diamond ring in a pile of leaves for somebody.

Speaker D: Right.

Speaker A: Or finding a hidden snake in, you know, a pile of wood, um, which I think is really fun.

Speaker D: Well, uh, I'm gonna find those. And can you send me some German memes? I'm really curious.

Speaker A: Yeah, I will.

Speaker D: Thank you.

Speaker F: Gab.

Speaker A: Foreign

Speaker B: this episode was sponsored by Basis, the leading intelligent operating system for autonomous advertising. Its enterprise AI solution transforms campaign briefs into strategies and media plans that integrate directly into omnichannel activation. Learn more@basis.com.

Speaker A: Sam m.