Confronting the Disinformation Age: A Conversation with Renée di Resta

Speaker A: Foreign.

Speaker B: Hey everyone, and welcome to today's episode of Think Bad, Do Good, published here@attack IQ. It is my great pleasure to have my friend and colleague Renee Diresta here on the line. Hey, Renee.

Speaker A: Hey there.

Speaker B: So for those of you that are tuning in that don't know who Renee is, and I'd be surprised if any of you are in that category. But for those of you that don't know who she is, Renee is the technical Research manager at the Stanford Internet Observatory at Stanford University. It's a cross disciplinary program of research, teaching and policy engagement for the study of abuse and current information technologies. Obviously very important. Renee investigates the spread of narratives across social media networks with an interest in understanding how platform algorithms and affordances intersect with user behavior and factional crowd dynamics. She's also a very prolific writer and she publishes quite a bit. I'll put some of the links to her work in the podcast so you can just click on it. But she studies how actors leverage the information ecosystem to exert influence. From domestic activists promoting health misinformation like anti vax theories and conspiracy theories, to the full spectrum of information operations executed by state actors, which is obviously a very big deal. She's advised President Barack Obama, many members of Congress, and she writes for the public through tons of outlets. I saw one, um, the digital Maginot line. We'll put in the link, uh, uh, from Ribbon Farm was the first one that I read of hers. Gosh, when was that, like four or five years ago?

Speaker A: Yeah, 2018. 2019.

Speaker B: Yeah, yeah, that was a big piece that she published that really got her ideas out there. But now she publishes a lot with the Atlantic and Wired and you can find a ton of her work on the Atlantic website. And she publishes a lot of research through Stanford, which includes, includes her new report, Unheard Voices. Unheard Voice, um, published jointly by Stanford and Graphica. You can find that on the Stanford website and very exciting. Publishers Weekly has recently announced that she's working on a new book, which I promised not to ask her about because there's nothing worse than asking someone about their book while they're writing it. It's called Agents of Influence. So it's going to come out, what, in about a year and a half?

Speaker A: No pressure, God willing.

Speaker B: So, um, so keep your eyes out, folks, and we'll have her back on if, if she's willing once that comes out. Um, so welcome Renee.

Speaker A: Thanks for having me. Jonathan.

Speaker B: Yeah, thanks for joining. So let's dive in. You have said that propaganda has always been a part of Violent conflict, which I think is something that people probably know in the back of their head, but maybe they don't recognize constantly. And that's. So. That's a great point. But in the modern era, it takes on a different point than it has historically. And you've written about the role of propaganda in history. You introduced this concept of ample gander. And I wonder if you could just talk about that.

Speaker A: Yeah, I, um, hate making up terms. I feel like that's one of these terrible, uh, terrible things that you have to do in essays sometimes. Um, but what I was trying to get at in that piece was that propaganda evolves to fit communication technology of the day, right? And so many of the reasons, um, for conflict have not changed in the span of human existence. Pursuit of territory, of power, of, uh, riches, m. Various motivations that have always been there. Um, you have the human, uh, nature, which has not really changed very much in many ways over time either. A lot of the kind of psychological motivators, um, have been consistent. What do people need? What do they want? What are they looking for? Um, but so what really does change is the communication technology. And when we're talking about propaganda, which really is referring to messaging, we're talking about ways in which entities, uh, who are trying to achieve a particular objective use communication to send messages to the public. Sometimes that's for purposes of persuasion, right? They want to change people's minds. They want to make them believe a certain thing. Um, they want to create, uh, a type of, you know, particular type of patriotism, a particular type of demonization of some group. You know, there's different persuasion motivations there. Um, but another. Another use for propaganda is really activation. Uh, so just riling up the people who already believe that thing and making them feel compelled to take an action. And that's where you start to see some of the things around revolutions, perhaps, or, um, or social movements that. That get violent at times, or even just mass protests. So you have these different motivations. But the thing that has changed over time is the technology by which those messages are given to the public and who controls that technology. And so for a very long time, when we talked about propaganda, what we were talking about was those who controlled broadcast media. And broadcast, of course, in its various forms, has evolved over time too, right? We have print, we have radio, we have television. Um, but who controls the means of information dissemination? Who controls that technology? It's traditionally for a very long time been those who already have power. Uh, governments, political leaders, authority figures working in conjunction with media to Push information to the public. I think most of the public tends to think about propaganda in the context of like Noam Chomsky, the idea of manufacturing the consent of the governed, um, this sort of 1980s type work that was done talking about uh, ways in which top down messages reach the public and how the public doesn't fully understand the motivations behind how that information is filtered. But then all of that really gets upended in 2000 with the kind of um, the arrival of the Internet, which starts as blogs and just ordinary people posting their thoughts, doesn't really solve distribution. So you know, there's uh, false information, misleading information, persuasion, all that stuff is out there. But what happens with social networks is people also then gain control over the distribution channels. And this is a fundamentally different thing in history. So instead of it just being from the top down, people who control the um, the broadcast channels deciding what the public will hear, the public actually shapes what the public hears as social media enables both creation and dissemination. And so what I was talking about with ampaganda was this idea of um, ways in which crowd amplification which really comes from the bottom up, sets the agenda for what people at the top talk about. So that amplification, that, that idea, um, that this is what the public is talking about today in turn shapes the media coverage of what shows up on the news at night. That's really very, very fundamentally different. And so anybody who is influential, uh, among the public on social media has that power, has that reach of mass media that previously was um, really limited to a very, very small group of people, uh, now that that dynamic is democratized. And so a lot of the work that I've been doing for the book, but also just um, in my work at sil, is asking how does that change society? Right? How does it change society when anyone can do this? Everyone is. And you know, you alluded to Maginot line. Um, the metaphor that I was trying to use then was like this kind of war of all against all, right? This, anybody who wants to put out a narrative can put out a narrative. And we all kind of sit in these online spaces, um, seeing these various factions fight each other for who is the main character of the day, what is going to trend, what is, you know, what are the major pundits going to talk about on Fox News or MSNBC at night. And that's just a very fundamentally different information environment than we've ever had before. And I think that that shift is um, is really just kind of foundationally um, transformative for society.

Speaker B: Yeah, I think. I think people generally have the sense, given the events of the last five years in particular, that something is deeply wrong. But I was like, you know, I'm a dilettante in this issue set at best. But, uh, as, like, as people think about it, they probably can't quite pin down exactly what is happening. So I wonder if you could break down some of the negative effects of this change in concrete terms for folks to think about, like, what are some of the top negative outcomes of this, of the problem that you just outlined?

Speaker A: So I think that the main challenge, it's not. I want to be really clear that it's not what is the message. You know, there are certain messages that are obviously demonstrably bad, right? There are certain types of things that, um, that we've heard over time, um, you know, that are, uh, you know, calls for genocide, calls for, um, calls for, uh, abuse or calls for hate. That, um, kind of stuff is demonstrably bad. But what I'm trying to get at with this shift is, is that the real, I think, insidious thing is that because we now all have the means for capturing attention, right? Because that's what. What is ultimately happening on most of these social channels, um, everything is a battle to capture attention. And so a lot of the dialogue, a lot of the discourse, um, has really just devolved down in these online spaces into one faction fighting against another in order to capture attention. And so we have created a system of incentives in which ordinary people who now do have reach, who can make their message heard, nonetheless have to do so, have to compete in this space by resorting to sensationalism, by resorting to harassment to try to push other people out of the conversation so they can take on more share of voice. Those fights become spectator sport, right? You know, there's like a kind of gladiatorial element. You see something trending on Twitter, and you know that when you click in, it's going to be, you know, two groups of people fighting each other, right? Because the topic is some sort of culture war, um, you know, hot button issue, some sort of flashpoint. And so you, as the even nominally, um, casual observer, receives this nudge that directs you to go pay attention to this very sensational, very polarizing argument. And that's the kind of content that we're being barraged with at this point. That's where the incentives have taken us. Um, there's always some sort of incentive structure in communication. In the older propaganda models, you know, Chomsky articulates these. The, uh, Five filters he calls them, one of which is advertising. He makes the argument that media is not going to cover certain things because it's not good for them because they take advertising from those industries. And so we had this understanding, this kind of intuitive sense of in prior media environments how those incentives shaped what we saw, how those incentives shaped what was covered. And that's what we're experiencing again. But I don't think that we spend as much time thinking about how does the incentive here, the platform's incentive in its design structure. Right. Keeping people on site, um, the design structure of an algorithm, something like the trending topics feature. How did the incentives intersect with the content to put us into particular mindsets and create particular types of exchanges, ah, between people and you know, this sort of the so called virtual public square.

Speaker B: Yeah. What do you think about the filter bubble problem? Could you talk a little bit about that?

Speaker A: So there's some interesting, there's kind of like two different um, sets of arguments in social science on this. But I'll start by saying filter bubble was something that Eli ah, parser, ah coined the term, I think back in 2012 so 10 years ago now, um, arguing that because people were seeing personalized search results there was a potential for um, people to see things that were kind of um, in line with their prior biases or targeted to them and that this might lead to um, people moving into filtered existences in which they and their neighbors were not seeing the same thing. So the question that he asks is what happens? What does this do to society? And that was again that was about search results in uh, in 2012. So where we've gone since then is interestingly the, a lot of the social media companies again from an incentive standpoint, were really incentivized to grow whole new social spheres for their user base because that would keep them on site. And so you went from bringing the social graph that you had in real life to Facebook, meaning these are my friends who I go out with, you know, I go to bars with them. And now we're going to post our bar pictures on Facebook, which is a thing that we all kind of did in the early 2000 and tens to uh, um, hey. Our machine learning algorithms have inferred demand and we think that because you, this is a specific example from my life. You just had a baby, um, you are in a baby food group. You must be a crunchy mom. Have you considered going and joining this anti vaccine group? Right. And so nudge comes at you right now. It's not an unreasonable Assumption for the.

Speaker B: Really got the wrong person on that one.

Speaker A: They did really get the wrong person with that one. Um, but I thought it was so interesting, right, because I, you know, I was thinking about it with my, like, um, user interface design and, like, you know, hat on, and I was like, why am I getting these nudges? This is insane. Um, but it's because, you know, the. In collaborative filtering, like I am in San Francisco, I make my own baby food, I cloth diaper my kid. I, you know, obviously, I must be an anti vaxxer. You know, it's not a. It's not totally unreasonable, but the algorithm, in fact, you know, the platform is incentivized to give me that little nudge. Now, maybe I don't take it, maybe I ignore it, and then the next time the recommendation engine serves me something, it recognizes that I, you know, the signal I gave back was not engaging with that nudge. Ergo, maybe that was a bad nudge. So here's this other thing maybe you want to see. Like, backyard chickens was another thing that it was pushing me for a while. Maybe you want to start a chicken coop in your backyard, hippie, you know,

Speaker B: that really causes rats.

Speaker A: But by the way, I actually did join the backyard chickens group. So I was like, oh, pictures of chickens. Like, what are people doing in the backyard chickens group? Um, I enjoyed it until the sick chicken pictures started. And then I was like, let me get the out of here, you know? Um, but it was one of, you know, it was one of these things where I was like, okay, there's this little nudge, and, um. How serendipitous is this? Let me go. Let me go down this, uh, down this. This rabbit hole. And sometimes it's just harmless and, um, and, you know, and fun or, you know, again, I had my backyard chickens for about a week. Um, but then other times, you do see how that nudge, that kind of like, finding social communities for people, if you're not making recommendations that are cognizant of certain, um, dynamics, like around radicalization or around extremism, which the algorithms really weren't at the time, um, you can actually kind of set people off down some pretty bad paths. And so what happened with things like QAnon was exactly that pathway. It was this series of nudges where the algorithm thinks it's just bringing together all these highly engaged people who really want to talk about political things and puts them in these groups. And then it wonders, as the community turns towards violence in some ways, what do we do about that? After the fact? What do we do after those network connections have been made? The interesting unintended consequences there with things like filter bubbles is that there is a potential to filter and sort and put people into groups. Um, there is some research that suggests that the filter bubble problem is uh, overstated. And Jonathan Haidt, um, and Chris Bell have been keeping up a Google Doc kind of a living, ah, lit review. Um, these are two social scientists, uh, John's, um, at nyu and I'm trying to remember where Chris is at the moment. Um, but Chris wrote a book called Breaking the Social Media Prism and his book actually goes into these questions. What happens when people are sorted and assembled into particular communities? And there are some theories that very, very, very small percentages of um, users on social media are sorted into these groups. Um, the problem is that a small percentage can still be a big number. Right. And so, or that a small percentage, uh, even if a relatively small number, um, can be harmful in some way. So that's where the social science on filter bubbles, um, is that they're probably not as big of a problem for the average person. The average person is still seeing stuff outside of their particular, you know, niche interest or niche political point of view. Um, but for some people, um, these suggestions go a bit haywire. The other interesting theory is that one of the reasons for polarization, for affective polarization, where groups actively don't like each other, is not so much the filter bubbles. There, um, was a new paper that came out I think just in the last couple of weeks, um, and I forgot the scientist name unfortunately. But um, what they found was actually this argument that's sort of intuitive, which is that it's actually that you are constantly seeing people who are in this opposing political class or this opposing political position, which makes politics the thing that people see so often. And so it's actually the kind of factional battles that are happening, um, because hyper partisan communities are encountering each other on these platforms and kind of clashing. So sort of makes almost the opposite point of view that people have been assembled into factions we now have. This is like such a strong identity, uh, and the norms of behavior are that harassing and dunking and owning your opponents is how we conduct ourselves in political debate today. And so having that arena, something like Twitter or something like Facebook, provides um, an opportunity to do this in ways that again, in past communication environments, um, we never really would have been able to do.

Speaker B: Yeah, so historically like, well, when I got into national security, part of it was because of the Rwandan Genocide, Right. And you think about radio millicholines, which is the radio platform that the Hutus used to galvanize the population to commit a genocide of, you know, where they killed a million Tutsis and more, uh, and sympathizers. This is like the. Like the original release from 30 years ago. The violent filter bubble of the radio function. And it's.

Speaker A: Radio is. Yeah, there was, um. I've been spending a lot of time, as I've. The interesting thing about books, I think, is, uh, the, um, extensive amounts of research that never actually makes it into the book. You know, um, I've been. I've read, I think, almost the entirety of the archives. I, like, went and dug them up. Of the Institute for Propaganda Analysis, which was this body in the 1930s. A bunch of academics, actually, I was curious about, in these prior media environments, what was the response? As I've been trying to think through, what is there, you know, how do I write about, um, past responses? And these were academics and, um, they were producing educational material for, uh, high school and, um, middle school kids mostly. And the person whose work they were analyzing was Father Coughlin was, um, this priest who had a radio show that reached 30 million people.

Speaker B: Yeah, Catholic priests in New York, right?

Speaker A: Yep. Reached 30 million people. Um, and it's. He's very interesting to me, as I've been writing about, um, influencers and responses and, you know, um, this person who had this just extraordinary capacity. Everybody at the time largely listened to a very small number of channels. So his audience wasn't fragmented in the way you do see on social media today. Right, where influencers amass, you know, a couple hundred thousand, maybe a million. But a million is like a big deal. But this person had just absolutely extraordinary influence. Um, and this was a group of academics who were arguing that you had to teach people to recognize the signs. Um, because Father Coughlin was moving into fascism, right? Was this sort of American fascist at a time when the Nazi party was rising, um, in Germany. And so his. The things that they're putting out are trying to help the public see this, see the calls for what they are, but by teaching them the skills to recognize how this rhetoric actually works. And I found it really fascinating because I think that. That. That sort of, like, rhetorical analysis almost, um, like, you know, meta education, if you were. That just teaches people to understand, like, what are the signs that somebody's playing on your emotions, what are the signs that you're being manipulated, um, are largely lost. But if you look at how people talk today, A lot of the hyperpartisan political influencers, in particular, the, the rhetoric is, it's quite similar. It's the same like they don't want you to know, you know, who the hell is they? You know, why, why wouldn't they, you know. And once you kind of start to flag how these flourishes work, you hear them everywhere. And so I've actually found that process of, um, just going back and reading these old archives, um, really interesting from the standpoint of, you know, we think the Internet is, um. The Internet didn't cause these problems. The Internet changed the problems. It changed the manifestation of how the message got there. But radio, you know, as you know, has been, uh, just incredibly powerful.

Speaker B: Mhm. Yeah. I mean one of the, one of the data points that I like to say is that the Internet's the same age as Chris Hemsworth and Nicki Minaj and it went from zero to 5.2 billion in their lifespan. I think maybe one of them just turned 40, um, this year actually. So it's like, that's such a rapid, um, incredible growth. Um, so what, what would be some of the recommendations you would make to the world for how to, for how to respond to manipulative language? Right, that's, that's an interesting point. I think, like if you were to tell people right now, like, what are some of the signs that you'd watch out for?

Speaker A: I think it's, it's teaching people. I, I think the um, the glittering generality is uh, one of the terms that they used in the 1930s. The way in which groups, um, of people are painted with a broad brush. You know, when you hear somebody reducing um, millions and millions of your fellow citizens down to one label, uh, what is that doing? What is the point of that? What is the intended effect of the speaker when they do that? Um, or the use of um, uh, words like they, where there is some implied boogeyman, you know, and trying to think through, through who is the, you know, is, is there, is this plausible? Is the, is they really, Is there a plausible statement? Is there a person they. Who. You know, why, why, why isn't the speaker giving a name? Why aren't they articulating who is theoretically doing or saying this? Um, again, ties into the generality. Another thing that we see a lot in our work looking at what makes things go viral is you will see small accounts will make a very, very concrete claim and then, you know, and they'll say something. We can use the context of an election right now because it's top of Mind for me, um, there is a suitcase with. There's, uh, a suitcase outside of my polling place. That's how they're delivering the stolen ballots. Right. As a representative example. Um, so someone will make a claim. And this is the sort of thing that most people would just ignore. Um, you know, people make crazy claims on the Internet all the time. Uh, but what starts to happen is that you'll see incentivized influencers with very large followings who will pick up that claim. And. But they do it in a really interesting way. They don't say anything concrete about it. They say, big if true. Is somebody looking into this? Somebody should look into this. Who is investigating this? This is what the media is not investigating. So all of those things, you'll notice they're not actually endorsing the claim in a particular way. They're not saying this is happening. They're just saying somebody should really look into this. So it's layering on suspicion, uh, suspicion to a claim that most people would largely disregard. But there's almost like a short circuiting where somebody with a very large following that you've come to trust over time is the person who's putting out this message implying a degree of suspicion. And by implying that degree of suspicion, you'll see then many of their followers go pick up the claim and begin to tweet it or share it, and they will actually revert back to saying something concrete about it. Here is how XYZ is stealing the election, right? And so there's this interesting middle ground, um, that we see where the people who are most powerful in the amplification chain actually kind of take a step back and hedge what they're saying. They don't want to get sued. You know, they don't want to make a concrete claim a lot of the time, uh, or they don't want to stake their reputation on being wrong. Uh, but there is that interesting dynamic. And so once you start to see, I think, how these things are assembled, how these, um, well, these types of language really trigger our proclivity just as people to look at things like rumors or look at conspiracy theories and be intrigued by them and want to know more and want to share. And so that dynamic, again, rumors are not new, Conspiracy theories are not new. Propaganda is not new. It's just that now we're very active participants because we in turn get to decide, are we going to click that share button and propagate along big if true, they don't want you to know, uh, or does it kind of Stop there with that, um, with that influencer who's just, who's just chosen to share it. So I think this dynamic is really, it's very interesting to me. Anyway,

Speaker B: also with some of the things that the Russians did around vaccine hesitancy early on and what they did during the 2016 campaign where they weren't saying vote for X person, they weren't being directive, they're saying, well, isn't this an interesting question? Like, who's behind your suffering? Is it this other racial group? Or if you take this vaccine, like, maybe you'll die? Like someone should be looking into that. So it's, it's actually sowing doubt as opposed to offering a directive. Is that right?

Speaker A: That's, that's very much how it manifests, I would say, um, just an overwhelming amount of the time. You mentioned the, the Russians, um, when we looked at the stuff from 2016 to 2018, they were, early on, they were trying to kind of create their own content and, um, you know, play both sides and um, boost, you know, uh, boost their own, their own messaging. But what they started to do is just pick up hyper partisan media and influencers that were real Americans kind of, they would, sometimes, this was always really funny to me. They would slap their own logo on the meme that had been branded with some hyper partisan American publication. And uh, and so they would try to kind of grow their audiences by cribbing off that content. But what they did most of the time, again, so the point is, they're using real American narratives. It wasn't like they had to make up something for Americans to believe. This is where we get at that propaganda of activation, right? These are people who believe this thing already. You just want to galvanize them, harness them, and then pit them against each other. You're not trying to persuade anybody to a new or novel point of view here. Um, so what we saw with the Russian stuff was constant emphasis on your identity as an American and what your distinct American identity was. Were you lgbt? Were you black? Were you white? Were you a, uh, descendant of Confederate soldiers? Were you a Texan? You know, there were all of these different. Were you a feminist? Were you the wife of an incarcerated spouse? I mean, they got real niche on some of these things. Um, and what you started to see was creating pride and all of these different identities, but then pitting them against each other. The argument being that these identities could not all coexist in America. Your identity was, you know, if, if your identity group, um, was, uh, experiencing some sort of grievance whatever that was. Veterans who are not receiving benefits. Rather than thinking of this as a collective problem to be solved, the problem for veterans was that too many benefits were going to refugees. So then you had the United Muslims page that they made that was pro, you know, pro refugee, pro Islam. And then the, um, whenever they had a handful of these veterans oriented pages, Veterans today, I think the name was, um, and they would just kind of like pit these groups, uh, oppositionally because there wasn't enough, um, it was a scarcity, scarcity dynamic. If your group had a problem, it meant that some other group had taken something away from you. So this is not something we could solve collectively as Americans. It meant that society had to fracture along these different identity points. So that kind of factionalization of American identity, um, was where they chose to go. And they just did it over and over and over again on all platforms, um, for a very long time. And you know, we were kind of doing it to ourselves. So I'm not saying that, you know, it would be, I think, a gross overstatement to say that the Russians were the first people to realize that Americans had some fraying of the social fabric. Um, but it was that they recognized it as a viable strategy for interference.

Speaker B: Yeah, it's like a very light touch. They could just intervene with a small amount of investment and exacerbate what already existed. Um, so we're in an election right now. People are. The voting is already happening. Um, your interventions today, um, to some degree probably could change some folks minds. But if you were to think about over the course of the next like, uh, month to six months, like, what are some of the trends, what are some of the trend lines, the positive trend lines that you would like to see accelerate to improve our current situation. And what are some concepts that haven't really been adopted in managing the problem that you've described right now that you would like to see adopted.

Speaker A: So I see my role and our role at SIO as um, understanding what is happening and how it happens. Right. Maybe why it happens to some extent. We do a lot of interdisciplinary work with, um, people in psychology or people in rhetoric or communications, the very interdisciplinary team. Um, but we do a lot of chronicling of like, what is happening and we make recommendations for how should government think about this. The question of what government should do is a whole other. I mean, we could spend an hour on that alone, um, because we have hit a really interesting point in society in the US where trust in government is so low that the idea that government should have a role in understanding social media narratives around elections or pandemics is seen by some segments of the population as some sort of ministry of truth or some sort of surveillance as opposed to um, the government taking the pulse of the, you know, where the, where the American public is, um, or having a role in American elections. Yeah, I mean that whole thing has, um. It's really been the acceleration over the last um, two to three years of, you know, should the government even have a role in this is, has been really, uh, a bit surreal to see. Um, but I think where. What we can do as academics is we can say, um, empirically this is our best understanding of where the problem is and how it manifests. We can also say, and we did um, during COVID during the election. Here is the specific narrative in the specific region. Um, this is showing signs that it is being adopted. It is getting engagement. That's maybe a proxy for impact. We don't know for sure. We don't know if it's again changing minds or just ah, activating people. But either way this is a thing that seems to be important in this moment. So we can do that work. But then comes the question of what do you do about it? So there's two time horizons, there's the short term, which is when the rumor is going viral, who is best equipped to counter, speak against it, to say, this is what we know right now and here's what we think the truth of this matter is. Um, what we saw during COVID over and over and over again is that people who had a lot of experience would often not participate in those conversations until they had a degree of certainty. So you saw this around the masking conversation with the CDC very, very early on in 2020 or this question of was Covid airborne? Number of things where influencers, including some scientists who were just not, um, you know, not CDC people coming out and saying this is what we think is true right now. When you have a rumor going viral, you're not always going to have a neat thing, fact check and a certainty with which to speak against it. But that doesn't mean that you should just let it snowball. So that question in that immediate moment is who is actually responsible for doing that counter speech? And more importantly, who is a trusted messenger capable of actually persuading or reaching the audiences that are most concerned about that particular rumor in that particular moment. And that piece, that sort of um, that rapid response back really requires some sort of like partnerships and infrastructure where you know, we, we worked with a couple of Groups of doctors during COVID who wanted to put out content as physicians, right? They wanted to say, there's a group called this is our Shot. They wanted to say, look, you know, we're a group of a couple thousand physicians. Um, we want to show ourselves getting vaccinated. We want to show, you know, we want to answer people's questions about the vaccine, the disease, all of these different things, but we don't know who's talking about what at any given time. So how can we kind of bridge this knowledge and that expertise and that desire to be communicators? Um, how can we bring those things together in the short term, um, in the longer term time horizon? That's where this is why my interest in the Institute for Propaganda Analysis, um, why I've been reading all this stuff is because they're doing it on a longer term time horizon, right? They're not trying to fact check Father Coughlin's speech from the night before. They're saying, we're going to create resilience in the population against these, you know, these types of demagogues and this type of propaganda by helping people understand these tricks, these rhetorical tricks so that they can recognize them anywhere at any time. And so that's an educational response. Not something that has to be done in the moment, but something that has to be done so that as these moments continue to pop up, because they will. Because this is. The information environment's not going anywhere, right? So this is the new normal from a Com, you know, from a communications perspective, then you have that short term rapid response thing. But over time, you're helping the public understand what is new, what is different. And to be educated about how they, as citizens in America in 2022, um, should think about the things that they hear and see. How does an algorithm work? Why is this in your feed? Is that trend something that is created by, um, you know, highly incentivized people who just want you to fight with each other?

Speaker B: You know, I think as a last question, you've been very generous with your time. Um, uh, I'm sort of mostly interested in how national level violent narratives spread and things like a January 6th event. And so if you take that as like there's a lie being told that the election was stolen, it spreads online. Um, when you look at platforms like Twitter, Facebook, others, and the development of their ability to blur the line between, to either flag, individuals that are spreading mis or disinformation or blur or to identify blurred lines between truth and fiction, what do you think is the Possibility for enforcing that kind of information at scale to prevent things like what? I mean, obviously the insurrection has within it a whole other investigation, that there's elements of conspiracy, there's national political leaders who are involved in spreading it, that that accelerates the problems that have been in place for years before then. But if you take, um, interventions like m. Uh, Donald Trump and Marjorie Taylor Greene were only removed from Twitter around the January 6th, or they were only removed or filtered in the days immediately before, on the day of, or immediately after. Right? But if you take. Consider in advance you want to prevent these narratives that you've talked about from spreading and becoming a movement. What. What is it? What do you look. What do you look for in the platforms and their ability to actually enforce the kinds of changes that need to happen?

Speaker A: What you're asking about there, it's a really challenging question, and I'm sure you know that because it's this question of, uh, um, there are a few things that happened. First, there's progressive buildup over time, right? What kinds of, um, what kinds of, uh, speech is not incitement in the moment, but is there something that is, um, demonizing or othering a community or population over time? You know, again, to get at some of the other topic areas we've talked about, um, immediate incitement to violence. The platforms have rules about, uh, that sort of slow buildup, that dehumanizing language. One of the things that we saw in the Facebook leaks, the Francis Haugen, uh, papers, um, was that they are trying to understand that internally, what. What are the signals that indicate that a community or group has a proclivity to violence. And what we did see was that around election day in 2020, um, in the days after, immediately after. So President Trump, um, makes his announcement on the night of, alleging that, you know, the election has been stolen and. Or heading in that direction. You know, I'm not 100% sure when that exact claim was made. Um, but as he begins to head in that direction, many, many groups, the Stop the Stop Steel groups on Facebook grow extraordinarily quickly. Hundreds of thousands of people in a day, you know, and what you start to see is the platforms taking steps to take them down. As the rhetoric in the groups heats up and they're caught off guard, they are blindsided by the extent to which this happened. And so you do see them, um, kind of stumbling around and being a bit, um, you know, reactive. I think that, you know, it's something of an unprecedented. That particular lead up, I think, was, um, A bit jarring. I don't, I don't. You know, I think some people who study extremism, um, uh, have made some, you know, made some arguments that January 6th was actually very predictable. If you were watching certain online spaces, um, the platforms have tried to figure out where to strike that line between freedom of expression, your right to express anger or distrust and ah, elected official or an outcome or um, versus the extent to which the buildup of those things leads people to then go and act. Um, I don't have a neat answer. I think that this kind of dynamic is the sort of thing where unfortunately that research happens internally and we can't see it from outside. We only see it when it leaks. Right. And so we don't have visibility into those signals. When, you know, when my team looks at stuff we can see, oh, this group is growing very fast. Uh, if it's a public group, we can only see it if it's a public group. As academics, we don't go and join secret groups or anything like that. But um, we can only see these public groups growing and so we can say it seems like this is really a thing that a lot of people are uh, caring about and agitated about. We can't make a leap from that. And that's where I think these questions again, the short term versus the long term time horizon, I think what you'll see, and then Election day is about two weeks out. Um, I think you'll see the platforms act um, very decisively this time around. Uh, if there's indications that, you know, these sorts of unrest is growing in particular locales. Um, we don't have a national election this time. So I think it'll look a little bit different than 2024 did. But if there are these, um, if there's indications that that rhetoric is heating up, I do think you'll see, uh, I think you'll see takedowns happen. Um, beyond that though, again in the longer term, arguing for researcher access and things like that so that we can as, as ah, you know, as social, um, scientists and others, uh, have visibility into what is happening and what kinds of um, the momentum, you know, by which these groups grow is something that we just don't have particularly strong visibility into.

Speaker B: Yeah, you're often looking at it historically, um, uh, and doing a marvelous job. Um, yeah, it's great. I mean you've really done incredible, incredible work in shaping the debate and the country's understanding of the problem set and the world's understanding. Um, congratulations on all you've written recently and all that's coming down the line. Certainly, um, everyone's excited to see. To see what you continue to come up with. So keep up the great work. Um, uh, and thanks for coming on, I should say. Is there anything you've left unsaid that you wanted to say?

Speaker A: Thanks so much for having me.

Speaker B: Yeah. It's wonderful to see you. Thanks, Renee.