How The Rhetoric Of Women In The Alt-Right Broadens The Movement’s Appeal | Using Text As Data In Policy Analysis

>> Justin Grimmer: Hello, everyone, and welcome to the Hoover Institution workshop on using text as data and policy analysis. In this workshop, we feature applications of natural language processing, structured human readings, and machine learning methods to text as data to examine policy issues across the social sciences, economics, history, national security, political science, and other fields.

I'm Justin Grimmer, and Steve Davis and I co organize the workshop. Today we're thrilled to have Rich Nielsen, who's an associate professor of political science at MIT, presenting joint work with Eliza Oak called how the rhetoric of women in the alt right broadens the movement's appeal. Just before Rich starts, just some quick ground rules.

Rich is going to speak for about 30 to 40 minutes. If you have any questions, please place them in the Q and a feature. Steve and I might interject with some pressing questions, and it's possible that there could be some live responses in the Q&A, though unlikely. After this 30 to 40 minutes period, Steve and I might recognize you and have you ask your question live.

After about an hour, we're gonna turn the recording off and we'll go to a more informal Q&A session where we can ask some more nuts and bolts style questions. So with that, Rich, take it away.

>> Richard A. Nielsen: Great. Thanks so much for having me. Pleasure to be here, virtually, of course.

And let me share my screen so I can get those slides up. So this is, I wanna highlight joint work with Eliza Oak, who's a PhD student at Yale. We are right at the point where feedback is useful. We also have follow on things we want to do with the data and with the methods.

So all comments, criticisms, thoughts for things. Very, very welcome. And I put a link to the paper just on this title slide here too, but you can also find it on my website. I say that in part because we're gonna show some graphs that I'll admit I didn't have as much time to figure out how to make them fit on a slide as I would like, and so I've cut them into pieces.

But if you're looking for the whole graph that might be in the paper, great. Diving in, I will say that this is a talk about white supremacists. So there may be some disturbing content ahead. I've tried to remove the most disturbing content and visuals, but if something is sitting on the screen that is making you uncomfortable, put a comment in.

I'm happy to just move ahead. And we don't have to keep things up on the screen for any longer than you want them there. So just flagging that. So if you google a white supremacist. And do it with an image search. You get something that looks like this.

Now, this is a bit outdated, meaning I did this last year when I put together this slide deck for the very first time. But I don't think it's changed terribly much in the following way. You see a lot of flags, imagery, a bunch of symbols, but also mostly what you're gonna see are a lot of men.

And this has been true of white supremacist movements in the US and internationally for a very long time, that they're male dominated movements where men are kind of the prime actors seemingly pushing things forward, right? And we're going to talk a little bit about the Unite the right rally that happened in Charlottesville in 2017.

That's where we opened the paper. And this is one of the images that was taken of that rally. You can see, again, a bunch of angry young men carrying torches, although these are tiki torches, which seems to, I don't know, trying to achieve a certain image with what was available, again, very male dominated type of space, a lot of anger.

But one of the speakers at that rally was supposed to be Eylah Stewart, who's shown here in an interview with Lana Lochtev on Lana's YouTube channel called Red Eyes TV. And she actually, Isla Stewart blogged under the name wife with a purpose. And this is her writing about her experience just a couple days after where she was supposed to speak at the unite the right rally.

And you can see in the second paragraph here, when I agreed to speak, it was barely more than a few friends getting together. There had already been a tiki torch gathering in the same park earlier, fairly peaceful and small. And then she talks about how her perspective as a speaker at this event, it kind of, in her words, ballooned out of control.

And she claims she decided not to speak, although actually, none of the speakers, if I understand correctly, really gave their speeches as planned because it descended into some really terrible violence. So the fact that women are speaking in the white nationalist alt right movement has attracted a lot of interest by journalists.

So there was, around this time, a whole genre of articles in the popular press. So this is from the Atlantic. This is NPR. That's a photograph of Lana Lochtev, who I just had a shot up from her video. Again, a lot of emphasis on Lana Lochtef and red eyes TV actually, and headlines like, why are women joining the alt right?

And you can see here in the kind of header of the transcript, the ideology rejects Jewish people, people of color, LGBTQ people and immigrants and is dominated by white Mendez. So why are some women joining it? I think that's been the dominant frame in the call it journalistic takes on women in the alt right, again, also characterized by this headline from Sayward Darby, who's written a book about it.

Now, how can alt right women exist in a misogynistic movement? And I actually think that that's not the most interesting question in this space. So this question has already largely been answered. The easy answer is that it's a patriarchal bargain. Patriarchal bargain is where women trade security and autonomy in exchange for supporting a system of gender based oppression.

And this has been documented, theorized. This goes back to Candy Ode, 1985. It's been documented and theorized pretty well, and I think so Eliza and I are less interested, kind of in this first order question, although I'm happy to talk about Lena. Like, why might these women be participating?

It's not nearly as puzzling a phenomenon as the journalistic writing makes it seem, right? What I think is more puzzling, or where we have less clear answers, are how do women get authority in a male dominated movement, and what effects do they have? And you can see here, I've put up pictures of.

Nine of the 13, or, sorry, 11 women that we're analyzing in this particular paper, there is a very strong performance of gender that's happening visually on the screen. Fair amount of priority towards a certain look, certain appeal, also a certain style of video. If I had time to sit and watch videos with you, you would see some similarities in addition to the kind of general appearance.

And we want to theorize this a little bit, but also it's pretty data-driven exercise of digging into a large, and, as far as we know, previously unexplored corpus of videos that these women, and then counterpart men in the movement have produced. And that's what the paper is, is looking at women's rhetoric, looking at men's rhetoric.

And comparing it to see, okay, what can we glean from the rhetoric they use about how they're constructing their authority in this space? And then what can we glean from reactions to these videos about what effects they might be having? This is mostly observational, we will show you a small experiment at the end.

But our answer is that, contra what a lot of the journalistic writing says, we find women are at the core of the ideas of the alt right movement, that they're producing and provoking more racist content than the men. Okay, so outline of the talk, broadly speaking, we'll do some theory, some data analysis, talk about implications.

The bulk of the talk really is on the data and analysis, but we welcome thoughts on all of it, of course. So, thinking theoretically, these are our motivating questions, although we think there are others that could be answered with this data as well. How do women claim authority in male-dominated politics, especially conservative or misogynistic social movements, which I'm quite interested in?

And then, what effects do they have when they're there as leaders in these social movements? On the question of women claiming authority, the dominant frame in the journalistic literature, and then also with a few researchers who started working on this. So this is a New York Times Op-ed, which I'd call journalism, but it's by Annie Kelly, who's a PhD student, who's been working on these women, and their content really focuses on what I call differentiation.

So, the idea that women contribute to the movement, or claim authority in the movement by doing something very different from what the men do. And of implicitly the claim is also pulling the movement away rhetorically from its kind of real roots, something like that. So, what this is drawing on is the self-presentation of a lot of these women as, quote, trad wives.

So this means that, this short for traditional wife, this is a label that Ila Stewart, who I talked about, who was gonna speak at the Unite the Right rally, claimed for herself. So this is not somehow disconnected from the data, even though we're gonna show you some evidence that it's certainly not the whole story.

But the idea then is that, women emphasize a particular performance of gender in the movement that's very distinct from what men do, and that helps the movement by creating a kind of softer, gentler frame, something like that. An alternative view, coming more outside of the social movements literature and more on the literature on how women enter electoral politics, is the idea of outbidding.

So that women actually perform gender in ways that maybe are counter stereotypical to how women are often thought to perform gender. Or they use rhetoric that's typically associated with men, or they garner nicknames like Iron lady, as you can see here, basically emphasizing toughness and what are often seen as stereotypical masculine traits.

And so, in this context, then, we think that the sort of outbidding theories would predict women really proving that they can be just as white supremacists as the men, and even more so. And as I previewed, that's actually what we're gonna find in their rhetoric. Although, as I said, I mean, these women do often claim the label of trad wife.

And so, in some sense, our data analysis leads us to say, well, even as we're juxtaposing these theories against each other, maybe that's not the right conclusion. The authority construction is happening in perhaps both ways. Turning to what effects women might have. The literature recently has benefited from an experimental intervention by Elihi Ben Shitrit, Elad-Strenger, and Hirsch-Hoefler, where they argue for Pinkwashing.

They do a series of experiments, where in the Israeli context, people from the general population are more open to far right ideas. Far right means, probably something a little different in the Israeli context than the white supremacists here. In particular, many of the white supremacists we're looking at are quite skeptical of Jews and anti-Semitic.

But that finding is that the argument's basically that women create a more palatable feel in the viewer when they're hearing the same argument and that this is through, they argue that it's mediated. They have some data on this, too, that's mediated through a feeling of warmth. So, subjects feel warmer towards the female presentation of the same idea, and so, they are less critical of it or more in favor of it.

I've argued in the piece that was kind of the precursor to this. It was primarily about female Muslim preachers in a patriarchal movement called Salafism. But actually, thanks to a helpful reviewer, too, who asked for an entirely different case, there was a preview of some of the white nationalist content here, that women do expand the audience in kind of the same idea as Pinkwashing.

But it's not so much about being palatable, it's that they actually reach different audiences and may or may not soften the rhetoric of a movement or change it. It's simply that, if you're a social movement and different spokespersons speak to different audiences and can access different audiences, even misogynistic movements want as diverse a group of spokespersons as possible, then, Then coming from recent literature and we Zenrico document evidence of a backlash effect against women in the far right.

And I think there's a lot of other kind of literature and common sense we could turn to say, maybe the reaction, the primary reaction to these women will be negative. They'll get a lot of abuse online for what they do. And then of course, there's a possibility that there's really no effect and that this primarily about influencers on social media trying to aggrandize themselves.

But not particularly about actually helping a social movement or hurting a social. So what are we doing data wise? And I'm sorry, I can't actually see the comments, so I trust that someone else is taking a look at those. If there's something pressing, I always feel a little disembodied in a Zoom talk, so feel free to ask clarifying questions.

If something's not making sense, what are we doing? The big thing is an observational analysis of YouTube videos. And then we also complement it with an experiment which we fielded in 2021 on mechanical Turk with simulated YouTube comments. But I kind of have some issues running experiments with white nationalist content on MTurk, and so we've been cautious about going further in that direction.

Happy to talk more in detail about why. Also, there's actually just some mismatch between. It's as impossible, as far as we can tell, to confirm or deny the observational results with the experimental results. Because there's both no way to go back in time and run the experiment on when the observational data were created.

And it's really difficult to actually sample. Somehow the group of people that are precisely were interacting with these videos at the time. And so there's just some limits there of what we can learn from combining experiments and observational analysis. So the observational data, we sampled 29 Alt-Right YouTubers in the paper, we talk about the process of getting to the sample.

There's no rolodex or even official list of all of the Alt-Right or white supremacist individuals at the top of the movement. And so that makes it difficult to say that these results are representative because it's essentially tried to design a sampling frame. And then get all the people we could who were on YouTube at that time.

But I'll just say upfront that that's a limitation of this is like thinking about what is the pocket rate?

>> Steven Davis: Can you at least say what your hypothetical frame is?

>> Richard A. Nielsen: Yeah.

>> Steven Davis: Give us the definition of alt right that you are after how you would sample from that frame?

 

>> Richard A. Nielsen: Yeah, okay, so, shoot. Do we have a bunch of things got cut as we tried to shorten the paper. I'm like, do we have a definition of Alt-Right in the paper at this moment? We'll define Alt-Right as holding a right wing set of ideas and being open to, or more than open to racist interpretations of world politics.

And this isn't just American politics. Many of the people actually are based in Europe. They are. We think we can identify the racism when we see it. I'll show you some examples of it. But we sampled more broadly and where we drew the line. So people who didn't make the cut, for example, are like Steven Crowder with the louder with Crowder YouTube channel, who's pretty right wing, but has.

And some people hear dog whistling racism and what he's done with this channel, but he says, no, I'm not racist. I'm not alt right. Most of these people self identify as alt right, and many of them self identify as white supremacists. There are a few that are in the network that we saw referenced over and over again that we include who might deny that label.

 

>> Steven Davis: But it just because whether they identify as alt right or not doesn't seem very helpful to me. So the only content I hear you say is they're racist. And that's fine, if that's how you want to define, Alt-Right, then that's fine. But otherwise, it's just this very vague notion, which is, apart from how you sample the frame, even the frame is vague, which is what's bothering me.

 

>> Richard A. Nielsen: Yeah, okay, so the other thing that I think is important is we intentionally grab people. Okay, sorry. So let me describe procedurally how we got people, is we went to several lists of essentially think tank reports who have tried, like southern poverty Law center. And others who have tried to list out who are the major actors, influencers, movement leaders in the Alt-Right.

And we started from some of those lists. So-.

>> Steven Davis: Do they have a well-defined definition of what it means to be Alt-Right? I will desist here. But I'm just, there's a vagueness to the enterprise here that I think makes it hard to understand what we should make of the results.

 

>> Richard A. Nielsen: Yeah, I should have a better answer for you right now but let me ruminate on it as I keep going with the talk and I'll see if I can make it more precise as we go on. Because I do think that we have a sense, a stronger sense of what we mean in our heads.

But I'm doing a bad job of giving a concrete definition to you right now. Okay, so we have about 12,000 videos from those 29 people. We then went back and collected comments about six or eight months later, and about 2600 of those videos had been censored. So just flagging up front that there is a selection effect in terms of what we're able to do with some of the comment analysis I'm gonna show you, we have a few.

We collected comments from a very small, random sample of the videos before they were censored. But YouTube did a big push and then has continued, actually, around the time we were collecting data to censor a number of these actors. I'll try to flag where I think that's relevant.

We get similar results with the small random sample as with the censored sample, which obviously has a massive amount of data. And YouTube did not actually do an amazing job of censoring all of the racist content, including some videos that are still up right now, which is flagging that.

Here are the actors that we're talking about. So with the women, we basically identified a pretty comprehensive set of women who were acknowledged by multiple sources, internal and external, as being part of the Alt-Right. That doesn't, again, get to. We did not start, Steve, to your question with a definition of this discourse is the alt right.

We will now sample everyone on YouTube who has this discourse. We instead started with, this is a recognized. A social movement. Here are the people that multiple sources say are involved in that social movement. Here is where they put out their content. We're going to grab the content and somewhat remain agnostic to what actually the content is.

And what you'll see is that actually the men, the women engage in rhetoric, that there are some things the men do that actually, you think, that doesn't seem super alt right to me. They spend a lot of time streaming video games, for example. And so we have a very movement based approach to thinking about who's in and who's out.

But we wouldn't want to, I think, sample people based on content directly, because we're asking about how they contribute to a movement. And I worry that we would misunderstand what the content of the movement actually is. That streaming, etc., may actually be important to what the men contribute to the movement.

Sucking in gamers, for example, we have eleven women, 18 men. You can see a number of them. All are connected with this red ice tv channel. We're gonna use that at some point because it has the same subscribers and viewer pool, but then different people are posting their videos as kind of sub channels within that single YouTube channel.

So you can think of it as sort of holding fixed, a number of things that might vary across the channels otherwise. Okay, so what do these videos look like? Here's where some of the really racist quotes come in. I'm going to speed up because I took too long to talk about the data.

This is from someone named Stefan Molyneux. You can see he's talking about Hispanics having low iq and that this is going to cause crime. There is also a lot of antifeminism, misogyny, and women are themselves purveyors of this part of the ideology. In fact, it's kind of what my earlier paper was about, that women are especially good at arguing for anti woman stances within a social movement because they have identity authority to do so.

I can say, as a woman, I don't want feminism. So this is Brittany Pettibone, who's in the data set, and this is someone named Jean-Francois Gariepy. And you can see he's streaming here. And there's this quote about how employers ought not to be forced to hire black people if they're worried they'll commit crimes.

But what's more horrifying is this is zooming in on the chat. This is like the chat that's streaming on the screen. And you can see, I mean, there's just. It's about ethno states in Europe in this particular frame. And it's just and some anti-semitism. Pretty terrible stuff. And then finally back to the frame that I showed you before.

Ayla Stewart was famous for this white baby challenge that she made. And this was the interview with Lana where she talked about how America does not need incoming babies from foreign lands causing problems. Okay, so we're gonna deal with the multidimensional video data. And I think the way that's interesting to this workshop.

So the audio we take as the transcripts, we get the VTT files from YouTube, which transcribes them all automatically. There's mistakes in there for the closed captions, we'll acknowledge that, but do a pretty good job. And then we take sampled video frames. We actually sampled fairly infrequently. It's just one frame a minute at the moment, although we could modify that and then we ocr the text of the sampled frames.

Obviously we're missing text that's on frames we didn't sample, right? So we're not catching all of the visual content of these videos, we're catching a random sample of some of that content. We still have a lot from the OCR there. And then we're gonna do Google object detection, Google label detection, which is saying, what search terms does Google think you would wanna associate with this image?

Then we're going to use an API from a firm called face to do face gender and emotion recognition on these because Google no longer does that through their API. So this is a visual display of what that's gonna look like for the frame we've been looking at. So in the optical character recognition is gonna be in the orange here, pulling out things on the screen.

You can see there's some mistakes. For example, it thinks the Twitter bird is a y, but those don't tend to matter too much. The face algorithm is in the light blue there. It's pulling out female, neutral. It says no smile here for Lana Lokteff. She's smizing. I would call it in green you have the object detection.

So here it's really showing you. It's just grabbing person, person. It didn't even grab the microphone, for example. The labels actually do a little better. So we get a lot of really useful things about, especially kinda the female gender presentation in the red there. And you can see the boxes for the whole frame.

Okay, so how are we gonna analyze this? We're going to have a topic model where we combine the text and image data in the same topic model. Nuts and bolts of that is we come up with matrices from the transcripts, matrices from the visual, from turning the visual aspects of the frames into words via that process I showed you.

And then we append them and there's a question of weighting. We're actually just gonna weight them one to one at the moment, but acknowledging that this is kind of new methodological terrain of exactly how you should weigh images versus text. And I'm trying to do some theoretical thinking about that on the statistical side as well.

Happy to talk about it. This is very packed together. I'll say. Then we run regressions with author gender as a predictor. Sorry, I'm thinking of each of these as authors here on the content of comments they get on the YouTube videos, and then measures of engagement, the gender of the commenters and bringing in new commenters.

And throughout we're gonna use some conditioning on the content of the videos to capture the counterfactual idea of imagine that a man had presented the same content, how would the reactions have been different? Of course, that counterfactual is actually quite tricky to think about. So do you think about giving a man the same script as the woman, or do you think about giving the man the same visual presentation as the woman?

You get different counterfactual quantities depending on which counterfactual you want to know about. Okay, so this is a summary of a topic model where what we've done is we're showing the words that are most associated with different topics in the rows, and then point estimates where the black dots are showing us the percentage of words that women devoted to that topic, and the white dots are showing us the percentage of words that men devoted that topic.

And they're ranked so that At the top are the most women emphasized topics in the videos. At the bottom are the most men emphasized topics. And this is a 50 topic model. So I've actually cut out the middle of this plot, and it's still kind of unreadable. So what I'm gonna do is, we can come back to this with the estimates if you wanna see them, but just remembering that they're ranked this way.

So the very top is the things that women emphasize most, and then I'm going to zoom in a bit on what those keywords are. So the thing we've done is when it's all caps, that's coming from the visual words. When it's capitalized, but otherwise lowercase, that's coming from the OCR on the frames.

And when it's all lowercase, that word is coming from the closed captions. And so these top two topics here, 12 and 16, those are the visual display of gender. And in particular, this lip, blonde, necklace, beauty, hairstyle thing captures what we think we're seeing visually and what we showed you visually.

But what's important to us is that topics 44, so immigration, culture, identity, topic 33, black, white, racism, racist, race, racial, Africa, African, etc. 45, speech, hate, opinion, Nazi, critic, 15, police, crime, attack, shoot, terrorist, murder. When we go through and say, okay, just looking at the words, which ones are about race and which ones are about gender?

So you can see 46 there is woman, men, sexual, woman, male, etc. These ones are actually emphasized more in women's rhetoric. And we go through an exercise of working on labeling all the topics. That includes looking at the visual aspect, too, which is kind of new of thinking, how do you validate some topic when you have these frames to look through?

So this is what the visuals look like on the anti-black racism topic you saw there. And I'm happy to talk about them, but for the sake of time, I'm just gonna move to the punchline, which is that from our topic model, women are emphasizing race and gender, or racism and misogyny and homophobia more than men.

Moving on to what reactions then they provoke, it's very similar. So what we do is also fit a topic model, a new topic model, to the comments on these videos. And then, again, break that down by men and women as a predictor variable there. And so this is, again, just the top of that topic model.

So what we're looking at now is not content produced by the YouTuber. It's content that's in the comments of the videos. So there's no visual information here anymore. But what you can see again is at the top here are a lot of topics about gender, misogyny, homophobia, the top two there.

48 is actually a congratulations topic. So what we're gonna see is there's a lot more praise on the women's videos. And then 37 here, 25, 12, 44. Even this 49, this Antifa topic, turns out to actually be about race quite a bit. These are all topics that are generated or appearing more in the comments on women's videos than on men's videos.

That's us zooming in, sorry. Okay, for the sake of time, what do I want to show you? The men's comments, just to give you a flavor of what men get that is different from women. Starting from the bottom, there's a lot more about American politics. So the biggest differentiating topic in the comments, and this is true actually also for the topic model on the video content, is on American politics.

And then there's a lot about some conspiracy theory stuff in here, some philosophy stuff, some libertarianism stuff, some politics stuff. And then in here are some gaming and streaming video stuff. So combining all of those topics, we could run a series of regressions with different predictor variables. These are all descriptive to me in the sense that I think the different control sets are not really making this like a coin flip in any sense.

They're just allowing us to reason about different counterfactual qualities. But we have kind of lurking confounding in mind for all of them. So we do it with not conditioning on anything, conditioning just on the topics that are primarily loading onto the text words, which we mean the transcripts.

So the counterfactual there is, imagine giving the men a similar transcript and having them deliver the video. So a more racist transcript in this case, on average. We can also condition on the image topic. So that's like conditioning on all topics at once. We add a few other video controls, and then we can do this, kind of think of it as a channel-fixed effect, so just on that Red Ice TV.

And what we're seeing is in general, there's a trend where women are provoking more misogyny and racism in the comments, kind of regardless of what the control set is, with some variation. We also find that they get more engagement in other ways. So it's not just more racism and misogyny in the comments.

Predicted video views are, there's no difference between men and women. That's statistically significant there. Men get a few more on everything else. Women get more. So they get more likes, they also get more dislikes. They get more comments per view, actually. This is conditioning on views, so they're able to generate more engagement like that.

They get more praise. They get fewer misogynist slurs, actually, although our list of misogynist slurs may be a little limited there, we tried to be imaginative. But YouTubers are more imaginative. They get more female commenters. But what's striking to me here is that still the majority, vast majority of commenters on women's videos are men.

Now, this is based on trying to guess people's gender from their YouTube handles, which is not an error-free enterprise, I'll say. And so there's a huge degree of uncertainty here. This is just people who used usernames on YouTube that we felt like we could label with some certainty as indicating some kind of gender.

And then, finally, we looked at first-time commenters. So these are people who comment on one video and then never enter the dataset. And women attract a lot more of those comments, too. We tried to take this to an experimental setting to replicate the pink washing type of experiment.

So what we did, as a cautious first step, is made simulated YouTube comments. And what you can see is that there's an anonymous version. So these are taking ideas that are almost verbatim from transcripts in the videos, posing them as comments to people, and then varying just what the stereotypical gender associated with the name is.

Putting it. Twice on there, so that there's some hope that they will notice gender. But that's not entirely clear, that that's the only thing that's going on here. And we had five statements. So in addition to this one about white babies, we have mixed race relationships. The risk of crossing the road that I showed you before this a version of this anti Hispanic racist one, and then a version of this antifeminist one.

And what we find is if there were effects, we would have expected to see some statistically significant differences for male username and gender neutral username. The idea there being that the female baseline should be lower. Sorry. The outcome I should have explained is asking people how offensive do you find these statements and not showing them.

We just show them a random ordering from male or female randomizing as we go. They don't see the same statement posed by one and then by the other, and we overarchingly find null effects, especially when you put a Bonferroni correction on this. So we don't find evidence of pink washing, kind of contra the benchy treat thing.

But we've been cautious about doing experiments that are expansive as ours. I think our content is rougher than what they were doing in those experiments. So implications, as I kind of indicated, this differentiation or outbidding frame we started the project with. Our answer now is kind of like yes.

It's like yes and they do a bit of both, but more complicated than just one or the other. But the image of alt right women is tradwives and pulling things just towards gender and home life, etc is pretty woefully incomplete. Women are talking about why white nationalism's core ideas more than men.

They're provoking more racism when they do, although maybe not through pink washing. Our best evidence is simply that women, when they talk more about race, generate more racism. And counter factually, if men talked more about race in the movement, they would also generate more racism. So broad conclusion is that this isn't just about making white nationalism seem friendly with pretty faces.

We don't think that we've fully exhausted some analysis of the performance of gender here. It's quite interesting, and it's in our topic models, but there's a lot more you could do. But overarchingly, we think the takeaway is that women are effective at eliciting support for the movement's most dangerous ideas.

And whatever you think about the flaws of the observational analysis, which I'm very happy to hear about, we are trying to make it better. There reality is just what you can infer from an observational analysis like this. It should horrify you that many of these racist comments are still up on YouTube.

So if we're thinking about how this relates to policy, I'll just say that I have some thoughts there. But for the moment this is pretty research focused and I'm happy to talk what I would recommend policy wise in Q&A, but I haven't made it part of the talk.

Thanks very much. Looking forward to your thoughts.

>> Justin Grimmer: Okay, I'm gonna ask a couple of questions, then I'll let Steve hop in and then we can open it up to the audience. So I had a similar reaction to Steve when thinking about the population under study here, and either a way to justify it is by establishing a sampling frame or by bolstering the importance of the speakers that are in your data set.

I wanted to suggest some ways to broaden it that would be perhaps a bit more reflective of where this content's being created. I would be surprised if the hotbed of white nationalists or racist rhetoric is happening on YouTube. Alternative sources would be like Rumble, which has a much less aggressive censoring policy, which would leave a lot more content up and is a place where much more sort of far right content.

You could also imagine juxtaposing speakers you've identified as being important because perhaps they have big audiences with more right leaning news broadcasters for the content, including the opinion there. So for example, Mike Lindell's TV network, Frank speech or right side broadcast news is another place where a lot of sort of right leaning speakers go.

Right leaning news broadcasters far to the right, what's on Fox News would broadcast. But the interesting problem, I think, in the assessment here is if there's just fundamentally different ways different speakers are conveying their content. So if the YouTube videos is where the women are sort of bringing people in, you could have some different presentation there.

Whereas if it's like the Twitch stream is where the men say incredibly racist things as a way to sort of normalize racist speak in reaction to events in a video game, which is the thing that happens, you can imagine that you could end up with the reverse conclusion.

 

>> Richard A. Nielsen: Yeah. So I mean, agree, I admit that some of the challenge we face is simply in cross platform data collection being difficult or impossible. That's an annoying response, but it is a real one for text as data researchers, because I will say I've never presented something where someone couldn't think of more places I should go get text from.

Let me justify the YouTube thing for a minute, which is that now rumble and gab and some of these other places where white nationalists are putting out content. But in 2019, YouTube was the premiere. They were very pissed when they were kicked off of YouTube. They complain about it all the time.

The men and women both were using YouTube to put up their primary content. And in fact, I actually get a little salty when people do Twitter analyses of white nationalists for example, because the Twitter was prime. Again, it's because it's accessible, right? Like, because you could scrape Twitter.

And so we're guilty as charged there too. It's like YouTube is somewhat accessible, although not as accessible as Twitter. But all of that Twitter content was primarily pointing people to the YouTube videos. The YouTube videos are where they ramble on and they put a lot like their Twitch streams they're putting up on YouTube.

So I would defend that as of 2019, this was the place where the premiere content was. And it's because in Tamar Mitt's new book, she's still working on it, but I've seen it. She argues that basically extremists want to be at the apex of impact and low content moderation, and actually, nobody's on rumble.

So, like YouTube is still where they want to be, and I actually see them reposting this content under alias. Accounts and trying to get a few views here and there. That said, I agree, with more money and more time, I absolutely would do a collection on the whole ecosystem.

I'd like to head that way. I sort of see this as a very time consuming but still pilot analysis of what's out there. How's that for caving in the face of a question?

>> Justin Grimmer: Steve, do you wanna ask and we can open it up?

>> Steven Davis: Yeah, let me, let me make a comment.

And I guess it's a request for framing things in the next draft of the paper. And I'll set aside the questions we've already discussed about sampling frame and sampling. This paper, as you presented it, kinda sits out there in the ether all by itself, and there's a lack of comparison to other settings that might be helpful.

And I'll give you three examples. First, just a terminological point. This isn't really about women in male dominated politics. There's lots of women in politics in the United States, and we live in a society where men and women have equal political and legal rights. But you can go back in history in the United States, especially before women had to vote.

And there's other forms of legal rights where there was inequality before then. And you could ask, not you personally, but what does the literature say about how women achieved authority in either political or social movements when they didn't have the same political and legal rights as men? You can do that today in many countries around the world where women are subjugated, often as part of a legal regime, and they have less in the way of political and legal rights than men.

Nonetheless, they often play important roles in social movements, if not political ones as well. How do they do that? And then finally, something that's really close to home and might be a useful point of comparison cuz it's very doable. But if you watch commentators coverage of the NBA and the NFL and other major sports, those are only two I look at occasionally.

You can see there's been a radical change in the past two decades. They used to be entirely male dominated, and now almost all these shows, the leading ones, seem to have at least one female commentator along with the men. So are the women doing something different in those settings than the men to achieve, not authority here, although that's really an input to viewership and ratings.

Those settings strike me as quite useful, partly because they're not so politically charged, but also because there are very clear ways you would sample and assess performance. You just on the relevant sample frame is all the most watched shows. The relevant metric is viewership and maybe advertising revenue and something about commentary.

So I find this one sits in isolation and there's a sense of which you only have one data point. You got one particular movement, and then you've got some somewhat noisy observations on how women achieve whatever level of success you wanted to characterize it as relative to men.

But I don't know what to make of it because there's no context.

>> Richard A. Nielsen: Yeah, so some of this may be an issue with the talk. We do try to bring in a bit more in the paper of this kind of broader literature. You're making me think of Don Teal's work, though, which I think we're not citing, about how women got votes in various places.

We do think a lot about Anna Catalano Weeks's work on women entering political parties and drawing from that how women's presentation. But I watch some of these sports shows too, and I am fascinated actually by the shifts you're saying. I sometimes worry that it's in fact not authority, that actually in the shows the women are still treated as less authoritative, although maybe, maybe that's a function of just some of the shows that I'm watching.

Often the woman is there as a facilitator and in this setting the women are not facilitators. So a thing that's a little different is that these are authorities in an open marketplace vying for influence without an overarching structure, like an employer who tells them they're in or out.

And can basically subsidize them even if they're not drawing in views or whatever. For me, this is connected to other social movements. So the other data point I have in mind is, as I mentioned, women in the Islamic Salafi movement. And so for me then, the two data points are lining up that basically women are involved in these, what I would call patriarchal movements because they actually help the movement reach more people.

That's what I would say is kinda my underlying takeaway. And because they communicate core ideas of the movement in better ways than men can, either because their gender performance allows them to do that so they can, for example, make anti woman statements better, or because they expand the audience, though not necessarily through pink washing, because I don't think the Islamists care about pink washing very much, at least not directly.

They just care about expanding the movement. And here I also think that there's, we even find quotes where the male leaders are like, well, we don't actually think that women should be leaders in this movement. On the other hand, they bring in a lot of interest. So what do we do about that?

And some say keep them, some say ditch them, etc. But point taken. Some of that is stuff that really was not in the presentation of the paper, and it's not the first time. I've also managed to make something feel free floating when it shouldn't be. I think that's partly because I'm just really interested in this particular phenomenon as well, trying to slot it into a theoretical lens.

But I will say that, like, the driving force for me is, like, racist comments on YouTube. How did those get there? What can you do about them? If we can put it into a theoretical frame, great. That's what I'm trying to do. But I think that's where some of the free floatingness comes from.

 

>> Justin Grimmer: Elizabeth Elder has a question, so can we give her the mic, please? Elizabeth, go ahead.

>> Elizabeth Elder: Thank you so much for this really interesting talk. I was thinking related to Steve's point, you know, that this might be kind of one observation rather than a bunch of different ones.

It seems like this is, even if it's not an organization with a hierarchy, it's still a network or an ecosystem where it matters a lot to the extent to which someone is cross referenced by other already popular people. There's a lot of kind of. Of capital being shared by maybe the men who were dominated by this network at the beginning.

So I was curious a little bit. To what extent are these particular women being kind of selected by the elites that already, like, the power players of content producers within this network? Versus to what extent is this kind of a grassroots way of appealing to the masses or the people who are the consumers here?

 

>> Richard A. Nielsen: Yeah, good. So a mix, great question. Thank you. And sorry, I'm going on too long, so I'll try to be concise. I'm not good at being concise. Some of them are clearly, like, driving. Some of these women are driving their own ship or driving their own vehicle, I guess.

They're in charge. They know what they want to do. Others do seem to have been, like, selected to some extent by the movement. So, for example, Ayla Stewart being invited to be a speaker at the Unite the Right rally was at the behest of men, but she got there by putting herself out there, by having this blog and being kind of one of the prominent people in the movement.

So I see it as a mix. There are other powerful people, mostly men in the movement, who can elevate and amplify and vouch for a new entrant into the authority space. And they actually, they do exactly what you're describing, appearing on each other's shows. I mean, they're doing this interview that you can see on the screen right here with each other.

This is Lana using her authority to try to amplify Ayla's authority, because Ayla is now famous for saying something that Lana wants to amplify. Lana has the reach. She's essentially in this, like, her and her husband ran red ice tv together. But I can see it over time, actually, in my inference is that they learned from the engagement side that Lana's videos got more engagement, and they actually, then, over time, keep doing more to make Lana more prominent in the channel as a result.

That also brings up that there's a business aspect to all of this, which is partly why they were so irritated about getting kicked off of YouTube, because they were making money. I don't have a clear answer to, are they selected from above or do they grassroots from below?

I think it's a mix. And I actually think authority construction is typically a mix of both hierarchy and kind of grassroots, like putting yourself forward as a potential authority.

>> Justin Grimmer: All right, so it looks like we're at time, so we're gonna go ahead and stop the recording. And feel free to stick around.

We can ask Rich a couple more questions. Thank you all. Thank you, Rich.

>> Richard A. Nielsen: Thank you.