Benevolent Billionaires? Evaluating The Influence Of Wealthy Owners On Local News | Using Text As Data In Policy Analysis

>> Justin Grimmer: Hello everyone, and welcome to the Hoover Institution workshop on using text as data and policy analysis. In this workshop we feature applications of natural language processing, structured human readings and machine learning methods to text as data. To examine policy issues and economics, history, national security, political science and other related fields.

I'm Justin Grimmer. Steve Davis and I co organize the workshop. We're thrilled today to have Michelle Torres who is an assistant professor of political science at UCLA. She's presenting work that's joint with Alison Archer, who's an assistant professor at the University of Houston. Their paper is titled Benevolent evaluating the influence of wealthy owners on local news.

Before we get started, let me just set some quick ground rules. Michelle is gonna speak for about 30 to 40 minutes. If you have any questions, please put them in the Q&A feature. Steve and I might interject as Michelle's presenting with any pressing questions. And after Michelle wraps up, if you have a question we might recognize you and ask you to speak your question or ask your question live.

After about an hour of Michelle's presentation and then total Q&A, we'll turn the recording off and we'll go to a more informal session. And in this informal session we can ask Michelle some more nuts and bolts style questions to really understand how the research was done. So with all that being said, Michelle, take it away.

 

>> Michelle Torres: Thank you. Thank you so much, Justin. I will quickly share my screen here. So thank you again, Justine, and thank you Steve, for having me here, for inviting me, and thank you to everyone for joining to hear this presentation. This project is co authored with the amazing and incredible Allison Archer at UoH.

And in this paper we are like really interested in kind of disentangling the relationship between local news billionaire owners and basically political accountability. It's a very, it's a kind of unorthodox project because it really came up like, or we like the motivation comes from observing the world science.

And we discussed a question. We thought it was really interesting that we were observing these billionaires purchasing newspapers and we were like, okay, what are the consequences from this? So that's how we embarked in this project and we are trying to be very rigorous in terms of the empirics and the data that we are using.

But we don't have super strong theoretical expectations for what we're gonna fight. So it's like an exploration that has been really fun and we are really looking forward to your feedback, questions and comments. So hopefully this is a crowd that doesn't need any convincing that news matter, mass political knowledge is shaped by the public's opportunities to learn from the media.

And media is gonna provide a lot of information and also hold actors accountable. Media outlets can raise awareness of those with low political interests. And research has found that they have potential effects, even if minimal, on polarization, trust and other attitudes. So overall, media has the potential and this power of really affecting things that we care about political outcomes that are part of a healthy and vibrant city culture.

But we also know from research and from experience and from traditional media that they are in a crisis, right? For some years, we've been seeing decline of traditional local media. And it's been well documented that there's a decline of the local news industry. Which is alarming if we consider that there's also been a lot of research talking about the importance, the political importance of local news.

So we have these pieces like from Eric Peterson on how increasing the number of staff cuts actually lead to a decrease of political coverage. Which eventually translates in less political participation and lower turnout rates. So based on these challenges that local media and local outlets are facing, they really have to come with difficult decisions to deal with these difficult times.

And some of these difficult decisions have to do mostly with ownership changes and ways of literally rescuing some of these newsrooms. So who are these new owners or these people coming to save these newsrooms? And what kind of rules are they gonna impose? And for that, we need to consider that there are different types of owners.

We have anything from hedge fund vultures and ideologically motivated corporations. They're really just coming to crush and get as much as possible from these dying businesses. We also have non-profit news organizations and local business people who had a deep commitment to these news outlets. Because this is the newspaper that they've been reading since they were kids and that have almost an emotional attachment with all of these things.

But we also have like this very interesting group of people, billionaire owners, entrepreneurs, that actually have nothing to do with the journalism area or the media space. And we are interested in basically analyzing what do they want, or at least what do they claim that they want, because there might be a disconnection between these two things.

What they want and what they are telling the world that they want to do with these newspapers. And what we are actually observing terms of the decisions that they are making for the newspaper once they take over it. So why billionaires? It's an interesting group to analyze, for sure, but probably now experience is telling us that they have resources, they have power.

And they had this potential of really playing a key role in shaping public's opinion, political outcomes, attitude formation, and some social dynamics are of that are important. So I will tell you about a typology of billionaires and billionaire owners that we identified, at least based on what they say or based on the motivations that they claim to have for purchasing newspapers.

And you might be familiar with this typology for other things in life. It's the good, the bad, and the ugly. So the good is people like Jeff Bezos, who, for example, acquired the Washington Post in 2013. And that came with these ideas and these statements about the commitment that he had to the Washington Post because it was a crucial part of a functioning democracy and a vibrant community.

So he had this statement that this is the newspaper in the capital city of the most important country in the world. The Washington Post has an incredibly important role to play in this democracy. There is no doubt in my mind about that. So really acknowledging the crucial role that a newspaper like the Washington Post plays in our normal dynamics.

And unlike him, we have other people, like John Henry, who acquired the Boston Globe, Glenn Taylor, the Star Tribune in Minneapolis, or Patrick Soon- Shiong, the LA Times. And the San Diego Union Tribune. Again, all of these people acknowledging the importance of the newspapers and saying that they really want to contribute and they really want to make them better.

You also have the bad in the sense that they don't have these grandiose motivations or objectives of making the papers vibrant, but rather that take them as businesses. So one example is Sam Zell, who acquired the Tribune Co. He is considered, or is named, or was named because he passed away, the graveyard dancer of real estate, because he was able to bring businesses back to life.

And he purchased a tribune code with this idea of really just make profit out of it. So it's not that I don't have opinions, and it's not that I don't feel strongly about things, but I didn't make this investment for any other reason than economic gain. So very open, very candid about his intentions.

At the end, it turned out to be the worst deal in the history of the world. So, if you're interested, we can talk about it in the Q&A. But that was very clear from his objectives. Now we have the ugly, which is, for example, Sheldon Adelson, who acquired the Las Vegas Review Journal.

And who has a magnate in Las Vegas, and he has, a very clear intention of being ideological and shaping people's opinions towards this newspaper. So this is a quote from another journal is saying, coverage of issues adults don't care about pretty much comes out of public relations. The idea that the review Journal is run as an independent body is laughable.

So people that are, I am actively using the newspapers to imprint or, fulfill their own ideological agenda. So this typology motivates the question of what's the real influence that these wealthy owners have on local news? And in a way, or, more broadly, how can we think of a framework for evaluating this influence?

So we know that ownership is important. There's a lot of research on this. We know that ownership changes and structure affect what and how information becomes news content. We also know that owners might be ideologically driven and that might affect the news content and the way in which voters, or people, think about political events.

And we also know that owners may have philanthropic goals regarding the role of a newspaper in promoting democratic values. And as you can see here, this one doesn't have, research citations because this is mostly the news. These billionaires owners coming and saying, yes, that's what we want. And we are philanthropists, but there's really not much evidence about it.

Okay, so what can billionaires do? They have disadvantages, right? Why are these important key players that we should look at? They have the resources to invest in the newsroom and to actually save these, dying businesses. They can withstand short term profit losses with a night towards long term success.

Something that is important, especially in journalism, where it's gonna take some time to actually see a utility. And often view their media assets as philanthropic ventures. So they are motivated like that. However, they do not have, most of the time, journalistic backgrounds or longstanding commitment to news. So it could be just a toy or an impulse purchase that they do.

And sometimes they are not really aware of the complexities of the media industry, especially the modern one. So this could lead to some frustrations and some abandonment of the whole project. And they also are really busy, have other interests and business to attend. This is not the main thing that they care about.

Okay, so we are gonna focus on the case of the LA Times and the San Diego Union Tribute and Patrick Son-Shiong. In June 18, 2018, Doctor Patrick Soon Shiong purchased the LA Times and the San Diego Union Tribune together with other few other community newspapers. Who is Patrick Soon-Shiong?

He is a transplant surgeon and billionaire entrepreneur who had enjoyed a lot of success in other fields that are not related to media, which is biotechnology and pharmaceuticals. Although he took a crash course on media and journalism, he was really proud of it and he paid 500 million for acquiring these California newspapers.

He has a wide coverage of the philanthropic and entrepreneurial reasons behind the purchase. He gave a million of interviews talking about how this was like his little new project and he was really passionate about. So basically, we're gonna take him as our test. We wanna see like hold him accountable for what he said in those interviews and see whether his objectives are getting actually translated into content of the newspaper articles, okay?

So in order to develop this evaluation framework of these billionaires, but in particular Patrick Soon-Shiong. We're gonna literally take the quotes that he had from all of these interviews and summarize all of the objectives and then put them to test. So one, he pledged to continuing the great tradition of award winning journalism, inject stability to save the newsroom, recommitted paper's investigative reporting work and turn the LA Times into a resource for the community.

Strengthen the presence of the paper in LA by increasing digital subscriptions and increase the coverage that holds other actors accountable. We're gonna focus on those highlighted in orange. And we're gonna study them by a studying the news quality, basically the typology of hard versus soft news and how prevalent they are in the things that the LA Times and the San Diego Union Tribune are publishing.

We're gonna also inspect the investigative reporting prevalence in the newspaper. We're gonna evaluate whether he's focusing or the news content is focusing on local topics more than non local, like national or international topics. We're gonna see if he's increasing his efforts to reach viewers and also analyze the prevalence of accountability related language in the news articles of the LA Times and the San Diego Union Tribune.

So this is when I said it was a bit unorthodox. We have basically three hypotheses. One, he has improved the content, he has worse in it, or he had no impact. So we're just gonna try to find evidence to give support to one of these three hypotheses. Our empirical strategy is the following Son Soon announced the purchase of the LA Times and San Diego Union Tribune in February of 2018.

So this is gonna be our treatment group, the LA Times and the San Diego Union Tribune. Treatment understood as purchased by a billionaire. Then the deal was finalized in June 2018. So this is gonna define our post treatment period start. We are interested in assessing changes before and after the purchase with some caveats that I will tell you about based on each of the analyses that we will have.

And assuming that before the treatment, before the purchase of Son Shiong, the LA Times, the San Diego Union tribute and a bunch of comparable newspapers publishing behavior had similar trends. We're gonna estimate the effect of this purchase on content and this is gonna be the average treatment effect among the treated.

Our approach is gonna be a difference in difference approach and we're gonna vary the post treatment period like the start and end and length to also see changes over time. Our data and variables first, I will start with the variables. We're gonna focus on indexes of investigative reporting, language related to accountability.

We are gonna try to classify news in two categories, local versus non local. And we are also gonna inspect the topics that we can find in this news with the idea that we identify hard versus local news, hard versus soft news, and local versus national or international. For the news articles, we basically have a big dataset with all the news pieces published in the front page or A1 Section of several newspapers.

This was compiled by News bank and because news bank didn't have the LA Times, we had to also complement this dataset with this information from Newseum, all the front pages that Alison has been collecting since 2012. So it's a great thing that I don't think anyone else has.

We also scraped back then it was still Twitter. You can tell that this project has been going for some time. We got the newsfeeds of all the California newspaper news feeds, their official Twitter feed. And we're also using some indexes of investigative reporting and accountability published in an article by Turkey let al in PNAs that uses a series of methods.

Our newspapers are eight California newspapers. So the first Nobel, the Sacramento be, LA Times, San Diego Union tribute, San Francisco, Chronicle, and for comparable newspaper. We wanted to have like a control or like a benchmark for California and like the things that these newspapers talk about in California, but also like a benchmark for other cities that are comparable to LA.

And that's why we're including the Houston Carnival, the Tampa Bay Times, and the St. Louis Dispatch, and another one and the New York Times. Okay, so we have almost 28,000 articles from California papers and 30,000 from other newspapers and around 75,000 tweets. We are focusing on different parts that are like a lot of moving parts.

In each of the analysis that I will show you, we are using the full text article of many sections for the investigative reporting with the idea that we really want to exploit and analyze the content of published articles. This is a central element of the newspaper, but it's also basically exposed to only their exclusive audiences or people who subscribe to them.

So that's the reason why we're using Twitter as well. First of all, because the cost of distribution are lower, it's easier to actually make rapid or sudden changes, and it's not that costly, and they will reach broader audiences. And we are also focusing on the front pages and A1 sections, one for data availability reasons, it's been very hard to get our hands on this data.

But also to signal, because these sections signal the newspaper's priorities and they get greater exposure even if you don't read the full newspaper. Okay, so I will start. These are gonna be three analyses. The first one is gonna inspect how specialized and high quality the content that we can find in the LA Times and San Diego Union tribute are.

And we are gonna do this through the exploration of investigative scores and accountability language for these. The measurement comes from Torquel et al, they developed this investigative scores, which is basically an index telling you how investigative an article is. This is an output of a neural network trained on 562 articles that were manually coded as investigative and takes a bunch of features as input.

Again, the score is basically the probability that a given article article is investigative. And the criteria or like part of the features that compose this index are, for example, whether they want like an award in like relevant journalism contest. Whether it was an article was entered into the database of the investigative reporters and editors for a regular contest, or other features like the article.

Actual Vettings the influence course, which is one of my favorite parts of seeing how some articles might set the seed. And if they get cited or they, there's some spillovers to other news that's their influence, because they start the conversation. Indicators of investigative reporting terms like audits, FOIAs and investigative related word count and section names for the accountability related word count.

This captures the proportion of words associated with characteristics such as accuse, allege, complain or denounce things that are like calling for attention to like these topics. This measure is one I want to like highlight that this word count is one of the 320 features that compose the investigative reported index.

But we are not super concerned about whether we have a huge overlap here. First of all, because this is just one of the features we are using a transformation and we even saw that the correlation is not super high. So the investigative reporting is capturing one dimension and this accountability related work count is capturing another one.

So we're gonna use the sum of investigative scores and the weekly proportion of accountability related words for as our outcome variables here. And here we're gonna define the post treatment period starting six months after the purchase because articles that are supposed to have, for example, this investigative, high investigated scores or more accountability related language take time.

It's not that in one month after he purchased the newspaper you can like start seeing changes. This is gonna take some time because of the process of creating these articles. And this is the distribution of the two indexes, the accountability words proportion on the right and the sum of investigative reported index on the left.

Okay, so I will show you here we plotted the raw data and like did some smooth transformations to assess whether we are fulfilling a parallel trends assumption before engaging in this diff and diff approach. And here is the index of investigative reporting. Again, the sum of it, the red line shows the time of the start of the post treatment period.

We are here already removing the six months after treatment. So here you can see that actually before the treatment we have some evidence that the things are working in parallel. The yellow line is the line depicting those newspapers owned by a billionaire. Our treatment group and those that were not owned by a billionaire are the blue line.

And here before the treatment, we see like this parallel trend, but we see that after that they basically split. And the difference, well, there's a gap between them. And this one is for the accountability words here. It's not as neat, as beautiful, but this is for transparency. We have some bumps here that it's also related that we have just two newspapers in this treatment group.

So it's more sensitive to some changes, but at least, at least some trend there. Okay, so these are the results. These are the regressions of investigative reporting on our what? This is the definitive models for investigative reporting and accountability words. What we find is that Dun Xiang, or like his ownership, definitely increased or had a positive effect in the content of investigative reporting.

And this effect is reliable and different from zero. But we don't observe the same for this count of accountability related words. What it's interesting here is we try to dig deeper and really dissect what was going on. And here, for example, instead of using just one indicator of post treatment and comparing just the six months after and before, what we're doing is basically an indicator by week.

That's how we are defining the post treatment period. And it's interesting because on average you can see that this is all like, yes, getting to zero, but over time, this accountability count is increasing. So I don't want to say that there could be a reversal or there could be that on average, it doesn't matter if we're not seeing that change that strongly.

But it's interesting to see that there are some dynamics here that suggest that things are changing in the newspaper. Okay, the second part of the project is trying to analyze the prevalence of local news and whether Sung Soon is actually fulfilling this objective or giving back to the community and focusing on the community's issues.

And for that, we're gonna classify the local news found in the front page and A1 section of different newspapers as local or non local. For that, we're gonna use bi directional encoder representations from Transformer Bert for both the tokenizing process of, like these articles. And also we're gonna use a model and we're gonna fine tune it.

BERT is learning the co occurrence of words from left and right. So when you're reading, it's like trying to learn the structure. And how these words are happening together in these sentences, not only when you read from left to right, but if you do it from the other side.

So it has proven to have very good results. We chose this one for this task because it was relatively easy, or so we thought, of course, but we actually had really, really good results. That I will show you in a sec. The tokenizer, the bird tokenizer was pre trained on Wikipedia articles, so not exactly our corpus with news articles.

But we thought that it was like a good match because the language and the things that we expect to find in these two corpora were similar. The birth model was retrained on 200 new species from the LA Times that a group of coders classified, two undergrad students, one grad student, and Alison and myself.

We had some validation processes that I'm very happy to talk about in the Q&A. And I'm putting here red Star, because ideally we should have retrained Bert on more newspapers, but at this point in time, we just had access to the LA Times. And we also had some restrictions on for how long we could access the server with other data, so we had to actually be very quick.

But I'm curious about your thoughts as text analysis experts here on this particular step. The codebook that we used to classify these news articles as local or non local was based on a codebook that we developed based on whether national figures were mentioned. Whether the topic mentioned different cities, whether the main focus was actually the city, even if there were comparisons with other cities in the country.

And that's how we trained also our coders.

>> Michelle Torres: And the model reached an accuracy level of 0.92, which we thought it was very, very, very good. But we found some interesting mistakes that actually made sense and that we are planning to correct and to retrain the model for that.

This is a pipeline of BERT, we had our training set that was just LA Times articles. We clean up the data, tokenized it, fine tuned the model, and then we evaluated, and after that we applied it to the tokens that we also extracted with BERT from our full corpus.

Okay, so the result are that, these are the results of regressing this classification of local versus non local on our diff and if approach, right? So these are the coefficients over time, increasing the post treatment period of this definitive effect. And what we see is like basically no results.

So at least in this categorization of local versus non local, we see no effect in the content, even after many months of sunshine being in charge of the LA Times. Okay, so the third and final analysis is related to inspecting the quality and impact of some of the news articles and the social media posts that we see from these news news outlets.

And for that, we're creating this typology of news topics, trying to identify hard versus sublocal news, and as well as like some interaction with local versus non local. For that, we are gonna focus on soft versus hard news, that's our first dimension. Soft news include the sensationalist crime or really impactful crimes committed in the city, sports, weather and traffic.

Things that are part of the community, but that are not super relevant. For example, political discussions, policy discussions, etc, whereas hard news actually focus on that discussion of politics, economic issues and policy formulation. We are using climate change here because a lot of the pieces that we could find, although they were talking, let's say, about the California fires, they also had this climate change angle.

So traditionally the field wouldn't put some things related to weather here. But for this particular question, we think it's important. And also investigations of high profile cases. For local versus non local news, basically state plus, city versus national and international, and for this analysis, we're only using newspapers from California.

We run two separate structural topic models, one on the tweets and one on the news articles with ten topics each. And our prevalence covariates are the treatment group, the treatment period, and the interaction between the two to capture the definitive coefficient here. These are the topics for the tweets, and we are trying to classify them according to their focus and type.

It was interesting to get this customer feedback, so these were tweets, like calling for like, hey, subscribe to the LA Times. And have you already subscribed to our paper and support your local journalist actor? So we found a lot of those customer feedback tweets. We also found some hard local news talking about policies, talking about political figures, political scandals.

But from an investigative reporting perspective, sports, soft local news like sensationalist crimes and fires, soft local news related to weather. Entertainment industry here, notice that we're coding it as both, because for the LA Times, the entertainment industry might be actually not that soft, given that it's like the heart, the economic heart of the city.

National and California interactions, high profile cases, other sensationalist news. And these were front pages that were just like getting tweeted to say, hey, go and read the newspaper from today. For the articles, we have local policymaking, also arts and sports, natural disasters, content in spanish, health and education policies.

A lot of these were also related to like, hey, the dodgers visited this hospital or like, hey, we're talking about that we need more diversity in our rooms. But generally with like a local story or like a local profile. Technology, housing, homelessness, national politics and international news. And of course, I'm cherry picking and I'm gonna show you the plot that is the prettiest with respect to the parallel trends that we found in some of these topics.

This is the proportion of topic customer feedback over time for the tweets. And I really like it because before Patrick Sunshine took over it, basically it's like the trend is identical for those papers that were owned by a billionaire versus not. But after he takes over, you start seeing really, really strong differences or disparities between the trends.

And this is enlightened with what we were reading or observing in the newspapers. So I will show you the coefficients basically for this DID coefficient the difference in difference to capturing the ATT of soon taking over the newspapers for the different topics that we estimated in both the tweets and the art and the news articles.

Here we see that there's a positive effect of Sunshine's tenure on the entertainment industry news or the prevalence of this news on hard local news and customer feedback. So again, based on like the qualitative exploration of our corpus, this definitely resonated and like a decrease in the prevalence of sub-news or more national news.

However, we don't see the same thing in the actual articles published. So this was like the public face in Twitter, but in the articles we see some positive effect in local issues like public safety, policing and arts and sports, less so on, for example, national politics. But in general, we just see these increase in like soft local news as well.

So not a focus on like the things that are like high quality that he claimed he wanted to focus on. But really more about local politics overall with like this very interesting decrease in the prevalence of the topic housing and homelessness. Which is one of the things that I think everyone wants to talk about and actually see a discussion of it.

So just to conclude, we actually find some support for this hypothesis one, that he had a positive effect on the content of the paper. We see increased levels of investigative reporting and a delayed increase in accountability words. We also see an increase or increased focus on hard local news and a decreased focus on soft local and national news, but only in social media.

So there are some caveats here. We see increased focus on local topics, but mostly on soft news when it comes to the actual news articles of the paper. And we see an increased emphasis on customer feedback and enrollment efforts. So this is actually one of the most striking findings that he's really trying to attract customers, to attract followers.

And he's probably failing at that, because remember when I said that these billionaires might be volatile? He ended up selling the San Diego Union Tribune. And there are rumors that he wants to again resell the allied times. So that cannot be captured by our data. But it's basically the conclusion after analyzing some of these things that might just.

It just might not be this profitable business that they are all envisioning. It could be so but we still want to conclude that while some billionaires cause harm to their newspapers. Some research finds that they cause harm to their newspapers and that might contribute to increase inequality. We find here some partial evidence that other type of actors, more civic minded billionaires, can actually improve some aspects of the paper.

And can at least help to promote local political learning and can at least save this important piece of our civic culture and democratic culture. And hopefully we also provided a framework to evaluate other billionaires and other type of owners now that this is becoming a more recurrent phenomenon that we see in local media and journalism.

And with that, thank you so much. And I'm really looking forward to your comments and feedback.

>> Justin Grimmer: Thank you, Michelle. I'll start off with a couple questions and I'm sure Steve will hop in and we already have at least one question in the Q&A. Okay, so my first question and comment is thinking about what is the optimal or best production function for investigative reporting?

And let me just offer a distinction between two models. One model gives an army of investigative reporters space to investigate deeply and produce a small number of very consequential pieces. So think like the Theranos Reporting didn't result in all that many stories but was super consequential. Or you could imagine a different model that says, I want to give the impression that we're ramping up lots of investigations.

So I'll send my reporters to a lot of local meetings, and I'll ask hard questions to local politicians, but never dive deep. And the reason that distinction is consequential, I think, is if we think that the deep investigations matter more, it could be the case that the share of investigative words would go down.

But the amount of socially desirable investigative reporting that is happening could be going up, if that makes sense. So I was just sort of curious how you're thinking about that, or maybe how reporters think about that possible distinction.

>> Michelle Torres: These are super good points. Thank you. I think we were thinking about it not as sophisticated as you're thinking it, because basically what we notice is not really a lot of nuance in this.

Well, it exists. What you're saying it definitely exists, but here it's more about it's either some variation of this or nothing, right? And I like, that's especially what we see with these hedge fund vultures coming. They just remove the entire department of investigative reporting is fired and there's just a lot of importing from newswire sources.

So leading also the local part and focusing on these cheap stories that. That are yeah that you can get in the community and with almost zero effort. So we were less concerned about the nuance. I would say, though, that I like the investigative scores that Turkey Liddell developed because it's trying to capture different dimensions that you're mentioning, right.

The influence for sure, is one of them, but the fact that a lot of people consider these a very influential, a very important piece. Or high-quality investigative reporting, and that's the reason why it won this award. Or the embeddings, the actual content, or all of these different components of the index are trying to reflect that.

But what I'm thinking about is that we could still try to parse out all of these components and maybe try to get these scores excluding different components at the time. Or like almost trying to, to capture these two dimensions that you're talking about and see whether there's an effect.

That would be super interesting, and I think you're right and we should see it. But probably at a higher level among these newspapers are already established and doing something unless so about some of these that are dying. And just need to be safe, basically because this is when the very drastic tactics just come.

 

>> Justin Grimmer: And just a very brief follow up on that before I let Steve ask a question. It seems you talked about sort of a first stage effect there. The billionaires take over and sometimes they completely get rid of the investigative unit and sometimes they leave it or they may beef it up.

Knowing the number of reporters who are on the investigative beat and how they're assigned within potential investigations would be a way to sort of get at exactly that idea. It's not taxed, but it sort of begins to explain why you get your results.

>> Michelle Torres: Yes, thank you. That's a great idea.

Yeah, I'm definitely putting it down and talking to Allison about it. Yes, thank you.

>> Justin Grimmer: Steve, do you want to chime in?

>> Steve Davis: Yeah, thanks. Interesting talk. I've got some critical comments and questions. So first let me just start with an observation which you can disagree with if you like.

Profit motivated owners have strong incentives to provide the kind of newspaper content that their readers want to read. And so is that bad? You implicitly take the stand that it is. But if so, why? You ought to articulate that you don't really make it. You don't really make a compelling argument in my view, it's more like argument by innuendo.

Profit motivated owners are vultures. Benevolent billionaires are doing things in the public interest. So I find that part of your setup for the presentation quite unsatisfactory. Second comment is it's odd to see the focus on local newspapers without. An effort to situate them in the broader context of news sources when the way people access and consume news is undergoing such profound changes.

It's like the printing press comes out, I guess was well after the first printing press, but when newspapers started being circulated. And well, the local town crier is now out of business, and isn't that bad? Well, the local newspapers may be the analog to the local town criers.

Maybe, maybe not. But at least it would be useful to try to situate your study in the context of some information about how it is that people now access and consume news, and particularly news about local issues in particular. That's just missing. And without that, it's very hard to know what to make.

And there's a related thing, which is, maybe this is being made up by other sources of news. And Twitter, you look at Twitter, but Twitter's not really a very good medium for long form or even anything more than tweet form journalism. It's mainly used to like, go look at this, go look at that.

So you need to look at other sources. I don't know if Substack has a large local news content that actually do something that is more comparable to what we think of as the longer form article supplied by journalism. So I just found that just missing. There's a lot of value judgments in the way you present things that I think the research would be stronger without.

You talk about, did this one particular owner worsen or improve coverage? I just would restate that in neutral terms. It's not clear whether local. You seem to think that local news coverage is better than national news coverage for local newspapers. I don't know that you can argue it both ways.

It's true that it might be better for the local news market quality. On the other hand, if the LA Times does a great investigative piece of national import that has benefits throughout the country, and there was a time, it's not so true anymore. There was a time when the major regional newspapers in the United States did aspire.

They did aspire to writing not only important national stories, but important international stories. And now there's only a handful of newspapers which are national scope that really see that as a central part of their mission. So I just think, why don't you just couch in terms of, do they do more local versus national, do they do more soft versus hard, do they do more long form, investigative or not?

Whether that's good or bad requires a lot of assumptions, either explicit or implicit, to judge. So that's just a comment about how you present things. And then I'll just make one more comment at this point. You got to raise the question of external validity. You've got one, you got one.

This is a case study. And if it's a template for how to do many case studies and we're going to accumulate them over time, that's one thing. But it's not hard to imagine that I might come to a very different assessment about how rich owners influence, directly or indirectly, newspaper performance and coverage.

I could look at the Murdoch family, I could look at the Salzburger family. You mentioned Bezos. So those would be very different case studies, I guess. I'd like you to speak to just what, if any, are the broader messages of this study? So I gave you a lot, I'll let you respond.

 

>> Michelle Torres: Yes, no, but thank you. This is great. And a lot of these comments are really tapping into things that somehow, yeah, we've been challenging with including the external validity question. And trying not to be normative and be, okay, what's good and what's bad? We're just observing these trends, but I would still say some things in my defense, in our defense there.

So with external validity, yes. So we started this, as I said, we saw it. It's like now hitting close to home the fact that it's in LA. And I get very excited about this. But we started it with like, let's have this test, let's try to learn as much as we can from it based on this figure that is kind of like representing a good portion of them, like Henry and, and Glenn and others, and then we get more data about them.

And we do this for Bezos, we do these for Glenn, and we do this for Henry, who acquired the. We started with this because we already had some data about it on it. It's been very, very, very hard to get our hands on this type of data over time or going back many years, especially with the objective of mining it or getting the full text.

And it's been very, very expensive. Eventually, the goal is to extend it like that and have a bigger project where we really needed to start relatively small because it was not small. We still spend a lot of money trying to get some of this data. So it was our first test.

And really with this idea of what can we learn about this, how can we shape some expectations and actually develop some theories based on this particular case? I mean, also because it's an important case, right? It's like the LA times, like one of like, and it's California, one of the most important states.

So that's a justification. But I'm with you there. This is providing some meat for what we eventually wanna do and extend, and trying to even start developing these theoretical things. Now for the assumption, as I said, we've been struggling to not be super normative, and we presented these a couple of times in the past.

And people had a similar concern of, why are you taking this localization as a good thing? And this is also based on what other people have found in their research. This is actually one of the few pieces that have a very strong theoretical basis, which is like, a local newspaper should be serving their local community.

This doesn't mean that it should have 100% of local news. It's perfectly fine. I mean, even some of these news, especially in LA, it was hard to. To tease apart what part was national and what part was local when they were talking about interactions between the levels. So this is by no means saying it should be 100% local, but if it's your local newspaper, it should be talking about the things that you consider important and that are affecting you directly, right?

And if we believe the research that has found that more local knowledge leads to less polarization or like to more participation, more engagement with the community, higher levels of knowledge, then we can here put the thread on how we might actually want to see more of that, rather than just newsware sources at the national level, very cheap news that are not really talking about your community, but about isolated cases here and there that just look very sensationalist.

So that's the motivation, or at the core of why we are still trying to do some kind of typology in this case.

>> Steve Davis: But yeah, you can do the typology without the normative overlay of whether it's good or bad. I'm not persuaded by what you said. You might be right, but what you said so far doesn't persuade me that on the margin, more local is.

Is better than less local. Again, it depends on the context within which people are acquiring their news and so on, and that's a very complex issue.

>> Michelle Torres: Yes, agree, agree, and even the quality, right? It could be, it doesn't matter if they're gonna only talk about the pumpkin patch-

 

>> Steve Davis: The wire services would object to your characterization of what they're supplying. I'm not here to defend the wire services, but.

>> Michelle Torres: And I work with, with a lot of their data, so I don't want to antagonize them. But it's true that, I generally work with images from wire sources, and you can tell that it's like very focused on particular national cases or things that are very disconnected from the community.

Even when you talk about protests, for example, you don't get a picture from your protests, like your local protests, but you get a picture from a protest in Washington or a protest in DC. Which is actually a little bit harder to connect to, or it's like providing less information about this local activity in your community.

But no, I fully agree with that. I don't wanna make it cartoonish and say it's just about that. That's why we have these different distinctions of soft versus hard, local versus non-local. But yes, very aware that it might, it might depend and probably we need to do a better job at like tapping some of these things.

Like, maybe local hard is better. And again, not like all of that, but a higher proportion so the communities get informed, but we will tone it down and think, I like the idea of, leave it more, more, more open. And then I would just quickly say something about the newspapers.

Yes, part of the motivation is to use Twitter, and I deleted that slide. But I even had a screenshot of a tweet saying, I don't subscribe to the New York Times. This is how I consume news. People saying those things where their newspaper is actually Twitter. So it was a way of trying to get at that, knowing that not a lot of people are going back and reading, reading the printed versions.

But yes, I mean, we're just assuming that they are here. Some people are consuming them, for better or worse. There's some research that suggests that it's important. So I like the idea of substack, I will need to think more about, well, how, first of all, can justify how people are consuming this news and whether this is still relevant.

And second, if I can think of other sources that could be interesting to include in the analysis. So thank you so much.

>> Justin Grimmer: Okay, we got a couple questions in the Q&A. First, could we give Erin Carter the microphone? Let her ask her question.

>> Erin Carter: Hello, hi, guys!

Sorry, I'm a bit sick today so I apologize for my voice. So, really, really interesting presentation. I really enjoyed it. So I have two questions. Please choose whichever you feel interested in. So the first one is, I'm curious about whether there's a calendar to this sort of thing.

In our propaganda book, we found that autocrats, when they bought or acquired newspapers in some way, they in general maintain pretty objective coverage, actually, to get people to buy in and view the newspaper as credible. And then they spent it right before political crises and elections, most importantly.

And I'm curious if you see some sort of similar business calendar for these billionaires, right? So are they just letting the LA Times do its own thing until right before a big deal, the billionaire is implicated in some way or some sort of other political business cycle moment that you think is important?

Or even elections themselves, right? So maybe these billionaires are supporting political candidates because they want a certain set of policies out of the LA government. So do you see that sort of differential coverage, you know, according to business or political calendars? I'm just curious if you looked into that.

The second thing I was wondering about, so, like, less charitably, one might think that billionaires are funding investigative journalism to investigate their rivals. So I'm really curious, if you would like, did you look into any sort of real world heterogeneous effects? Do you see their business arrivals engaging in less fraud or different behavior, or investment decisions after the acquisition?

Thank you, really enjoying the presentation.

>> Michelle Torres: Thank you so much. I love this question. I would say we haven't done any of those, but I will start working on that, and then identifying even some periods. I mean, we are kind of limited here in terms of, given that the treatment period is just fixed, we don't have that much.

I mean, we can go before, but I haven't gone that far after, but that's something that we can explore. So I will definitely go there. I mean, one thing that I should say, and it's kind of related to both questions that I find super fascinating, we haven't done that deep exploration to see, okay, what are they publishing about their rivals?

And identifying even this in the first place. But, for example, we tried to look for the number of times that Soon Xiong was mentioning his own businesses. Or he was mentioning the sector in which his businesses are to see if he was using this as a platform for self-promotion.

And there, surprisingly, there was nothing. It was really, really hard to even identify that. The few things that we see is with, for example, the topic technology that we see, there was, like, there were some changes, but overall, in that one, we didn't see anything. But I will dig deeper into what you said and actually try to code it, because it's a super good idea.

And I think very interesting angles that tell us about how rich these interactions, and how this can be used as a platform. This is super interesting, thank you.

>> Justin Grimmer: Okay, and Elizabeth? Can we give Elizabeth Elder the microphone, ask her question?

>> Elizabeth Elder: Hi, thank you so much. So I was thinking about the pattern of results that you found, and thinking that it would be really, really hard to differentiate them between someone who was trying to improve the quality of coverage in the paper in some way versus someone who was just taking a pretty long view of the potential resale options.

Maybe he wants to hold onto the paper for ten years. And in that case you don't wanna gut the staff and move everything. Do what the Cleveland Plain dealer did. You want to maintain the brand. You wanna win some awards, you wanna increase circulation. These seem like the kind of things that you would want to do if you had kind of, just a not particularly short term view of making a profit, and not necessarily being charitable.

Just thinking longer term. So I was just curious if you had generally thinking about maybe ways that these owners might be thinking about economic value that they get from this differently, and if there's any way to identify whether there is. What would it look like to have a purely philanthropic motive here?

What would we expect to see if Jeff Bezos really did just want to spend money on owning the Washington Post and never intended to make it back, what kind of patterns would we expect to see in order to identify someone who is being philanthropic versus people with different pictures in their minds of how to get the most value out of these investments?

 

>> Michelle Torres: Yeah, this is a super interesting question. And I think it's gonna be related to something that Steve mentioned and that I didn't. It's like, we're also cynical. We're not saying that a philanthropic objective, or these philanthropic statements, really evidence their true objectives and are just going to do it for, the love to this art, or, for love to their communities.

I definitely think that there's something about profit and about business. These are businessmen, right? This is all about how they've been successful at selling things to people. So there's definitely this component that I'm not entirely sure how to disentangle, because in a way, one of the ideas that we had is to include also, circulation rates or, Like money invested in some of these things over time.

It's kind of difficult, again, because of the data points, and it will also be difficult to basically estimate the utility curve. So you have some investment in terms of resources. Let's say there are also some even ideological resources that you might invest, or emotional resources, or all the resources that you have, and the costs, versus what you're getting out from this.

So we are struggling to find that, and I think that's why it makes this particular question really hard. What I can say, though, is that what we were trying to find and that we got glimpses of it is, for example, this idea of, well, if you're really trying to increase the number of subscribers, it's giving you some indication that they are like, yes, of course they want to survive, they need to do this.

In this particular case, for example, we know that he already invested a lot of money on the LA Times. He really wanted these to work out. And then the rest, we are basing this evaluation based on what he said, that he was doing this because it was more of an internal thing than a business, but we should probably add the other component.

I can't promise that we will be able to disentangle these two dimensions there, but at least it would be very good to have that part, too.

>> Justin Grimmer: Okay, we have about a minute left, so I'm gonna make one more suggestion just to give a plus one on something Steve said.

I also had written down a question, why is profit motive bad? Cuz you imagine lots of good things can come from profit motives as well. But another way, if there's one way to do this, which is to pivot away from some of the claims that are made in the paper, just to focus on exactly what you can with the newspaper.

Another thing that you could try to show, though, is if you established outcomes that you cared about and that you think newspapers, in some way, cause those outcomes or influence those outcomes. So, for example, behavior of elected officials, something like discovery of fraud in an area, information that citizens have when they vote, it's obviously a hard thing to measure.

But if you can show, however you might do it, that different ownership structures affect these outcomes, that you can make a strong case. I think we can all agree, more fraud from politicians, that we're worse off, less fraud, we're better off. Citizens knowing more about what they're gonna vote on when they go to the ballot box, that might be a thing that we can all agree that is probably better.

If you can point to those, then I think you're in a position where you can make these sorts of stronger arguments. And absent those, it's challenging.

>> Michelle Torres: Okay, okay, perfect, no, makes sense, makes sense.

>> Justin Grimmer: Yep.

>> Michelle Torres: And just to be clear, I share the skepticism and I don't think that making profit is bad per se.

It's just that when they interview them, there are some honest ones, like Zelle being like, I'm just here for the money. And there are others that I'm like, I'm here because I love LA. But I know that they are probably not doing it just for that, but it's basically what the public discourse is and what they are self-proclaiming to be interested in.

But I love this idea. Thank you, Justin.

>> Justin Grimmer: Yeah, well, just to be clear, I'm here because this was a great paper, not because I was being paid to be here. And Michelle, that was outstanding, thank you so much. So folks can stick around. We can ask Michelle some more nitty gritty questions and we can now thank her for this great presentation.

Thank you.

>> Michelle Torres: Thank you, everyone for being here. Thank you for the invitation and for this great feedback. Stay tuned for second version of this paper, thank you.