This interview focuses on a chapter from A Nation at Risk +40, a report by the Hoover Education Success Initiative (HESI) that looks back at the birth, struggles, and future of the modern school reform movement. (Download the report here.)
John D. Singleton, an associate professor of economics at the University of Rochester, is a visiting fellow at Hoover who focuses on school choice, education productivity, and different systems of organizing school systems. He spoke with Chris Herhalt about school choice models, how to measure performance, and evidence for the benefits school choice has brought to public school districts in the United States.
Chris Herhalt: You write that the most important element of introducing school choice is to improve the quality of education in traditional public schools. Which model, school-choice voucher or charter, do you think provides the bigger benefit to traditional public schools? Is that even settled?
John D. Singleton: No, and I think there are two ways to look at it, considering the different incentives created by different school-choice programs. A voucher program will defray costs for students to attend a school, but those vouchers may be available only to students whose family income falls below a certain threshold. In other words, there are barriers to exercising choice for at least some families. So, you might think that competition from vouchers may not be as strong as it would be from charter schools that students would be free to attend. For various reasons, charter schools also tend to be located in high-need areas, so as for distance, there’s going to be a lot more direct competition. But what potentially pushes against that is charter schools are often oversubscribed. They have no space. And obviously, if there’s no space available, they’re not really creating incentives.
There’s a large literature about how you should design these programs with such questions in mind. But looking at it empirically, what do the data show? Voucher programs get launched, and we can see what’s happening to nearby public schools. Charter schools come into play. What’s happening at public schools exposed to that competition? There is probably twenty years of research that’s fairly consistently found small, positive impacts on nearby public schools.
A lot of the best evidence is coming out of Florida and the Florida Tax Credit Scholarship program, in research conducted by one of my colleagues, David Figlio, and his co-authors. The evidence, especially if you wind back ten years, was a little more ambiguous on the charter-school side. But if you look at the more recent evidence coming out of places like North Carolina, Florida again, and New York City, I think you’re seeing that at least under certain conditions, the effects are positive. And there’s not a lot of evidence that they’re negative. So, under certain conditions at least, both programs are generating these positive externalities.
So, which is generating bigger ones and who is it generating impacts for? We don’t have a great handle on that right now. It’s difficult to make comparisons across studies. My sense is that the order of magnitude for these impacts is pretty similar between vouchers and charter schools in terms of their spillover impacts on public schools, which is in some ways surprising when you go back to the theoretical literature, which I think gives you reasons to think charter schools will be more effective competitors. But that’s not necessarily what’s been borne out by the evidence. There’s still a lot to learn, but the evidence suggests both kinds of programs can have competitive effects on public schools.
Herhalt: You write about the growth of both charters and vouchers, as well as homeschooling being way up. But charters are the biggest share of this new school choice movement. Is that an endorsement of it as the best model, or is that because it’s the easiest to implement? Why are charters growing so much faster than the rest?
Singleton: I think a big part of it is charters had a head start. Historically, people will tell a story about interest in vouchers morphing into a lot of policy momentum for charter schools, which is true; ten or fifteen years ago, there was a lot of bipartisan consensus, and a lot of consensus across states, behind the charter-school movement, among people in the Obama administration and continuing to Betsy DeVos’s support. But I think that’s waned a lot. Now, the big push is for school vouchers. The number of students going to charter schools is almost an order of magnitude larger than the number of students who go to a private school using a school voucher, but five years from now, that may look quite different. New voucher programs have expanded in places like Florida and Arizona. And it will be interesting to see, because these are also states that have very vibrant charter-school systems, what that’s going to mean. Will it draw students out of charter schools, not just out of public schools? What does it mean when you have these multiple school-choice programs competing against not only each other but also the public schools? It’s going to be an interesting area for research.
Herhalt: What’s the biggest piece of information lacking today when you’re assessing the performance of these various types of schools?
Singleton: From a researcher’s perspective, we often use things like test scores to measure what we might call school productivity. Students come into a school, and we see them leave with these scores, maybe we see them leave whether they graduate or not. And test scores—less so now, but certainly in the past—were relatively easy kinds of data to acquire that you could examine in a quantitative way and which we could use statistical analyses to try to understand. Is the charter school boosting test scores, is the school-voucher program boosting test scores? But I think that all along, people have criticized that measure—rightly—because test scores themselves are not the thing we care about.
What we care about is acquiring durable human capital: the skills and productivity that will translate into success in the labor market and in life. And maybe test scores are a proxy for that, but there are lots of reasons to think they might not be. So, what we would really like to do is be able to track students outside of the schooling system. That may be in addition to tracking them in other ways within the schooling system, expanding beyond test scores to pick up measures of non-cognitive ability or engagement in school and class. Are they attending school, are they getting suspended? Things like that.
But then, once they leave secondary school, are they in the labor market? Are they holding a job? Are they interacting with the criminal-justice system? Are they interacting with the welfare system and welfare agencies? These are all, in some sense, proxies for the kinds of things we care about, but there’s good reason to think of these as more directly related than tests to the thing we’re interested in, which is improving students’ long-run success and contributing to it. I think that’s where the frontier is. That’s where we want to evaluate these kinds of programs: in terms of longer-run success stories, which may or may not be correlated with test scores.
It often takes a while to be able to connect some kind of reform or intervention that happened in, say, primary school with some later-life outcome. But talking about the charter-school sector, it’s been twenty-five, almost thirty years now that charter schools have existed in certain places. This analysis is becoming more and more feasible, but it also requires the ability to put together datasets. These data exist. You go through the school system, you enter the labor market, all of that gets tracked. Education data often is held by a department of education. Earnings data can go through the IRS, and can also go through the unemployment-insurance agencies. All of those records exist, but only in a few places in the United States has there been a meeting of the minds to bring these data sources together in a way that researchers can use to develop better evidence-based policy, or to facilitate parents’ decision making.
Wouldn’t you be interested to know what fraction of the students who attended this school went on to attend a selective university, or what fraction of them went on to have success in the labor market and work in a high-paying firm? It’s very difficult to put all those pieces together, except in select situations. And that’s unfortunate, because I think it makes for bad policy and makes it harder for parents to make informed decisions about different schooling alternatives.
Herhalt: If you and others researching education have trouble bringing together those factors, what does that say about what the average parent faces when trying to make a decision of where to put their kid?
Singleton: It says a lot. You can go look for real estate listings. It’s fairly easy to do, and a lot of platforms where you can look at listings will have a school finder and show you attendance-zone boundaries. Some of them will even integrate a school-rating system. Some organizations rate public schools, and I think there are others for private schools; they offer one-to-ten scores or even national rankings of high schools. But if you ask yourself, what are these rankings based on? I think it’s right to conclude that they may be not especially well correlated with what we might call productivity, or a school’s value-added, or its ability to improve a student’s long-run outcomes. By contrast, it may be highly correlated—and there’s a lot of evidence consistent with this—with certain kinds of background advantages for students. These kinds of rankings, and you can even see this in the way people talk about schools, what they’re really picking up, reflect measures of socioeconomic advantage. This school has a lot of resources. Those resources attract very advantaged families, and those students go on to do very well. But is that because it was a good school or because it was a very advantaged school in the first place? It builds in this kind of feedback loop where if a school has these kinds of resources, it’s going to attract these kinds of students. Success is going to feed on itself.
Of course, the tragedy of urban schools is that in a lot of places you have the opposite.
So, the lack of information about which schools are actually doing a good job in terms of improving outcomes, as opposed to which schools are just attracting highly resourced families and highly resourced students, is part of perpetuating that cycle.
Herhalt: You write about generating a multidimensional measure of school and teacher quality. It could combine factors like GPA, attendance, likelihood of college entry and completion, and so on. And you say bureaucracy is keeping this from coming together. Is it more than that? Are there entrenched interests that don’t want that kind of disclosure?
Singleton: There’s been a big pushback against what we might call the accountability movement. And one piece of the accountability movement—you can see this in the No Child Left Behind legislation—were the testing and reporting requirements. We needed accountability and we needed consequences. And part of that consequentiality piece would be to publicly release information about which schools were doing well and their students improving in test scores year to year. And, even in some cases, about specific teachers, whether their students were improving year to year. For a while, and I think this might still exist, you could look up every public-school teacher in Florida on a website and look at these rankings of value added.
A lot of that has fallen by the wayside. And if you’re looking for a villain or culprit in the story, I think a lot of it is teachers’ unions. Not to say that there aren’t important criticisms of accountability and some of the incentives that it creates, or that test scores don’t have limitations. But to the degree that there is pushing—insulating educators who are not highly effective at their jobs as opposed to promoting teachers who are highly effective at their jobs—then one place to look for that resistance is certainly in teachers’ unions. You can look across states and see that pattern quite clearly.
Herhalt: Is research under way into the long-run impact of school choice on student outcomes?
Singleton: Yes, there is some work, including really well-done studies using lotteries for students to get into charter schools. Charter schools are oversubscribed; we can look at a student who was randomly offered admission and compare them to a student who was randomly not offered admission. These kinds of studies have a lot of internal validity. There’s also been some work using those kinds of studies to look at college entrance and even college completion. More work needs to be done, but these findings are really quite encouraging, and I think continue to paint a picture of at least certain types of charter schools being highly successful at improving students trajectories.
Two things are still missing. One is on the voucher side. Going back ten years now, there were a handful of states that put in statewide voucher programs. And in terms of test-score impacts, there were alarmingly negative effects from those programs. I’m thinking about the Louisiana program in particular, but there was also evidence from Ohio and Indiana, again for students using vouchers in terms of their test-score outcomes, showing them not doing as well as a counterfactual student who stayed in a public school as opposed to attending a private school. But an important limitation for those kinds of studies was exactly what we were talking about earlier: the difference between test-score impacts and long-run skill acquisition. And that’s especially true when we’re comparing private schools, whose curriculum may not be aligned with the kinds of state tests used to evaluate students in the state public schools.
The other missing piece, as I mentioned, pertains to my view that there’s an emerging consensus around the evidence that these kinds of school-choice programs create competitive incentives for public schools—that at least under some conditions they can improve performance in public schools. I think there’s some work looking at attendance and other outcomes I described, but by and large, typically this means reading and math scores.
So, we could find a setting and a research design that allowed us to say, here are some students who didn’t go to charter schools but who grew up in a world where charter schools were competing with the public schools they were attending, and compare them to students who didn’t grow up in that same competitive environment. Fifteen or twenty years down the road, would we see any differences in students’ success and trajectories that we could attribute to that?
