The evolution of charter school quality in Texas

Charter schools are an increasingly popular alternative to traditional public schools for families across the United States. Charter schools were initially hailed as sources of educational innovation and as facilitators of school choice because they brought market mechanisms into educational markets. But after the rapid expansion of the charter sector over the past two decades, these schools remain controversial. The reason stems in part from the mixed empirical evidence of the effectiveness of charters at raising student achievement. On the one hand, lottery studies focused on specific oversubscribed urban schools generally find positive effects among students who attend charter schools. But on the other, studies using a broader set of schools tend to find smaller or even negative impacts on student achievement.

The problem with both sets of studies is that they generally evaluate schools at one point in time, which tells educators and policymakers alike relatively little about the overall performance of the charter school sector. Evaluating the market-oriented reforms that were originally explicit in the charter movement requires examining the dynamics of their educational achievement over time. In our working paper “The Evolution of Charter School Quality,” my co-authors and I study these charter school sectoral dynamics using longitudinal microdata on students and schools made available by Texas Schools Project, a research institution interested in educational policy questions. These data allow us to characterize the full distribution of school quality as measured by school-level value added for both charter schools and traditional public schools.

Figure 1 below presents the 25th, 50th, and 75th percentiles of the distribution of school effectiveness in mathematics for charter schools relative to the corresponding percentiles for their traditional public school counterparts. The figure highlights the steady improvement of charter schools relative to traditional public schools in Texas between 2001 and 2014. Using our preferred model (see our working paper), we observed substantial improvement in charter school value—roughly 0.25 standard deviations in math and 0.22 standard deviations in reading at the mean.

Figure 1
The improving relative performance of charter schools versus traditional public school in Texas
Charter school educational quality over time relative to traditional public schools in mathematics, 2001–2014

My co-authors and I next examine how the dynamics of charter schools’ entry, exit, and persistence in the educational marketplace in Texas drives the evolution of school quality. We observe three features of these dynamics that are consistent with market forces. First, the schools that closed either voluntarily or by the Texas Education Agency were, on average, the worst performers. Among schools that were open over the entire period, there was improvement in mathematics relative to their traditional public school counterparts. (See Table 1.)

Table 1
Charter schools that closed were typically the worst performers
Average value-added, enrollment, and length of operations of charter schools in mathematics for 2001 and 2011, by status of school operations

Finally, among the charter schools that were new entrants, their performance, on average, was much closer to those that persisted in the marketplace compared to those that closed. This last finding probably reflects that during the period we study, most new charter schools in Texas were due to expansion within so-called charter management organizations, or CMOs, which oversee the rollout of new schools and overall operations of the system. These results suggest that relatively successful CMOs may be, in many cases, figuring out what works in their schools and replicating it.

We then look deeper into three possible sources of this observed improvement considered in the educational literature on school:

• Reductions in school turnover
• Increases in the share of charter schools that adhere to a “no excuses” educational philosophy
• Pre-enrollment differences among charter school enrollees

In the paper, we first present evidence that the number of new schools accelerated in the latter part of our observation period, yet the fraction of students attending new schools declined steadily over the period. This suggests that charter schools are maturing as a sector over time.

Second, and consistent with existing literature that points to the relative success of the “no excuses” style curriculum in a number of settings, we find that the improvement of charter schools in Texas has coincided with such schools capturing an increasing share of the charter school market. Several important studies report evidence of the strong performance among charter schools that set high expectations, required uniforms, or more broadly adopt a “no excuses” philosophy, among them a 2012 study published by Mathematica Policy Research, a 2013 study by Joshua Angrist and Parag Pathak at the Massachusetts Institute of Technology and Christopher Walters at the University of California, Berkeley, and another study that same year by Will Dobbie of Princeton University and Roland Fryer of Harvard University.

Third, we present evidence of changes over time in the composition of who attends charter schools. Our results suggest that, in comparison to demographically similar students who attended the traditional public schools from which these charter school students were selected, these new charter-school students are increasingly positively selected on the basis of their pre-charter achievement scores, as well as a lower likelihood of having a disciplinary infraction in the previous year. Moreover, in comparing students who remained at the charter school with students who left after a year, we find evidence again that those students who remained exhibited higher pre-charter enrollment scores.

Importantly, regression evidence reported in the paper shows that including controls for increasing positive selection of charter school student entrants and reductions in student turnover does not mitigate the robust positive relationship between charter school effectiveness and adopting a “no excuses” curriculum. In short, the curriculum and disciplinary structure of the school seem to be a key part of improving charter school quality.

Notwithstanding the suggestiveness of these findings, these potential sources of improvement explain only a fraction of the improvement in the effectiveness of charter schools. The findings suggest the value of taking a longer-term perspective when evaluating the impact of a major educational reform such as the introduction of charter schools, especially when the success of the reform ostensibly depends on parental decisions and market forces. But the findings also point to the necessity of further research on the role of principals and other aspects of school operations, as well as the behavior of families with children attending charter schools in leading to better educational outcomes for those students.

—Marcus Casey is an assistant professor of economics at the University of Illinois at Chicago and the David M. Rubenstein Fellow at the Brookings Institution.