Charter Schools, Test Outcomes, and Behavioral Change
...Promise Academy students display some evidence of lifestyle changes: a decline in the rate of teen pregnancy for female students and in the rate of incarceration for males.
A number of recent studies have shown that high-performing charter schools increase test scores for many urban students, but there is relatively little evidence on whether this translates into changes in other dimensions of human capital acquisition such as college enrollment, reduction in the incidence of risky behavior, and improved health.
In The Medium-Term Impacts of High-Achieving Charter Schools on Non-Test Score Outcomes (NBER Working Paper No. 19581), authors Will Dobbie and Roland Fryer, Jr. analyze survey data from hundreds of students who applied to the Promise Academy, a charter middle school in the Harlem Children's Zone in New York City. They find that students who had the opportunity to attend this school not only scored higher on tests but were also more likely to enroll in college. Moreover, Promise Academy students display some evidence of lifestyle changes: a decline in the rate of teen pregnancy for female students and in the rate of incarceration for males. However, for a range of other health-related behaviors there were no apparent changes.
The authors recognize that charter schools are heterogeneous and that many charter schools are no more effective at improving test scores than traditional public schools. But some studies show that higher-quality charter schools - those with extended school days and school years, aggressive recruitment of high-quality teachers, intense data-driven monitoring of student progress, and group tutoring - do achieve higher test scores. Promise Academy is a charter institution with all these attributes and with a lottery process for admission, which allows the authors to compare the academic performance and the other behaviors of admission lottery winners and losers. This provides "treatment" and "control" groups of otherwise comparable students.
The authors find that six years after being admitted to Promise Academy, the lottery-winning students scored 0.28 standard deviations higher on national math achievement tests compared to lottery-losing students, and 0.12 standard deviations higher on reading exams. These students were also more than twice as likely to take and pass advanced exams in chemistry, geography, and other subjects. They were 14.1 percentage points more likely to enroll in college and 21.3 percentage points more likely to enroll in a four-year college, a 102 percent increase from the control mean.
The authors find mixed results with regard to "risky behaviors." Female lottery winners were 12.1 percent less likely to report getting pregnant during their teen years, a 71 percent drop from the control group mean. Male students were 4.3 percent less likely to be incarcerated. But Promise Academy lottery winners reported similar drug and alcohol use and criminal behavior as students who were not selected in the admissions lottery. On health-related issues, going to Promise Academy appears to have little impact on the incidence of asthma, obesity, and mental health problems, though lottery winners were more likely to report eating more nutritious foods.
The authors stress that their conclusions are limited to medium-term outcomes of Promise Academy students, and that longer-term consequences cannot yet be evaluated because of the relatively short time span of the research project. The authors conclude that "the cross-sectional correlation between test scores and adult outcomes may understate the true impact of a high quality school, suggesting that high quality schools change more than cognitive ability."
--Jay Fitzgerald
