Assessing Pandemic Learning Losses and Recoveries

Test scores at the end of the 2020–21 school year revealed dramatic declines in student performance during the COVID-19 pandemic. In a data sample including schools from 21 states, only a fraction of those losses — 20 percent in English language arts and 37 percent in math — were recovered by the end of the next school year.

Those nationwide averages mask wide variations in student performance among individual states, according to Post COVID-19 Test Score Recovery: Initial Evidence from State Testing Data (NBER Working Paper 31113) by Clare Halloran, Claire E. Hug, Rebecca Jack, and Emily Oster.

Analysis of test scores in 21 states shows less than half of the losses have been regained overall, with wide variations across states.

The researchers focused on grades 3–8 and restricted their sample to states that maintained the same testing and scoring practices over the 2017–22 study period. The sample includes Arkansas, Colorado, Connecticut, Georgia, Idaho, Kansas, Louisiana, Massachusetts, Minnesota, Mississippi, Missouri, New Hampshire, Ohio, Pennsylvania, Rhode Island, South Carolina, South Dakota, Virginia, West Virginia, Wisconsin, and Wyoming.

Across states, recovery rates in English language arts varied much more than those in math. Student performance in two states, Mississippi and South Carolina, fully recovered from pandemic losses in English language arts while that in Kansas and Massachusetts, among other states, continued to decline. In math, student performance in all states experienced recovery, but none recovered completely. Mississippi and Rhode Island math scores recovered by over 70 percent, while in Arkansas and Minnesota the recovery was under 20 percent.

The researchers analyzed data for individual school districts and found that recovery rates were highly idiosyncratic. They did not significantly correlate with demographic characteristics, baseline achievement, schooling mode during the pandemic, or how states chose to allocate federal recovery funds.

The researchers were particularly interested in whether the way a state allocated its share of the $122 billion in federal relief funds that were targeted toward education affected initial losses and subsequent recovery. States varied widely in how they addressed students’ academic, social, emotional, and mental health needs. The researchers did not find any statistically significant links between intervention techniques and recovery rates, but they caution that some school districts may not yet have fully realized the benefits of programs financed by federal relief money, complicating the analysis of links between recovery rates and federal grants.

The researchers found a potential correlation between English language arts recovery and how long states have had legislation implementing reading improvement programs. The only two states that fully recovered pandemic learning losses by 2022 were also the earliest adopters of the science of reading (SOR) program: Mississippi (2013) and South Carolina (2014). Both states also took other steps to improve reading, including enhanced teacher preparation, professional development, and instruction. States that implemented the program more recently show mixed recovery rates.

—Steve Maas