In 2009, the federal government made an unprecedented investment in the country’s lowest-performing schools when it sent them $3.5 billion with an order: turn things around. Sufficient time has now passed for researchers and policymakers to begin examining how well the School Improvement Grant program (SIG) is working.
So far, the evidence has been largely anecdotal. In a recent series of stories on the program, The Hechinger Report, together with the Education Writers Association, Education Week and a group of news organization around the country, found mixed reports on how the schools in the program have fared.
More coverage
Education Week, the Education Writers Association and The Hechinger Report partnered with 18 news outlets in 16 states to investigate how $4.6 billion in federal School Improvement Grants are being used to revitalize some of the nation’s lowest-performing schools. Interviewing scores of students, teachers, researchers and education officials at all levels of government, participating reporters set out to determine how the money is being spent and whether the changes it sparks are likely to last.
A new study looking at preliminary data indicates that California schools receiving SIG money made significant gains as measured by test scores, to the surprise of the study’s author, Thomas Dee, a professor of public policy and economics at the University of Virginia. By using data from schools right above and below the eligibility cut-off for SIG money, Dee found that SIG reforms raised a school’s score on California’s Academic Performance Index (API) by 34 points. Before the program, SIG-eligible schools had an average API of 650, while the state’s performance target is 800, so an increase of 34 points translates into a narrowing of this gap by 23 percent. (API scores range from 200 to 1000.)
In California, annual SIG awards averaged about $1,500 per pupil, or $1.5 million per school.
States identified their bottom 5 percent of schools in the SIG program, and those meeting both the “lowest-achieving” and the “lack of progress” criteria could apply for a grant. To get the money, schools had to follow one of three improvement strategies: “restart,” “turnaround” or “transformation.” As a fourth option, districts could take a small amount of money and shut down a failing school.
Nearly three-quarters of SIG schools nationally and 60 percent of those in California opted for the transformation model, which required schools to replace their principals, develop new teacher evaluation systems and lengthen the school day. About 20 percent nationally and a third of California schools picked the turnaround option, which mandated that they fire the principal as well as at least half of the teaching staff.
The amount of buy-in required to make these reforms work was just one reason that Dee was skeptical of the SIG program’s chances for success. The checks came in right as the school year was starting, meaning there was limited time to plan for changes. And, Dee says, it’s just hard to find reforms that work when evaluated rigorously.
Once he analyzed his data, Dee said, “I was staring at these results and exploring alternative interpretations for basically the last five or six months.” But in the end, he said, his conclusions hold up.
In his research, Dee compared schools just above and below the benchmarks set by California to determine SIG eligibility. For example, elementary schools needed to have under a 29.97 percent proficiency rate to apply for a grant. Dee analyzed the test scores of schools just shy of this mark that got the SIG money along with the scores of schools just above it (which made them ineligible for the money).
In the paper, Dee writes that “there were significant improvements in the test-based performance of schools on the ‘lowest-achieving’ margin but not among schools on the ‘lack of progress’ margin.” And the improvements were mostly evident in schools that followed the less-popular “turnaround” model, which required far more staffing changes than the “transformation” model.
As for whether the multibillion-dollar federal investment has proven worthwhile, Dee compares the SIG program to class-size reduction efforts, including the famed Project STAR experiment in Tennessee in the 1980s. He says his findings suggest that “SIG-funded school turnarounds are cost-effective relative to expensive class-size reductions, generating roughly half of the achievement gains associated with Project STAR but doing so at a third of the cost.”
