Salvaging Accountability
George W. Bush rode to the White House pledging high standards for all students. He’ll leave Washington with the nation’s public education system focused on teaching basic skills to disadvantaged student populations, with the United States lagging in international comparisons of educational attainment, and with his signature education law plagued by so many problems and mired in so much controversy that it has put at serious risk two decades of work to improve public schooling by making educators accountable for their students’ success.
The most important thing Barack Obama or John McCain could do quickly to salvage the accountability movement is change the way that the federal No Child Left Behind Act judges schools. Not by abandoning NCLB’s focus on students’ meeting standards, a move that would be unwise on both policy and political grounds, but by making the law a more legitimate report card of school performance, one that provides a fair and accurate gauge of educators’ contribution to their students’ achievement. Since its inception, NCLB has instead held schools responsible for factors they can’t control and perversely encouraged states to set standards low.
It’s critical in any accountability system that the metrics used to judge performance reflect accurately the contributions of those being judged. In education, that means measuring how much progress a school’s students make during the school year, a “value added” approach that accounts for the disadvantages (or advantages) students may bring to school because of the quality of prior instruction or their family backgrounds. It’s a strategy that pressures schools working with disadvantaged students to work hard in their students’ behalf without penalizing educators for taking on tough assignments. And it’s a strategy that doesn’t reward rich schools merely for having privileged students.
Yet NCLB currently demands a different system of rating schools, one that gauges their contribution to learning much less accurately and far less fairly—a system that has given the law’s many opponents an easy avenue of attack. Rather than measure how much a school’s students learn in a year, NCLB requires that school ratings be based on how many students in several groups (white, Asian, black, and Latino students and English-language learners, for example) pass state tests that are given once a year. Civil rights groups argue that this is the best way to get educators to finally pay attention to traditionally underserved students.
As a measure of school performance, however, this snapshot strategy is flawed. Because student populations vary greatly from school to school, and because family income, parental education, and a host of other non-school-related factors have a major influence on students’ learning, some schools have to improve student achievement a lot more than others to get their students up to state standards. The federal law is unforgiving of such schools. As a result, it gives an unfair advantage to schools with students from privileged backgrounds, and it fails to measure what matters most: how much students learn during the school year.
The consequences have been severe. While the No Child Left Behind law’s accountability system has undeniably cast light into some dark corners of American public education, it has labeled as failures many high-performing schools serving very disadvantaged students, demoralizing their teachers and administrators and turning them against the law, while giving an undeserved seal of approval to thousands of middling schools with more affluent students. In Dallas, one of the few places to rate its schools using both value-added and NCLB metrics, researchers found that schools ranked 94th, 77th, 83rd, and 107th among the city’s 206 schools under NCLB placed second, fifth, eighth, and 16th under the city’s value-added ratings, where achievement gains are most important.
NCLB’s failure to credit schools for improving student achievement has led many states to abandon the law’s pursuit of high standards to avoid the political embarrassment of having large numbers of failing schools. In effect, NCLB has placed a glass ceiling on the public education system and the students it serves: By incentivizing states to lower their sights, the law has created an environment that encourages schools to focus simply on getting enough students over the achievement hurdles. But there are vast numbers of students who clear state standards without difficulty and who could and should be educated to higher levels than they are today. At the same time, there’s evidence that the law has caused schools to pay little attention to very low-achieving students who have scant chance of meeting state standards. Ironically, the low bar in many states has led some NCLB defenders to reframe the law’s purpose. NCLB, they now say, is about ensuring students’ basic proficiency.
The law’s flawed school ratings have contributed to a related but broader problem. National education policy can focus on disadvantaged students or high achievers, some argue, but we can’t do both, and focusing on those who traditionally have been left behind is a defensible educational and social strategy, they say. But it’s a false choice. A number of other industrialized nations have fewer low achievers and more high achievers, and thus our schools should be able to produce stronger results across the achievement spectrum, if we give them the right incentives.
The architects of the No Child Left Behind Act were aware of the shortcomings of the law’s school ratings. The Bush administration’s point person on NCLB, Sandy Kress, had led the development of the Dallas value-added system a decade earlier, and told a writer in 2001 that it was “a whole lot fairer” to schools than the rating system of the legislation Congress was about to approve. But the value-added model was too little known and required testing and data systems available in too few states to impose it nationally under NCLB, Kress and others reasoned. Still other voices in the debate opposed it on philosophical grounds.
Only in 2005 did U.S. Secretary of Education Margaret Spellings, under pressure to address the weaknesses of NCLB’s accountability system, institute rules that have allowed 11 states to include measures of learning gains in their school ratings. But, as researchers like Michael J. Weiss at mdrc have pointed out, the states’ new “growth models” don’t improve on the law’s rating system very much. (See The ‘Growth Model’ Pilot Isn’t What You Think It Is, June 18, 2008.)
That’s because Spellings requires states using growth models to judge schools on whether they produce enough yearly achievement gains to get their students up to state standards within several years—a so-called growth-to-proficiency requirement. A school with students a year below grade level and a typical three-year window to get them to state standards would get a thumbs-up rating only if it produced one-and-a-third year’s worth of growth in its students annually. But this still requires schools with disadvantaged students to make greater gains than schools with affluent kids, perpetuating the problems that plague states using the NCLB proficiency model. Weiss’ research reveals that most schools fare the same under the growth-to-proficiency models as they do under the NCLB system.
A wiser strategy would be to rate schools using both the NCLB how-many-kids-are-meeting-state-standards model and a value-added, how-big-are-the-school’s-achievement-gains approach. Holding schools accountable for both types of performance would preserve the law’s commitment to getting low performers up to proficiency, while giving schools meaningful incentives to improve the achievement of all students.
There’s more than one way to combine the proficiency and value-added strategies. One would be to reserve the toughest sanctions for those schools that fail on both metrics, and reward schools that earn high marks in each category. Schools in the middle tiers, with either high value-added scores or high proficiency scores, would still be required to address their shortcomings, but would face less-stringent sanctions.
A second strategy, already in use in Florida as part of a state accountability system that predates NCLB, combines proficiency and value-added ratings by awarding schools points for reaching achievement targets and for learning gains, especially among low-achieving students. And, unlike NCLB, it includes rewards as well as sanctions. Studies have shown that the Florida approach has lifted achievement among low performers and encouraged educators, who tend to trust the system much more than NCLB’s, to embrace school improvement. Former Gov. Jeb Bush urged his brother’s administration to incorporate the Florida model into NCLB, without success. New York City launched a combined rating system in 2006.
Value-added calculations have larger margins of error than NCLB’s proficiency ratings, but because they measure what’s most important in judging schools—student learning gains—their statistical shortcomings are more than worth tolerating. And combining value-added and proficiency metrics would both improve error margins and ensure that schools remain committed to helping 100 percent of students achieve state standards (at least in the mostly elementary and middle school grades where the law requires student testing), without encouraging them to focus only on students near the standards, as NCLB does now.
To help promote higher standards, schools would only get passing grades on the value-added portion of this combined annual rating system if they achieved at least a year’s worth of learning. The proficiency portion of the rating system would give schools with very low-achieving students incentives to make even larger gains. And while most states lacked the technology required for value-added ratings when Congress drafted NCLB seven years ago and the law didn’t require them to create the necessary capacity, building both data systems and statistical safeguards today wouldn’t be unduly difficult.
It’s clear in the face of a deepening backlash against the No Child Left Behind Act that its survival—and perhaps the prospects of the accountability movement in public education—hinge on making the law’s measurement of school performance a fairer and more effective enterprise. With so much at stake, the next administration shouldn’t wait for the reauthorization of the law to act. It should begin to introduce value-added measures in NCLB’s school ratings the same way Secretary Spellings introduced her inadequate growth-to-proficiency models—through regulations.
More here.