Hey, Arne Duncan! Be Progressive!

A comment from Jean at Bridging Differences. Well done!

Everyone who thinks that more testing is the answer, or more testing tied to even more serious consequences for more people, should read this report from the National Center on Performance Incentives authored by Richard Rothstein. Apparently, economists (!), business management theorists (!), sociologists and historians have known for decades what teachers and other educators have been saying about the use of quantitative output performance measures for accountability and performance incentive plans. (Merit pay is the latter--it has nothing to do with accountability.)

Specifically, there are negative consequences to 1) exclusive reliance on quantitative measures because they cannot accurately capture the complexity of human endeavors and 2) making the quantitative measures high-stakes for the people whose performance is being measured. At the very least, these negative consequences have to be weighed against the benefits in formulating policy. Preferably, the quantitative measures have to be balanced with the use of qualitative measures in order to mitigate the negative consequences of each.

To simply ignore this reality--pretend we don't know about the downside of, in our case, exclusive reliance on high-stakes testing for accountability and performance incentives--only guarantees that those negative consequences will have full sway and will play out until they bring the system down, or at least become so obvious that the most determined can't ignore them.

Rothstein cites three problems, with much supporting evidence extending back to the early 20th century. All three are familiar to those of us living daily with the testing-and-accountability craze and its consequences. They are:

1) conventional definitions and measurements of educational outputs are so oversimplified that they cannot support valid accountability or performance incentive systems. Goal distortion results, . . . .
2) Adjusting expectations of performance for the characteristics of inputs has proven more difficult than anticipated. With students at different risks of failure because of the varied background characteristics, accountability and incentive systems can be credible only if sanctions and rewards can be adjusted for these variations. . . . .
3) Untrustworthy statistics undermine the credibility of accountability and incentive systems. They would do so even if measurement of outputs and inputs could be defined more precisely. Inadequate data reliability is one impediment: . . . . Because standardized test items are too few to fully represent the curriculum, sampling corruption results. . . . . explicit gaming can manipulate accountability data:. . . .

Rothstein goes on to say

These challenges--in defining outputs and inputs and in the accuracy of data themselves--surprise many education policy makers. Some conclude that the problems stem only from the inadequacy of publich educators. {emphasis added; sound familiar?} For example, one critic argues, good teachers "can and should" integrate subject matter so that raising mathe and reading scores need not result in diminished attention to other curricular areas. But this expectation denies the intent and power of incentives which, if successful, should redirect attention and resources to those outputs that are rewarded.

The corruption of performance incentive systems stimulated by a too-heavy reliance on quantitative measurement is not peculiar to public education. It has been extensively documented in other fields by economists, business management theorists, sociologists, and historians.

The bulk of the report documents these claims, not only from the public sphere but also from private enterprise.

The thing that gripes me about this is that all these economists and business experts that have been weighing in so heavily on education policy lately either do or should know about all this already, since so much of the research is from economists and business.

I urge everyone to read the whole report, and maybe send copies to Duncan and Obama by special delivery. Maybe if several humdred landed on their doorsteps, they'd actually read it also, and take the results seriously.