Campbell's Law And High Stakes Testing

Joe Bower is a new blogger. He teaches in Canada. You should read his blog, often.  This is his latest, sans links:

High Stake Testing's Kryptonite

The effects of high-stakes testing should not come as a surprise to us. That some very good teachers feel the pressure to cheat for their students in a kind of Robin Hood act to save their children and their school from undue harm should make sense. With the proper pressure, even very good people can be forced into doing 'bad' things.

A well-known (but not well-known enough) social-science law called Campbell's Law helps to explain why high-stakes testing will NEVER work the way it was intended. David Berliner and Sharon Nichols explain Campbell's Law in their book Collateral Damage: How High Stakes Testing Corrupts America's Schools.

Campbell's law stipulates that "the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor. Campbell warned us of the inevitable problems associated with undue eight and emphasis on a single indicator for monitoring complex social phenomena. In effect, he warned us about the high-stakes testing program that is part and parcel of No Child Left Behind.

Campbell's Law should disturb anyone who uses data to make decisions. If the stakeholders responsible for caring through with the day to day doing that the data measures feel like their work is attached to a high stakes indicator, they will work to corrupt the validity and reliability of the measurement.

Berliner and Nichols summarize:

Apparently, you can have (a) higher stakes and less certainty about the validity of assessment or (b) lower stakes and greater certainty about validity. But you are not likely to have both high stakes and high validity. Uncertainty about the meaning of test scores increases as the stakes attached to them become more severe.

The high stakes reward-punishment nature of today's testing regime has contributed to its own demise. Everytime someone places more emphasis on testing, the more likely the results gathered will be comprimised - making the data less valid and any decisions based on that data less reliable.

This is a complicated idea with huge implications for policy makers. We can't afford to ignore this law anymore.

No matter how valid or reliable we think certain data is, if high-stakes reward-punishment consequences are to follow the data, then that data becomes more and more invalid and unreliable.