Why Rewards Don't Work (Neither Does Merit Pay)

...

The Overjustification Effect
...

In 1973, Lepper, Greene and Nisbett met with teachers of a preschool class, the sort that generates a steady output of macaroni art and paper-bag vests. They arranged for the children to have a period of free time in which the tots could choose from a variety of different fun activities. Meanwhile, the psychologists would watch from behind a one-way mirror and take notes. The teachers agreed, and the psychologists watched. To proceed, they needed children with a natural affinity for art. So as the kids played, the scientists searched for the ones who gravitated toward drawing and coloring activities. Once they identified the artists of the group, the scientists watched them during free time and measured their participation and interest in drawing for later comparison.

They then divided the children into three groups. They offered Group A a glittering certificate of awesomeness if the artists drew during the next fun time. They offered Group B nothing, but if the kids in Group B happened to draw they received an unexpected certificate of awesomeness identical to the one received by Group A. The experimenters told Group C nothing ahead of time, and later the scientists didn’t award a prize if those children went for the colored pencils and markers. The scientists then watched to see how the kids performed during a series of playtimes over three days. They awarded the prizes, stopped observations, and waited two weeks. When they returned, the researchers watched as the children faced the same the choice as before the experiment began. Three groups, three experiences, many fun activities – how do you think their feelings changed?

Source: Benedikte on Flikr

Well, Group B and Group C didn’t change at all. They went to the art supplies and created monsters and mountains and houses with curly-cue smoke streams crawling out of rectangular chimneys with just as much joy as they had before they met the psychologists. Group A, though, did not. They were different people now. The children in Group A “spent significantly less time” drawing than did the others, and they “showed a significant decrease in interest in the activity” as compared to before the experiment. Why?

The children in Group A were swept up, overpowered, their joy perverted by the overjustification effect. The story they told themselves wasn’t the same story the other groups were telling. That’s how the effect works.

Self-perception theory says you observe your own behavior and then, after the fact, make up a story to explain it. That story is sometimes close to the truth, and sometimes it is just something nice that makes you feel better about being a person. For instance, researchers at Stanford University once divided students into two groups. One received a small cash payment for turning wooden knobs round and round for an hour. The other group received a generous payment for the same task. After the hour, a researcher asked students in each group to tell the next person after them who was about to perform the same boring task that turning knobs was fun and interesting. After that, everyone filled out a survey in which they were asked to say how they truly felt. The people paid a pittance reported the study was a blast. The people paid well reported it was awful. Subjects in both groups lied to the person after them, but the people paid well had a justification, an extrinsic reward to fall back on. The other group had no safety net, no outside justification, so they invented one inside. To keep from feeling icky, they found solace in an internal justification – they thought, “you know, it really was fun when you think about.” That’s called the insufficient justification effect, the yang to overjustification’s yin. In telling themselves the story, the only difference was the size of the reward and whether or not they felt extrinsically or intrinsically motivated. You are driven at the fundamental level in most everything you choose to do by either intrinsic or extrinsic goals.

Intrinsic motivations come from within. As Daniel Pink explained in his excellent book, Drive, those motivations often include mastery, autonomy, and purpose. There are some things you do just because they fulfill you, or they make you feel like you are becoming better at a task, or that you are a master of your destiny, or that you play a role in the grand scheme of things, or that you are helping society in some way. Intrinsic rewards demonstrate to yourself and others the value of being you. They are blurry and difficult to quantify. Charted on a graph, they form long slopes stretching into infinity. You strive to become an amazing cellist, or you volunteer in the campaign of an inspiring politician, or you build the starship Enterprise in Minecraft.

Extrinsic motivations come from without. They are tangible baubles handed over for tangible deeds. They usually exist outside of you before you begin a task. These sorts of motivations include money, prizes and grades, or in the case of punishment, the promise of losing something you like or gaining something you do not. Extrinsic motivations are easy to quantify, and can be demonstrated in bar graphs or tallied on a calculator. You work a double shift for the overtime pay so you can make rent. You put in the hours to become a doctor hoping your father will finally deliver the praise for which you long. You say no to the cheesecake so you can fit into those pants at the Christmas party. If you can admit to yourself that the reward is the only reason you are doing what you are doing – the situps, the spreadsheet, the speed limit – it is probably extrinsic.

Whether a reward is intrinsic or extrinsic helps determine the setting of your narrative – the marketplace or the heart. As Dan Ariely writes in his book, Predictably Irrational, you tend to unconsciously evaluate your behavior and that of others in terms of social norms or market norms. Helping a friend move for free doesn’t feel the same as helping a friend move for $50. It feels wonderful to slip into the same bed with your date after getting to know them and staying up one night making key lime cupcakes and talking about the differences and similarities between Breaking Bad and The Wire, but if after all of that the other person tosses you a $100 bill and says, “Thanks, that was awesome,” you will feel crushed by the terrible weight of market norms. Payments in terms of social norms are intrinsic, and thus your narrative remains impervious to the overjustification effect. Those sorts of payments come as praise and respect, a feeling of mastery or camaraderie or love. Payments in terms of market norms are extrinsic, and your story becomes vulnerable to overjustification.

Marketplace payments come as something measurable, and in turn they make your motivation measurable when before it was nebulous, up for interpretation and easy to rationalize.

The deal the children struck with the experimenters ruined their love of art during playtime, not because they received a reward. After all, Group B got the same reward and kept their desire to draw. No, it wasn’t the prize but the story they told themselves about why they chose what they chose, why they did what they did. During the experiment, Group C thought, “I just drew this picture because I love to draw!” Group B thought, “I just got rewarded for doing something I love to do!” Group A thought, “I just drew this to win an award!” When all three groups were faced with the same activity, Group A was faced with a metacognition, a question, a burden unknown to the other groups. The scientists in the knob-turning study and the child artists study showed Skinner’s view was too narrow.

Thinking about thinking changes things. Extrinsic rewards can steal your narrative.

...

Why So Few Posts?

I have a new job. Not enough folks have the money to hire me as a consultant, even with my sliding scale of fees, so I took a teaching job. Now that I am working for someone else I have much less time for blogging. It's kind of a bummer, but a steady income is not a bummer.

There does seem to be lots of movement in favor of questioning the current edreform policies and ideas spouted by know-nothings like Rhee, Gates and the rest (yeah, you too Tilson). Rhee's organization, StudentsFirst, is now touting partnerships with teachers because they realize that they need the support of teachers if they are going to get their agenda enacted. What they don't seem to realize is that hiring TFA quitters and former teachers who have chosen to leave kids and become policy gurus does nothing for Rhee and her organization's desire to become loved by actual teachers who have remained inside classrooms despite Rhee's unfounded vitriol against us.

Even Arne Duncan is trying to get on our good side by saying we should double teacher salaries. This sounds nice, but is just more of the same teacher-bashing--this time it's just veiled. There is no money to do such a thing, and even if there was enough money, higher-paid teachers cannot overcome the devastating effects of poverty on our low-scoring students. Hence, offering more money is a veiled claim that poorly paid teachers are slouching on the job. Um, no, we're not. Fuck you.

The movement to opt out of standardized testing is also gaining traction. We have seen some prominent educators publicly claim that they have opted their own kids out of the state test. If my kid's mom is cool with it, we will also opt our son out. The oligarchs don't need or deserve all that data on my kid. They are abusive as it is, and I don't need their abuse aimed directly at my son. Fuck those assholes.

My radio show on BTR is also suffering from my new job, as I can no longer do it in the afternoon. If I am to do more radio I must do it during prime-time, which requires a pro account that I cannot afford, hence the donation widget in my sidebar on your left there. I would love to do more radio, and if you want me to do more, I need your financial help to do it. Consider a donation, won't you? I am in contact with a few interesting education leaders who are interested in interviews, and I think they would be a valuable addition to the counter-reform movement that is gaining steam. Anyone who donates is invited to be a guest on the show!

Remember, poverty is the reason for the achievement gap. We've known it for 60 years, and we've been in denial for that long. The reformers have done a good job defeating this idea by saying poverty is not an excuse. They are right. Poverty is not an excuse, it's a goddamned diagnosis.

Why Incentives Don't Always Work, Like Merit Pay, For Example

h/t Jersey Jazz Man

Merit Pay For Teachers Doesn't Work

The most rigorous study of performance-based teacher compensation ever conducted in the United States shows that a nationally watched bonus-pay system had no overall impact on student achievement—results released today that are certain to set off a firestorm of debate.

Steven Sawchuck

Sunday Cartoon Fun: Accountability Edition

Merit Pay: It Won't Work II

Merit Pay: It Won't Work I is here.

Retro reform idea - Merit Pay
Thursday, May 28, 2009

Now and then I like to post about writers who have contributed to our knowledge about progressive education. It would be nice to talk about new ideas, but if we’re going to discuss old ideas, we should at least know what’s already been said so we can stop repeating ourselves and either move the discussion forward or change the subject.

EdSec Duncan, for example, has a big pile of money he wants to use to “incent” and reward excellence “based on student achievement” because he believes that a quality education for every student is a civil right. That’s a nice idea, but we need to agree on some key details before we can expect to see much progress there. Prof. Daniel Willingham posted a video on You Tube, offering six reasons why merit pay will not work. Three reasons are about why test scores won’t give us valid information about teacher effectiveness, and the other three are about social factors that make some classes more challenging than others.

James Herndon covered this topic in his own special way 25 years ago in Notes From a Schoolteacher:

The idea that if you’re paid more you’ll work harder may apply to selling encyclopedias. If you’re a lion-tamer, you’re not going to work any harder just because you’ll be paid more. The job of a teacher is more like a lion-tamer, I think.
-Al Shanker, President
AFT, AFL-CIO

I’ve tried hard to find something to say, pro or con, about merit pay - something that has not already been said hundreds of times. Shanker’s remark, above, is one point of view. You must work hard, as a schoolteacher, simply in order to avoid being eaten alive. Subduing the lion’s natural appetite comes first - after that is assured, maybe you’ll be able to teach him a trick or two.

Merit pay has been around a long time in the corporate / industrial world, but even there no one seems satisfied with it. No research can be found which agrees that the salesman works harder or is more successful at his trade if he is given extra pay for “merit.”

It is, anyway, quite beside the point whether one works hard or not. Success is the point. But even there, sales managers report that no one is satisfied if the person who demonstrably sells the most of whatever product it is, is paid more. The other salesmen argue that they had bad territories, mix-ups in their deliveries, no cooperation from the front office, storms - otherwise they would have been right up there.

Teachers, like salesmen, all believe that they are among the very best at their job. You simply must believe that in order to continue teaching (and probably selling).

You begin to teach as a lion-tamer, to be sure and, if not eaten up, go on to ask other teachers what they do here and there, what “works” for them, and quite soon, by some curious amalgam, you develop a way to work in the classroom which suits you and which you think is best … best, considering the various and vast distances between what you must do, want to do, and can do.

You think it best, for you and the students, or for the students and you.

I certainly think that my “style” or “strategy” in the classroom is the best. That’s why I do it that way. I also know that my opinion is not shared by the other teachers at Spanish Main, each of whom, quite rightly, prefers his own.

The whole idea of merit pay, then, seems to founder at this point. If we all think that we are among the best, how are we to reward the best?

If we must decide who is the best, then who is to decide, and on what basis?

Herndon doesn’t say anything about test scores, presumably because nobody had the genius idea of using them to compare teachers. Instead, he tells us that the “plans suggest a committee” of roving teachers who would visit schools and rate them based on their observations. Herndon wonders about the inferences these people would draw if they paid him a surprise visit.

The visiting team, concluding that this teacher is not teaching at all, let alone well, is not dedicated, doesn’t give a damn, certainly deserves no merit pay (if he deserves to be paid at all!) - the team has just missed out on one of the best teachers in the world! They are unaware of it.

Too late, then, for my thoughtful discourse on what teaching is, how students learn, etc.!

Has something been left out in this discussion? I want to cover everything about this now; I never want to return to it.

Well, the basis is left out. The standard, criterion, measure, rule of thumb … anything, any way by which to tell the great teachers from the simply OK teachers. The standard, etc., by which to tell the wonderful teaching strategies from the mediocre ones.

Are the great teachers more entertaining? Have they better intellectual command of their subjects? Have they greater rapport with the students? Are they more efficient, provide more time on task? Are they more aware of their students’ ethnic backgrounds, social class, personal or family problems? All of the above? Well, some of the above?

No one knows.

Does anyone know whether students actually learn more from great teachers, if you could ever find out who were the great teachers?

No one knows that either. The sentence just above sounds insane (p. 85).

It would be so much more interesting to talk about that.

h/t Doug Noon

Merit Pay: It Won't Work

Diane Ravitch on merit pay:

What's Wrong With Merit Pay

Dear Deborah,

Over time we have developed a very solid and smart community of readers who like to argue with us and with each other. That is as it should be. And of course we need to bridge differences—or disagree—with them, too, as we do with each other.

So the subject today is merit pay. This is an important topic because it has become clear that President Obama has decided to hang his hat on this idea. It has not yet been explained just what he means by merit pay. Does he mean that teachers should be paid more for teaching in what is euphemistically called “hard-to-staff” schools? Or paid more for teaching in areas where there are shortages, like certain kinds of special education or subjects such as math and science? Or paid more for mentoring other teachers? Or paid more for teaching longer days?

I would call such compensation “performance pay,” rather than “merit pay,” because teachers are paid more for doing more.

But I have a feeling that what the Obama administration has in mind is paying teachers more based on their students’ “value-added” test scores. So if their students see increases in their scores, they will get “merit pay” to reward their supposedly superior teaching.

I believe that this is the direction the administration is heading and that it is the purpose of the millions that will be spent on data warehouses in every state. And it is why Secretary Duncan has told the governors that they will get their stimulus money only if they collect and report data to the U.S. This was an odd request because some of the data he asked for is already available, such as the gap between state and NAEP scores (previously published in Education Week, for example, and no secret).

There are several reasons why it is a bad idea to pay teachers extra for raising student test scores:

• First, it will create an incentive for teachers to teach only what is on the tests of reading and math. This will narrow the curriculum to only the subjects tested.
• Second, it will encourage not only teaching to the test, but gaming the system (by such mechanisms as excluding low-performing students) and outright cheating.
• Third, it ignores a wealth of studies that show that student test scores are subject to statistical errors, measurement errors, and random errors, and that the “noise” in these scores is multiplied when used to make high-stakes personnel decisions.
• Fourth, it ignores the fact that most teachers in a school are not eligible for “merit” bonuses, only those who teach reading and math and only those for whom scores can be obtained in a previous year.
• It ignores the fact that many factors play a role in student test scores, including student ability, student motivation, family support (or lack thereof), the weather, distractions on testing day, etc.
• It ignores the fact that tests must be given at the beginning and the end of the year, not mid-year as is now the practice in many states. Otherwise, which teacher gets "credit," and a bonus for score gains, the one who taught the student in the spring of the previous year or the one who taught her in the fall?

I believe that our readers are right when they predict that merit pay of the stupidest kind is coming. I predict that it will do nothing to improve our schools. A few weeks ago, the conservative Manhattan Institute released a study showing that merit pay had no impact on test scores in 200 schools in New York City that are trying it. In fact, scores went down in larger schools that offered bonuses. This little experiment in schoolwide bonuses is costing taxpayers $20 million a year.

Now it is possible that scores may go up in later years; this is only the first year, after all. But what is most interesting is the subdued release of this study. When the Manhattan Institute releases a study, it often holds a press conference to announce the results. This study, however, had no fanfare; its study was quietly posted on MI's Web site; no press conference, no press release. Somehow I suspect that the study would have been released with bells and whistles if the scores had flown upward.

Here is my prediction: Merit pay of the kind I have described will not make education better, even if scores go up next year or the year after. Instead, it will make education worse, not only because some of the "gains" will be based on cheating and gaming the system, but because they will be obtained by scanting attention to history, geography, civics, the arts, science, literature, foreign languages, and all the other studies that are needed to develop smarter individuals, better citizens, and people who are prepared for the knowledge-based economy of the 21st Century. Nor will it identify better teachers; instead, it will reward those who use their time for low-level test preparation.

Is it possible to have an education system that mis-educates students while raising their test scores? Yes, I think it is. We may soon prove it.

Diane

Merit Pay: It Doesn't Work!

I've said it before!

Following the wrong example

Houston teacher Laura Taylor examines proposals for merit pay for teachers in light of the experience in her school district.

March 24, 2009

IN HIS recent speech, President Barack Obama spoke about the importance of education in our country and declared that under his administration, "good teachers will be rewarded with more money for student achievement."

Obama gave two examples of school districts already doing this, one being the district that I teach in, Houston Independent School District (HISD). The largest school district in Texas, HISD has been at the forefront of using standardized test scores to determine bonuses.

Given this, you might expect merit pay and standardized testing to be universally effective and accepted within HISD. In reality, they are anything but.

In this economy, it's difficult for any person to turn down extra money. Yet a survey done by HISD right after bonuses were awarded in January found that only 45 percent of teachers and other school employees liked the system.

Every year, it seems, the HISD administration rolls out yet another version of ASPIRE, the program that determines these bonuses. And every year, confusion and frustration reigns among teachers.

Though the district pours in money for professional development to explain the program, the merit pay "awards" still feel arbitrary to many teachers. As Houston Federation for Teachers President Gayle Fallon told the Houston Chronicle, "They're still comparing it to winning the lottery."

Part of what makes the system so controversial is the tremendous amount of bonus money that goes to principals and administrators.

For the most recent round of bonuses, School Superintendant Abelardo Saavedra gave himself the largest, paying himself $77,500 out of a possible $80,000, in addition to his annual salary of $327,010. The next highest paid were executive principals, many earning bonuses of well over $10,000. Teachers who did earn a bonus got nowhere near that amount. And more than 2,100 eligible employees earned nothing.

- - - - - - - - - - - - - - - -

EVEN IF the merit pay distribution appeared to be fair to those involved, it would still be a failure.

On the surface, it might appear like a good idea to reward teachers--who are underappreciated and underpaid--with more money. However, this is not the role merit pay plays in the school system. Merit pay is another way to divide teachers, fostering competition between teachers instead of collaboration.

It also goes against what I have experienced as a teacher. Teachers do their best not because they could earn more money if they do, but because they care about their students. Barbara Falcon, a high school teacher who opted out of the system this year, agrees. "I am against ASPIRE because, as an insider, I clearly see that you cannot improve the quality of teachers or teaching unless teacher salaries in total are raised," Falcon told the Houston Chronicle.

On top of the problems inherent within merit pay is the way in which money is divvied up. Merit pay schemes reward not good teachers, but those who are most effective at test preparation.

When merit pay is involved, student achievement becomes simply student scores on standardized tests. Any teacher can tell you that standardized tests are not the best measure of student achievement. Not only are they culturally biased, but they typically represent the lowest levels of thinking, via multiple-choice answers, and leave no room for creative and critical thinking.

Relying on standardized testing removes the trust we have in our teachers to determine student achievement. What is a better measure of student learning--an assessment by the teacher who spends every day with a student or high-stakes, fill-in-the-bubble standardized tests?

Beyond this, because standardized testing has come to determine everything from bonuses to student promotion, teachers are increasingly pressured to "teach to the test." Learning is focused on figuring out the right answer in the mind of test creators, rather than the most creative answer.

In many schools, entire lessons are devoted to teaching young children how to appropriately fill in the bubbles for these standardized tests. Principals and superintendents, who are looking forward to big merit pay bonuses, push teachers to focus on this test preparation rather than genuine learning.

The use of merit pay in HISD is not an example for others to follow, but rather a cautionary tale. Those who seek to improve our public school--teachers, parents, students and community members alike--should oppose merit pay in favor of paying all our teachers a fair wage and fully funding teaching, not test preparation.

The Howler On Gladwell

The Howler is concerned that Education Journalism is populated by people who have never been in a classroom. Funny, seems just about nobody who enjoys the "reformer" label has taught in public school.

Special report: Schools daze!

Part 1—Gladwell, unblinking: Who will Obama pick to be Secretary of Education? Some slightly-odd writing has surfaced of late as big mainstream news orgs ponder this question. The writers often have little background in education issues—and their lack of experience often shows. One other attribute tends to show up: The way these mainstream scribes sometimes seem to be in thrall to “conservative” educational notions.

There’s nothing automatically “wrong” with conservative educational ideas, of course. But something is a little bit wrong with uninformed public ed writing.

For starters, consider this piece by Malcolm Gladwell in last week’s New Yorker. Gladwell ponders a worthwhile question: How could school districts improve their performance in deciding which new teachers to hire? According to Gladwell, it’s hard to review a college graduate’s resume and determine if he or she will become a good teacher. How might school districts do a better job picking applicants who turn out to be top-notch teachers?

As he starts, Gladwell compares this to a problem from the world ofsports: Football scouts have a hard time knowing which collegequarterbacks will succeed at the NFL level. How might school districtsaddress their version of this problem? This is the perfectly sensiblequestion Gladwell attempts to address.

Gladwell discusses a serious issue—but does he have the chops to do so? He starts with ruminations about quarterbacks—but what follows is his very first paragraph about public education. And his reasoning here strikes us as odd. Frankly, it makes us wonder if he might be somewhat over his head discussing public school issues:

GLADWELL (12/15/08): One of the most important tools in contemporary educational research is “value added” analysis. It uses standardized test scores to look at how much the academicperformance of students in a given teacher’s classroom changesbetween the beginning and the end of the school year. Suppose that Mrs.Brown and Mr. Smith both teach a classroom of third graders who score at the fiftieth percentile on math and reading tests on the first day of school, in September. When the students are retested, in June, Mrs.Brown’s class scores at the seventieth percentile, while Mr. Smith’s students have fallen to the fortieth percentile. That change in the students’ rankings, value-added theory says, is a meaningful indicator of how much more effective Mrs. Brown is as a teacher than Mr. Smith.

Let’s see if we have fully grasped the reasoning found in that passage:

According to Gladwell, two classes were even at the start of the year—but by the end of the school year, one of the classes was doing much better. Our question: Why would it take “one of the most important tools in contemporary educational research” to deduce that this group’s teacher had been “more effective as a teacher?” Why would we need an “important tool in educational research”—a “theory,” no less—to draw such an obvious conclusion? Has any principal ever lived who wouldn't have reached this obvious judgment? The conclusion here is comically obvious. But it’s buried beneathsome ponderous talk about “contemporary research” and“important research tools.”

But then, we’re often struck by writing like that when mainstream journalists proclaim about public schools. In fairness, we might say that Gladwell has merely constructed an exceptionally simple example to illustrate some larger point—though we’re not sure what that point might be. Who wouldn’t “use standardized test scores to look at how much the academic performance of students in a given teacher’s classroom changes between the beginning and the end of the school year?” We started teaching fifth grade in Baltimore in 1969. And sure enough! Not being the dumbest humans on earth, everyone in our low-income school was doing this, even back then.

So that opening paragraph made us wonder a bit about Gladwell’s competence in this area. But we were also struck by his third paragraph about the schools. Our view? In this passage, the gentleman’s lack of background really does seem to show through:

GLADWELL: Eric Hanushek, an economist at Stanford, estimates that the students of a very bad teacher will learn, on average, half a year’s worth of material in one school year. The students in the class of a very good teacher will learn a year and a half’s worth of material. That difference amounts to a year’s worth of learning in a single year. Teacher effects dwarf school effects: your child is actually better off in a “bad” school with an excellent teacher than in an excellent school with a bad teacher. Teacher effects are also much stronger than class-size effects. You’d have to cut the average class almost in half to get the same boost that you’d get if you switched from an average teacher to a teacher in the eighty-fifth percentile. And remember that a good teacher costs as much as an average one, whereas halving class size would require that you build twice as many classrooms and hire twice as many teachers.

In that passage, Gladwell starts considering a serious policy question: Should money be used to reduce class size? Or would such money be better spent attracting more capable teachers? On that question, we have no view. But we’re not real sure that Gladwell’s the man to help us sort it out.

In the sentences we have highlighted, Gladwell claims to be paraphrasing Hanushek, who is actually a senior fellow at the conservative Hoover Institution, not a lowly faculty member at miserable Stanford itself. There’s nothing “wrong” with working at Hoover, of course, and Hanushek’s research and views are surely well worth considering. (In recent weeks, he’s certainly had a lot of success getting mainstream scribes to recite them!) But does that slightly puzzling, highlighted passage really reflect something Hanushek said? “The students of a very bad teacher will learn, on average, half a year’s worth of material in one school year?” And: “The students in the class of a very good teacher will learn a year and a half’s worth of material?” We’ll take a guess: That might mean that an average student (a kid near the fiftieth percentile in reading or math) will typically learn that much in those situations—although the statement means almost nothing until we’re told how many teachers qualify as “very good” and “very bad.” Did Hanushek really say something like this: On average, students will learn three times as much from a very good teacher? We have no clue, but Gladwell’s presentation is mired in the murk and the gloam.

In short, that presentation—by Gladwell, not Hanushek—is thoroughly lacking in clarity. Does this reflect a lack of chops on Gladwell’s part when it comes to educational issues? We have no

way of judging that just from this piece. But in the mainstream upper-end press corps, journalists often orate at length about public schools—even though they seem to have no background in the area at all. And uh-oh! Such people may be inclined to believe whatever dang-fool thing they get told.

As we’ve said, Gladwell ponders a worthwhile question—and that may be the problem. He ends up making a somewhat eccentric suggestion about teacher recruitment—a suggestion he seems to source to no one but himself. And the validity of his suggestion turns, almost completely, on his unblinking acceptance of (paraphrased) claims about the fruits of Hanushek’s research. This is the passage where Gladwell’s rubber really starts hitting the road:

GLADWELL: Hanushek recently did a back-of-the-envelope calculation about what even a rudimentary focus on teacher quality could mean for the United States. If you rank the countries of the world in terms of the academic performance of their schoolchildren, the U.S. is just below average, half a standard deviation below a clump of relatively high-performing countries like Canada and Belgium. According to Hanushek, the U.S. could close that gap simply by replacing the bottom six per cent to ten per cent of public-school teachers with teachers of average quality. After years of worrying about issues like school funding levels, class size, and curriculum design, many reformers have come to the conclusion that nothing matters more than finding people with the potential to be great teachers.

Could the U.S. really produce some sort of major change “simply by replacing the bottom six per cent to ten per cent of public-school teachers with teachers of average quality?” We don’t havethe slightest idea, although we’re hugely dubious. But we note that Gladwell’s claim is based on a paraphrased account of something Hanushek supposedly said—a claim Hanushek supposedly based on “a back-of-the-envelope calculation.” Once again, we’re forced to rely on Gladwell’s basic chops in such matters.

Does Gladwell know what he’s talking about? Does he have a suitable background for such ruminations? We’re not sure—but in the world of mainstream journalism, reporters and editorialists often expound on educational matters, often without showing the slightest sign of anything like expertise. And oh yes: In the current climate surrounding the schools, they will often be found recommending “conservative” views—and showing that the word “reform” now extends to conservatives only.

When it comes to public education, there’s absolutely nothing “wrong” with “conservative” ideas and perspectives. But in the world of the mainstream press, many things are often wrong with the way these ideas get reviewed. In recent weeks, a bit of a tipping point has been reached in the way this familiar old game is played. Did Gladwell know whereof he spoke? We’re not sure—but then again, how about Time’s Amanda Ripley?

Another "No" To Merit Pay

OPEN LETTER TO PRESIDENT-ELECT OBAMA

Dear President-Elect Obama:

Congratulations on the overwhelming mandate issued by the American voters on November 4. In the months and years ahead, I look forward to your leadership and vision, and those of your Vice President, Joe Biden.

Over the past several months, issues related to teacher attraction, retention, and compensation have been discussed by the candidates and campaign spokespeople. In addressing these issues, Senator McCain espoused the concept of providing bonuses for teachers based on the standardized test score gains of their students. Your campaign platform recognized the need to attract more teachers into the profession and proposed to pay them more once they arrive. The respective campaigns touched on ideas and initiatives that are central to the growing national debate over these complex and controversial issues.

As you and Vice President Biden formulate your agenda in this arena, please consider the following perspectives. The ideas and assertions that formulate these beliefs are the product of more than a decade researching, analyzing, and bargaining teacher compensation. These suggestions are not intended to promote any political agenda or ideology. They are intended to advance the profession of teaching and learning in America. Among the things recent research has indicated, effective teaching is the single most important school-related factor in determining student success.

Much has been said and written about the need to reward the Nation’s “best” teachers. While such an idea may seem logical, understandable, and worthy, it is but a simplistic attempt to address very complex issues. Even if we were to set aside the profound difficulties in defining the “best” teachers, such an approach will not provide the type of systemic changes needed to effect positive advances in the teaching profession.

The effectiveness of a population of teachers, like that all other workers, tends to fall along a continuum or “bell curve.” A handful will be underachievers, a handful exemplary performers, and the vast majority will be found clustered in the middle. Those clustered in the middle are hard-working teachers doing what they can to deliver high quality instruction to students.

Most school districts and unions have negotiated systems to address the underachievers. If attempts to help them improve fail, they are escorted out of the profession.

The exemplary performers will remain such with or without the promise of a bonus. Instead of offering bonuses, the money is better spent building systems that put these highly effective educators in positions of teacher leadership.

For systemic change to occur, however, we must build systems that provide the majority of teachers clustered in the middle the ability and opportunity to further advance and improve the effectiveness of their teaching. The enticement of a bonus, limited and fleeting in nature, will not give them the additional skills and knowledge they need to become even more effective teachers.

With the above serving as the foundation, I respectfully request that you consider the following five point plan:

1. $40,000 Minimum Starting Salary
As proposed by Governor Richardson and others, we must provide economic incentives to encourage the best and brightest to consider teaching as a viable professional option.

2. Teacher Residency
In cooperation with institutes of higher learning, school districts and their teacher associations would be encouraged to implement teacher residency programs, similar to that used for training physicians, for first year teachers. These programs would assign a full-time first year teacher with less than a full-time student caseload. Such an approach would permit the new teacher time and opportunities to reflect on their practices, model and observe exemplary teaching from more experienced colleagues, and work closely with a mentor. In fact, Title II of the recently enacted Higher Education Opportunity Act authorizes a teacher residency program. One quick way to demonstrate your support for this program would be to include $300 million for Title II, it’s new authorized level, as part of the Fiscal Year 2009 education appropriations bill as well as in your proposed Fiscal Year 2010 budget.

3. Teacher Leadership
It is not enough to build systems designed to attract new and talented people into the profession; we must devise systems to retain them once they arrive. Studies have indicated that up to 50% of new teachers leave the profession within the first five years, at an estimated cost of nearly $3 billion annually. Leadership opportunities would permit alternatives for highly qualified teachers. Currently, educators either remain in the classroom, move into administrative roles, or leave
the business altogether. Teacher leadership would provide economic incentives and alternatives to those not wishing to move into administration, while providing extremely valuable contributions to the school district’s mission. Examples of leadership roles would likely include:

□ Mentoring
□ Peer Coaching and Assistance
□ Curriculum Development
□ Content Specialists
□ Research Coordination

4. Action Research
Adding to the research-based body of knowledge about effective teaching practices is an imperative component of effective school reform. Individual teachers and groups of teachers would be provided economic incentives to conduct action research in their classrooms. Such research would help to discover and promote the most effective teaching practices. The findings of this research would be warehoused in an electronic database available to teachers across the
country. The outcomes of such a database, for example, would permit a teacher in Maine to learn from the research conducted by a teacher in California.

5. Professional Growth Tied to Classroom Objectives and District Mission
There are scores of school districts and their teacher unions that have developed exemplary professional development programs. Many of these are manifested in intra-district learning communities, where teachers gain the skills and knowledge they need to effectively teach 21st Century skills to an increasingly diverse student population. These models can and must be successfully adapted and implemented across the country.

Upon implementation of the above-prescribed initiatives, comprehensive assessment measures must follow. To adequately and effectively measure the impact of these initiatives, measures must include:

□ Classroom and school-based measures of student growth in those competencies
identified by the Partnership for 21st Century Skills.
□ Graduation rates
□ Student satisfaction
□ Rates of teacher attraction
□ Rates of teacher attrition
□ Participation rates in action research
□ Participation rates in teacher leadership
□ Participation rates in professional development
□ School climate and culture

Comprehensive and effective reform of the Nation’s teacher compensation systems is a highly complex and controversial issue. Simple prescriptions will not effect the type of systemic change needed to successfully achieve positive school reform.

I encourage you and your administration to avoid the temptations of politically-charged quick fixes. Please consider the above perspectives. I am convinced they will result in the type of positive evolution of the teaching profession and resultant improved education results that we all want.

Thank you for your time and consideration. And again, congratulations.

Sincerely,

Jim Carlson
President
November 10, 2008

Merit Pay For Teachers? Not Going To Work!

Merit pay for teachers is not a fix:

"In the 1980s, school districts dabbled with programs that offered teachers cash inducements, such as bonuses or raises, for doing their jobs well.

But those merit-pay programs were mostly short-lived, hotly debated, and understudied. Even after all this time, no one knows definitively whether children learn more when teachers are paid extra for boosting their students' achievement."

Go read the entire piece.

Merit Pay For Teachers? Don't Go There...

Michelle Rhee is the Devil.

Anyway, merit pay for teachers is a bad idea for so many reasons; teacher evaluation and assessment come to mind. Ezra Klein has a post with this quote:

My point is this. It seems like reformers keep wanting to jump in and start doing things, because education is so stagnant. Which I understand. But the place to start isn't merit pay, it is finding methods of assessing teachers that aren't totally fucking useless and irrelevant. I think if we had such assessment, people would find teachers much more amenable to merit pay and a lack of tenure.

Go check it out. The folks like Klein who think merit pay is a good idea get smacked pretty hard in the comments (yours truly could only cheer!).

Update: Here is the NYT article referenced in Klein's piece.

New Teacher: Find Other Work!

I am really frustrated. Everyday Math, Lucy Calkins, Words Their Way Spelling Inventory, and the rest are a waste of time and money, and they create frustration at the expense of educating the kids.

Here's what I mean. We teachers have to use these materials whether they are useful to us or not. We have to use them because of NCLB; accountability is now more important than education, joy, love, life, community and family.

Schools must show improvement, and new materials are almost never always the way to improve things. Indeed, we adopted Everyday Math because apparently we had to adopt something; it was time! Seriously! Not because EDM is better. Not because Scott Forseman was worse. It was just because it was adoption time.

I have mentioned my scores many times because they are at the core of my argument against forcing teachers to use certain materials. Most of the curricular materials a school uses have been produced for a huge market; many school districts nation-wide adopt identical materials. This nationalization of materials makes for watered down materials. They can't be rich and specific because some things may not go over well in certain places. So, we get lean materials, especially in history, social studies and science. Math, less so.

With math, because we all know if we do not lead the world in math and science we will not continue to lead the world, we adopt new materials--that are research based--hoping knowing they will improve scores. There is only one problem with this: Everyday Math was rejected by many school districts because of its spiraling sequence and overly complicated teacher guide, the plethora of silly materials that are embedded in the instruction making for less than rigorous lessons, and all the games of "math self-discovery". It is a bad program, and one that I do not need; look at my scores!

So, my high scores may go down if I have to implement Everyday Math. If my scores go down, and my candidate of choice is elected, I will not get my merit pay. However, if I refuse to use EDM, my scores will remain high, and I will get merit pay, unless I get fired for not using the curriculum provided, regardless of its lack of efficacy.

And don't get me started on Lucy Caulkins. We were given the writing assessment materials today. We were supposed to be teaching "How to" (the real words are "expository text") because that is what the first writing assessment will be on. Well, we have not been teaching what the assessment will be assessing because the assessment has a requirement in the rubric that is not a part of "How To" writing, except sometimes. Confused? I think the kids will be too.

This is the kind of negligence administration constantly foists on us, and then we look like the idiots. Teachers have complained about the lack of information this assessment provides for years. Each year our literacy leader (oxymoron) says they will be fixing it. They haven't, yet. It's only been my whole career, so, maybe they will get to it. Things take time, right?

Well, I cannot do it anymore. I sit in those damned meetings, make valid and important points by exposing the silliness, lies, or whatever else is being obfuscated by the principal and administration, then get a letter of reprimand in my personnel file for doing so.

I should be congratulated for my students' scores. I should be asked how I do it! But no, I am being told to shut up, regardless of the substance of my remarks (principal actually said that. My points are valid and substantive, but shut up). Shut up, sit down, be quiet, play dumb, and shut up again.

Well, fuck you. Got it!? Fuck. You.

I'm looking for other work. If you are a new teacher, or thinking of getting into teaching, you might want to think again. The future is not bright for teachers. Just like the taxpayers taking final responsibility for the greedy bastards, teachers are going to continue to take responsibility for the outcomes of kids they have no control over. If it seems unfair, and a little stoopid, well, you might be on to something.

Update: I forgot to mention that I cannot quit mid-year without being in danger of having my credential revoked. Teaching has changed, for the worse...

Can I Get Paid For My Students' Great Scores? Yes!

The scores of my students cannot be used in any evaluation of my performance (look at my scores at the top of the sidebar). Well, this my not be the case for long. It's fine for me, but not for oher teachers who are stuck with the lowest performing kids. Read the NYT story after expansion (though linkless)...

October 2, 2008
Teachers to Be Measured Based on Students’ Standardized Test Scores
By JENNIFER MEDINA

New York City is beginning to measure the performance of thousands of elementary and middle school teachers based on how much their students improve on annual state math and reading tests.

To avoid a contentious fight with the teachers’ union, the New York City Department of Education has agreed not to make public the reports — which described teachers as average, below average or above average with various types of students — nor let them influence formal job evaluations, pay and promotions.

Rather, according to a memo to principals from Chancellor Joel I. Klein and Randi Weingarten, the president of the United Federation of Teachers, sent on Wednesday night, the reports are designed to be guides for the teachers themselves to better understand their achievements and shortcomings.

“They won’t be used in tenure determinations or the annual rating process,” the memo said. “Many of you have told us how useful it would be to better understand how your efforts are influencing student progress.”

Still, even without formal consequences for teachers, the plan is likely to anger teachers and parents who are already critical of the increasing emphasis on standardized test scores as a substitute for judging school quality. It follows the city’s much-debated issuance of report cards labeling individual schools A through F largely on the basis of student improvement on state exams.

The State Legislature this spring prohibited the use of student test scores in teacher tenure decisions. The new measurement system — called “teacher data reports” — is an expansion of a pilot program that the city began in January involving about 2,500 teachers at 140 schools. The pilot program was so controversial that several participating principals did not tell teachers they were being monitored.

Christopher Cerf, the deputy chancellor overseeing the program, said it was important to get teachers “comfortable with the data, in a positive, affirming way.”

“The information in here is a really, really important way to foster change and improvement,” he said. “We don’t want people to be threatened by this.”

In introducing the pilot program, Mr. Cerf said it would be a “powerful step forward” to have the teacher measurements made public, arguing, “If you know as a parent what’s the deal, I think that whole aspect will change behavior.” But this week, he said that for now the reports will be treated as personnel records not subject to public-records laws.

Principals interviewing prospective teachers from other schools would be permitted to ask candidates for their reports, but the candidates would not have to provide them.

Ms. Weingarten said that the assurance that there would not be a public airing of individual teachers’ information made her more comfortable with the idea of the reports, which she said could help teachers identify their strengths and weaknesses.

“This can be used to inform instruction and advance it,” she said in an interview. “If this is something that becomes a ranking facility, opinions will be very, very different. That door has now been closed.”

Still, Ms. Weingarten said the reports answer only “a very narrow question” of how a particular teacher’s students do on tests. She and others have long argued that there are many other criteria on which teachers should be evaluated.

The new reports are part of a broader bid by the city to improve the ways teachers are recruited, trained and measured. Last year, the Education Department began a push to get rid of subpar teachers before they earned tenure, forming a team of lawyers and consultants to help principals amass enough information to oust those who are deemed deficient and do not show signs of improvement.

There have been similar efforts across the country, as politicians and academic experts say that teachers are the most important element in improving student performance and closing the gap in achievement between white and minority students. School systems in Texas and Tennessee, for example, have used student performance and improvement as a tool to evaluate teachers.

New York City plans to generate reports for roughly 18,000 teachers — every math and English teacher in fourth through eighth grades.

Amy McIntosh, the Education Department’s chief talent officer, who helped develop the system, said that her team would continue to explore ways to monitor the effectiveness of the city’s nearly 60,000 other public school teachers, but that for now the state tests were the only data on which to reliably base evaluations of them.

The teacher data report balances the progress students make on state tests and their absences with factors that include whether they receive special-education services or qualify for free lunch, as well as the size, race and gender breakdown of the teacher’s class.

Using a complicated statistical formula, the report computes a “predicted gain” for each teacher’s class, then compares it to the students’ actual improvements on the test. The result is a snapshot analysis of how much the teacher contributed to student growth.

The reports classify each teacher as average, above average or below average in effectiveness with different categories of students, like those who score in the top third or the lowest third on the test, and those still learning English or enrolled in special-education programs. It also contains separate measurements on effectiveness in teaching boys and girls, though it does not distinguish performance by students’ race or income level. Teachers will also be given a percentile ranking indicating how their performance compares to those who teach similar students and to a citywide pool.

“When we have talked to teachers about this, there is real insight about the students,” Ms. McIntosh said. “They will say, ‘I didn’t realize I was teaching to the bottom,’ or, ‘I am really great with boys, and less so with girls.’ ”

Last year’s pilot program also attempted to measure how well a principal’s perception of teachers aligned with the student test score data. According to the Education Department, about 69 percent of the teachers whom principals rated “exceptional” were in the top half on the reports. And 73 percent of those whom principals called “fair, poor or very poor” were in the bottom half.

Frank Cimino, the principal of Public School 193 in Brooklyn, which participated in the pilot program, said he was still uncertain about how useful the reports were.

“I would like to make a comparison to see what it shows this year to what it showed last year,” he said. “I don’t think anything can replace getting into the classroom.”

Prove Your Premise!!

I read a snippet of a story over at Schools Matter that just bolsters the correct notion that NCLB is a policy with a head-up-its-ass problem. Here is the mythbusting story from NewsReview, Reno!

Policy myths cause a lot of government’s problems, and education is especially damaged
Can a school system built on imaginary premises possibly succeed?

By Dennis Myers

Richard Rothstein spent three years as education columnist for the New York Times, giving a popular audience an unaccustomed look at the work of a scholar in sharp, to-the-point essays that challenged conventional wisdom and corrected public policy assumptions. Reading them and his current work as an associate at the Economic Policy Institute in Washington, D.C., one gets the impression of a very patient man. Although he must deal constantly with public policy myths—things we all know to be “true” but are not—he keeps plugging away trying to dispel them."All I can do is keep on telling the truth as I see it, and others have to do the same,” Rothstein says. “In the long run, I have to hope the truth wins out.”

The number of myths on which so many public policies are built raise serious questions about whether those policies have any hope of succeeding. In education, there are a number of myths that have been repeated so incessantly by press and politicians that they have become “true” in the public’s mind. Samples:

• Business uses performance pay, so schools should do the same

• Schools are violent

• Parents and students are fleeing public for private schools

• Schools should use the kind of numerical goals that business uses

• Charter schools outperform public schools

In fact, business generally avoids performance pay, schools are the safest places children frequent, private school enrollment is declining, business recommends against numerical goals, and public schools generally perform better than charter schools.

Take just one of them—numerical goals, which were actually written into the No Child Left Behind Act in the belief that they are an accepted business practice. In fact, it reflects a practice that has long since passed out of fashion in the business world, which found that it focused workers on process instead of outcome and generated fear in the workplace, low productivity, and customer alienation.

Using modern research methods, the business community discovered that when a worker must have numbers to show, she or he will crank them out by some means, at the expense of careful workmanship and productivity. Legendary statistician W. Edwards Deming, who in the postwar years gave Japanese management the methods of design, product quality and sales that brought that nation to commercial dominance and later swept the United States, wrote in his book Out Of the Crisis, “A numerical goal leads to distortion and faking, especially when the system is not capable of meeting the goal. Anybody will meet the quota (goal) allotted to him. He is not responsible for the losses so generated. Sears Roebuck waded into trouble in 1992 by allotting goals to their Auto Service Centers. Agents tried to meet the goals set for them. They did, to the detriment of the customer and of the reputation of the company.”

Insurance consultant John Pryor tells the companies he advises, “Focus first on the underwriting or claims or audit processes—and the quality of their delivery from the customer’s perspective—and then it can be determined if productivity is optimum.”

Yet numerical goals—what Rothstein calls “goals distortion” are a basic part of U.S. education policy.

Our peaceful schools
In 1998 during the spate of heavily publicized school shootings around the nation, the Center on Juvenile and Criminal Justice checked some figures on school violence against figures provided by the National Climatic Data Center. “To give the reader a sense of the idiosyncratic nature of these [school violence] events,” the CJCJ reported, “the number of children killed by gun violence in schools is about half the number of Americans killed annually by lightning strikes.”

Nothing has changed since then. There were more deaths on school grounds 32 years ago, when there were 80 million fewer people in the United States than there are today—usually less than 50 annually. Of home, the 7-Eleven, the park, school is their safest environ—far safer than the home, where as many children are killed in family violence every three days as died at Columbine. But the myth lives on.

Rothstein has been a one-person myth-buster in the education field by doing what reporters are supposed to do—checking the facts and statistics before reporting a “trend” (which may be the most abused word in journalism).

When in the early years of the Bush administration members of Congress were arguing that the private sector uses performance pay, so schools should do likewise, Rothstein went looking for companies that did so. He talked to firms like Wal-Mart and Cisco Systems, consulted Harvard Business School, called private and commercials schools. All told him the same thing. “The private sector does nothing of the sort,” he said. He even called John Chubb at Edison Schools Inc., the largest firm that tries to get contracts to commercially operate public schools. Chubb said using test scores to influence pay was a mistake. (Rothstein does acknowledge that “stockbrokers and sales clerks are paid on commission” but says the hardball sales tactics the practice fosters “should be intolerable where children are concerned.")

What George Bush, in his 2000 campaign, called parents voting with their feet—taking kids out of public and into private schools—did not escape Rothstein’s notice. He found that the actual number showed enrollment falling in private schools at every income level.

The charter school myth has been examined in depth, with Rothstein a part of it. In their book The Charter School Dust-Up, authors Martin Carnoy, Rebecca Jacobsen, Lawrence Mishel, and Rothstein examined 19 studies in 11 states and the District of Columbia, and then folded in data from the 2003 National Assessment of Educational Progress (NAEP) tests. Their conclusion: “There is no evidence that, on average, charter schools outperform regular public schools. In fact, there is evidence that the average impact of charter schools is negative.” (Their study also found that a claim that charter schools serve more disadvantaged students than public schools was false.)

In some cases, legislators have enacted legislation knowing full well that it was flawed. Rothstein recalls that during the debates over No Child Left Behind, economists Thomas Kane and Douglas Staiger produced a paper showing that, under the legislation, schools would be rewarded or penalized entirely because of dubious statistics produced by inadequate testing. Months went by while members of Congress wrestled with the problem. Then, they just gave up and passed the bill anyway.

“It’s a mistake to adopt policies that you know, based on the science, cannot work effectively,” Rothstein said.

The cost
None of this would matter much if it was harmless, but it is very harmful. Though the pendulum is now swinging back, for a decade precious education dollars were diverted to expensive high tech security gear, more police and weapons, expansion of juvenile jails. Various panaceas were legislated. In Virginia, the governor proposed eliminating after-school programs, an established violence preventive.

No Child Left Behind has exacerbated long-standing education problems, such as teacher shortages.

Can a school system succeed when its policy premises are false?

Former Nevada school superintendent Eugene Paslov says policy myths are complicated—he says they are like onions, with merit to be found in some levels as they are peeled—"Some of it has merit and much of it is mythology.” It’s hard to imagine hard pressed school administrators having the time or resources to sort things out.

Washoe County School District spokesperson Steve Mulvenon says he has to spend unnecessary amounts of time helping reporters do stories about school violence in the hope that they’ll get it right, though at times he has become so exasperated that he has considered cutting off assistance to such stories. “I guess I need to keep putting the message out there on each one of these incidents, that it is an aberration, that it is unusual.”

I Had An Original Thought...

About merit pay for teachers: If we give money to a teacher because his students did better on the test one year, do we take the money away when the students don't do better?

And if, per chance, student scores in that teacher's class were to vary from year to year, forget financially, what are we to make of that conundrum pedagogically, statistically, educationally, socio-economically, policyily, personally, professionally, confusingly?