KIPP Study Makes A Dubious Claim

KIPP Works

It’s relatively obvious to anyone who looks that the Knowledge Is Power Program (KIPP), the nation’s largest charter management organization, produces results. Just by seeing its classrooms you start to figure this out: the students are in matching uniforms, they chant and seem energized about learning, and, other than the chants, they’re orderly and respectful. To prove that KIPP works mathematically, up until now we’ve had to rely on pretty low-level analyses that show very high numbers of their students pass state accountability exams.

A new, rigorous analysis for the National Bureau of Economic Research changes that. Using a quasi-experimental research design that capitalizes on the large number of students applying to get into but ultimately rejected from one KIPP school in Lynn, Massachusetts, the researchers were able to compare students who entered the school with those who wanted to attend but were rejected due to space restrictions. This design helps the researchers isolate KIPP attendance from motivation, parental education, environmental factors, or any other variable that might be difficult to detect.

After these controls, KIPP attendees gained .35 of a standard deviation every year in math and .12 standard deviations each year in English. Results were even more positive for Limited English Proficiency and special education students (the demographics of KIPP Lynn lottery winners matched lottery losers and the district as a whole)...

Is it so obvious KIPP works?  The study claims to deal with "creaming"--the effect that self-selection creates a superior student body--so a comparison can be made between those who got into KIPP and those who didn't.  What the study doesn't deal with is the environment at the two different schools students ended up in.  On one hand you have the students who got into KIPP along with all the other self-selected students who got in.  Great learning environment there, I assume.  Then there are the kids who ended up in non-KIPP schools with all the other students whose parents couldn't or wouldn't attempt to get them into KIPP.  Not as great a learning environment there, I assume.

The study is flawed due to this rather blatant omission of reality.  A KIPP school demands proper behavior from its students, as well as high parental participation among other things.  A non-KIPP school takes all comers, regardless their motivation.  That makes for a less ideal learning environment.

So, it's not that KIPP works; it's that KIPP has "better" (creamed) students as we have been saying forever.