Educational Research: Mostly Crap

Here is a little nugget from a 2007 journal article (or pdf) regarding the re-authorization of NCLB. In it they question the SBR (scientifically based research) requirement that all materials must meet to conform to NCLB. I have said many times that educational research is some of the poorest, least robust, most maligned research. Here is a good reason why:

A basic research goal for educational research is to determine causation. The advantage of using SBR is that having a control group allows a better explanation of the variance in the data. Even with a control group, however, educational research must account for the intentionality of both students and teachers. This is not trivial [emphasis mine]. Intentionality refers to the ability of students and/or teachers to behave in a way that is not norm-regulated (Howe, 2005, p. 311). To determine the effectiveness of an intervention, researchers must assume that control and test group teachers deliver the same content in exactly the same way to the same mix of children. Research design options can minimize errors for these assumptions, but not without considerable efforts and collaboration (National Research Council, 2005, p. 30). Student populations vary widely in their ethnicity, socio-economic status, and family backgrounds. These factors can be controlled for. The student motivation, the intentionality of students and teachers and the variety of administrative philosophies make it impossible to control for all variability [emphasis mine]. Many of these constructs cannot be measured or scaled as a data point; however, each can strongly influence educational achievement. The variance inherent within each of the constructs does not lend itself well to SBR, even with randomization and a control group.

We can't even measure this shit! We know the problems are societal. We have big things to fix before anything is going to improve. All this nonsense is a reaction, and it is a knee-jerk reaction, providing a lot of people with a whole lot space for their self-serving, career-advancing, kid-under-the-bus-throwing, teacher blaming, policy excusing nonsense they decided to name NCLB. TAKE A STAND!!!!!!