Pages

3/22/12

Why The "Tests" Are Actually Completely Useless

...
So, what’s the problem? In short, it’s that year-to-year changes in proficiency rates are not valid evidence of school or policy effects. These measures cannot do the job we’re having them do, even on a limited basis. This really has to stop.

The literature is replete with warnings and detailed expositions of these measures’ limitations. Let’s just quickly recap the major points, with links to some relevant evidence and previous posts.

  •     Proficiency rates may be a useful way to present information accessibly to parents and the public, but they can be highly-misleading measures of student performance, as they only tell you how many test-takers are above a given (often somewhat arbitrary) cutpoint. The problems are especially salient when the rates are viewed over time – rates can increase while average scores decrease (and vice-versa), and rate changes are heavily dependent on the choice of cutpoint and distribution of cohorts’ scores around it. They are really not appropriate for evaluating schools or policies, even using the best analytical approaches (for just two among dozens of examples of additional research on this topic, see this published 2008 paper and this one from 2003);
  •     The data are (almost always) cross-sectional, and they mask changes in the sample of students taking the test, especially at the school- and district-level, where samples are smaller (note that this issue can apply to both rates and actual scores; for more, see this Mathematica report and this 2002 published article);
  •     Most of the change in raw proficiency rates between years is transitory – i.e., it is not due to the quality of a school or the efficacy of a policy, but rather to random error, sampling variation (see the second bullet) or factors, such as students’ circumstances and characteristics, that are outside of schools’ control (see this paper analyzing Colorado data, this one on North Carolina and our quick analysis of California data).

... Shanker Blog (embedded links there)

3/21/12

A Concerned Father And 8th Grade Test Prep Classes: Updated Again & Again & Finally Again

A father is concerned for his 8th grade son's new-found worry over high-stakes tests.  Sorry about all the italicized question marks posing as names.
Dear Principal ?????? of ?????? Middle School,

I have a comment and a question or four.  My son, ?son?, brought home a flier that asked him to participate in a test prep class because he scored a 3 in ELA, but it was a low 3.

The class was presented as an opportunity for him to improve his score.  Why would he need to bring up his score?  He scored proficient.  And the test has no bearing on his life.  None whatsoever.  Why would you tell him that his score was low and that he needs a class to improve his score?  And if his score was low in ELA, why isn't he being tutored in ELA as opposed to being tutored on how to take a test?  Who is teaching the class? How many kids are in the class? What is the curriculum of the class?

I am very concerned that he is being used as a way for ?school? to raise its AYP due to fears of PI status.  I want to know the real reason for the class, why my son was asked to participate, and I would like to know if kids who scored Below Basic or Basic are being asked to participate in the test prep class.

The simple act of telling him he should take the class has instilled in him the notion that he is a bad test taker and not the good student he actually is; he is an A student, as you know.  He is active in school leadership and has always performed very well academically.  He is now not sure if he is a good student.

High-stakes tests cause kids to question themselves, all for nothing.  The state tests have no impact on my son, but have a huge impact on the school, unfairly in my opinion.

I am very upset with the decision to offer the class to my son--effectively telling him he needs it--without first consulting his parents. 

I am aware my son can opt out of the state test, as can any child if their parent wants them to.  I am aware that teachers are not allowed to offer this information.  I am free to offer it, and I will encourage all parents of public school students to kindly refuse to take the test.

A await your response.

--?father?, father of ?student?
Okay, there is the email from the concerned father. And now for the Principal's response.
Mr. ????, Thank you for you email. You are correct, ?son? did score in the low Proficient range last spring. The class is being offered to the students who we feel can benefit from working in small groups on test taking strategies and dissecting some of the pre=release questions from the California Department of Education. This is not intended to be a negative experience for our students, we hand select the students that we are extending an invite to for participation. I understand clearly how this has impacted ?son? and will let him know that we did not intend make him feel negative and that the class is totally optional.

?Principal?
Principal, ????? Middle School
Notice not one question was answered, but another offer was made to make the son uncomfortable. And the father's response to the response:
I would prefer you not mention anything to ?son?. I know the class is optional, as does he. But he is now convinced he needs it.

Again, do not mention anything about our correspondence to ?son?.

--?father?
And a bit more from the father because that previous email needed to go out immediately to stave off more unwanted nonsense from the Principal to the son:
The flier said he would be tutored in the area he scored poorly in, ELA. But that is not true? It is pure test prep?

And I still would like to know which kids were invited, and why.

Again, this conversation between you and me (and those cc'd) is not for ?son?.

--?father?
You knew this response was coming from the Principal:
Mr. ?father?, I am sorry, but I already spoke to ?son?. We will support what ever decision is made regarding his participation.

Principal, ????? Middle School
I don't know about you, but this is exactly why the high-stakes tests do nothing of value for kids. Please, opt your kids out of the high-stakes tests!

Update: Some more of the exchange is dribbling in. This from Dad:
I am very upset you would talk to my son before talking to me. I am also concerned that none of my original questions were answered.

Please, answer my previous questions about who was invited to participate (broken down by B, BB, P, or A), who the class actually helps (students or school's AYP), who is teaching it, what the curriculum is, and also please tell me the status of ?school? in terms of Program Improvement.

I am concerned that the class is a response to not making AYP last year and has little to do with helping educate my son but instead is a method to raise the average score of ?school? students on the CST--I hope I am wrong, but as a former teacher I am pretty sure I am right. I really want, and deserve, answers to these questions.

--?father?
And the response from the Principal:
Mr. ?father?, I will be glad to discuss the class with you and the teacher Ms. ?teacher? in person not thru a shared email. I am not at liberty to share with you who was selected or how. If you would like to know how the class is structured and what material will be used, please schedule a time with Ms. ?teacher? to come in and observe the class. It is unfortunate that you do not see the value in what we are doing at ?????, and ????? Class is only one of the many positive experiences that we offer to our students. If you want to set-up an appointment with me, please email your contact information.

Thanks,
Principal, ????? Middle School
There you go. The principal won't answer the questions. Do you know why? She isn't allowed to, because the class is for the school, not the kids, and to admit that would be an admission of malpractice. But that's what high-stakes tests do, they cause educators to commit malpractice, and then lie about it.

Shame! Shame! Shame!

Update II: Response from Dad:
That is an incredibly unsatisfying response, and a bit condescending. I am not asking for the names of students participating, I am asking for their rank only.

When I taught I attended a meeting where the principal told us to focus on Basic students because raising them would help AYP, and focusing on Below Basic students would not help because they were unlikely to make Proficient. I was astounded at the time and said something. She didn't like that. I am now concerned this was not limited to my principal, but is a district directive. Your statements sound very similar to what I heard a few short years ago. I would love to be wrong.

I have told you that the class has caused my son to question himself, yet you claim that I "do not see the value in what [you] are doing at ?????," as if the damage to my son's sense of self was a benefit to him. Please.

I would like a meeting with you and whoever is in charge of the class, its participants, its structure, its intended purpose and knows its history.

Please let me know a time this week.
Update Again: And the final response from the principal:
Mr ?father?, I will arrange for a meeting on Thursday. We will be glad to share with you an overview of the program. You will not be given the other information that you have requested. I am concerned that you have stated that we have harmed your son with our invitation and I must state that ?son? has not been forced to participate, and he indicated to me that he wants to attend. Since you have such strong feelings about the opportunity, we will be glad to resend the invitation. You do not have to agree with how we support and engage our students at ?school?, but it is unfair of you to indicate that we are only focused on a select student population and not engaged in addressing all of our students needs. You did not send your contact information, please email a phone number where you can be reached tomorrow and I will call you with a meeting time to confirm our meeting. This will be my last email, I will contact you tomorrow.

Principal, ???? Middle School
Notice how she phrases that part about the father's feelings about the "opportunity" she has provided. Tell me, is the principal wrong here when she tries to spin the class as an opportunity, or is it actually an opportunity?  And does she mean rescind or resend?  She is concerned that the father stated there was harm done to the son; she is not concerned about the harm, but about the statement.

Also, she has the contact information for the father--it's all over the son's file, as is the mother's information, as all students' files should have.

And what kid, who is concerned about doing well, as this son is, wouldn't want to take the class the teacher says he should take because he fucked up the test so badly last time?  So yeah, I'm not surprised the kid, like most bright, interested, grade-centric kids, wants to take the class.  That's why the parents should be asked first, not the kid.

I would love comments.

Update III: The dad chose not to have a meeting.  He realized it would do nothing except make him mad and possibly put the son in an uncomfortable situation.

But the kid went to the class on Tuesday and Thursday.  I will paraphrase for you what the kid reported back to the father about the 2 sessions.  The kid pretty much confirmed the dad's fears that it was about AYP.  The teacher apparently told the prep class exactly why they were chosen: because they were all low 2s and hi 3s hi 2s and low 3s and could impact the school's AYP.  This was told to the class, explicitly, and the teacher referred to the class as her "special team" of kids who are going to raise the school out of PI status, or keep them from going into PI status.

So, here we have one of the greatest examples of the horrors of high-stakes testing.  Schools now use students as a marketing tool.  I hope the kids are getting paid and not working for free.

Now would be a good time to check out the bartleby project.
Final Update: Yes, this is actually about my son and me.

Tonight At 7pm(PST) TFT Interviews Michael Butz


Tonight at 7pm (pst) is the big show. This has been in the works a loooong time.

Getting Mike to do the show took nearly a year. Why? He didn't like me, at first. I was mean.

But after sharing some killer music with him after finding out we had similar tastes, he softened and acquiesced.

There a number of us who seem to dominate Students First's Facebook page, and this show is really an opportunity for all of us to hear each others voices. My hope is that a few of them will call in and participate. Folks like Eileen, Jane, Diane, Sahila, Jon, Neil, Stu, Bronx, Michelle, Mafara, Hari and a few I am forgetting.

3/20/12

The Hiatus Is Over



I took an unexpected hiatus from blogging the last few days.  Why you ask? Because my computer died. Well, the hard drive died. But, because I back everything up and because I am a PC guy, all I had to do was drop a new hard drive in the machine and reload everything.

Done.

The hardest part is the time it takes, especially getting stuff back down to earth from the cloud. But well worth it.

Make back ups often. Be your own genius.

Also, don't forget that today at 5pm (PST) I am playing some cool music on the radio show!

TFT Plays Some Killer Music Again 03/20 by The Frustrated Teacher TFT | Blog Talk Radio