The New York State Education Department on Aug 10 released a few reading and math questions that appeared on this year’s third- through eighth-grade standardized tests, the New York Times reports.
One passage from the third-grade reading test and a few of the accompanying questions stumped half the adults who took the little quiz in the interactive feature. See whether they stump you, too.
In fact, the Times contacted the author of the passage, who was unaware the passage had been used on the test because Pearson, the test publisher, had obtained the rights directly from his publisher. He was stumped initially by the correct answer on the question about the passage’s theme but upon reconsideration said the answer could possibly be considered correct.
Which statement best describes a lesson learned from this story?
- A. Secrets can often be entertaining.
- B. Secrets are not very useful.
- C. Rewards should not be taken for granted.
- D. Rewards can be more trouble than they are worth.
A fine point of order: The use of the word “best” is common in questions that have more than one possibly correct answer. This question isn’t bad, however, because (D) isn’t the best answer among those listed; it’s bad because the “best” lesson learned from the story isn’t even one of the choices, and that is frustrating to kids who know how to read well. This momentary frustration often ruins the next few questions on the test for those kids, adding a degree of invalidity to their score.
From the excerpt quoted, it’s not easy for me to see how the “lesson learned” question even lists the right answer. I’ll leave that to you to decide, but be warned: “You can sort of read different things into it,” the Times quoted the author, Saviour Pirotta, as saying about the excerpt from his book entitled Around the World in 80 Tales.
But my reason for reporting this story isn’t so much that one question on a test might have been a bad one—or at least open to multiple correct answers depending on the many diverse ways 9-year-olds read stories. No, my point in reporting it is to say tests are only as good as the questions that make them up. Questions about the theme of a story or its takeaway lessons are subject to interpretation and are better asked in a constructed-response format, where kids write short paragraphs or essays.
Of course, it costs more money to score essays than it does to score multiple-choice questions, and that’s a problem worth considering since testing is a federal mandate and states are bound to do the least they can get away with while still complying with the law.
But on this question, if I were writing an essay about the lessons learned, I wouldn’t have said, “Rewards can be more trouble than they are worth.” The man in the story never regrets being given the reward for helping the snake, even when he loses his ability. Then, even though the fun wears off, he takes a second reward in a way that indicates he has learned a completely different lesson about being rewarded: that he should try harder to keep secrets.
It seems this public release from New York, then, wasn’t about reading at all; it seems to be about determining how well kids can predict what question writers are thinking. I learned some of the same tricks when I was in school, but we’re spending a lot of money to ask kids to learn how people, who appear to be stupider than half of New York’s third graders but work at a test-development company, think. And then, we’re counting on these invalid scores to tell us a lot.
We’ll never know how many bad questions like this were on the test. It needs to be said, though, that the state of New York has dropped its association with the test-production company behind this mediocre exam, Pearson. Now if only we could stop politicians and business executives from using students’ scores from these tests to fire, promote, and pay teachers, abuses that give these scores more weight than they deserve, that would be something.