Testing, 102: Test Question Standardization

So a standardized test, because it is ‘standardized’, ensures fairness, equity and justice, right? Since every student taking that test is tested on the exact same content under the exact same circumstances, it stands to reason that that gives everyone taking the test an equal opportunity to do well, right? And that the results will be an unbiased way to compare one student to another, apples to apples, right?

But that is not always as easy as it sounds

Yes, the content is standardized, so each student taking that test will answer the same questions as every other student taking that test.

The trouble is, students are human, and like snowflakes, each one is different.

Sometimes, there is inadvertent bias built right into the test questions themselves, even though the test developers are aware of this and try to control for it. Here are a couple of oversimplified possibilities, just to give you the idea. Test developers look for common ground, so no one test taker has an advantage over another because of, say who they are. So let’s say we create a question to assess a student’s ability to identify sequence, a common reading skill.

So the test developers decide to use an excerpt from the story of The Three Little Pigs, because everybody knows that, right? Well, not necessarily. If the child comes from a culture other than the dominant one, the set of bedtime stories that the child heard could be totally different and ‘foreign’ to the dominant culture’s. The bedtime stories may even have been told/read in a completely different language. Or maybe the child’s parent works two jobs and doesn’t have time to read to the children every night, even though the parent knows he/she should and would, in fact, relish the time spent with the children. Would a test taker not familiar with the story of the three pigs be at a disadvantage?

Or maybe the test could contain a reference to sibling rivalry. Could that put an only child who never played with brothers and sisters at a disadvantage when answering that question compared to a child who has had first hand experience interacting with siblings on a regular basis ?

It has been shown that the more affluent the student’s family, the more closely the prior knowledge of the test taker matches the general content from which the test developers draw.

So even if the questions on a given test are standardized, individual students could interpret those questions differently because of influences over which the student has no control: gender, race, religious affiliation, family socio-economic status, and education levels of the parents.

The quest for a fair, equitable, and just test, the results of which can be used to compare apples to apples is a noble one. It is also very difficult to achieve.

Leave a Reply

Your email address will not be published. Required fields are marked *