[Each day in October, I analyze one of the 31 item writing rules from Haladyna, Downing and Rodriquez (2002), the super-dominant list of item authoring guidelines.]
Style concerns: Edit and proof items.
We’re just gonna ignore that they left it at “proof” instead of “proofread,” right? I shouldn’t be that petty, right?
Because this rule is one of the style concerns, we know that this is not about substantive editing. It is not in the content section, nor in the stem or choices sections. However, the 2002 article does not provide any explanation. Their 2004 book revises this rule to “Proofread each item,” but the 2013 book goes back to “Edit and proof items.”
That 2013 book addresses style guides at length, and this is…somewhat important. Consistency, while sometimes the hobgoblin of little minds, is something that will be looked for by many people. Others within testing organizations and/or sponsors, in addition to teachers and other members of the public, can notice and pick on inconsistency in style. So, yes, items should be compliant with the relevant style guides. Such style guides ought to end discussion and debate on how something should be presented or how a word is spelled (e.g., email, e-mail, Email, E-Mail, emails, etc.).
However, when most people think about proofreading, they are thinking about…spelling, grammar, punctuation and word choice (i.e., often formality of register). Yes, items should be edited for style and these concerns. And yes, proof-reading is important. But here’s the thing: this rule is not a description of anything about the items. This is the only rule that is only about test developers actions. And we know that this rule is not about “grammar, punctuation, capitalization and spelling” because that is the next rule.
Note that this rule does not actually say anything about style guides in any version of the Haladyna rules. One of the explanations (2013) mentions the importance of style guides, but even that version does not mention them in the rule itself. This is entirely about process, and not about product. This is about all published writing and is in no way particular to items.
It is probably dumb to include this in a set of item writing rules or guidelines, and it certainly is particularly dumb to include it in this kind of list of rules or guidelines. In fact, only one-third of their 2002 sources even mention it because it probably should go without saying!
[Haladyna et al.’s exercise started with a pair of 1989 articles, and continued in a 2004 book and a 2013 book. But the 2002 list is the easiest and cheapest to read (see the linked article, which is freely downloadable) and it is the only version that includes a well formatted one-page version of the rules. Therefore, it is the central version that I am taking apart, rule by rule, pointing out how horrendously bad this list is and how little it helps actual item development. If we are going to have good standardized tests, the items need to be better, and this list’s place as the dominant item writing advice only makes that far less likely to happen.
Haladyna Lists and Explanations
Haladyna, T. M. (2004). Developing and validating multiple-choice test items. Routledge.
Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. Routledge.
Haladyna, T., Downing, S. and Rodriguez, M. (2002). A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Applied Measurement in Education. 15(3), 309-334
Haladyna, T.M. and Downing, S.M. (1989). Taxonomy of Multiple Choice Item-Writing Rules. Applied Measurement in Education, 2 (1), 37-50
Haladyna, T. M., & Downing, S. M. (1989). Validity of a taxonomy of multiple-choice item-writing rules. Applied measurement in education, 2(1), 51-78.
Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied measurement in education, 15(3), 309-333.
]