[Each day in October, I analyze one of the 31 item writing rules from Haladyna, Downing and Rodriquez (2002), the super-dominant list of item authoring guidelines.]
Writing the stem: Avoid window dressing (excessive verbiage).
If you haven’t read my analysis of Rule 13 (Minimize the amount of reading in each item), please go back and read that. It applies here. But there is more, following a fantastic meme.
Yes, excessive verbiage is bad. Afterall, that’s what “excessive” means. So, this rule is somewhat tautological. I think that item developers should not make bad items. But that is not helpful advice.
The question is what counts as excessive. At this point, it is not surprising that the 2002 article makes no effort to explain this. Their 2004 book really offers no meaningful explanation for its version, Make the stem as brief as possible. Their 2013 book combines this rule with Rule 13, and does say quite a bit more. But it is not quite helpful.
Their example (2013, p. 98) is, “Which of the following represents the best position the vocational counselor can take in view of the very definite possibility of his being in error in his interpretations and prognoses?” Yes, that is clearly excessively wordy, but it is practically a straw man argument. Has anyone ever suggested that such a stem might be appropriate, or that such a question would be well writing in any circumstance?
Stems should be clear. They should include all the information needed for the test taker to understand what is being asked of them. Extra adverbs, adjectives and degree modifiers should not be included (e.g., “very” and “definite” in the example above). Filler words and phrases that do not contribute meaning or information should not be included. Phrases and words that can be replaced with simpler, more common and shorter equivalents without a loss of meaning should be so replaced. (e.g., replacing “represents the” with “is,” and replacing “in view of” with “given” in the example above).
My usual co-author offers, “Ensure you use as much verbiage as needed to make the task clear, no more and no less.” This emphasizes that clarity is the guiding principle. Of course, it highlights the reality that once one has Rule 14 (Ensure that the directions in the stem are very clear), the verbiage rule really does not add very much, perhaps not anything at all.
If the explanation of this rule included how to recognize excessive verbiage, the rule would not seem tautological. I understand why a simply stated rule might require further explanation to really be understood, but the articles quite often do not do that, and the quite rarely do it well.
[Haladyna et al.’s exercise started with a pair of 1989 articles, and continued in a 2004 book and a 2013 book. But the 2002 list is the easiest and cheapest to read (see the linked article, which is freely downloadable) and it is the only version that includes a well formatted one-page version of the rules. Therefore, it is the central version that I am taking apart, rule by rule, pointing out how horrendously bad this list is and how little it helps actual item development. If we are going to have good standardized tests, the items need to be better, and this list’s place as the dominant item writing advice only makes that far less likely to happen.
Haladyna Lists and Explanations
Haladyna, T. M. (2004). Developing and validating multiple-choice test items. Routledge.
Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. Routledge.
Haladyna, T., Downing, S. and Rodriguez, M. (2002). A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Applied Measurement in Education. 15(3), 309-334
Haladyna, T.M. and Downing, S.M. (1989). Taxonomy of Multiple Choice Item-Writing Rules. Applied Measurement in Education, 2 (1), 37-50
Haladyna, T. M., & Downing, S. M. (1989). Validity of a taxonomy of multiple-choice item-writing rules. Applied measurement in education, 2(1), 51-78.
Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied measurement in education, 15(3), 309-333.
]