One of the most famous Sherlock Holmes stories is Silver Blaze, the one in which he deduces the identity of the guilty party because a dog did not bark. Negative information is often worth as much as positive information.
There certainly have been times in my life, however, when I felt that those who judged me were more concerned with negative information that with positive information. They were more concerned with that I did not do than with what I did do. Or were more concerned with what I could not do than with what I could do. As a teacher, I certainly wanted to identify and celebrate what my students could do.
Positive information feels positive and perhaps celebratory. Negative information feels negative and perhaps even mean.
But negative information can be invaluable.
In some places, there is an explicit effort to focus on what students can do, rather than what they cannot do. This even gets to how some people and organizations talk about assessment. Rather than saying. “Assessments should identify what students can and cannot do,” they say “Assessments should identify what students know and can do.” I really do understand this urge. I used to be a teacher and I cared deeply about my students.
And yet, educators often say that what they really need is high quality formative assessments. That is, they need assessments that help them to provide instruction and to identify where students need more instruction. Of course, formative use of assessment requires times to go back and provide additional instruction and support to students after the assessment (and therefore timely results). But it also requires negative information. It requires tests and items that highlight what students do not know and what they cannot do.
This means that items used in formative assessment must be incredibly careful about false positive results (Type I errors). They cannot provide alternative paths to a successful response that avoids use of the targeted cognition. They cannot be so unstructured that it is not even clear what KSAs a test taker used to produce a successful response. Nor can they focus on the integration of skills such that it is not clear which KSAs broke down when test takers failed to produce a negative response.
The kinds of activities that I would rather teach with and want my students to be able to succeed with are not likely to be very useful for identifying what they need further help with.Yes, many students’ shortfalls might be clear, but many students will be able to steer around their weaknesses and lean further on their strengths. This kind of compensatory approach works to actively hide the information that formative assessments are intended to uncover.
The backlash against standardized tests is based upon many ideas, but one of them is surely that standardized tests and their results can feel mean. Such tests are often designed to designed to reveal shortcomings, deficits and lack of proficiency. Alternatives to traditional standardized tests, therefore often focus on the kinds of activities that I would rather teach my students with and with which I want my students to be successful. Such tasks seem to have more potential to feel celebratory. But such test simply cannot provide high quality information for formative purposes (and the information is not really valid for summative purposes either).
Formative assessment is just as demanding as summative assessment. It requires just as much skill and rigor to produce. Though we do not focus on formative assessment in our Rigorous Test Development (RTD) model of practice, just about everything in RTD applies to formative assessment as well as it does to summative assessment.