The 2014 Standards for Educational and Psychological Testing define validity as, “The degree to which evidence and theory support the interpretations of test scores for proposed uses of tests,” unchanged from 1999. The wording has changed since 1985 and 1966, but the idea that validity refers to the inferences made from tests in their various uses goes back longer than my lifetime. This century, that wording as included, “for the proposed uses of tests.”
This has long prompted the question, “Who gets to propose test uses.” But as I read the standards, it is pretty clear that anyone gets to propose a use, and the validity question is whether there is sufficient evidence and theory to support that as a valid use.
However, there are many measurement professionals—including psychometricians—who read The Standards. They ignore the word “proposed” and replace it with sometime like “officially sanctioned.” To the degree that they consider validity at all, they believe that they can lay out in the fine print of technical documents which uses are valid and which are not.
But that approach is like sticking your head in the sand. That approach ignores reality.
We all know the uses that motivate test sponsors to invest in developing assessments. Those are perhaps the most important uses of tests. Those are the uses to which the test most certainly will be put. Furthermore, there are often other uses that we know are inevitable. Those uses are important, too. And they all are proposed uses of tests. Heck, they are proposed and accepted.
Some act as though the only test uses that matter are the ones that they bless, as though that is somehow relevant to whether tests will be misused. And then when tests are misused, they wash their hands of it. Amidst all the the finger pointing, they point their fingers at test sponsors or other test users and blame them for the unsanctioned uses—as though they did everything they were supposed to do and therefore have the moral authority to declare the uses to which tests might be allowed to be used.
But this usually constitutes a failure to live up to terms of contracts. It is poor customer service. It is incredibly unprofessional. It is almost unimaginatively arrogant. And, frankly, it is immoral.
Test developers should prioritizing the expected uses of tests. They should be laboring mightily to meet the needs of virtually inevitable uses of tests. They should not act like prima donnas artists, saying “This is what I create, and you can buy it or not.” Rather, they should be meeting the needs of tests users and their test uses. As the assessment experts, it is on them—on us—to develop tests than can be validily used for purposes that make them worth the time, money and other resources that sponsors and test users invest in them.
The finger pointing from test developers to test sponsors and test users should stop. Test users are right to point their fingers at the test developers who sell products that are not appropriate for their actual intended uses, as predictably and inevitably proposed by the actual test users.