There is good reason to think there is only one type of test validity: the degree to which evidence and theory support the intended uses of a test. We can ask whose intended uses count—the developer's, the state's, the district's, the teacher's—but the idea that test validity is test-use validity is strong and well established. We agree with it entirely.
But items exist before there are tests.
Items are developed, reviewed, and banked as items. They are bought, sold, and licensed as items. They carry labels: aligned to this standard, measuring that skill, at a certain level of proficiency or perhaps even at a level of cognitive complexity. Each label is a claim that the item produces a particular kind of evidence. Items, too, have intended uses.
None of this impugns the unitary view of test validity. Item validity is not a challenge to the established unitary view of validity because it is not an alternative view of test validity. It is the validity of a different object. It is a different level of analysis. Because items are so important in the processes of developing tests and the processes of taking tests, they merit rigorous scrutiny. Thus, mirroring The Standards’ idea of test validity, we can define item-level validity: the degree to which evidence and theory support the use of an item for its intended purpose.
This is a more straightforward idea than test validity. A test's intended uses can be contested, multiplied, and extended far beyond what its developers imagined. An item's intended use is already on its label: to elicit evidence of the cognition it targets.
Thus, we can operationalize item validity with a more directly useful definition: valid items elicit evidence of the targeted cognition for the range of typical test takers.
Each clause matters. Elicit: the item must actually prompt the cognition and produce observable evidence of that cognition. The targeted cognition: what the alignment reference specifies, not something adjacent to it. For the range of typical test takers: not just the idealized student the item writer pictured, but the full range of test takers who will actually encounter the item.
That is item validity.