There is a hierarchy in educational measurement. Quantitative work is seen as superior. This is so thoroughly naturalized that many of the people who benefit from it cannot see it as a hierarchy at all—they experience it as a simple fact about what rigorous evidence and reasoning looks like. This is an epistemology at work.
We can watch how it operates in practice. Psychometric flags—low point biserials, suspect discrimination indices—are supposed to function as alerts, prompts for human judgment about whether something is wrong with an item. A negative point biserial should not automatically exclude an item. But in practice, they function as verdicts. Low point biserial? The item simply cannot make the operational form. The flag overrides the judgment of the content experts who built the item, reviewed it, and know the domain. Cronbach's alpha has achieved something close to supreme authority—a single number that seems to settle questions that involve far more than internal consistency. The quantitative output doesn't inform the judgment; it replaces it. Some deny that this happens, while others acknowledge the fact that this is standard practice.
Meanwhile, the people holding those quantitative outputs frequently feel entitled to opine about things about which they lack expertise. Where exactly is the line between fifth and sixth grade mathematics? What distinguishes two sixth grade standards from each other in ways that matter for item development? What does it mean for a student to have genuinely mastered a standard versus having picked up a surface procedure? These are not trivial questions. They require deep content knowledge, experience with how students think, and serious engagement with domain models. They are the core expertise of content development professionals. And yet many others don’t even see the tools and lenses required to build and hold that expertise—therefore think that there is nothing wrong with offering their views as though they might be of similar standing. Their epistemological framework does not see this other expertise, and they don’t either. Certainly, it is not as meaningful, challenging or rigorous as their own.
They don't see it. And what you can't see, you can't respect.
This often extends to qualitative methodologies generally. There is a pervasive assumption—rarely stated, but deeply held—that qualitative work is not really disciplined. That it has no real methodology. That its findings are, at best, suggestive and, at worst, just opinion. The possibility that qualitative research might have rigorous standards, systematic procedures, and criteria for what counts as a valid finding simply does not register. Not because anyone has examined qualitative methodologies and their foundational epistemologies and found them wanting. Rather, the epistemologies are ignored and therefore the methodologies viewed as foundationless.
Here is what makes this genuinely strange: it is not a symmetric problem. Qualitative researchers in this field are typically intimidated by quantitative work. They defer to it. They actually have experience with it from their schooling days, and they respect that there is something rigorous there. They rarely question whether their quantitative colleagues' work actually meets its own standards. They just assume it does.
But it often doesn't. Underpowered studies. Violated methodological assumptions. Too much dependence on linear regression. Ordinal data treated as interval data. Multiple comparisons not accounted for. Proxies standing in for constructs they don't actually measure. The quant work that commands all this deference frequently falls short of the very standards of rigor it claims.
The asymmetry is that quantitative practitioners assume their own rigor without interrogating it, demand that others respect it, and simultaneously refuse to recognize the possibility of rigor in other traditions. They don't look for it. They don't ask what it would look like. The question doesn't arise, because within their epistemological framework, the answer is already settled. All of this despite the fact—or perhaps because of it?—that they rarely have any experience or training in rigorous qualitative work from their school days.
Every discipline—every epistemological tradition—has its own requirements for rigor. All of us should lean into those requirements, interrogate our own work against them, and resist the temptation to assume that producing the right kind of output means the work is actually good. And all of us should do the harder thing, too. We should learn to see what rigor looks like in traditions other than our own, and respect the expertise it takes to achieve it—especially when our colleagues in our collaborations come from different epistemologies.
Good multi-disciplinary collaborative work requires doing what our epistemologies cannot do for us. It requires looking for—and allowing—other ideals of rigor. It requires humility about our own epistemologies and their limits. It requires respecting the capabilities of others, rather than expecting deference.