What Is Criterion Validity?
Criterion validity is the degree to which scores on a test, survey, or measurement instrument correlate with an external, real-world outcome (the criterion) that the measure is supposed to predict or reflect. Unlike construct validity, which asks whether you're measuring the right abstract concept, criterion validity asks a more concrete question: do the scores relate to something tangible that matters? If a customer satisfaction score doesn't correlate with retention rates, or if a hiring assessment doesn't predict job performance, the measure lacks criterion validity, regardless of how well-constructed its items are. Criterion validity comes in two forms depending on timing: concurrent validity (the criterion is measured at the same time as the test) and predictive validity (the criterion is measured later). Both provide evidence that the measure connects to outcomes in the real world.
Why Criterion Validity Matters in Research
Research measures ultimately need to mean something beyond the survey itself. Criterion validity is the bridge between measurement and action. When a decision-maker asks "So what?" after seeing satisfaction scores or engagement metrics, criterion validity is the answer, it's the evidence that the scores predict churn, purchasing behavior, productivity, or whatever outcome the organization cares about. Without it, measurement becomes an academic exercise with no practical payoff.
How Criterion Validity Works
Criterion validity assessment compares measure scores to external outcomes using correlational or predictive methods.
Concurrent Validity
Concurrent validity is established when the measure and the criterion are assessed at the same point in time. For example, you might correlate a new depression screening tool with a clinician's diagnosis made during the same visit, or correlate a customer effort score with actual support ticket data from the same period. The correlation between the measure and the simultaneous criterion provides evidence that the measure captures what's happening right now.
Concurrent validity is useful when you're building a measure that's meant to serve as a proxy for something that's harder or more expensive to measure directly. If a 5-minute survey correlates strongly with a 2-hour clinical interview, the survey has practical value as a screening tool.
Predictive Validity
Predictive validity is established when the measure forecasts a criterion that's assessed at a future point in time. An employee engagement score measured in January that correlates with voluntary turnover six months later demonstrates predictive validity. This is the more practically powerful form of criterion validity because it shows the measure has forward-looking value, it doesn't just describe the present, it anticipates the future.
Predictive validity studies are harder to conduct because they require longitudinal data collection. You need to administer the measure, wait for the criterion to occur, then analyze the relationship. The payoff is evidence that directly supports using the measure for decision-making.
Choosing and Measuring the Criterion
The quality of a criterion validity study depends entirely on the criterion. It should be relevant (theoretically connected to what you're measuring), reliable (measured consistently), and practical (actually available). Common criteria in applied research include behavioral outcomes (purchase, churn, adoption), performance metrics (sales figures, error rates), clinical assessments, and objective records. Self-reported criteria are weaker than behavioral or objective criteria because they introduce additional measurement error.
Interpreting Criterion Validity Coefficients
Criterion validity is typically reported as a correlation coefficient (r) between the measure and the criterion. What counts as "good" depends on the context. In personnel selection, validity coefficients of r = 0.30-0.50 are considered practically significant. In clinical screening, higher thresholds apply because the stakes are higher. criterion validity coefficients are constrained by the reliability of both the measure and the criterion, unreliable criteria will suppress the coefficient even if the measure is excellent.
When to Use Criterion Validity Assessment
- Validating screening or selection instruments where the measure's value depends entirely on its ability to predict real outcomes
- Justifying the use of proxy measures: when the outcome you care about (retention, clinical diagnosis, job performance) is expensive or slow to measure, and you want evidence that a faster alternative works
- Evaluating competing measurement tools: if two instruments claim to measure the same thing, comparing their criterion validity coefficients tells you which one is more practically useful
- Building the business case for measurement programs: showing stakeholders that survey scores predict the outcomes they care about
- Regulatory or accreditation contexts where instruments must demonstrate criterion-referenced evidence before use
Common Mistakes to Avoid
- Using a weak or unreliable criterion and concluding the measure is invalid when the real problem is criterion quality, always evaluate the criterion's reliability before interpreting low validity coefficients
- Conflating concurrent and predictive validity: demonstrating that a measure correlates with a simultaneous criterion doesn't prove it predicts future outcomes; if prediction is the goal, you need a time-lagged study
- Ignoring range restriction: if your sample is homogeneous on the measure or the criterion (e.g., only studying high performers because low performers have already left), the correlation will be artificially suppressed; correct for range restriction when possible
How Quali-Fi Supports Criterion Validity
Quali-Fi's panel management and longitudinal study tools let research teams administer measures at one point and track criterion outcomes over time within the same platform. For concurrent validity studies, the platform's survey tools can collect measure data alongside behavioral or attitudinal criteria in a single session, with real-time analytics showing the relationship between scores and criterion variables as data comes in.
Frequently Asked Questions
What's a good criterion validity coefficient?
It depends on the domain. In educational and personnel testing, r = 0.30-0.50 is considered useful. In clinical screening, r > 0.50 is typically expected. The practical significance also depends on the decision context, even a modest correlation can have meaningful impact when applied to large populations. Always compare your coefficient to established benchmarks in your field.
How is criterion validity different from construct validity?
Construct validity asks whether the measure captures the intended abstract concept (through convergent, discriminant, and structural evidence). Criterion validity asks whether the measure relates to a concrete, external outcome. A measure can have good construct validity (it measures satisfaction, as confirmed by factor analysis and convergent correlations) but poor criterion validity (satisfaction scores don't predict retention). Both are important, and they answer different questions.
Can the same criterion be used for concurrent and predictive validity?
Yes, but the study designs differ. Measuring satisfaction and churn rate at the same time point tests concurrent validity. Measuring satisfaction now and churn rate six months later tests predictive validity. The criterion variable is the same; the timing and the type of evidence differ.
Related Topics
- Predictive Validity
- Construct Validity
- Convergent Validity
- Discriminant Validity
- Face Validity
- Reliability in Research
Need to connect survey scores to real-world outcomes? See how Quali-Fi's longitudinal and panel tools support criterion validation research over time.