What Is the Minimum Detectable Effect?
The minimum detectable effect (MDE) is the smallest true effect size that a study can reliably detect given its sample size, alpha level, and desired power. It answers the question: "If there's a real difference, how large does it have to be for my study to have a reasonable chance of finding it?" For example, if your A/B test's MDE is 3 percentage points, you can detect a difference of 3 points or larger with your target power (typically 80%), but differences smaller than 3 points will likely be missed. MDE is the critical link between sample size planning and business relevance, it forces you to define what "big enough to matter" means before you collect data. Every experiment and survey comparison should have an explicit MDE, because without one, you can't know whether a non-significant result means "no effect" or "not enough data."
Why the Minimum Detectable Effect Matters
MDE is what makes power analysis actionable. Saying "we need 80% power" is meaningless without specifying what effect size you're trying to detect. A sample of 50 has 80% power to detect a massive effect; a sample of 50,000 has 80% power to detect a tiny one. MDE forces the conversation about what effect size would actually change a business decision, then sizes the study accordingly.
How the Minimum Detectable Effect Works
MDE for Comparing Two Proportions
The most common use case in market research and A/B testing is comparing conversion rates between two groups:
MDE = (z_α/2 + z_β) × √[2p̄(1-p̄)/n]
Where z_α/2 is the critical value for your alpha level, z_β is the critical value for your desired power, p̄ is the average proportion across both groups, and n is the sample size per group.
For α = 0.05 (z = 1.96) and power = 0.80 (z = 0.84):
MDE = 2.80 × √[2p̄(1-p̄)/n]
Worked Example: A/B Test
Your website has a 10% baseline conversion rate. You plan to run an A/B test with 1,000 visitors per variation.
MDE = 2.80 × √[2 × 0.10 × 0.90 / 1,000]
MDE = 2.80 × √[0.18 / 1,000]
MDE = 2.80 × √0.00018
MDE = 2.80 × 0.01342
MDE = 0.0376 or about 3.8 percentage points
Interpretation: With 1,000 visitors per group, you can reliably detect a conversion rate increase from 10% to at least 13.8%. If the true improvement is only 2 percentage points (10% to 12%), your study has less than 80% power to detect it.
Rearranging for Sample Size
If you know the MDE you need, solve for n:
n = 2 × [(z_α/2 + z_β) / MDE]² × p̄(1-p̄)
For an MDE of 2 percentage points with a 10% baseline:
n = 2 × [2.80 / 0.02]² × 0.10 × 0.90
n = 2 × 19,600 × 0.09
n = 3,528 per group
Cutting the MDE in half roughly quadruples the required sample, a critical planning consideration.
MDE for Comparing Two Means
MDE = (z_α/2 + z_β) × σ × √(2/n)
Where σ is the standard deviation of the outcome measure.
Example: You want to detect a difference in satisfaction scores (σ = 15) with n = 200 per group.
MDE = 2.80 × 15 × √(2/200) = 2.80 × 15 × 0.10 = 4.2 points
You can detect a 4.2-point difference on the satisfaction scale. If a 2-point improvement is what you consider meaningful, you need a larger sample.
MDE and Business Relevance
The MDE should be set based on what effect size is practically meaningful, not what's statistically convenient. Ask:
- What's the smallest improvement that would justify the investment? If a new campaign costs $100K and needs a 2% lift in conversions to break even, your MDE should be 2%.
- What effect size is realistic? Prior research, benchmarks, or pilot data should inform your expectations. Optimizing a button color might yield 0.5% lift; redesigning the entire funnel might yield 5%.
- What can your budget support? If the smallest meaningful effect requires 10,000 per group and you can only afford 2,000, either accept a larger MDE or don't run the test.
MDE Tradeoff Table
For comparing two proportions with baseline = 10%, α = 0.05, power = 0.80:
| Sample per Group | MDE (percentage points) |
|---|---|
| 250 | 7.5 |
| 500 | 5.3 |
| 1,000 | 3.8 |
| 2,500 | 2.4 |
| 5,000 | 1.7 |
| 10,000 | 1.2 |
Relationship to Other Parameters
MDE doesn't exist in isolation. It's one variable in a four-way relationship:
| If you want to... | You need to... |
|---|---|
| Detect smaller effects | Increase sample size |
| Use less sample | Accept a larger MDE |
| Increase power (lower beta) | Increase sample or accept larger MDE |
| Use stricter alpha | Increase sample or accept larger MDE |
When to Calculate MDE
- Before every A/B test to verify the test can detect effects large enough to matter
- During study design for survey experiments, concept tests, and product tests
- When evaluating past studies: calculating the MDE retrospectively helps you understand whether a null result was informative
- During budget discussions to show stakeholders the relationship between investment in sample size and the precision of your findings
- When comparing study designs: MDE lets you compare the sensitivity of different approaches (e.g., paired vs. Independent designs)
Common Mistakes to Avoid
- Not calculating MDE at all: running a test without knowing what it can detect is like driving without knowing your fuel range
- Setting MDE based on what sample you can afford rather than what effect matters, this gets the logic backwards; start with business relevance, then determine the sample
- Confusing MDE with expected effect size: MDE is a sensitivity threshold, not a prediction; the actual effect may be larger, smaller, or zero
How Quali-Fi Supports MDE Planning
Quali-Fi's Research plan ($1,061/month) includes MDE calculators for common research designs, letting you input your baseline metric, desired alpha and power, and available sample to see exactly what effect size your study can detect. The platform generates tradeoff curves showing how MDE changes with sample size, so you can find the sweet spot between budget and sensitivity.
Calculate your MDE with Quali-Fi
Frequently Asked Questions
Is MDE the same as effect size?
No. Effect size is the true magnitude of the difference in the population, it's unknown. MDE is the smallest effect your study design can reliably detect. If the true effect is larger than your MDE, you'll probably find it. If it's smaller, you probably won't. MDE is a property of your design; effect size is a property of reality.
What MDE should I target for market research?
It depends on the context. For A/B tests on conversion rates, MDEs of 1-5 percentage points are typical. For satisfaction or NPS comparisons, an MDE of 3-5 points is often practical. The key question is always: "What's the smallest change that would alter my decision?" Let that answer drive the MDE.
How does MDE change with a one-tailed test?
A one-tailed test reduces MDE by about 15-20% for the same sample size, because all of the alpha is concentrated in one direction. However, one-tailed tests can't detect effects in the opposite direction, so they're only appropriate when the direction is pre-specified and effects in the other direction are irrelevant.