What Is the Design Effect?
The design effect (DEFF) is a ratio that measures how much less (or more) statistically efficient a complex sampling design is compared to a simple random sample of the same size. It's calculated as the variance of an estimate under your actual design divided by the variance that estimate would have under simple random sampling with the same number of observations. A DEFF of 2.0 means your complex design produces twice the variance of a simple random sample, or equivalently, your 1,000 interviews have the statistical precision of only 500 simple random interviews. A DEFF below 1.0 (which happens with efficient stratified designs) means your complex sample is actually more precise than a simple random sample of the same size. The concept was formalized by Leslie Kish in 1965, and it remains the standard metric for quantifying how design choices, clustering, stratification, weighting, oversampling, affect the real precision of your estimates.
Why the Design Effect Matters
Most sample size calculators assume simple random sampling. But most real surveys use complex designs, clustered, stratified, weighted, or some combination. If you plan for 1,000 interviews using a simple random sampling formula but your clustered design has a DEFF of 2.5, you'll achieve the precision of only 400 simple random interviews. Your confidence intervals will be wider than expected, your significance tests will be underpowered, and your findings may not support the conclusions you planned to draw. The DEFF bridges the gap between textbook formulas and real-world designs.
How the Design Effect Works
DEFF captures the net impact of every design choice that deviates from simple random sampling.
The Formula
DEFF = Var_complex(θ̂) / Var_SRS(θ̂)
Where Var_complex is the variance of your estimate under the actual design and Var_SRS is the variance under simple random sampling of the same size.
The effective sample size, the SRS-equivalent number of interviews, is:
n_eff = n / DEFF
If n = 1,000 and DEFF = 2.0, your effective sample size is 500. This is the number you should use when assessing whether your study has enough power for the planned analyses.
Sources of Design Effects
Different design features push the DEFF in different directions.
Clustering increases DEFF. When you sample clusters (geographic areas, schools, clinics) and interview multiple people within each cluster, those people tend to be more similar to each other than to people in other clusters. This intra-cluster correlation (ICC or rho) reduces the unique information each additional within-cluster interview provides. The clustering DEFF for a one-stage cluster sample is approximately:
DEFF_cluster ≈ 1 + (m - 1) × ρ
Where m is the average cluster size and ρ is the intra-cluster correlation. With 20 interviews per cluster and an ICC of 0.05, the DEFF is 1.95, nearly doubling your required sample size.
Unequal weighting increases DEFF. When disproportionate allocation or differential non-response creates unequal weights, the effective sample size shrinks. The weighting DEFF is:
DEFF_weight = 1 + CV²(w)
Where CV²(w) is the squared coefficient of variation of the weights. More extreme weights produce larger design effects.
Stratification decreases DEFF. Proportionate stratification on a variable correlated with the outcome reduces variance below what SRS would produce, creating a DEFF below 1.0. This is one of the few design features that improves efficiency.
Combined Design Effects
Real surveys combine multiple design features. A stratified, clustered, disproportionately allocated survey has a design effect that reflects all three influences. The total DEFF isn't simply the product of individual components (they interact), but it's often approximated that way for planning purposes.
In practice, estimate the DEFF from pilot data or similar previous studies rather than trying to derive it from first principles. Published surveys in your field often report design effects that you can use as benchmarks.
Using DEFF for Sample Size Planning
Adjust your SRS-based sample size requirement by multiplying by the expected DEFF:
n_required = n_SRS × DEFF
If a simple random sample of 400 would achieve your desired precision and you expect a DEFF of 2.0, plan for 800 interviews.
For multi-stage clustered designs, you can also work the formula backward: given a target effective sample size and an expected DEFF, determine how many clusters and how many interviews per cluster you need. More clusters with fewer interviews per cluster reduces the DEFF, but increases travel and logistical costs.
DEFT: The Square Root Alternative
Some reports use DEFT (the design effect square root), which is the ratio of standard errors rather than variances. DEFT = √DEFF. A DEFT of 1.4 corresponds to a DEFF of 2.0. DEFT is more intuitive for margin-of-error calculations since margins of error scale with the standard error, not the variance.
When to Calculate the Design Effect
- Sample size planning for any non-SRS design: clustered, stratified, weighted, or multi-stage, to ensure adequate effective sample size
- Reporting precision for complex surveys so readers can assess the real statistical power of the study, not the nominal power
- Comparing designs during study planning to choose the most efficient approach for a given budget
- Evaluating the cost of oversampling or disproportionate allocation by quantifying how much aggregate-level precision is lost
- Quality control after data collection to verify that the achieved design effect is consistent with planning assumptions
Common Mistakes to Avoid
- Ignoring the design effect in sample size calculations. This is the most common and most consequential mistake. An underpowered study wastes the money spent on it because it can't answer the research question with adequate precision.
- Assuming a DEFF of 1.0 for online panel surveys. Even simple quota-based online surveys have design effects from unequal weighting. And if you oversample any subgroup, the weighting DEFF increases. A DEFF of 1.2-1.5 is common even for straightforward designs.
- Reporting nominal sample sizes without effective sample sizes. Telling stakeholders "we surveyed 2,000 people" when the effective sample size is 800 misrepresents the study's precision. Report both numbers.
How Quali-Fi Supports Design Effect Estimation
Quali-Fi's sample planning tools incorporate design effect estimates into sample size calculations, so your survey targets reflect effective, not just nominal, interview requirements. The platform's post-collection analytics calculate achieved design effects from your data, comparing planned vs. Actual efficiency and flagging studies where the DEFF exceeded expectations.
Frequently Asked Questions
What's a typical design effect for a clustered survey?
For face-to-face household surveys in developing countries, DEFFs of 1.5-3.0 are typical. For telephone or online surveys with geographic clustering, 1.2-2.0 is common. For simple quota-based online surveys, 1.1-1.5. The value depends on cluster size, intra-cluster correlation, and weighting variability.
Can the design effect be less than 1?
Yes. Proportionate stratified sampling on a variable strongly correlated with the outcome produces a DEFF below 1.0, meaning the stratified sample is more efficient than SRS. This is the payoff from good stratification.
How do I calculate the design effect from my data?
Use survey analysis software that accounts for the complex design. In Stata, the estat effects command after svy estimation produces variable-specific design effects. In R, the survey package's svymean function reports DEFF. These calculate the actual DEFF from your data rather than relying on planning assumptions.
Related Topics
- Finite Population Correction
- Disproportionate Stratified Sampling
- Oversampling
- Area Probability Sampling
- Proportionate Stratified Sampling
Plan for real precision, not nominal sample sizes. Start a free trial with Quali-Fi and use DEFF-adjusted sample planning and post-collection efficiency analysis to ensure your studies deliver the precision you need.