Why is Multiplicity a Problem?
Each statistical test has a probability α, for example, 0.05 or 5% of producing a false positive/false finding/false discovery (rejecting H₀ when it is true).
- If you perform only 1 statistical test, the risk of false findings remains at α = 5%.
- If you perform 10 independent tests, the risks multiply, and the probability of observing at least one false finding increases considerably to 40%. Not uncommon in preclinical studies.
- With 20 independent tests, the probability of observing at least one false finding increases considerably to 64%. Not uncommon in preclinical studies.
- With 100 independent tests, the probability of observing at least one false finding increases considerably to 99%. Not uncommon in omic studies.
- Therefore, the more tests you perform, the greater the risk of incorrectly concluding that an effect is significant.

Typical Situations That Generate Multiplicity
Multiple Endpoints
Example: A study evaluates a new drug to reduce mortality, but also measures quality of life, blood pressure, blood sugar, etc.
If each endpoint is tested separately without correction for multiplicity, the probability that at least one test will be significant by chance increases greatly.
Comparison of Multiple Treatment Groups
Example: Three doses of a drug compared to a placebo.
This involves several statistical tests: low vs. placebo, medium vs. placebo, high vs. placebo, etc.
Analyses at Multiple Time Points
Example: Efficacy is examined at 3 months, 6 months, and 12 months.
Each assessment is an additional statistical test.
Multiple Subgroups
Example: The effect of the treatment is examined in men, women, young people, the elderly, etc.
If the effect is tested in each subgroup, the number of tests multiplies.
Interim Analyses
In sequential trials, data can be reviewed before the end of the study to decide whether to continue or stop.
Each interim analysis increases the risk of prematurely reporting an effect.
Post-hoc Exploration or Unplanned Analyses
Example: After the primary results are obtained, other variables, interactions, or subsets are explored.
These unplanned tests are not controlled for the overall risk of error.
Multi-Omics or High-Throughput Analyses
In pharmacogenomics, transcriptomics, or imaging, thousands of genes/voxels can be tested.
How to Correct for Multiplicity?
- Plan primary and secondary analyses in advance.
- Group or prioritize outcome measures (hierarchy, sequential tests).
- Adjust p-values (Bonferroni, Holm, Hochberg, FDR, FWER, etc.).
- Interpret exploratory analyses with caution.
Tips
It is a good idea to consult a statistician about this, ideally when planning a study or experiment.
Multiplicity increases the risk of false positives. This must be recognized and corrected to avoid drawing misleading conclusions.
References
Menyhárt O, Győrffy B. Multiplicity corrections in life sciences: challenges and consequences. Int J Epidemiol. 2025 Jun 11;54(4):dyaf098. doi: 10.1093/ije/dyaf098. PMID: 40579911; PMCID: PMC12205177.
Wason, J.M.S., Stecher, L. & Mander, A.P. Correcting for multiple-testing in multi-arm trials: is it necessary and is it done?. Trials 15, 364 (2014). https://doi.org/10.1186/1745-6215-15-364
