Definition
- “The winner’s curse refers to the phenomenon that studies that find evidence of an effect often provide inflated estimates of the size of that effect.
- Such inflation is expected when an effect has to pass a certain threshold — such as reaching statistical significance — in order for it to have been ‘discovered’.
- Effect inflation is worst for small, low-powered studies, which can only detect effects that happen to be large.
- If, for example, the true effect is medium-sized, only those small studies that, by chance, estimate the effect to be large will pass the threshold for discovery (that is, the threshold for statistical significance, which is typically set at p < 0.05).
Consequences
- In practice, this means that research findings of small studies are biased in favour of inflated effects.
- By contrast, large, high-powered studies can readily detect both small and large effects and so are less biased, as both over- and underestimations of the true effect size will pass the threshold for ‘discovery’.
Illustration
The figure shows simulations of the winner’s curse (expressed on the y‑axis as relative bias of research findings). These simulations suggest that initial effect estimates from studies powered between ~ 8% and ~31% are likely to be inflated by 25% to 50% (shown by the arrows in the figure). Inflated effect estimates make it difficult to determine an adequate sample size for replication studies, increasing the probability of type II errors.”
@ See Button, K., Ioannidis, J., Mokrysz, C. et al.
Reference
- Button, K., Ioannidis, J., Mokrysz, C. et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 14, 365–376 (2013). https://lnkd.in/eex83shH
- Seth Lindstromberg. The winner’s curse and related perils of low statistical power − spelled out and illustrated. Research Methods in Applied Linguistics, Volume 2, Issue 3, 2023. https://doi.org/10.1016/j.rmal.2023.100059.