The issues discussed in this paper are clearly not only prevalent in neuroscience… I know that this is a 2013 paper and hopefully the situation has improved, but allow me to seriously doubt it. Nevertheless, I decided to post it because of the general nature of the problems and also because this is an easy to read paper.
Facts
- A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also reduces the likelihood that a statistically significant result reflects a true effect.
- Authors of this paper show that the average statistical power of studies in the neurosciences is very low.
- As a consequence:
- Overestimation of reported effect size
- Low reproducibility of results
- Unreliable, inefficient and wasteful research
- Ethical issues
Improving reproducibility in science is a key priority and requires attention to well-established but often ignored methodological principles.
@See reference
Conclusions
- In order to work around these issues, statistical thinking must be integrated into the planning of studies/trials/experiments.
- Research scientists, reviewers must be trained in statistics.
- Changes should be embraced to make the scientific method robust again
- Requiring ≥ 80% power results in more reliable results and replicable experiments; blinded studies, determine minimal detectable difference and biological relevance in advance
- Eliminate positivity bias – negative results need to be published
- Embrace a more collaborative scientific model
- The current system is in ‘over-drive’, publish fewer, but more transparent, robust and reproducible studies
Reference
Button, K., Ioannidis, J., Mokrysz, C. et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 14, 365–376 (2013). https://doi.org/10.1038/nrn3475