Simple cheat sheets are popular, but many statisticians consider them misleading or even erroneous. They often oversimplify statistical reasoning and can lead to incorrect analyses.

They reduce statistical methods to a oversimplistic decision tree

Most cheat sheets look like this:
“Two groups → use a t-test
More than two groups → use ANOVA
Categorical variables → use chi-square”
Real life problems are rarely that simple. Important factors are ignored, such as:
– Study design
– Distribution of the data
– Underlying assumptions
– Missing data
– Clustering or repeated measures
– Interactions
– Covariates
– Etc.
Two studies with the same “two groups” structure might require completely different models.

They ignore the study design

The design of the experiment usually determines the correct analysis more than the variable types.
For example:
– Repeated measurements on the same subject → paired test or mixed model
– Clustered data (schools, hospitals, families) → hierarchical models
– Longitudinal data → time-series or mixed models
Cheat sheets usually only classify:
number of variables
variable type but not the dependency structure, which is critical.

They reinforce “test shopping”

Cheat sheets encourage people to ask:
“Which test should I run?” instead of the better question “What model describes my data-generating process?” Modern statistics treats most classical tests as special cases of statistical models.
Examples:
t-test → general linear model (special case of ANOVA)
ANOVA → general linear model with categorical predictors
logistic regression → generalised linear model
Thinking in terms of models is much more flexible and correct (allows accounting for covariates, interactions, polynomial effects, etc.)

They hide assumptions

Every statistical test has assumptions, such as:
independence, normality, equal variances, linearity, correct link function, etc.
Cheat sheets rarely emphasise checking these assumptions. They imply:
If your data fits the boxes → the test is valid. But violating assumptions can invalidate results.

They promote outdated practice

Some cheat sheets recommend things many statisticians now discourage, such as normality testing to decide between parametric/nonparametric tests
– default reliance on p-values
– using nonparametric tests instead of modeling
Modern approaches often prefer:regression frameworks, mixed models, Bayesian models, resampling methods

They may change the question one tries to address

Simple statistical-test cheat sheets are problematic because they


– Oversimplify data analysis
– Ignore study design
– Randomisation restriction
– Hide underlying assumptions
– Encourage test shopping
– Obscure the fact that many tests are just regression models
– Do not promote statistical thinking