Statistical Terminology for ANOVA (lm), glm, glmm & Beyond

I would like to propose alternatives to what is unfortunately very common in scientific papers “An ANOVA was done/…” Hopefully, the terminology below should help investigators improving the reporting of statistical modelling in R&D. This goes beyond ANOVA.

Terminology

• ANOVA (Analysis of Variance) examines the effect of one or more categorical independent variables, known as ‘factors’, on a dependent variable. Examples of factors might include age (young vs. old), hypertension (hypertensive vs. normotensive) or time since drug administration (baseline, 1 hour, 6 hours, 12 hours, and 24 hours).
• The groups or conditions within each factor are called levels. In the example above, age has two levels (young vs. old), hypertension has two levels, and time since drug administration has five levels.
• Each factor can be entered into the ANOVA as a between-subjects factor or a within-subjects factor. Measurements on unrelated participants, specimens or samples (i.e., age, hypertension) are entered into the ANOVA as between-subjects factors. Longitudinal measurements (i.e., time since drug administration) are entered into the analysis as within-subjects factors.
• Other types of non-independent measurements may also be entered as within-subjects factors. This includes related measurements that are performed in the same participants (i.e., arm vs. leg), or groups in which subjects are matched (i.e., normotensive and hypertensive participants matched for sex and age).
• An ANOVA that includes at least one within-subjects factor is called repeated measures ANOVA.
• A repeated-measures ANOVA with two or more factors may include between-subjects factors, depending on the study design. For example, a two-way repeated measures ANOVA with one between and one within-subjects factor might be used for a study in which men and women completed four weeks of exercise training. Sex (female vs. male) would be a between-subjects factor, whereas exercise training (baseline vs. post-training) would be a within-subjects factor. A two-way repeated measures ANOVA with two within-subjects factors might be used for a study in which vascular function was measured in both arms (right vs. left), before and after 4 weeks of right arm handgrip training (baseline vs. post-training).
• Post-hoc tests are used to determine which groups in the ANOVA differ from each other. These tests are used when p-values for the ANOVA show that there is a significant effect of any factor, or a significant interaction between two or more factors. There are many different types of post-hoc tests. These tests use different procedures to determine which groups to compare and adjust for multiple comparisons; some are more conservative than others.

Reference

Tracey L Weissgerber, Oscar Garcia-Valencia, Vesna D Garovic, Natasa M Milic, Stacey J Winham (2018) Meta-Research: Why we need to report more than ‘Data were Analyzed by t-tests or ANOVA’ eLife 7:e36163