Replications in Animal Experiments
An important point regarding animal studies is the use of “technical repetitions” also referred to as “pseudo replications”, “technical repeats” instead of true biological repetitions.
Technical repetitions refers to a specific type of repetition where the same experimental unit/sample is measured multiple times, aiming to probe the noise associated with the measurement method or the device. No matter how many times the same sample is measured, the actual sample size will remain the same.
Illustration
To illustrate this, let us assume that a research group is investigating the effect of a therapeutic drug on blood glucose level. If the researchers measure the blood glucose level of 3 mice receiving the actual treatment “treatment group” and 3 mice receiving placebo “control group”, this would correspond to true biological repetitions.
On the other hand, if the blood glucose level of a single mouse receiving the actual treatment “treatment group” and the blood glucose level of a single mouse receiving placebo “control group” are each measured 3 times, this would correspond to technical repetitions.
Both designs will provide 6 data points to calculate p-value, yet the p-value obtained from the second scenario would be meaningless since each treatment group will only have one member. Multiple measurements on single mice are pseudo-replications; therefore do not contribute to n. This actually would correspond to an unreplicated experiment from a scientific standpoint.
Consequences of Using Pseudo-Replications
The effect of using pseudo-replications as opposed to true biological replications, that is treating pseudo-replications as independent measures will lead to the following issues in the statistical analysis of the collected data:
- Inflation of the true sample size
- Smaller standard errors
- Results erroneously significant (smaller p-values than they should be)
- Increase in both type I (false positive results) and type II (false negative results) errors
This Issue Cannot be Fixed in the Data Analysis
💣 No matter how ingenious, no statistical analysis method can fix incorrectly selected replicates at the post-experimental stage; replicate types should be selected accurately at the design stage. This illustrates the importance of carefully designing experiments. Statistical methods MUST match the experimental design.
💣 This problem is a critical limitation, especially in pre-clinical studies that conduct cell culture experiments. It is also very important for critical assessment and evaluation of the published research results.
💣 This issue is mostly underestimated, concealed or ignored. It is striking that in some publications, the actual sample size is found to be as low as one. Several studies mistaken pseudo replications (technical repeats) for genuine replications, while several do not provide sufficient information to enable evaluation of appropriateness of the sample size.
💣 Experiments comparing drug treatments in a patient-derived stem cell line are specific examples for this situation. Although there may be many technical repeats for such experiments and the experiment can be repeated several times, the original patient is a single biological entity. Biological variation is completely ignored. The underlying assumption here is that the single patient is representative of the entire target population, which is unlikely to hold.
💣 Similarly, when six metatarsals are harvested from the front paws of a single mouse and cultured as six individual cultures, another pseudo-replication is practiced where the sample size is actually 1, instead of 6.
Useful References
- Leonhard Held and Simon Schwab. Improving the reproducibility of science. Significance. Volume 17, Issue 1. Feb 2020. Pages 10-11. https://doi.org/10.1111/j.1740-9713.2020.01351.x
- Serdar CC, Cihan M, Yücel D, Serdar MA. Sample size, power and effect size revisited: simplified and practical approaches in pre-clinical, clinical and laboratory studies. Biochem Med (Zagreb). 2021 Feb 15;31(1):010502. doi: 10.11613/BM.2021.010502. Epub 2020 Dec 15. PMID: 33380887; PMCID: PMC7745163.
- Vaux DL, Fidler F, Cumming G. Replicates and repeats–what is the difference and is it significant? A brief discussion of statistics and experimental design. EMBO Rep. 2012 Apr 2;13(4):291-6. doi: 10.1038/embor.2012.36. PMID: 22421999; PMCID: PMC3321166.
Training Sessions
To learn more on the design of experiments