The misuse and abuse of statistics in biomedical research
Open accessAbstract
Statistics are the primary tools for assessing relationships and evaluating study questions. Unfortunately, these tools are often misused, either inadvertently because of ignorance or lack of planning, or conspicuously to achieve a specified result. Data abuses include the incorrect application of statistical tests, lack of transparency and disclosure about decisions that are made, incomplete or incorrect multivariate model building, or exclusion of outliers. Individually, each of these actions may completely invalidate a study, and often studies are victim to more than one offense. Increasingly there are tools and guidance for researchers to look to, including the development of an analysis plan and a series of study specific checklists, in order to prevent or mitigate these offenses.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many published articles fail to discuss statistical assumptions, specify the exact statistics used in analysis, and employ vague terms when reporting statistical tests.
“On the other hand, many published articles fail to discuss statistical assumptions, specify the exact statistics used in analysis, and employ vague terms when reporting statistical tests.14”
From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Yaşlı Kadınlarda Üreme Sağlığı
Kokanalı, D., & Engin Üstün, Y. (n.d.) · DergiPark (Istanbul University) · Cited by 170,090 · see citing works
Herne, H., Cooley, W. W., & Lohnes, P. R. (1973) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344428 · Cited by 35,801 · see citing works
Two new Later Stone Age sites from the Final Pleistocene in the Falémé Valley, eastern Senegal
Ndiaye, M., Lespez, L., Tribolo, C., Rassé, M., Hadjas, I., Davidoux, S., Huysecom, É., & Douze, K. (2024) · Monash University Research Portal (Monash University) · doi:10.3929/ethz-b-000667478 · Cited by 24,015 · see citing works
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020 · see citing works
Hanrahan, G. (2008) · CRC Press eBooks · doi:10.1201/9781420067972-10 · Cited by 8,196 · see citing works
What's wrong with Bonferroni adjustments
Perneger, T. V. (1998) · BMJ · doi:10.1136/bmj.316.7139.1236 · Cited by 5,900 · see citing works
(1996) · Institutional Research Information System University of Turin (University of Turin) · doi:10.1111/j · Cited by 5,486 · see citing works
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373 · see citing works
Calamita, E., Schmid, M., Kunz, M., Ndebele-Murisa, M. R., Magadza, C. H. D., Nyambe, I., & Wehrli, B. (2019) · Figshare · doi:10.26181/6153922fe919c · Cited by 4,222 · see citing works
Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative.
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00010 · Cited by 3,219 · see citing works
Hawkins, D. M. (1980) · doi:10.1007/978-94-015-3994-4 · Cited by 2,982 · see citing works
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755 · see citing works
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · Preventive Medicine · doi:10.1016/j.ypmed.2007.08.012 · Cited by 2,698 · see citing works
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 2× · see citing works
“Inappropriate study design is often the first actions in research, and has been discussed in the literature (3-5).”
BackgroundSystematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · see citing works
Publication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542 · see citing works
Reducing bias through directed acyclic graphs
Shrier, I., & Platt, R. W. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-70 · Cited by 1,394 · see citing works
Sauerbrei, W., Royston, P. J., & Binder, H. (2007) · Statistics in Medicine · doi:10.1002/sim.3148 · Cited by 1,177 · see citing works
Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127 · see citing works
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124 · see citing works
Comparison of Stopping Rules in Forward “Stepwise” Regression
Bendel, R. B., & Afifi, A. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10479905 · Cited by 885 · see citing works
Derksen, S. A., & Keselman, H. J. (1992) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1992.tb00992.x · Cited by 767 · see citing works
Lang, T. A., & Altman, D. G. (2014) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2014.09.006 · Cited by 662 · see citing works
Observational and interventional study design types; an overview
Thiese, M. S. (2014) · Biochemia Medica · doi:10.11613/bm.2014.022 · Cited by 615 · see citing works
Self-citationStatistical Methods and Scientific Induction
Fisher, R. A. (1955) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1955.tb00180.x · Cited by 611 · see citing works
The interplay of Bayesian and frequentist analysis
Bayarri, M. J., & Berger, J. O. (2004) · Statistical Science · doi:10.1214/088342304000000116 · Cited by 525 · see citing works
Comparison of Stopping Rules in Forward "Stepwise" Regression
Bendel, R. B., & Afifi, A. A. (1977) · Journal of the American Statistical Association · doi:10.2307/2286904 · Cited by 356 · see citing works
Biostatistics: how to detect, correct and prevent errors in the medical literature.
Glantz, S. A. (1980) · Circulation · doi:10.1161/01.cir.61.1.1 · Cited by 292 · see citing works
Hawthorne effects and research into professional practice
Holden, J. D. (2001) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2001.00280.x · Cited by 263 · see citing works
Misuse of statistical methods: critical assessment of articles in BMJ from January to March 1976.
Gore, S. M., Jones, I., & Rytter, E. (1977) · BMJ · doi:10.1136/bmj.1.6053.85 · Cited by 238 · see citing works
Methods of Covariate Selection: Directed Acyclic Graphs and the Change-in-Estimate Procedure
Weng, H., Hsueh, Y.-H., Messam, L. L. M., & Hertz‐Picciotto, I. (2009) · American Journal of Epidemiology · doi:10.1093/aje/kwp035 · Cited by 223 · see citing works
Statistical Evaluation of Medical Journal Manuscripts
Schor, S. (1966) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1966.03100130097026 · Cited by 207 · see citing works
Bayesians, Frequentists, and Scientists
Efron, B. (2005) · Journal of the American Statistical Association · doi:10.1198/016214505000000033 · Cited by 201 · see citing works
Statistical errors in medical research - a review of common pitfalls
Strasak, A. M., Zaman, Q., Pfeiffer, K. P., Göbel, G., & Ulmer, H. (2007) · Swiss Medical Weekly · doi:10.4414/smw.2007.11587 · Cited by 162 · see citing works
What do doctors know about statistics?
Wulff, H. R., Andersen, B., Brandenhoff, P., & Güttler, F. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060103 · Cited by 157 · see citing works
Statistics and ethics in medical research
Altman, D. G. (1981) · BMJ · doi:10.1136/bmj.282.6268.990-a · Cited by 151 · see citing works
STRengthening Analytical Thinking for Observational Studies: the STRATOS initiative
Sauerbrei, W., Abrahamowicz, M., Altman, D. G., le Cessie, S., Carpenter, J. R., & initiative, O. B. O. T. S. (2014) · Statistics in Medicine · doi:10.1002/sim.6265 · Cited by 135 · see citing works
Statistical Errors in Papers in the British Journal of Psychiatry
White, S. J. (1979) · The British Journal of Psychiatry · doi:10.1192/bjp.135.4.336 · Cited by 134 · see citing works
Statistics and ethics in medical research. Misuse of statistics is unethical.
Altman, D. G. (1980) · BMJ · doi:10.1136/bmj.281.6249.1182 · Cited by 132 · see citing works
Models and Statistical Inference: The Controversy between Fisher and Neyman–Pearson
Lenhard, J. (2006) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axi152 · Cited by 121 · see citing works
Low power, type II errors, and other statistical problems in recent cardiovascular research
Williams, J. L., Hathaway, C. A., Kloster, K. L., & Layne, B. H. (1997) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.1997.273.1.h487 · Cited by 85 · see citing works
Performance of using multiple stepwise algorithms for variable selection
Wiegand, R. E. (2010) · Statistics in Medicine · doi:10.1002/sim.3943 · Cited by 79 · see citing works
XXV. A survey of the statistical procedures in general medical journals
Feinstein, A. R. (1974) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt197415197 · Cited by 66 · see citing works
Assessment of errors and misused statistics in dental research
Kim, J. S., Kim, D., & Hong, S. J. (2011) · International Dental Journal · doi:10.1111/j.1875-595x.2011.00037.x · Cited by 50 · see citing works
Misuse of statistical methods in arthritis and rheumatism
Felson, D. T., Cupples, L. A., & Meenan, R. F. (1984) · Arthritis & Rheumatism · doi:10.1002/art.1780270908 · Cited by 40 · see citing works
MISUSAGE OF STATISTICS IN MEDICAL RESEARCH
Ercan, İ., Yazıcı, B., Yang, Y., Özkaya, G., Cangür, Ş., Ediz, B., & Kan, İ. (2007) · Electronic Journal of General Medicine · doi:10.29333/ejgm/82507 · Cited by 34 · see citing works
Misleading Statistics: Errors in Textbooks, Software and Manuals
BLAND, J. M., & Altman, D. G. (1988) · International Journal of Epidemiology · doi:10.1093/ije/17.2.245 · Cited by 32 · see citing works
Gore, A., Kadam, Y., Chavan, P., & Dhumale, G. (2012) · International Journal of Applied and Basic Medical Research · doi:10.4103/2229-516x.96792 · Cited by 28 · see citing works
Statistics usage in the American Journal of Obstetrics and Gynecology: Has anything changed?
Welch, G. E., & Gabbe, S. G. (2002) · American Journal of Obstetrics and Gynecology · doi:10.1067/mob.2002.122144 · Cited by 22 · see citing works
Wald, A. (1948) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177730246 · Cited by 18 · see citing works
Statistical Tests in Medical Research
Jamart, J. (1992) · Acta Oncologica · doi:10.3109/02841869209083860 · Cited by 18 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Guidelines for experimental models of myocardial ischemia and infarction
Lindsey, M. L., Bolli, R., Canty, J. M., Du, X.-J., Frangogiannis, N. G., Frantz, S., Gourdie, R. G., Holmes, J. W., Jones, S. P., Kloner, R. A., et al. (2018) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00335.2017 · Cited by 526 · see citing works
Bustin, S., & Nolan, T. (2017) · European Journal of Clinical Investigation · doi:10.1111/eci.12801 · Cited by 113 · see citing works
Alnaimat, F., AlSamhori, A. R. F., El Sharu, H., Othman, L., Oralbek, A., & Zimba, O. (2025) · Journal of Korean Medical Science · doi:10.3346/jkms.2025.40.e342 · Cited by 4 · see citing works
“3 Common examples include 13,14 : • Errors in statistical design: inappropriate assumptions while selecting a statistical test.”
- Authors
- Thiese, M. S., Arnold, Z. C., & Walker, S. D.
- Affiliations
- University of Utah (US)
- Corresponding author
- Thiese, Matthew S.
- Venue
- Biochemia Medica
- Published
- 2015
- Volume
- 25
- Issue
- 1
- Pages
- 5-11
- ISSN
- 1330-0962 (ISSN-L)
- 1846-7482
- PMC
- PMC4401313
- OpenAlex
- W2089130219
- Cited by
- 87
- References
- 51
- Citations by year
- 2026: 5 · 2025: 15 · 2024: 6 · 2023: 7 · 2022: 8 · 2021: 9 · 2020: 5 · 2019: 14 · 2018: 6 · 2017: 5 · 2016: 2 · 2015: 3
- FWCI
- 0.90
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Biochemia Medica (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Hrčak Portal of scientific journals of Croatia (University Computing Centre) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Keywords
- misuse of statistics, biomedical research, outlier exclusion, analysis plan, research integrity
- MeSH terms
- Biomedical Research, Biostatistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
