ChatGPT as a Tool for Biostatisticians: A Tutorial on Applications, Opportunities, and Limitations
Open accessAbstract
Modern large language models (LLMs) have reshaped the workflows of people across countless fields-and biostatistics is no exception. These models offer novel support in drafting study plans, generating software code, or writing reports. However, reliance on LLMs carries the risk of inaccuracies due to potential hallucinations that may produce fabricated "facts", leading to erroneous statistical statements and conclusions. Such errors could compromise the high precision and transparency fundamental to our field. This tutorial aims to illustrate the impact of LLM-based applications on various contemporary biostatistical tasks. We will explore both the risks and opportunities presented by this new era of artificial intelligence. Our ultimate conclusion emphasizes that advanced applications should only be used in combination with sufficient background knowledge. Over time, consistently verifying LLM outputs may lead to an appropriately calibrated trust in these tools among users.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 37 · Cited by 20
Research works printed but not linked: 16
- [1]M. F. Rabbi, A. I. Champa, M. F. Zibran, and M. R. Islam, “ AI Writes, We Analyze: The ChatGPT Python Code Saga,” in Proceedings of the 21st International Conference on Mining Software Repositories; International Conference on Software Engineering in Lisbon (ACM Digital Library, 2024), 177– 181.
- [3]A. J. Titus, “NHANES‐GPT: Large Language Models (LLMs) and the Future of Biostatistics,” Medrxiv. 2023:2023–12.
- [4]R. Komandur, J. McDunn, N. Nair, B. Fall, A. P. Dicker, and S. Khozin, “Artificial Intelligence in Biomedical Data Analysis: A Comparative Assessment of Large Language Models for Automated Clinical Trial Interpretation and Statistical Evaluation,” Medrxiv. 2025:2025–02.
- [5]Z. Wang, B. Danek, Z. Yang, Z. Chen, and J. Sun, “Can Large Language Models Replace Data Scientists in Clinical Research?” arXiv Preprint arXiv:2410.21591. 2024. · arXiv
- [16]S. Friedrich, G. Antes, S. Behr, et al., “ Is There a Role for Statistics in Artificial Intelligence?,” Advances in Data Analysis and Classification 16, no. 4 (2022): 823– 846.
- [17]A. Vaswani, N. Shazeer, N. Parmar, et al., “ Attention Is All You Need,” Advances in Neural Information Processing Systems 30 (2017): 1– 11.
- [19]A. B. Haidich, “ Meta‐Analysis in Medical Research,” Hippokratia 14, no. Suppl 1 (2010): 29– 37. 21487488 PMC3049418
- [20]M. Borenstein, L. V. Hedges, J. P. T. Higgins, and H. R. Rothstein, Introduction to Meta‐Analysis (John Wiley & Sons, 2009).
- [21]C. H. Schmid, T. Stijnen, and I. White, Handbook of Meta‐Analysis (Chapman and Hall/CRC Press, 2021).
- [32]J. J. Deeks, J. P. T. Higgins, D. G. Altman, J. E. McKenzie, and A. A. Veroniki, “ Chapter 10: Analysing Data and Undertaking Meta‐Analyses,” in Cochrane Handbook for Systematic Reviews of Interventions Version 6.5, ed. J. P. T. Higgins, J. Thomas, J. Chandler, et al. (Cochrane, 2024).
- [33]W. Viechtbauer, “ Conducting Meta‐Analyses in R With the Metafor Package,” Journal of Statistical Software 36, no. 3 (2010): 1– 48.
- [35]A. Veroniki, D. Jackson, W. Viechtbauer, et al., “ Recommendations for Quantifying the Uncertainty in the Summary Intervention Effect and Estimating the Between‐Study Heterogeneity Variance in Random‐Effects Meta‐Analysis,” Cochrane Database of Systematic Reviews no. Suppl 1 (2015): 25– 27.
- [39]M. Ikeda, S. Okada, K. Tokuuye, H. Ueno, and T. Okusaka, “ Prognostic Factors in Patients With Locally Advanced Pancreatic Carcinoma Receiving Chemoradiotherapy,” Cancer 91, no. 3 (2001): 490– 495, 10.1002/1097-0142(20010201)91:3¡490::AID-CNCR1027¿3.0.CO;2-L. 11169931 · DOI
- [53]M. O. S. N. Wolberg and W. Street, “Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository,” 1993, 10.24432/C5DW2B. · DOI
- [54]W. N. Venables and B. D. Ripley, Modern Applied Statistics With S, 4th ed. (Springer, 2002).
- [57]A. Woodruff, R. Shelby, P. G. Kelley, S. Rousso‐Schindler, J. Smith‐Loud, and L. Wilcox, “ How Knowledge Workers Think Generative Ai Will (Not) Transform Their Industries,” Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (2024): 1– 26.
- Authors
- Dobler, D., Binder, H., Boulesteix, A., Igelmann, J., Köhler, D., Mansmann, U., Pauly, M., Scherag, A., Schmid, M., Al Tawil, A., & Weber, S.
- Affiliations
- RWTH Aachen University (DE)
- University of Freiburg (DE)
- Munich Center for Machine Learning (DE)
- TU Dortmund University (DE)
- University Hospital Bonn (DE)
- Institut für Medizinische Informationsverarbeitung, Biometrie und Epidemiologie (DE)
- Jena University Hospital (DE)
- Institute for Medical Information Processing, Biometry, and Epidemiology LMU Munich Munich Germany
- Corresponding author
- Dobler, Dennis
- ORCID
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2025-10
- Volume
- 44
- Issue
- 23-24
- Pages
- e70263-e70263
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- PMC
- PMC12548020
- OpenAlex
- W4415468579
- Cited by
- 20
- References
- 37
- Citations by year
- 2026: 20
- FWCI
- 4.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Statistics in Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (acceptedVersion)
- FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
- RWTH Publications (RWTH Aachen) (submittedVersion, other-oa)
- Common Library Network (Der Gemeinsame Bibliotheksverbund) (submittedVersion, cc-by)
- Funders
- Deutsche Forschungsgemeinschaft (413270747, 499552394, SFB 1597)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- ChatGPT, large language models, biostatistics, LLM hallucination, report writing, artificial intelligence in research
- MeSH terms
- Generative Artificial Intelligence, Artificial Intelligence, Data Interpretation, Statistical, Humans, Language, Software, Models, Statistical, Biostatistics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
