ChatGPT for Univariate Statistics: Validation of AI-Assisted Data Analysis in Healthcare Research
Open accessAbstract
Background: ChatGPT, a conversational artificial intelligence developed by OpenAI, has rapidly become an invaluable tool for researchers. With the recent integration of Python code interpretation into the ChatGPT environment, there has been a significant increase in the potential utility of ChatGPT as a research tool, particularly in terms of data analysis applications. Objective: This study aimed to assess ChatGPT as a data analysis tool and provide researchers with a framework for applying ChatGPT to data management tasks, descriptive statistics, and inferential statistics. Methods: A subset of the National Inpatient Sample was extracted. Data analysis trials were divided into data processing, categorization, and tabulation, as well as descriptive and inferential statistics. For data processing, categorization, and tabulation assessments, ChatGPT was prompted to reclassify variables, subset variables, and present data, respectively. Descriptive statistics assessments included mean, SD, median, and IQR calculations. Inferential statistics assessments were conducted at varying levels of prompt specificity (“Basic,” “Intermediate,” and “Advanced”). Specific tests included chi-square, Pearson correlation, independent 2-sample t test, 1-way ANOVA, Fisher exact, Spearman correlation, Mann-Whitney U test, and Kruskal-Wallis H test. Outcomes from consecutive prompt-based trials were assessed against expected statistical values calculated in Python (Python Software Foundation), SAS (SAS Institute), and RStudio (Posit PBC). Results: ChatGPT accurately performed data processing, categorization, and tabulation across all trials. For descriptive statistics, it provided accurate means, SDs, medians, and IQRs across all trials. Inferential statistics accuracy against expected statistical values varied with prompt specificity: 32.5% accuracy for “Basic” prompts, 81.3% for “Intermediate” prompts, and 92.5% for “Advanced” prompts. Conclusions: ChatGPT shows promise as a tool for exploratory data analysis, particularly for researchers with some statistical knowledge and limited programming expertise. However, its application requires careful prompt construction and human oversight to ensure accuracy. As a supplementary tool, ChatGPT can enhance data analysis efficiency and broaden research accessibility.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Users can describe the structure of their dataset, specify normalization, request plots, or outline statistical comparisons, and the model can generate code in Python, R, or another preferred language.
“Users can describe the structure of their dataset, specify the desired normalization strategy, request plots, or outline statistical comparisons, and the model can generate code in Python, R, or another preferred language 22 .”
From Reimagining biomedical science workflows in the age of large language models (2026) · cites this paper as [22]
The Methods > Inferential Statistics section (page unavailable) shows users describing variables and research questions and specifying statistical tasks. The Introduction (page unavailable) discusses language-to-code translation, but the paper does not establish the full workflow claimed: specifying normalization, requesting plots, and generating code in Python, R, or any preferred language. Its use of Python, SAS, and RStudio in Methods > Response Grading was for comparison outputs, not evidence of user-selected code generation. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 19 · Cited by 24
Research works printed but not linked: 16
- [1]Bhargava DC Jadav D Meshram VP Kanchan T ChatGPT in medical research: challenging time ahead Med Leg J 2023 91 4 223 225 10.1177/00258172231184548 37802491 37802491 · DOI
- [2]Gödde D Nöhl S Wolf C Rupert Y Rimkus L Ehlers J Breuckmann F Sellmann T A SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis of ChatGPT in the medical literature: concise review J Med Internet Res 2023 25 e49368 10.2196/49368 37865883 v25i1e49368 37865883 PMC10690535 · DOI
- [3]Khlaif ZN Mousa A Hattab MK Itmazi J Hassan AA Sanmugam M Ayyoub A The potential and concerns of using AI in scientific research: ChatGPT performance evaluation JMIR Med Educ 2023 9 e47049 10.2196/47049 37707884 v9i1e47049 37707884 PMC10636627 · DOI · Paper page
- [6]Piccolo SR Denny P Luxton-Reilly A Payne SH Ridge PG Evaluating a large language model's ability to solve programming exercises from an introductory bioinformatics course PLoS Comput Biol 2023 19 9 e1011511 10.1371/journal.pcbi.1011511 37769024 PCOMPBIOL-D-23-00520 37769024 PMC10564134 · DOI
- [7]Wang L Ge X Liu L Hu G Code interpreter for bioinformatics: are we there yet? Ann Biomed Eng 2024 52 4 754 756 10.1007/s10439-023-03324-9 37482573 10.1007/s10439-023-03324-9 37482573 · DOI
- [8]Perkel JM Six tips for better coding with ChatGPT Nature 2023 618 7964 422 423 10.1038/d41586-023-01833-0 37277596 10.1038/d41586-023-01833-0 37277596 · DOI
- [9]Shue E Liu L Li B Feng Z Li X Hu G Empowering beginners in bioinformatics with ChatGPT Quant Biol 2023 11 2 105 108 10.15302/j-qb-023-0327 37378043 37378043 PMC10299548 · DOI
- [10]Xu D ChatGPT opens a new door for bioinformatics Quant Biol 2023 11 2 204 206 10.15302/j-qb-023-0328 37900935 37900935 PMC10609615 · DOI
- [12]Ignjatović A Stevanović L Efficacy and limitations of ChatGPT as a biostatistical problem-solving tool in medical education in serbia: a descriptive study J Educ Eval Health Prof 2023 20 28 10.3352/jeehp.2023.20.28 37840252 jeehp.2023.20.28 37840252 PMC10646144 · DOI
- [13]Jahangiri Y Can Chat Generative Pretraining Transformer (ChatGPT) be used for statistical analysis of research data? J Vasc Interv Radiol 2023 34 12 2242 2246.e2 10.1016/j.jvir.2023.09.010 37717655 S1051-0443(23)00669-3 37717655 · DOI
- [14]Ordak M ChatGPT's skills in statistical analysis using the example of allergology: do we have reason for concern? Healthcare (Basel) 2023 11 18 2554 10.3390/healthcare11182554 37761751 healthcare11182554 37761751 PMC10530997 · DOI
- [15]Ruksakulpiwat S Phianhasin L Benjasirisan C Ding K Ajibade A Kumar A Stewart C Assessing the efficacy of ChatGPT versus human researchers in identifying relevant studies on mHealth interventions for improving medication adherence in patients with ischemic stroke when conducting systematic reviews: comparative analysis JMIR mHealth uHealth 2024 12 e51526 10.2196/51526 38710069 v12i1e51526 38710069 PMC11106699 · DOI · Paper page
- [16]Zhu L Mou W Hong C Yang T Lai Y Qi C Lin A Zhang J Luo P The evaluation of generative AI should include repetition to assess stability JMIR mHealth uHealth 2024 12 e57978 10.2196/57978 38688841 v12i1e57978 38688841 PMC11106698 · DOI · Paper page
- [17]Miao H Li C Wang J A future of smarter digital health empowered by generative pretrained transformer J Med Internet Res 2023 25 e49963 10.2196/49963 37751243 v25i1e49963 37751243 PMC10565615 · DOI
- [18]Magalhaes Araujo S Cruz-Correia R Incorporating ChatGPT in medical informatics education: mixed methods study on student perceptions and experiential integration proposals JMIR Med Educ 2024 10 e51151 10.2196/51151 38506920 v10i1e51151 38506920 PMC10993110 · DOI · Paper page
- [19]Wang J Ye Q Liu L Guo NL Hu G Scientific figures interpreted by ChatGPT: strengths in plot recognition and limits in color perception NPJ Precis Oncol 2024 8 1 84 10.1038/s41698-024-00576-z 38580746 10.1038/s41698-024-00576-z 38580746 PMC10997760 · DOI
- Authors
- Ruta, M. R., Gaidici, T., Irwin, C., & Lifshitz, J.
- Affiliations
- University of Arizona College of Medicine- Phoenix (US)
- Corresponding author
- Ruta, Michael R
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2025-02-07
- Volume
- 27
- Pages
- e63550
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11845875
- OpenAlex
- W4407241386
- Cited by
- 24
- References
- 19
- Citations by year
- 2026: 14 · 2025: 10
- FWCI
- 5.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- ChatGPT, large language models, AI-assisted data analysis, univariate statistics, descriptive statistics, statistical inference, prompt specificity, National Inpatient Sample, exploratory data analysis, health care research
- MeSH terms
- Data Analysis, Generative Artificial Intelligence, Artificial Intelligence, Health Services Research, Humans
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
