A Comparative Evaluation of Statistical Product and Service Solutions (SPSS) and ChatGPT-4 in Statistical Analyses
Open accessAbstract
BACKGROUND: The objective of this study was to assess the accuracy of Chat Generative Pre-trained Transformer 4.0 (ChatGPT-4; OpenAI, San Francisco, CA ) compared to Statistical Product and Service Solutions (SPSS; IBM SPSS Statistics for Windows, Armonk, NY) in performing statistical analyses commonly used in medical and dental research. METHODS: The datasets were analysed using SPSS (version 26) and ChatGPT-4. Statistical tests included the independent t-test, paired t-test, ANOVA, chi-square test, Wilcoxon signed-rank test, Mann-Whitney U test, Pearson and Spearman correlation, regression analysis, kappa statistic, intraclass correlation coefficient (ICC), Bland-Altman analysis, and sensitivity and specificity analysis. Descriptive statistics were used to report results, and differences between the two tools were noted. RESULTS: SPSS and ChatGPT-4 produced identical results for the independent sample t-test, paired t-test, and simple linear regression. In one-way ANOVA, both tools provided consistent F-values, but post-hoc analysis revealed discrepancies in mean differences and confidence intervals. Pearson chi-square and Wilcoxon signed-rank tests showed variations in p-values and Z-values. Mann-Whitney U test had differences in interquartile range (IQR), U, and Z-values. Pearson and Spearman's correlations were consistent, with IQR differences in Spearman. Sensitivity, specificity, and area under the curve (AUC) analyses were consistent, though differences in standard errors and confidence intervals were observed. CONCLUSION: ChatGPT-4 produced accurate results for several statistical tests, matching SPSS in simpler analyses. However, discrepancies in post-hoc analyses, confidence intervals, and more complex tests indicate that careful validation is required when using ChatGPT-4 for detailed statistical work. Researchers should exercise caution and cross-validate results with established tools such as SPSS.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI helps non-experts analyze data, but it can be unpredictable for experts dealing with complex data analysis.
“AI helps non-experts analyze data, but it can be unpredictable for experts dealing with complex data analysis.”
From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [23]
When comparing AI to current statistical programs, AI produces similar results in simple analyses such as the sample t-test, paired t-test, simple linear regression, and Pearson and Spearman's correlations.
“When comparing AI to the current statistical programs, AI produces similar results in simple analyses such as the sample t-test, paired t-test, simple linear regression, and Pearson and Spearman’s correlations.”
From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [23]
When challenged with complex analyses, AI performance varies.
“When challenged with complex analyses, AI performance varies.”
From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [23]
Discrepancies between AI and current statistical programs were noted in post-hoc analyses, confidence intervals, and more complex tests.
“However, discrepancies in post-hoc analyses, confidence intervals, and more complex tests were noted.23”
From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430 · see citing works
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 29,639 · see citing works
Interrater reliability: The kappa statistic
McHugh, M. L. (2012) · Biochemia Medica · doi:10.11613/bm.2012.031 · Cited by 19,877 · see citing works
Correlation Coefficients: Appropriate Use and Interpretation
Schober, P., Boer, C., & Schwarte, L. A. (2018) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000002864 · Cited by 10,986 · see citing works
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.64134 · Cited by 10,460 · see citing works
Understanding bland altman analysis
Giavarina, D. (2015) · Biochemia Medica · doi:10.11613/bm.2015.015 · Cited by 4,022 · see citing works
The chi-square test of independence
McHugh, M. L. (2013) · Biochemia Medica · doi:10.11613/bm.2013.018 · Cited by 2,828 · see citing works
Ethical Considerations of Using ChatGPT in Health Care
Wang, C., Liu, S., Yang, H., Guo, J., Wu, Y., & Liu, J. (2023) · Journal of Medical Internet Research · doi:10.2196/48009 · Cited by 541 · see citing works
ChatGPT in healthcare: A taxonomy and systematic review
Li, J., Dada, A., Puladi, B., Kleesiek, J., & Egger, J. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108013 · Cited by 452 · see citing works
Hassani, H., & Silva, E. S. (2023) · Big Data and Cognitive Computing · doi:10.3390/bdcc7020062 · Cited by 401 · see citing works
Májovský, M., Černý, M., Kasal, M., Komarc, M., & Netuka, D. (2023) · Journal of Medical Internet Research · doi:10.2196/46924 · Cited by 301 · see citing works
ChatGPT: these are not hallucinations – they’re fabrications and falsifications
Emsley, R. (2023) · Schizophrenia · doi:10.1038/s41537-023-00379-4 · Cited by 258 · see citing works
khan, Z. F., & Alotaibi, S. R. (2020) · Journal of Healthcare Engineering · doi:10.1155/2020/8894694 · Cited by 235 · see citing works
Statistics review 1: presenting and summarising data.
Whitley, E., & Ball, J. (2002) · Critical Care · doi:10.1186/cc1455 · Cited by 129 · see citing works
Huang, Y., Wu, R., He, J., & Xiang, Y. (2024) · Journal of Global Health · doi:10.7189/jogh.14.04070 · Cited by 72 · see citing works
ChatGPT generates fake data set to support scientific hypothesis
Naddaf, M. (2023) · Nature · doi:10.1038/d41586-023-03635-w · Cited by 46 · see citing works
Ignjatović, A., & Stevanović, L. (2023) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2023.20.28 · Cited by 38 · see citing works
Exploring the use of ChatGPT in learning and instructing statistics and data analytics
Xing, Y. (2024) · Teaching Statistics · doi:10.1111/test.12367 · Cited by 34 · see citing works
Ordak, M. (2023) · Healthcare · doi:10.3390/healthcare11182554 · Cited by 22 · see citing works
Assessing the ChatGPT Accuracy Through Principles of Statistics Exam: A Performance and ImplicationsPreprint
AL-Qadri, A. H., & Ahmed, S. A. M. (2023) · Research Square · doi:10.21203/rs.3.rs-2673838/v1 · Cited by 6 · see citing works
Are ChatGPT and Copilot Reliable for Health Education on Statistical Testing?Preprint
Rovetta, A., & Mansournia, M. A. (2024) · medRxiv · doi:10.1101/2024.03.08.24304007 · Cited by 2 · see citing works
Statistics every orthodontist should know
Pandis, N. (2024) · Seminars in Orthodontics · doi:10.1053/j.sodo.2024.01.013 · Cited by 2 · see citing works
Assessing the effectiveness of ChatGPT in generating Python code
Adamson, V., & Bägerfeldt, J. (2023) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [8]SPSS Survival Manual: A Step by Step Guide to Data Analysis Using SPSS 4th Edition Pallant J Berkshire, UK Open University Press 2010 https://www.amazon.com/SPSS-Survival-Manual-Julie-Pallant/dp/0335242391
Cited by
Sorted by citations
Kabanda, M. (2025) · EIKI Journal of Effective Teaching Methods · doi:10.59652/jetm.v3i1.404 · Cited by 23 · see citing works
Wang, Q., & Xu, M. (2025) · Biological Research For Nursing · doi:10.1177/10998004251314017 · Cited by 4 · see citing works
Chang, L., Huang, C., Yu, H., Hung, L., Lin, H., Liao, L., & Lai, I. (2025) · International Nursing Review · doi:10.1111/inr.70076 · Cited by 4 · see citing works
Alnaimat, F., AlSamhori, A. R. F., El Sharu, H., Othman, L., Oralbek, A., & Zimba, O. (2025) · Journal of Korean Medical Science · doi:10.3346/jkms.2025.40.e342 · Cited by 4 · see citing works
“10,11,21,23 23 As noted, their performance is inconsistent on complex analyses, they risk “hallucinating” or misinterpreting information, and their “black-box” nature makes their reasoning difficult to verify.”
Confirming SPSS Results With ChatGPT-4 and o3-mini Models
Strale, F., Riddle, I., Geng, B., Oxford, B., Kah, M., & Sherwin, R. L. (2025) · Cureus · doi:10.7759/cureus.82005 · Cited by 2 · see citing works
Kumar, A. R., Saha, S. K., & Choudhary, M. (2025) · Newborn · doi:10.5005/jp-journals-11002-0134 · Cited by 2 · see citing works
Strale, F., German, R. M., Mendes-Kramer, V., Hammer, S. B., Ehrman, R. R., & Sherwin, R. L. (2026) · Cureus · doi:10.7759/cureus.111207 · Cited by 1 · see citing works
Bağcı, B., Ramazanoğlu, E., Maden, Ç., & Gündoğdu, N. (2025) · Egyptian Journal of Bronchology · doi:10.1186/s43168-025-00469-0
Hossain, M. M. M., Rahman, M., Hasan, M. M., Suhag, N. I., Sen, B. K., Ghosh, R. R., Ferdous, M. J., Lopa, N. J., Jeba, J. T., Alarfaj, A. A., et al. (2026) · Environmental DNA · doi:10.1002/edn3.70317
Monteiro, F. H., & de Sousa, R. R. (2026) · RECIMA21 - Revista Científica Multidisciplinar · doi:10.47820/recima21.v7i7.8395
- Authors
- Shahrul, A. I., & Syed Mohamed, A. M. F.
- Affiliations
- National University of Malaysia (MY)
- Corresponding author
- Shahrul, Al Imran
- ORCID
- Shahrul, Al Imran
- Venue
- Cureus
- Published
- 2024-10-28
- Volume
- 16
- Issue
- 10
- Pages
- e72581-e72581
- ISSN
- 2168-8184 (ISSN-L)
- PMC
- PMC11602406
- OpenAlex
- W4403821593
- Cited by
- 9
- References
- 23
- Citations by year
- 2026: 3 · 2025: 6
- FWCI
- 0.84
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Cureus (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Mathematics › Statistics and Probability › Probabilistic Statistics in Medicine
- Keywords
- ChatGPT-4, SPSS, large language models, automated statistical analysis, confidence intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
