The DOI matches its Crossref record; 9 of the 10 references it cites were found in a registry or an index.
Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis
Open accessAbstract
We reflect on our experiences of using Generative Pre-trained Transformer ChatGPT, a chatbot launched by OpenAI in November 2022, to draft a research article. We aim to demonstrate how ChatGPT could help researchers to accelerate drafting their papers. We created a simulated data set of 100 000 health care workers with varying ages, Body Mass Index (BMI), and risk profiles. Simulation data allow analysts to test statistical analysis techniques, such as machine-learning based approaches, without compromising patient privacy. Infections were simulated with a randomized probability of hospitalisation. A subset of these fictitious people was vaccinated with a fictional vaccine that reduced this probability of hospitalisation after infection. We then used ChatGPT to help us decide how to handle the simulated data in order to determine vaccine effectiveness and draft a related research paper. AI-based language models in data analysis and scientific writing are an area of growing interest, and this exemplar analysis aims to contribute to the understanding of how ChatGPT can be used to facilitate these tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“ChatGPT’s GPT-3.5 iteration has already shown promise in several routine medical tasks and medical research [4-7], even raising ethical concerns in the literature [2,3,8].”
From Comparison of the Performance of GPT-3.5 and GPT-4 With That of Medical Students on the Written German Medical Licensing Examination: Observational Study (2024) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, utilizing GAI tools introduces the risk of plagiarism, when text is duplicated from data sources.12 33”
From Publishers’ and journals’ instructions to authors on use of generative artificial intelligence in academic and scientific publishing: Bibliometric analysis (2024) · cites this paper as [12]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Abstracts written by ChatGPT fool scientists
Else, H. (2023) · Nature · doi:10.1038/d41586-023-00056-7 · Cited by 611
Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse?
O’Connor, S., & ChatGPT (2023) · Nurse Education in Practice · doi:10.1016/j.nepr.2022.103537 · Cited by 585
AI bot ChatGPT writes smart essays — should professors worry?
Stokel-Walker, C. (2022) · Nature · doi:10.1038/d41586-022-04397-7 · Cited by 543
Huh, S. (2023) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2023.20.1 · Cited by 278
Rapamycin in the context of Pascal's Wager: generative pre-trained transformer perspective
Generative Pre-trained Transformer, C., & Zhavoronkov, A. (2022) · Oncoscience · doi:10.18632/oncoscience.571 · Cited by 203
Are ChatGPT and AlphaCode going to replace programmers?
Castelvecchi, D. (2022) · Nature · doi:10.1038/d41586-022-04383-z · Cited by 138
Daily briefing: will ChatGPT kill the essay assignment?
Graham, F. (2022) · Nature · doi:10.1038/d41586-022-04437-2 · Cited by 79
Zielinski, C., Winker, M. A., Aggarwal, R., Ferris, L. E., Heinemann, M., Lapeña, J. F. F., Pai, S. A., Ing, E. B., & Citrome, L. (2023) · The National Medical Journal of India · doi:10.25259/nmji_365_23 · Cited by 73
JoGH policy on the use of artificial intelligence in scholarly manuscripts
Marušić, A. (2023) · Journal of Global Health · doi:10.7189/jogh.13.01002 · Cited by 9
Cited by
Sorted by citations
Ray, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463
ChatGPT and Open-AI Models: A Preliminary Review
Roumeliotis, K. I., & Tselikas, N. D. (2023) · Future Internet · doi:10.3390/fi15060192 · Cited by 877
Haleem, A., Javaid, M., & Singh, R. P. (2022) · BenchCouncil Transactions on Benchmarks, Standards and Evaluations · doi:10.1016/j.tbench.2023.100089 · Cited by 694
Athaluri, S. A., Manthena, S. V., Kesapragada, V. S. R. K. M., Yarlagadda, V., Dave, T., & Duddumpudi, R. T. S. (2023) · Cureus · doi:10.7759/cureus.37432 · Cited by 403
ChatGPT in healthcare: A taxonomy and systematic review
Li, J., Dada, A., Puladi, B., Kleesiek, J., & Egger, J. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108013 · Cited by 452
Metadata
- Authors
- Macdonald, C., Adeloye, D., Sheikh, A., & Rudan, I.
- Affiliations
- University of Edinburgh (GB)
- Corresponding author
- Rudan, Igor
- Venue
- Journal of Global Health
- Published
- 2023-02-17
- Volume
- 13
- Pages
- 01003-01003
- ISSN
- 2047-2978 (ISSN-L)
- 2047-2986
- PMC
- PMC9936200
- OpenAlex
- W4321167341
- Cited by
- 243
- References
- 10
- Citations by year
- 2026: 18 · 2025: 44 · 2024: 87 · 2023: 93
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Global Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- ChatGPT, generative AI, scientific writing, vaccine effectiveness, population-level analysis, simulated data, healthcare workers, hospitalization
- MeSH terms
- Vaccine Efficacy, Computer Simulation, Confidentiality, Health Personnel, Humans, Software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
