ChatGPT can pass the AHA exams: Open-ended questions outperform multiple-choice format
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
The study by Fijačko et al. tested ChatGPT's ability to pass the BLS and ACLS exams of AHA, but found that ChatGPT failed both exams. A limitation of their study was using ChatGPT to generate only one response, which may have introduced bias. When generating three responses per question, ChatGPT can pass BLS exam with an overall accuracy of 84%. When incorrectly answered questions were rewritten as open-ended questions, ChatGPT's accuracy rate increased to 96% and 92.1% for the BLS and ACLS exams, respectively, allowing ChatGPT to pass both exams with outstanding results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Can ChatGPT pass the life support exams without entering the American heart association course?
Fijačko, N., Gosak, L., Štiglic, G., Picard, C. T., & John Douma, M. (2023) · Resuscitation · doi:10.1016/j.resuscitation.2023.109732 · Cited by 206
Cited by
Rossettini, G., Rodeghiero, L., Corradi, F., Cook, C., Pillastrini, P., Turolla, A., Castellini, G., Chiappinotto, S., Gianola, S., & Palese, A. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05630-9 · Cited by 103
“Fifthly, we inputted the test questions only once to simulate the test execution conditions in real educational contexts [32], although previous studies have prompted the test questions multiple times in AI chatbots to obtain better results [68].”
MethodEmbracing ChatGPT for Medical Education: Exploring Its Impact on Doctors and Medical Students
Wu, Y., Zheng, Y., Feng, B., Yang, Y., Kang, K., & Zhao, A. (2024) · JMIR Medical Education · doi:10.2196/52483 · Cited by 92
Abouammoh, N., Alhasan, K., Raina, R., Malki, K. A., Aljamaan, F., Tamimi, I., Muaygil, R., Wahabi, H., Jamal, A., Al-Tawfiq, J. A., et al. (2023) · doi:10.1101/2023.07.13.23292624 · Cited by 44
The Evaluation of Generative AI Should Include Repetition to Assess Stability
Zhu, L., Mou, W., Hong, C., Yang, T., Lai, Y., Qi, C., Lin, A., Zhang, J., & Luo, P. (2024) · JMIR mHealth and uHealth · doi:10.2196/57978 · Cited by 38
“Compared to earlier results, ChatGPT’s performance in this study significantly improved, and it could pass the Basic Life Support exam [18], further underscoring the importance of repetitions.”
ResultSelf-citationEmbracing the future: Integrating ChatGPT into China’s nursing education system
Ni, Z., Peng, R., Zheng, X., & Xie, P. (2024) · International Journal of Nursing Sciences · doi:10.1016/j.ijnss.2024.03.006 · Cited by 35
“Furthermore, ChatGPT software has been successfully used to pass the American Heart Association (AHA) Basic Life Support (BLS) test [27] and the Plastic Surgery In-Training Exam (PSITE) [28].”
Method
Metadata
- Authors
- Zhu, L., Mou, W., Yang, T., & Chen, R.
- Venue
- Resuscitation
- Publisher
- Elsevier BV
- Published
- 2023-07
- Volume
- 188
- Pages
- 109783
- ISSN
- 0300-9572 (ISSN-L)
- OpenAlex
- W4381431616
- Cited by
- 65
- References
- 1
- FWCI
- 2.25
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT can pass the AHA exams: Open-ended questions outperform multiple-choice format. https://app.cochat.ai/papers/10.1016/j.resuscitation.2023.109783
- The paper
- Zhu, L., Mou, W., Yang, T., & Chen, R. (2023). ChatGPT can pass the AHA exams: Open-ended questions outperform multiple-choice format. Resuscitation. https://doi.org/10.1016/j.resuscitation.2023.109783
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.