Performance of ChatGPT on Stage 1 of the Taiwanese medical licensing exam
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Introduction: Since its release by OpenAI in November 2022, numerous studies have subjected ChatGPT to various tests to evaluate its performance in medical exams. The objective of this study is to evaluate ChatGPT's accuracy and logical reasoning across all 10 subjects featured in Stage 1 of Senior Professional and Technical Examinations for Medical Doctors (SPTEMD) in Taiwan, with questions that encompass both Chinese and English. Methods: In this study, we tested ChatGPT-4 to complete SPTEMD Stage 1. The model was presented with multiple-choice questions extracted from three separate tests conducted in February 2022, July 2022, and February 2023. These questions encompass 10 subjects, namely biochemistry and molecular biology, anatomy, embryology and developmental biology, histology, physiology, microbiology and immunology, parasitology, pharmacology, pathology, and public health. Subsequently, we analyzed the model's accuracy for each subject. Result: In all three tests, ChatGPT achieved scores surpassing the 60% passing threshold, resulting in an overall average score of 87.8%. Notably, its best performance was in biochemistry, where it garnered an average score of 93.8%. Conversely, the performance of the generative pre-trained transformer (GPT)-4 assistant on anatomy, parasitology, and embryology was not as good. In addition, its scores were highly variable in embryology and parasitology. Conclusion: ChatGPT has the potential to facilitate not only exam preparation but also improve the accessibility of medical education and support continuous education for medical professionals. In conclusion, this study has demonstrated ChatGPT's potential competence across various subjects within the SPTEMD Stage 1 and suggests that it could be a helpful tool for learning and exam preparation for medical students and professionals.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“In the work of Kung and colleagues, 9 where the questions from the United States Medical Licensing Examination (USMLE) in 2022 were used, ChatGPT yielded passing performance with moderate accuracy.”
MethodKung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Bhayana, R., Krishna, S., & Bleakney, R. R. (2023) · Radiology · doi:10.1148/radiol.230582 · Cited by 460
Early applications of ChatGPT in medical practice, education and research
Sedaghat, S. (2023) · Clinical Medicine · doi:10.7861/clinmed.2023-0078 · Cited by 163
Humar, P., Asaad, M., Bengur, F. B., & Nguyen, V. (2023) · Aesthetic Surgery Journal · doi:10.1093/asj/sjad130 · Cited by 155
Performance of ChatGPT on the pharmacist licensing examination in Taiwan
Wang, Y.-M., Shen, H.-W., & Chen, T.-J. (2023) · Journal of the Chinese Medical Association · doi:10.1097/jcma.0000000000000942 · Cited by 109 · cited 1×
“Another study regarding the Taiwanese Pharmacist Licensing Examination 19 yielded a non-passing result.”
BackgroundChatGPT failed Taiwan’s Family Medicine Board Exam
Weng, T.-L., Wang, Y.-M., Chang, S., Chen, T.-J., & Hwang, S.-J. (2023) · Journal of the Chinese Medical Association · doi:10.1097/jcma.0000000000000946 · Cited by 100 · cited 1×
“Weng et al. 15 observed GPT-4 ’ s challenges with the Taiwanese Family Medicine Board Exam, achieving an accuracy of 41.6%, below the passing threshold of 60%, though its performance in certain question types was acceptable.”
BackgroundInfluentialPerformance of ChatGPT on registered nurse license exam in Taiwan: a descriptive study
Huang, H. (2023) · Healthcare · doi:10.3390/healthcare11212855 · Cited by 48
Al Kahf, S., Roux, B., Clerc, S., Bassehila, M., Lecomte, A., Moncomble, E., Alabadan, E., de Montmolin, N., Jablon, E., François, E., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0278673 · Cited by 42
Cited by
Lin, Y., Luo, Z., Ye, Z., Zhong, N., Zhao, L., Zhang, L., Li, X., Chen, Z., & Chen, Y. (2025) · JMIR Medical Education · doi:10.2196/71125 · Cited by 58
“If the data is insufficient or skewed, it may lead to potential biases and limitations in practical applications, causing underperformance in less-represented areas [33,59, 73,82,85,86,111,117,122,126,140-143].”
Jaleel, A., Aziz, U., Farid, G., Zahid Bashir, M., Mirza, T. R., Khizar Abbas, S. M., Aslam, S., & Sikander, R. M. H. (2025) · JMIR Medical Education · doi:10.2196/68070 · Cited by 25
Oladokun, B. D., Enakrire, R. T., Emmanuel, A. K., Ajani, Y. A., & Adetayo, A. J. (2025) · Journal of Web Librarianship · doi:10.1080/19322909.2025.2482093 · Cited by 7
“Huang et al. (2024) tested GPT-4 on Taiwan’s medical licensing exams, reporting an overall accuracy of 87.8%, with biochemistry being the best-performing subject.”
InfluentialKeskin, A., & Aygün, T. (2025) · Tıp Eğitimi Dünyası · doi:10.25282/ted.1716591 · Cited by 1
Metadata
- Authors
- Huang, C.-H., Hsiao, H.-J., Yeh, P.-C., Wu, K.-C., & Kao, C.-H.
- Venue
- DIGITAL HEALTH
- Publisher
- SAGE Publications
- Published
- 2024-01
- Volume
- 10
- Pages
- 20552076241233144-20552076241233144
- ISSN
- 2055-2076 (ISSN-L)
- OpenAlex
- W4391890163
- Cited by
- 32
- References
- 9
- FWCI
- 2.57
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Performance of ChatGPT on Stage 1 of the Taiwanese medical licensing exam. https://app.cochat.ai/papers/10.1177/20552076241233144
- The paper
- Huang, C.-H., Hsiao, H.-J., Yeh, P.-C., Wu, K.-C., & Kao, C.-H. (2024). Performance of ChatGPT on Stage 1 of the Taiwanese medical licensing exam. DIGITAL HEALTH. https://doi.org/10.1177/20552076241233144
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.