Performance of ChatGPT on registered nurse license exam in Taiwan: a descriptive study
Open accessPublished version of a preprint
Abstract
(1) Background: AI (artificial intelligence) chatbots have been widely applied. ChatGPT could enhance individual learning capabilities and clinical reasoning skills and facilitate students' understanding of complex concepts in healthcare education. There is currently less emphasis on its application in nursing education. The application of ChatGPT in nursing education needs to be verified. (2) Methods: A descriptive study was used to analyze the scores of ChatGPT on the registered nurse license exam (RNLE) in 2022~2023, and to explore the response and explanations of ChatGPT. The process of data measurement encompassed input sourcing, encoding methods, and statistical analysis. (3) Results: ChatGPT promptly responded within seconds. The average score of four exams was around 51.6 to 63.75 by ChatGPT, and it passed the RNLE in 2022 1st and 2023 2nd. However, ChatGPT may generate misleading or inaccurate explanations, or it could lead to hallucination; confusion or misunderstanding about complicated scenarios; and languages bias. (4) Conclusions: ChatGPT may have the potential to assist with nursing education because of its advantages. It is recommended to integrate ChatGPT into different nursing courses, to assess its limitations and effectiveness through a variety of tools and methods.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For students, studies documented improved personalisation and self-directed learning capabilities, where GenAI functions as cognitive scaffolding that helps students advance towards the performance of complex tasks.
“For students, studies documented improved personalisation and self-directed learning capabilities, where GenAI functions as cognitive scaffolding that helps ' students advance towards the performance of complex tasks '”
From Patterns, advances, and gaps in using ChatGPT and similar technologies in nursing education: A PAGER scoping review (2025) · cites this paper as Huang, 2023
The Introduction and Background (pages not available in our copy) describe personalized educational materials, step-by-step guidance, feedback, and the potential to strengthen students’ learning and understanding of complex concepts. But the paper tested ChatGPT on a nursing licensure exam, not students’ learning outcomes; its Conclusions say effectiveness in promoting comprehension still needs investigation. It does not document improved self-directed learning or students’ performance of complex tasks. (LLM, conf=0.87)
Experimental — accuracy not yet measuredStudent evidence supports cognitive scaffolding effectiveness and personalised learning support with GenAI.
“Complementing faculty applications, student evidence supports cogni tive scaffolding effectiveness and personalised learning support (Bumbach, 2024; Cox et al., 2023; Huang, 2023; Irwin et al., 2023; Kowitlawakul et al., 2024; Liu et al., 2023).”
From Patterns, advances, and gaps in using ChatGPT and similar technologies in nursing education: A PAGER scoping review (2025) · cites this paper as Huang, 2023
The Background (no page given) says ChatGPT can adapt educational materials to individual students and provide feedback, which supports the personalised-learning part of the claim. But this study tested ChatGPT on a nursing licensing exam, not with students; its Conclusions say further investigation is needed to assess its effectiveness in promoting comprehension. It therefore does not provide student evidence that cognitive scaffolding is effective. (LLM, conf=0.86)
Experimental — accuracy not yet measuredNewer GenAI versions demonstrated improved accuracy rates across multiple studies.
“Newer GenAI versions demonstrated improved accuracy rates across multiple studies”
From Patterns, advances, and gaps in using ChatGPT and similar technologies in nursing education: A PAGER scoping review (2025) · cites this paper as Huang, 2023
The Study Design section says this study tested GPT-3.5, not newer versions. The Limitations section says GPT-3.5 and GPT-4.0 could give different responses, but does not report improved accuracy. A search for version comparisons found no passage in our copy establishing improved accuracy across multiple studies. (LLM, conf=0.72)
Experimental — accuracy not yet measuredIn some cases, the output from ChatGPT could lead to "hallucination or confusion".
“and in some cases, the output from ChatGPT could lead to “hallucination or confusion” (Huang, 2023).”
From Current trends and future implications in the utilization of ChatGPT in nursing: a rapid review (2024) · cites this paper as Huang, 2023
ChatGPT passed the Taiwan Registered Nurse License Exam.
“and the Taiwan Registered Nurse License Exam (Huang, 2023)”
From Current trends and future implications in the utilization of ChatGPT in nursing: a rapid review (2024) · cites this paper as Huang, 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 34 · Cited by 48
Research works printed but not linked: 1
- [15]Kikuchi R. Application of Artificial Intelligence Technology in Nursing Studies: A Systematic Review 2021 Available online: https://www.himss.org/resources/application-artificial-intelligence-technology-nursing-studies-systematic-review (accessed on 10 September 2023)
- Authors
- Huang, H.
- Affiliations
- School of Nursing, College of Nursing, Tzu Chi University of Science and Technology, Hualien 970302, Taiwan
- ORCID
- Huang, Huiman
- Venue
- Healthcare
- Publisher
- MDPI AG
- Published
- 2023-10-30
- Volume
- 11
- Issue
- 21
- Pages
- 2855
- ISSN
- 2227-9032 (ISSN-L)
- PMC
- PMC10649156
- OpenAlex
- W4388101135
- Cited by
- 48
- References
- 34
- Citations by year
- 2026: 12 · 2025: 14 · 2024: 20 · 2023: 1 · 2022: 1
- FWCI
- 1.87
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Healthcare (publishedVersion, cc-by)
- Preprints.org (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Keywords
- ChatGPT, nursing education, artificial intelligence in education, exam performance, hallucinations, language bias, clinical reasoning, Taiwan
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
