Comparative analysis of NCLEX-RN questions: a duel between ChatGPT and human expertise
Abstract
Background: Artificial intelligence (AI) has the potential to revolutionize nursing education. This study compared NCLEX-RN questions generated by AI and those created by nurse educators. Method: Faculty of accredited baccalaureate programs were invited to participate. Likert-scale items for grammar and clarity of the item stem and distractors were compared using Mann-Whitney U, and yes/no questions about clinical relevance and complex terminology were analyzed using chi-square. A one-sample binomial test with confidence intervals evaluated participants' question preference (AI-generated or educator-written). Qualitative responses identified themes across faculty. Results: Item clarity, grammar, and difficulty were similar for AI and educator-created questions. Clinical relevance and use of complex terminology was similar for all question pairs. Of the four sets with preference for one item, three were generated by AI. Conclusion: AI can assist faculty with item generation to prepare nursing students for the NCLEX-RN examination. Faculty expertise is necessary to refine questions written using both methods. [ J Nurs Educ . 2023;62(12):679–687.]
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Faculty focused on content preparation, assessment development, and quality control, with an emphasis on content oversight.
“While faculty focused on content preparation, assessment development, and quality control (Bumbach, 2024; Cox et al., 2023; Irwin et al., 2023; Liu et al., 2023; Thakur et al., 2023), with an emphasis on content oversight (Sessions and Ness, 2024)”
From Patterns, advances, and gaps in using ChatGPT and similar technologies in nursing education: A PAGER scoping review (2025) · cites this paper as Cox et al., 2023
These tools demonstrated value in creating diverse assessment formats across multiple studies.
“these tools demonstrated value in creating diverse assessment formats across mul tiple studies”
From Patterns, advances, and gaps in using ChatGPT and similar technologies in nursing education: A PAGER scoping review (2025) · cites this paper as Cox et al., 2023
Student evidence supports cognitive scaffolding effectiveness and personalised learning support with GenAI.
“Complementing faculty applications, student evidence supports cogni tive scaffolding effectiveness and personalised learning support (Bumbach, 2024; Cox et al., 2023; Huang, 2023; Irwin et al., 2023; Kowitlawakul et al., 2024; Liu et al., 2023).”
From Patterns, advances, and gaps in using ChatGPT and similar technologies in nursing education: A PAGER scoping review (2025) · cites this paper as Cox et al., 2023
Benefits of GenAI-enhanced simulation are particularly evident in mental health and paediatric scenarios, where communication skills are paramount.
“These benefits are particularly evident in mental health and paediatric scenarios, where communication skills are paramount (Cox et al., 2023; Jung, 2023; Sharma and Sharma, 2023; Sharpnack, 2024).”
From Patterns, advances, and gaps in using ChatGPT and similar technologies in nursing education: A PAGER scoping review (2025) · cites this paper as Cox et al., 2023
ChatGPT was quick in developing NCLEX questions compared to nursing educators.
“Cox et al. (2023) compared the NCLEX developed by ChatGPT and nursing educators and found that ChatGPT was quick in developing the questions.”
From Current trends and future implications in the utilization of ChatGPT in nursing: a rapid review (2024) · cites this paper as Cox et al., 2023
ChatGPT can be useful in developing virtual patient simulation exercises for teaching students.
“or in developing virtual patient simulation exercises (Cox et al., 2023) for teaching students.”
From Current trends and future implications in the utilization of ChatGPT in nursing: a rapid review (2024) · cites this paper as Cox et al., 2023
Miscalculation and interpretation of laboratory values and limited timeframe of information generated are noted.
“Miscalculation and interpretation of laboratory values and limited timeframe of information generated (Cox et al., 2023) are also noted in these studies.”
From Current trends and future implications in the utilization of ChatGPT in nursing: a rapid review (2024) · cites this paper as Cox et al., 2023
ChatGPT application is useful for generating quick nursing care plans, developing and enhancing assessment modalities and providing feedback, and enhancing the structure of academic papers.
“Likewise, Dagcı et al. (2024) also found the ChatGPT application useful for generating quick nursing care plans, developing and enhancing assessment modalities and providing feedback (Chang et al., 2024), and enhancing the structure of academic papers (Cox et al., 2023).”
From Current trends and future implications in the utilization of ChatGPT in nursing: a rapid review (2024) · cites this paper as Cox et al., 2023
Cites
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · see citing works
Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse?
O’Connor, S., & ChatGPT (2023) · Nurse Education in Practice · doi:10.1016/j.nepr.2022.103537 · Cited by 585 · see citing works
The ChatGPT Storm and What Faculty Can Do
Sun, G. H., & Hoelscher, S. H. (2023) · Nurse Educator · doi:10.1097/nne.0000000000001390 · Cited by 263 · see citing works
Choi, E. P. H., Lee, J. J., Ho, M.-H., Kwok, J. Y. Y., & Lok, K. Y. W. (2023) · Nurse Education Today · doi:10.1016/j.nedt.2023.105796 · Cited by 172 · see citing works
Using ChatGPT to Teach Enhanced Clinical Judgment in Nursing Education
Seney, V., Desroches, M. L., & Schuler, M. (2023) · Nurse Educator · doi:10.1097/nne.0000000000001383 · Cited by 49 · see citing works
Bosher, S. (2003) · PubMed · Cited by 28 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Current trends and future implications in the utilization of ChatGPT in nursing: a rapid review
Kleib, M., Darko, E. M., Akingbade, O., Kennedy, M., Majekodunmi, P., Nickel, E., & Vogelsang, L. (2024) · International Journal of Nursing Studies Advances · doi:10.1016/j.ijnsa.2024.100252 · Cited by 40 · see citing works
Embracing the future: Integrating ChatGPT into China’s nursing education system
Ni, Z., Peng, R., Zheng, X., & Xie, P. (2024) · International Journal of Nursing Sciences · doi:10.1016/j.ijnss.2024.03.006 · Cited by 35 · see citing works
Amankwaa, I., Ekpor, E., Cudjoe, D., Kobiah, E., Fuseini, A.-K. J., Diebieri, M., Gyamfi, S., & Brownie, S. (2025) · Nurse Education Today · doi:10.1016/j.nedt.2025.106822 · Cited by 20 · see citing works
Glimpses of the Use of Generative AI and ChatGPT in Medical Education
Human Biology Division, S. O. M., Mitra, N. K., Chitra, E., & School of Health Sciences, I. U. (2024) · Education in Medicine Journal · doi:10.21315/eimj2024.16.2.11 · Cited by 15 · see citing works
- Authors
- Cox, R. L., Hunt, K. L., & Hill, R. R.
- Affiliations
- MGH Institute of Health Professions (US)
- Corresponding author
- Cox, Rachel L.
- ORCID
- Hill, Rebecca R.
- Venue
- Journal of Nursing Education
- Publisher
- SLACK, Inc.
- Published
- 2023-12
- Volume
- 62
- Issue
- 12
- Pages
- 679-687
- ISSN
- 0148-4834 (ISSN-L)
- 1938-2421
- OpenAlex
- W4389323382
- Cited by
- 30
- References
- 6
- Citations by year
- 2026: 12 · 2025: 12 · 2024: 5 · 2023: 1
- FWCI
- 1.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Nursing Education (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Social Sciences › Education › Health Education and Validation
- Keywords
- NCLEX-RN, ChatGPT, AI-generated test items, item generation, nursing faculty, item clarity, distractor quality, clinical relevance, nursing licensure examination preparation
- MeSH terms
- Artificial Intelligence, Education, Nursing, Education, Nursing, Baccalaureate, Educational Measurement, Humans, Licensure, Nursing, Students, Nursing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review