Cited by 199Artificial Intelligence in Healthcare and EducationInnovations in Medical EducationArtificial Intelligence in Education

Are ChatGPT's knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study

Open access
Huh, Sun·2023·Journal of Educational Evaluation for Health Professions

Abstract

This study aimed to compare the knowledge and interpretation ability of ChatGPT, a language model of artificial general intelligence, with those of medical students in Korea by administering a parasitology examination to both ChatGPT and medical students. The examination consisted of 79 items and was administered to ChatGPT on January 1, 2023. The examination results were analyzed in terms of ChatGPT’s overall performance score, its correct answer rate by the items’ knowledge level, and the acceptability of its explanations of the items. ChatGPT’s performance was lower than that of the medical students, and ChatGPT’s correct answer rate was not related to the items’ knowledge level. However, there was a relationship between acceptable explanations and correct answers. In conclusion, ChatGPT’s knowledge and interpretation ability for this parasitology examination were not yet comparable to those of medical students in Korea.

Integrity signals

Registry

Crossref · 10.3352/jeehp.2023.20.01

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Cites

Still being checked.

Cited by

Sorted by citations

Authors
Huh, S.
Affiliations
Hallym University (KR)
Corresponding author
Huh, Sun
Venue
Journal of Educational Evaluation for Health Professions
Published
2023-01-11
Volume
20
Pages
1
ISSN
  • 1738-1339 (ISSN-L)
  • 1975-5937
OpenAlex
W4315498620
Cited by
199
References
0
Citations by year
2026: 5 · 2025: 23 · 2024: 69 · 2023: 100 · 2022: 2
FWCI
7.87
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Educational Evaluation for Health Professions (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
Keywords
ChatGPT, large language models, artificial general intelligence, medical students, knowledge assessment
MeSH terms
Artificial Intelligence, Educational Measurement, Humans, Students, Medical, Knowledge, Republic of Korea

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review