Evaluation of ChatGPT’s Real-Life Implementation in Undergraduate Dental Education: Mixed Methods Study
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Background: The recent artificial intelligence tool ChatGPT seems to offer a range of benefits in academic education while also raising concerns. Relevant literature encompasses issues of plagiarism and academic dishonesty, as well as pedagogy and educational affordances; yet, no real-life implementation of ChatGPT in the educational process has been reported to our knowledge so far. Objective: This mixed methods study aimed to evaluate the implementation of ChatGPT in the educational process, both quantitatively and qualitatively. Methods: In March 2023, a total of 77 second-year dental students of the European University Cyprus were divided into 2 groups and asked to compose a learning assignment on “Radiation Biology and Radiation Protection in the Dental Office,” working collaboratively in small subgroups, as part of the educational semester program of the Dentomaxillofacial Radiology module. Careful planning ensured a seamless integration of ChatGPT, addressing potential challenges. One group searched the internet for scientific resources to perform the task and the other group used ChatGPT for this purpose. Both groups developed a PowerPoint (Microsoft Corp) presentation based on their research and presented it in class. The ChatGPT group students additionally registered all interactions with the language model during the prompting process and evaluated the final outcome; they also answered an open-ended evaluation questionnaire, including questions on their learning experience. Finally, all students undertook a knowledge examination on the topic, and the grades between the 2 groups were compared statistically, whereas the free-text comments of the questionnaires were thematically analyzed. Results: Out of the 77 students, 39 were assigned to the ChatGPT group and 38 to the literature research group. Seventy students undertook the multiple choice question knowledge examination, and examination grades ranged from 5 to 10 on the 0-10 grading scale. The Mann-Whitney U test showed that students of the ChatGPT group performed significantly better (P=.045) than students of the literature research group. The evaluation questionnaires revealed the benefits (human-like interface, immediate response, and wide knowledge base), the limitations (need for rephrasing the prompts to get a relevant answer, general content, false citations, and incapability to provide images or videos), and the prospects (in education, clinical practice, continuing education, and research) of ChatGPT. Conclusions: Students using ChatGPT for their learning assignments performed significantly better in the knowledge examination than their fellow students who used the literature research methodology. Students adapted quickly to the technological environment of the language model, recognized its opportunities and limitations, and used it creatively and efficiently. Implications for practice: the study underscores the adaptability of students to technological innovations including ChatGPT and its potential to enhance educational outcomes. Educators should consider integrating ChatGPT into curriculum design; awareness programs are warranted to educate both students and educators about the limitations of ChatGPT, encouraging critical engagement and responsible use.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Digital Natives, Digital Immigrants Part 1
Prensky, M. (2001) · On the Horizon · doi:10.1108/10748120110424816 · Cited by 11,104
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 4×
“Nevertheless, students thought that the LLM must evolve to provide images, videos, accurate and relevant citations, and browse the internet [31,41,42].”
InfluentialKung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1×
“…of this study lies in the fact that this was a very early attempt to implement legitimately and in vivo a language model in the teaching process as a partner in learning, in contrast to the large number of publications perceiving ChatGPT as a partner in cheating and academic dishonesty [12,59,60].”
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Students' voices on generative ai: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · arXiv · doi:10.48550/arxiv.2305.00290 · Cited by 2,504 · cited 7×
“Despite numerous publications about AI and large language models (LLMs), the majority involve discussion papers, viewpoint articles, and positions [3,13,16,20,21], with few being exploratory, cross-sectional, or questionnaire-based studies [11,17,19].”
InfluentialStudents’ voices on generative AI: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00411-8 · Cited by 2,301 · cited 7×
“Despite numerous publications about AI and large language models (LLMs), the majority involve discussion papers, viewpoint articles, and positions [3,13,16,20,21], with few being exploratory, cross-sectional, or questionnaire-based studies [11,17,19].”
InfluentialWhat Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842
A SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384 · cited 1×
“…perspective, students using LLMs have the potential to develop new competencies including 21st-century soft skills, such as self-reflection abilities, problem-solving skills, creative and critical thinking, and collaboration, thus becoming motivated and autonomous learners [3,4,16,33,49].”
What the Student Does: teaching for enhanced learning
Biggs, J. (1999) · Higher Education Research & Development · doi:10.1080/0729436990180105 · Cited by 1,332
Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00819-6 · Cited by 1,099
ChatGPT for Education and Research: Opportunities, Threats, and Strategies
Rahman, M. M., & Watanobe, Y. (2023) · Applied Sciences · doi:10.3390/app13095783 · Cited by 1,080 · cited 6×
“…perspective, students using LLMs have the potential to develop new competencies including 21st-century soft skills, such as self-reflection abilities, problem-solving skills, creative and critical thinking, and collaboration, thus becoming motivated and autonomous learners [3,4,16,33,49].”
InfluentialCelik, I. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107468 · Cited by 1,012 · cited 1×
“…have limited technological knowledge, skills, and pedagogical expertise to assess AI applications and successfully integrate them into education [12,47]; therefore, they should pursue professional development to develop new skills related to AI understanding, possibilities, and implementation…”
Open AI in Education, the Responsible and Ethical Use of ChatGPT Towards Lifelong Learning
Mhlanga, D. (2023) · doi:10.2139/ssrn.4354422 · Cited by 719
ChatGPT - Reshaping medical education and clinical management
Khan, R. A., Jawaid, M., Khan, A. R., & Sajjad, M. (2023) · Pakistan Journal of Medical Sciences · doi:10.12669/pjms.39.2.7653 · Cited by 628
ChatGPT in education: Strategies for responsible implementation
Halaweh, M. (2023) · Contemporary Educational Technology · doi:10.30935/cedtech/13036 · Cited by 607 · cited 3×
“In accordance with the above procedure determined by the authors and in affirming their decisions, Halaweh’s study [14] published in April 2023—2 months after the development of this study’s design and 1 month after its implementation—precisely described the same process when discussing the…”
Discourses of artificial intelligence in higher education: a critical literature review
Bearman, M., Ryan, J., & Ajjawi, R. (2023) · Higher Education · doi:10.1007/s10734-022-00937-2 · Cited by 545
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.23 · Cited by 421 · cited 1×
“To respond to the AI paradigm shift, higher education institutions, educators, and students must engage in constructive dialogue to develop policies, guidelines, and training opportunities for the implementation of innovative technological tools in the teaching process [16,34,55].”
Nazari, N., Shabbir, M. S., & Setiawan, R. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e07014 · Cited by 405
Lecler, A., Duron, L., & Soyer, P. (2023) · Diagnostic and Interventional Imaging · doi:10.1016/j.diii.2023.02.003 · Cited by 385 · cited 1×
“Despite numerous publications about AI and large language models (LLMs), the majority involve discussion papers, viewpoint articles, and positions [3,13,16,20,21], with few being exploratory, cross-sectional, or questionnaire-based studies [11,17,19].”
Fuchs, K. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1166682 · Cited by 353 · cited 2×
“…technological knowledge, skills, and pedagogical expertise to assess AI applications and successfully integrate them into education [12,47]; therefore, they should pursue professional development to develop new skills related to AI understanding, possibilities, and implementation [15,40,48,49].”
Ethical use of Artificial Intelligence in Health Professions Education: AMEE Guide No. 158
Masters, K. (2023) · Medical Teacher · doi:10.1080/0142159x.2023.2186203 · Cited by 302
Evaluating Academic Answers Generated Using ChatGPT
Fergus, S., Botha, M., & Ostovar, M. (2023) · Journal of Chemical Education · doi:10.1021/acs.jchemed.3c00087 · Cited by 300
ChatGPT: Is this version good for healthcare and research?
Vaishya, R., Misra, A., & Vaish, A. (2023) · Diabetes & Metabolic Syndrome: Clinical Research & Reviews · doi:10.1016/j.dsx.2023.102744 · Cited by 293
Will ChatGPT get you caught? Rethinking of Plagiarism Detection
Khalil, M., & Er, E. (2023) · doi:10.35542/osf.io/fnh48 · Cited by 259
The exciting potential for ChatGPT in obstetrics and gynecology
Grünebaum, A., Chervenak, J., Pollet, S. L., Katz, A., & Chervenak, F. A. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2023.03.009 · Cited by 234 · cited 2×
“Interest in ChatGPT applications encompasses both clinical practice [6,7] and higher education [3,8-11], with promising results.”
Will ChatGPT Get You Caught? Rethinking of Plagiarism Detection
Khalil, M., & Er, E. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-34411-4_32 · Cited by 146
ChatGPT in Medical Imaging Higher Education
Currie, G., Singh, C., Nelson, T., Nabasenja, C., Al-Hayek, Y., & Spuur, K. (2023) · Radiography · doi:10.1016/j.radi.2023.05.011 · Cited by 128 · cited 2×
“…agreement with the relevant literature, the authors would suggest that higher education institutions and dental schools could consider updating their curricula, policies, and teaching methods to prepare students for an AI-driven future, by including education on and with AI tools and LLMs [8,45].”
Artificial intelligence and clinical anatomical education: Promises and perils
Lazarus, M. D., Truong, M., Douglas, P., & Selwyn, N. (2024) · Anatomical Sciences Education · doi:10.1002/ase.2221 · Cited by 116 · cited 1×
“…will offer opportunities to improve its efficiency and quality: improved student performance, personalized learning, targeted and immediate feedback, increased accessibility, creativity and innovations, student engagement, lesson preparation, collaborative activities, and evaluation [4,40,54-56].”
AI in higher education: A literature review of ChatGPT and guidelines for responsible implementation
Perera, P., & Lankathilaka, M. (2023) · International Journal of Research and Innovation in Social Science · doi:10.47772/ijriss.2023.7623 · Cited by 85
Chat Generative Pre-trained Transformer: why we should embrace this technology
Chavez, M. R., Butler, T. S., Rekawek, P., Heo, H., & Kinzler, W. L. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2023.03.010 · Cited by 83 · cited 4×
“In agreement with Chan and Hu [11], they are ready to embrace this new technology but in a collaboration where people maintain control and are not replaced by AI [17,20,37,38].”
InfluentialMynlieff, M., Manogaran, A. L., St. Maurice, M., & Eddinger, T. J. (2014) · CBE—Life Sciences Education · doi:10.1187/cbe.13-05-0097 · Cited by 60 · cited 2×
“Writing assignments enhance retention of knowledge; when assignments include reflective thinking, for example, when students have to evaluate and synthesize information (as happens in this study), higher-order (critical) thinking is also enhanced as students work at a higher cognitive level [30].”
Is ChatGPT a Good Tool for T&CM Students in Studying Pharmacology?
Nisar, S., & Aslam, M. S. (2023) · doi:10.2139/ssrn.4324310 · Cited by 60
Cited by
Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies
Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2025) · Computers & Education · doi:10.1016/j.compedu.2024.105224 · Cited by 531
Tam, T. Y. C., Sivarajkumar, S., Kapoor, S., Stolyar, A. V., Polanska, K., McCarthy, K. R., Osterhoudt, H., Wu, X., Visweswaran, S., Fu, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01258-7 · Cited by 424
“28 , in which they assessedChatGPT for undergraduate dental education.”
BackgroundGenerative AI in education and research: A systematic mapping review
Yusuf, A., Pervin, N., Román‐González, M., & Noor, N. M. (2024) · Review of Education · doi:10.1002/rev3.3489 · Cited by 182
Cai, L., Msafiri, M. M., & Kangwa, D. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-13112-0 · Cited by 104
Pham, T. D., Karunaratne, N., Exintaris, B., Liu, D., Lay, T., Yuriev, E., & Lim, A. (2025) · Medical Education · doi:10.1111/medu.15746 · Cited by 100
Metadata
- Authors
- Kavadella, A., Dias da Silva, M. A., Kaklamanos, E. G., Stamatopoulos, V., & Giannakopoulos, K.
- Venue
- JMIR Medical Education
- Publisher
- JMIR Publications Inc.
- Published
- 2024-01-31
- Volume
- 10
- Pages
- e51344
- ISSN
- 2369-3762 (ISSN-L)
- OpenAlex
- W4389559003
- Cited by
- 133
- References
- 35
- FWCI
- 4.60
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.