Evaluation of ChatGPT’s Real-Life Implementation in Undergraduate Dental Education: Mixed Methods Study
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Background: The recent artificial intelligence tool ChatGPT seems to offer a range of benefits in academic education while also raising concerns. Relevant literature encompasses issues of plagiarism and academic dishonesty, as well as pedagogy and educational affordances; yet, no real-life implementation of ChatGPT in the educational process has been reported to our knowledge so far. Objective: This mixed methods study aimed to evaluate the implementation of ChatGPT in the educational process, both quantitatively and qualitatively. Methods: In March 2023, a total of 77 second-year dental students of the European University Cyprus were divided into 2 groups and asked to compose a learning assignment on “Radiation Biology and Radiation Protection in the Dental Office,” working collaboratively in small subgroups, as part of the educational semester program of the Dentomaxillofacial Radiology module. Careful planning ensured a seamless integration of ChatGPT, addressing potential challenges. One group searched the internet for scientific resources to perform the task and the other group used ChatGPT for this purpose. Both groups developed a PowerPoint (Microsoft Corp) presentation based on their research and presented it in class. The ChatGPT group students additionally registered all interactions with the language model during the prompting process and evaluated the final outcome; they also answered an open-ended evaluation questionnaire, including questions on their learning experience. Finally, all students undertook a knowledge examination on the topic, and the grades between the 2 groups were compared statistically, whereas the free-text comments of the questionnaires were thematically analyzed. Results: Out of the 77 students, 39 were assigned to the ChatGPT group and 38 to the literature research group. Seventy students undertook the multiple choice question knowledge examination, and examination grades ranged from 5 to 10 on the 0-10 grading scale. The Mann-Whitney U test showed that students of the ChatGPT group performed significantly better (P=.045) than students of the literature research group. The evaluation questionnaires revealed the benefits (human-like interface, immediate response, and wide knowledge base), the limitations (need for rephrasing the prompts to get a relevant answer, general content, false citations, and incapability to provide images or videos), and the prospects (in education, clinical practice, continuing education, and research) of ChatGPT. Conclusions: Students using ChatGPT for their learning assignments performed significantly better in the knowledge examination than their fellow students who used the literature research methodology. Students adapted quickly to the technological environment of the language model, recognized its opportunities and limitations, and used it creatively and efficiently. Implications for practice: the study underscores the adaptability of students to technological innovations including ChatGPT and its potential to enhance educational outcomes. Educators should consider integrating ChatGPT into curriculum design; awareness programs are warranted to educate both students and educators about the limitations of ChatGPT, encouraging critical engagement and responsible use.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Digital Natives, Digital Immigrants Part 1
Prensky, M. (2001) · On the Horizon · doi:10.1108/10748120110424816 · Cited by 11,104
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 4×
“Nevertheless, students thought that the LLM must evolve to provide images, videos, accurate and relevant citations, and browse the internet [31,41,42].”
InfluentialKung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1×
“…of this study lies in the fact that this was a very early attempt to implement legitimately and in vivo a language model in the teaching process as a partner in learning, in contrast to the large number of publications perceiving ChatGPT as a partner in cheating and academic dishonesty [12,59,60].”
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Students' voices on generative ai: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · arXiv · doi:10.48550/arxiv.2305.00290 · Cited by 2,504 · cited 7×
“Despite numerous publications about AI and large language models (LLMs), the majority involve discussion papers, viewpoint articles, and positions [3,13,16,20,21], with few being exploratory, cross-sectional, or questionnaire-based studies [11,17,19].”
InfluentialStudents’ voices on generative AI: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00411-8 · Cited by 2,301 · cited 7×
“Despite numerous publications about AI and large language models (LLMs), the majority involve discussion papers, viewpoint articles, and positions [3,13,16,20,21], with few being exploratory, cross-sectional, or questionnaire-based studies [11,17,19].”
InfluentialWhat Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842
Cited by
Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies
Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2025) · Computers & Education · doi:10.1016/j.compedu.2024.105224 · Cited by 531
Tam, T. Y. C., Sivarajkumar, S., Kapoor, S., Stolyar, A. V., Polanska, K., McCarthy, K. R., Osterhoudt, H., Wu, X., Visweswaran, S., Fu, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01258-7 · Cited by 424
“28 , in which they assessedChatGPT for undergraduate dental education.”
BackgroundGenerative AI in education and research: A systematic mapping review
Yusuf, A., Pervin, N., Román‐González, M., & Noor, N. M. (2024) · Review of Education · doi:10.1002/rev3.3489 · Cited by 182
Cai, L., Msafiri, M. M., & Kangwa, D. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-13112-0 · Cited by 104
Pham, T. D., Karunaratne, N., Exintaris, B., Liu, D., Lay, T., Yuriev, E., & Lim, A. (2025) · Medical Education · doi:10.1111/medu.15746 · Cited by 100
The Impact of AI on the Personal and Collaborative Learning Environments in Higher Education
Msambwa, M. M., Wen, Z., & Daniel, K. (2025) · European Journal of Education · doi:10.1111/ejed.12909 · Cited by 76
Daniel, K., Msambwa, M. M., & Wen, Z. (2025) · European Journal of Education · doi:10.1111/ejed.70036 · Cited by 71
Xia, Q., Li, W., Yang, Y., Weng, X., & Chiu, T. K. F. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100455 · Cited by 45
Thomae, A. V., Witt, C. M., & Barth, J. (2024) · JMIR Medical Education · doi:10.2196/50545 · Cited by 36
Assessing ChatGPT’s capability as a new age standardized patient: qualitative study
Cross, J., Kayalackakom, T., Robinson, R. E., Vaughans, A., Sebastian, R., Hood, R., Lewis, C., Devaraju, S., Honnavar, P., Naik, S., et al. (2025) · JMIR Medical Education · doi:10.2196/63353 · Cited by 33
“These results are broadly in line with those of other studies using LLMs in the role of SP [24-29].”
ChatGPT Interventions in Higher Education: A Systematic Review of Experimental Studies
Jin, Y., & Sercu, L. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70072 · Cited by 21
Jacobs, C., Johnson, H., Tan, N., Brownlie, K., Joiner, R., & Thompson, T. (2025) · JMIR Medical Education · doi:10.2196/70766 · Cited by 20
“Previous studies have used AI in primary care education [17,18]; however, publications using advanced language models to simulate VP encounters remain limited, as the vast majority are discussion papers and viewpoints [19].”
McBee, J. C., Han, D. Y., Liu, L., Ma, L., Adjeroh, D. A., Xu, D., & Hu, G. (2024) · JMIR Medical Education · doi:10.2196/51157 · Cited by 14
Teledentistry for Pediatric Dental Emergency: Comparison Between Experienced and Novice Users
Huang, C.-C., & Chen, J.-W. (2024) · Bioengineering · doi:10.3390/bioengineering11111054 · Cited by 8
Susena, Y. S., Arianto, F., Khotimah, K., Andrianto, R. E., & Syuhada’, M. (2026) · Reading Psychology · doi:10.1080/02702711.2026.2644973 · Cited by 1
Experimental Evidence on the Learning Impact of Generative AI
Contractor, Z., & Reyes, G. (2026) · arXiv · doi:10.48550/arxiv.2607.08849 · Cited by 10
Metadata
- Authors
- Kavadella, A., Dias da Silva, M. A., Kaklamanos, E. G., Stamatopoulos, V., & Giannakopoulos, K.
- Venue
- JMIR Medical Education
- Publisher
- JMIR Publications Inc.
- Published
- 2024-01-31
- Volume
- 10
- Pages
- e51344
- ISSN
- 2369-3762 (ISSN-L)
- OpenAlex
- W4389559003
- Cited by
- 133
- References
- 35
- FWCI
- 4.60
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.