Virtual Clinical Encounter Examination (VICEE): A novel approach for assessing medical students’ non-psychomotor clinical competency
Abstract
INTRODUCTION: The Corona Virus Disease-19 (COVID-19) pandemic disrupted medical education across the world. Online teaching has grown rapidly under lockdown. Yet the online approach for assessment presents a number of challenges, particularly when evaluating clinical competencies. The aim of this study was to investigate the feasibility, acceptability, reliability and validity of an online Virtual Clinical Encounter Examination (VICEE) to assess non-psychomotor competencies (non-procedure or manual skills) of medical students. METHOD: Sixty-one final year medical students took the VICEE as part of the final summative examination. A panel of faculty experts developed the exam cases and competencies. They administered the test online via real-time interaction with artificial intelligence (AI) based virtual patients, along with faculty and IT support. RESULTS: Student and faculty surveys demonstrated satisfaction with the experience. Confirmatory factor analysis supported convergent validity of VICEE with Direct Observation Clinical Encounter Examination (DOCEE), a previously validated clinical examination. The observed sensitivity was 81.8%, specificity 64.1% and likelihood ratio 12.6, supporting the ability of VICEE to diagnose 'clinical incompetence' among students. CONCLUSION: Our results suggest that online AI-based virtual patient high fidelity simulation may be used as an alternative tool to assess some aspects of non-psychometric competencies.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Online AI-based examinations may serve as alternative tools for assessing specific nonpsychomotor competencies.
“Online AI-based examinations may serve as alternative tools for assessing specific nonpsychomotor competencies [142].”
From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [142]
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · see citing works
Principles and Practice of Structural Equation Modeling
Tabri, N., & Elliott, C. M. (2012) · Canadian Graduate Journal of Sociology and Criminology · doi:10.15353/cgjsc.v1i1.3787 · Cited by 39,065 · see citing works
Structural Equation Modeling With AMOS: Basic Concepts, Applications, and Programming, Third Edition
Byrne, B. M. (2016) · doi:10.4324/9781315757421 · Cited by 4,653 · see citing works
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814 · see citing works
The Impact of E-Learning in Medical Education
Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006) · Academic Medicine · doi:10.1097/00001888-200603000-00002 · Cited by 2,240 · see citing works
Internet-Based Learning in the Health Professions
Cook, D., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008) · JAMA · doi:10.1001/jama.300.10.1181 · Cited by 1,552 · see citing works
Pei, L., & Wu, H. (2019) · Medical Education Online · doi:10.1080/10872981.2019.1666538 · Cited by 852 · see citing works
The Mini-CEX: A Method for Assessing Clinical Skills
Norcini, J. J., Blank, L. L., Duffy, F. D., & Fortna, G. S. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-6-200303180-00012 · Cited by 730 · see citing works
Kononowicz, A. A., Woodham, L. A., Edelbring, S., Stathakarou, N., Davies, D., Saxena, N., Tudor Car, L., Carlstedt-Duke, J., Car, J., & Zary, N. (2019) · Journal of Medical Internet Research · doi:10.2196/14676 · Cited by 600 · see citing works
Self-citationMassoth, C., Röder, H., Ohlenburg, H., Heßler, M., Zarbock, A., Pöpping, D. M., & Wenk, M. (2019) · BMC Medical Education · doi:10.1186/s12909-019-1464-7 · Cited by 318 · see citing works
Situational interest and learning: Thirst for knowledge
Rotgans, J. I., & Schmidt, H. G. (2014) · Learning and Instruction · doi:10.1016/j.learninstruc.2014.01.002 · Cited by 267 · see citing works
Self-citationPonti, R. D., Marazzato, J., Maresca, A. M., Rovera, F. A., Carcano, G., & Ferrario, M. (2020) · BMC Medical Education · doi:10.1186/s12909-020-02245-8 · Cited by 219 · see citing works
A history of assessment in medical education
Schuwirth, L., & van der Vleuten, C. P. M. (2020) · Advances in Health Sciences Education · doi:10.1007/s10459-020-10003-0 · Cited by 149 · see citing works
Ponti, R. D., Marazzi, R., Ghiringhelli, S., et al. (2011) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2011.02.063 · Cited by 101 · see citing works
Virtual patients for assessment of clinical reasoning in nursing — A pilot study
Forsberg, E., Georg, C., Ziegert, K., & Fors, U. (2010) · Nurse Education Today · doi:10.1016/j.nedt.2010.11.015 · Cited by 89 · see citing works
Kleinert, R., Heiermann, N., Plum, P. S., Wahba, R., Chang, D., Maus, M. K. H., Chon, S., Hoelscher, A. H., & Stippel, D. L. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.5035 · Cited by 65 · see citing works
Reliability and validity of the direct observation clinical encounter examination (DOCEE)
Hamdy, H., Prasad, K., Williams, R. G., et al. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01438.x · Cited by 54 · see citing works
Self-citationGunning, W. T., & Fors, U. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.642830 · Cited by 32 · see citing works
Virtual patients in problem‐based learning
Hamdy, H., Al-Moslih, A., Tavarnesi, G., & Laus, A. (2017) · Medical Education · doi:10.1111/medu.13293 · Cited by 19 · see citing works
Self-citationRossi, L. C., Penela, D., Doni, L. A., et al. (2018) · Pacing and Clinical Electrophysiology · doi:10.1111/pace.13508 · Cited by 13 · see citing works
Ullah, W., Hunter, R. J., Finlay, M., et al. (2016) · Simulation in Healthcare The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0000000000000197 · Cited by 9 · see citing works
Distance assessment of counselling skills using virtual patients during the COVID-19 pandemic
Thomas, D., Beshir, S. A., Zachariah, S., Sam, K. G., & Hamdy, H. (2020) · Pharmacy Education · doi:10.46542/pe.2020.202.196204 · Cited by 5 · see citing works
Self-citationMoments of Rupture: The Importance of Affect in Medical Education and Surgical Training
Mahendran, A. (2019) · Routledge eBooks · doi:10.4324/9780429455148 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A scoping review of artificial intelligence in medical education: BEME Guide No. 84
Gordon, M., Daniel, M., Ajiboye, A., Uraiby, H., Xu, N. Y., Bartlett, R., Hanson, J., Haas, M., Spadafore, M., Grafton-Clarke, C., et al. (2024) · Medical Teacher · doi:10.1080/0142159x.2024.2314198 · Cited by 543 · see citing works
Issa, W. B., Shorbagi, A., Al-Sharman, A., Rababa, M., Al-Majeed, K., Radwan, H., Refaat Ahmed, F., Al-Yateem, N., Mottershead, R., Abdelrahim, D. N., et al. (2024) · BMC Medical Education · doi:10.1186/s12909-024-06076-9 · Cited by 78 · see citing works
“The results demonstrated AI was a highly effective and acceptable assessment tool for both educators and students [13].”
BackgroundArtificial intelligence in undergraduate medical education: an updated scoping review
Simoni, J., Urtubia-Fernandez, J., Mengual, E., Simoni, D. A., Royo, M., Egaña-Yin, D., Hertog, O. L. A., López-Ortiz, L., Muñoz-Tomás, A., Santiago-Martínez, P., et al. (2025) · BMC Medical Education · doi:10.1186/s12909-025-08188-2 · Cited by 62 · see citing works
“Online AI-based examinations may serve as alternative tools for assessing specific nonpsychomotor competencies [142].”
Artificial Intelligence in Health Professions Education
Shankar, P. R. (2022) · Archives of Medicine and Health Sciences · doi:10.4103/amhs.amhs_234_22 · Cited by 40 · see citing works
- Authors
- Hamdy, H., Sreedharan, J., Rotgans, J. I., Zary, N., Bahous, S. A., Venkatramana, M., AbdelFattah Elzayat, E., Lamba, P., Sebastian, S. K., & Momen, N. K. A.
- Affiliations
- Gulf Medical University (AE)
- Nanyang Technological University (SG)
- Mohammed Bin Rashid University of Medicine and Health Sciences (AE)
- Lebanese American University (LB)
- College of Medicine, Gulf Medical University, Ajman, United Arab Emirates
- Corresponding author
- Hamdy, Hossam
- ORCID
- Venue
- Medical Teacher
- Publisher
- Informa UK Limited
- Published
- 2021-06-15
- Volume
- 43
- Issue
- 10
- Pages
- 1203-1209
- ISSN
- 0142-159X (ISSN-L)
- 1466-187X
- OpenAlex
- W3171229166
- Cited by
- 25
- References
- 23
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 10 · 2023: 3 · 2022: 3
- FWCI
- 3.46
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Teacher (publishedVersion)
- PubMed (publishedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Health Sciences › Medicine › Physiology › Simulation-Based Education in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- online assessment, medical students, virtual patients, artificial intelligence, high-fidelity simulation, COVID-19 pandemic, summative assessment, validity, reliability statistics, acceptability, clinical competence assessment
- MeSH terms
- COVID-19, SARS-CoV-2, Artificial Intelligence, Clinical Competence, Communicable Disease Control, Education, Medical, Undergraduate, Educational Measurement, Humans, Students, Medical, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
