Simulated patient judgements of medical student performance in OSCEs: A realist evaluation
Open accessAbstract
INTRODUCTION: Simulated patients (SPs) are widely used in Objective Structured Clinical Examinations (OSCEs) and increasingly contribute to summative scoring in high-stakes clinical assessments. While their judgments inform pass-fail decisions, concerns remain about variability and potential bias. There is limited theory-driven evidence explaining how SP involvement contributes to assessment decisions, and under what conditions this supports fair and credible assessment. This study aimed to develop and refine a programme theory explaining how SP involvement contributes to decision-making in high-stakes OSCEs. METHODS: We conducted a realist evaluation informed by an initial programme theory developed through a rapid realist review. Semi-structured interviews were undertaken with 19 SPs in a sequential OSCE at a UK medical school. Interviews explored how SPs approached scoring decisions and how features of the assessment environment shaped their judgments. Data were analysed iteratively using abductive and retroductive reasoning to identify context-mechanism-outcome configurations (CMOCs) and refine the programme theory. RESULTS: Eighteen interrelated CMOCs were constructed across nine domains describing how SP judgments were shaped through interactions between assessment-related factors (including assessment tools, assessment focus, logistical environment, and the sequential format) and SP-related factors (including background and experience, training and benchmarking, role positioning, fatigue, and interaction with examiner). When SPs felt prepared, role clarity was established, and assessment tools and conditions were supportive, they reported making judgments experienced as fair and authentic. In contrast, high cognitive load, role ambiguity, fatigue and insufficient preparation activated mechanisms that increased perceived variability and potential for bias. DISCUSSION: SP judgments in high-stakes OSCEs are complex and context-dependent. This programme theory provides an explanatory account of how SP judgments are produced and identifies conditions that support or undermine assessment. The findings offer practical implications for assessment design, SP training and faculty-SP collaboration to strengthen the use of SPs as assessors in high-stakes clinical examinations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283 · see citing works
A practical guide to reflexivity in qualitative research: AMEE Guide No. 149.
Olmos-Vega, F. M., Stalmeijer, R. E., Varpio, L., & Kahlke, R. (2023) · Medical Teacher · doi:10.1080/0142159x.2022.2057287 · Cited by 2,014 · cited 1× · see citing works
“These reflexive discussions allowed us to challenge one another ’ s interpretations and consider alternative explanations to theory development, which strengthens the credibility of our findings [26].”
RAMESES II reporting standards for realist evaluations
Wong, G., Westhorp, G., Manzano, A., Greenhalgh, J., Jagosh, J., & Greenhalgh, T. (2016) · BMC Medicine · doi:10.1186/s12916-016-0643-1 · Cited by 880 · see citing works
The craft of interviewing in realist evaluation
Manzano, A. (2016) · Evaluation · doi:10.1177/1356389016638615 · Cited by 691 · see citing works
A contemporary approach to validity arguments: a practical guide to K ane's framework
Cook, D. A., Brydges, R., Ginsburg, S., & Hatala, R. A. (2015) · Medical Education · doi:10.1111/medu.12678 · Cited by 644 · see citing works
The use of simulated patients in medical education: AMEE Guide No 42
Cleland, J. A., Abe, K., & Rethans, J.-J. (2009) · Medical Teacher · doi:10.1080/01421590903002821 · Cited by 509 · see citing works
Realist methods in medical education research: what are they and what can they contribute?
Wong, G., Greenhalgh, T., Westhorp, G., & Pawson, R. (2011) · Medical Education · doi:10.1111/j.1365-2923.2011.04045.x · Cited by 405 · see citing works
Simulated and standardized patients in OSCEs: achievements and challenges 1992-2003
Adamo, G. (2003) · Medical Teacher · doi:10.1080/0142159031000100300 · Cited by 281 · see citing works
Pawson, R. (1996) · British Journal of Sociology · doi:10.2307/591728 · Cited by 264 · see citing works
Using the realist interview approach to maintain theoretical awareness in realist studies
Mukumbang, F. C., Marchal, B., Belle, S. V., & van Wyk, B. E. (2019) · Qualitative Research · doi:10.1177/1468794119881985 · Cited by 202 · see citing works
Berg, K. T., Majdan, J. F., Berg, D., Veloski, J. J., & Hojat, M. (2011) · Academic Medicine · doi:10.1097/acm.0b013e3182224f1f · Cited by 112 · see citing works
Rapid reviews methods series: Guidance on literature search
Klerings, I., Robalino, S., Booth, A., Liquitay, C. M. E., Sommer, I., Gartlehner, G., Devane, D., & Waffenschmidt, S. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-112079 · Cited by 88 · see citing works
Duddy, C., & Wong, G. (2022) · BMJ Quality & Safety · doi:10.1136/bmjqs-2022-015236 · Cited by 85 · see citing works
Rater Errors in a Clinical Skills Assessment of Medical Students
Iramaneerat, C., & Yudkowsky, R. (2007) · Evaluation & the Health Professions · doi:10.1177/0163278707304040 · Cited by 82 · see citing works
Assessing students’ communication skills: validation of a global rating
Scheffer, S., Mühlinghaus, I., Fröhmel, A., & Ortwein, H. (2007) · Advances in Health Sciences Education · doi:10.1007/s10459-007-9074-2 · Cited by 82 · see citing works
Guiton, G., Hodgson, C. S., Delandshere, G., & Wilkerson, L. (2004) · Advances in Health Sciences Education · doi:10.1023/b:ahse.0000038174.87790.7b · Cited by 80 · see citing works
Davis, M. H. (2003) · Medical Teacher · doi:10.1080/0142159031000100292 · Cited by 65 · see citing works
Interventions to improve antimicrobial prescribing of doctors in training (IMPACT): a realist review
Papoutsi, C., Mattick, K. L., Pearson, M., Brennan, N., Briscoe, S., & Wong, G. (2018) · Health Services and Delivery Research · doi:10.3310/hsdr06100 · Cited by 59 · see citing works
Wright, B., McKendree, J., Morgan, L., Allgar, V. L., & Brown, A. K. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-199 · Cited by 51 · see citing works
Examiner fatigue in communication skills objective structured clinical examinations
Humphris, G. M., & Kaney, S. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00893.x · Cited by 44 · see citing works
The view from over there: reframing the OSCE through the experience of standardised patient raters
Johnston, J. L., Lundy, G., McCullough, M., & Gormley, G. (2013) · Medical Education · doi:10.1111/medu.12243 · Cited by 40 · see citing works
Patients' global ratings of student competence. Unreliable contamination or gold standard?
WILKINSON, T. J., & Fontaine, S. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01379.x · Cited by 39 · see citing works
Homer, M., & Pell, G. (2009) · Medical Teacher · doi:10.1080/01421590802520949 · Cited by 38 · see citing works
Rees, C. E., Davis, C., Nguyen, V. N. B., Proctor, D., & Mattick, K. L. (2024) · Medical Education · doi:10.1111/medu.15270 · Cited by 27 · see citing works
Roshal, J. A., Chefitz, D., Terregino, C. A., & Петрова, А. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02540-y · Cited by 26 · see citing works
Realist analysis of qualitative data in health professions education research
Rees, C. E., Proctor, D. W., Nguyen, V. N. B., Ottrey, E., & Mattick, K. L. (2024) · Medical Education · doi:10.1111/medu.15482 · Cited by 23 · see citing works
Performance pressure: Simulated patients and high‐stakes examinations in a regional clinical school
Harvey, P., & Radomski, N. (2011) · Australian Journal of Rural Health · doi:10.1111/j.1440-1584.2011.01231.x · Cited by 18 · see citing works
‘How can we help the students learn?’ A grounded theory study of simulated participants as educators
Sullivan, C., Doyle, A. J., O’Toole, M., Mulhall, C. M., McNaughton, N., & Eppich, W. (2023) · Medical Teacher · doi:10.1080/0142159x.2023.2171857 · Cited by 15 · see citing works
Chen, J. Y., Chin, W., & Tsang, J. P. Y. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1665635 · Cited by 11 · see citing works
Realist Reviews: A Brief How-To
Kent, F., & Ajjawi, R. (2022) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-22-00335.1 · Cited by 9 · see citing works
Dufayet, L., Piot, M., Geoffroy, P. A., Oulès, B., Brichant‐Petitjean, C., Peiffer‐Smadja, N., Bouzid, D., Tran‐Dinh, A., Mirault, T., Faye, A., et al. (2024) · Medical Teacher · doi:10.1080/0142159x.2024.2306840 · Cited by 8 · see citing works
Training framework for high‐stakes OSCE: Experience from volunteer standardized patients' bank
Feigerlová, E. (2024) · The Clinical Teacher · doi:10.1111/tct.13787 · Cited by 7 · see citing works
Corrigan, M., Reid, H., & McKeown, P. (2021) · Advances in Simulation · doi:10.1186/s41077-021-00179-9 · Cited by 5 · see citing works
Understanding students’ self-efficacy and motivation in sequential OSCE: a qualitative study
Lim, J. J., Roberts, C. J., Ali, S. N. J., Collini, P., Burney, A., & Hughes, D. (2025) · BMC Medical Education · doi:10.1186/s12909-025-08301-5 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Objective Structured Clinical Examination (OSCE) in the Training of Medical Specialists
Nykytyuk, S. O., Hariyan, T., Kyrychok, O., Martynyuk, L. P., & Potikha, N. (2026) · Academic Visions · doi:10.66556/2786-586x.55.nykytyuk-s
- Authors
- El-Sherif, A., Lim, J. J., & Roberts, C.
- Affiliations
- University of Sheffield (GB)
- Newcastle University (GB)
- Corresponding author
- Lim, Jun Jie
- ORCID
- Roberts, Chris
- Venue
- Medical Teacher
- Publisher
- Informa UK Limited
- Published
- 2026-03-18
- Volume
- 48
- Issue
- 10
- Pages
- 1715-1724
- ISSN
- 0142-159X (ISSN-L)
- 1466-187X
- OpenAlex
- W7138943060
- Cited by
- 1
- References
- 34
- Citations by year
- 2026: 1
- FWCI
- 6.51
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Medical Teacher (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Health Sciences › Medicine › Physiology › Simulation-Based Education in Healthcare
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Keywords
- standardized patients, OSCE, summative assessment, evaluation bias, medical student performance, role clarity, role ambiguity, assessment design, program theory
- MeSH terms
- Clinical Competence, Decision Making, Education, Medical, Undergraduate, Educational Measurement, United Kingdom, Humans, Interviews as Topic, Judgment, Students, Medical, Patient Simulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
