Enhancing the Reliability of the Multiple Mini-Interview for Selecting Prospective Health Care Leaders
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
PURPOSE: The David Geffen School of Medicine at UCLA Program in Medical Education (UCLA-PRIME) used a 12-station multiple mini-interview (MMI) circuit to assess applicants. The authors sought to determine the reliability of the MMI, potential bias in scores, and the degree of acceptance by interviewers and applicants. METHOD: In 2009, 28 interviewers interviewed a cohort of 76 applicants. An anonymous survey assessed interviewers' and applicants' satisfaction with the MMI process and perceived bias. Psychometric properties were determined with generalizability and decision theory. The process was repeated the following year with a new cohort of 78 applicants and minor modifications aimed at improving reliability. RESULTS: The MMI format was well received by both applicants and interviewers. No bias based on gender or disadvantaged status was found. The preliminary reliability of the MMI in 2009 was 0.58-lower than reported in previous studies-but improved in 2010 to 0.71 after an easy station was replaced with a more challenging one and a new scoring rubric was introduced. CONCLUSIONS: This interview technique proved to be reliable and was seen as transparent, uniform, and fair. The predictive validity of this process remains to be determined.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The preliminary reliability of the MMI in 2009 was 0.58 – lower than reported in previous studies – but improved in 2010 to 0.71 after an easy station was replaced with a more challenging.”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as Uijtdehaage S. et al., (30) 2011
In the citing paper
“Applicants indicated that MMI process was free from cultural bias (average 6.3) or gender bias (average 6.6) in 2009”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as Uijtdehaage S. et al., (30) 2011
In the citing paper
“2010 cohort rated the MMI process as less stressful (average 3.7 in 2010 versus 4.2 in 2009)”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as Uijtdehaage S. et al., (30) 2011
Citations
Cites
An admissions OSCE: the multiple mini-interview
Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004) · Medical Education · doi:10.1046/j.1365-2923.2004.01776.x · Cited by 617
Predictive validity of the multiple mini-interview for selecting medical trainees
Eva, K. W., Reiter, H. I., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G. R. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03407.x · Cited by 267
Multiple mini-interviews predict clerkship and licensing examination performance
Reiter, H. I., Eva, K. W., Rosenfeld, J., & Norman, G. R. (2007) · Medical Education · doi:10.1111/j.1365-2929.2007.02709.x · Cited by 235
Personality scale validities increase throughout medical school.
Lievens, F., Ones, D. S., & Dilchert, S. (2009) · Journal of Applied Psychology · doi:10.1037/a0016137 · Cited by 213
Engel, B. T. (1967) · Archives of General Psychiatry · doi:10.1001/archpsyc.1967.01730230058008 · Cited by 187
Investigating the Reliability of the Medical School Admissions Interview
Kreiter, C. D., Yin, P., Solow, C. M., & Brennan, R. L. (2004) · Advances in Health Sciences Education · doi:10.1023/b:ahse.0000027464.22411.0f · Cited by 184
On the generality of specificity
Eva, K. W. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01563.x · Cited by 178
Assessment of non-cognitive traits through the admissions multiple mini-interview
Lemay, J., Lockyer, J. M., Collin, V. T., & Brownell, A. K. W. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02767.x · Cited by 156
Harris, S., & Owen, C. (2007) · Medical Education · doi:10.1111/j.1365-2929.2007.02682.x · Cited by 101
Rosenfeld, J. M., Reiter, H. I., Trinh, K., & Eva, K. W. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9029-z · Cited by 88
Research works printed but not linked: 1
- Norman, 2010, March, 15, 1
Cited by
How effective are selection methods in medical education? A systematic review
Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F., & Cleland, J. (2016) · Medical Education · doi:10.1111/medu.12817 · Cited by 481
Pau, A., Jeevaratnam, K., Chen, Y. S., Fall, A. A., Khoo, C., & Nadarajah, V. D. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.829912 · Cited by 195
“2009), transparent (Uijtdehaage et al. 2011) and providing opportunities for the interviewees to regain composure if they had problems with a previous station (Kumar et al 2009).”
BackgroundRees, E. L., Hawarden, A. W., Dent, G., Hays, R., Bates, J., & Hassell, A. B. (2016) · Medical Teacher · doi:10.3109/0142159x.2016.1158799 · Cited by 139
“Uijtdehaage et al. (2011) found no difference in MMI score for (self-reported) economically disadvantaged applicants; findings from Reiter et al. (2012) corroborate this.”
BackgroundResultInfluentialMultiple mini-interviews: same concept, different approaches
Knorr, M., & Hissbach, J. (2014) · Medical Education · doi:10.1111/medu.12535 · Cited by 110
“Only McMaster University provides results for all three of these clusters (see Table 2).”
InfluentialInternet-based multiple mini-interviews for candidate selection for graduate entry programmes
Tiller, D., O'Mara, D., Rothnie, I., Dunn, S., Lee, L., & Roberts, C. (2013) · Medical Education · doi:10.1111/medu.12224 · Cited by 61
Metadata
- Authors
- Uijtdehaage, S., Doyle, L. “., & Parker, N.
- Affiliations
- University of California, Los Angeles (US)
- Corresponding author
- Uijtdehaage, Sebastian
- Venue
- Academic Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-08
- Volume
- 86
- Issue
- 8
- Pages
- 1032-1039
- ISSN
- 1040-2446 (ISSN-L)
- OpenAlex
- W2051596482
- Cited by
- 57
- References
- 20
- Citations by year
- 2024: 1 · 2023: 2 · 2022: 5 · 2021: 2 · 2020: 2 · 2019: 5 · 2018: 8 · 2017: 5 · 2016: 8 · 2015: 7 · 2014: 8 · 2013: 4
- FWCI
- 3.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Academic Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Education › Medical Education and Admissions
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Gender Studies › Diversity and Career in Medicine
- Keywords
- multiple mini-interviews, medical school admissions, healthcare leadership, applicant selection, inter-rater reliability, interview bias, gender bias, rubrics, applicant satisfaction, generalizability theory, decision theory
- MeSH terms
- Adult, Education, Medical, Female, Humans, Interviews as Topic, Leadership, Male, Psychometrics, School Admission Criteria, Reproducibility of Results, Cohort Studies, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Enhancing the Reliability of the Multiple Mini-Interview for Selecting Prospective Health Care Leaders. https://app.cochat.ai/papers/10.1097/acm.0b013e3182223ab7
- The paper
- Uijtdehaage, S., Doyle, L. “., & Parker, N. (2011). Enhancing the Reliability of the Multiple Mini-Interview for Selecting Prospective Health Care Leaders. Academic Medicine. https://doi.org/10.1097/acm.0b013e3182223ab7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.