Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37
Open accessThe DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index.
Abstract
BACKGROUND: In the 11 years since its development at McMaster University Medical School, the multiple mini-interview (MMI) has become a popular selection tool. We aimed to systematically explore, analyze and synthesize the evidence regarding MMIs for selection to undergraduate health programs. METHODS: The review protocol was peer-reviewed and prospectively registered with the Best Evidence Medical Education (BEME) collaboration. Thirteen databases were searched through 34 terms and their Boolean combinations. Seven key journals were hand-searched since 2004. The reference sections of all included studies were screened. Studies meeting the inclusion criteria were coded independently by two reviewers using a modified BEME coding sheet. Extracted data were synthesized through narrative synthesis. RESULTS: A total of 4338 citations were identified and screened, resulting in 41 papers that met inclusion criteria. Thirty-two studies report data for selection to medicine, six for dentistry, three for veterinary medicine, one for pharmacy, one for nursing, one for rehabilitation, and one for health science. Five studies investigated selection to more than one profession. MMIs used for selection to undergraduate health programs appear to have reasonable feasibility, acceptability, validity, and reliability. Reliability is optimized by including 7-12 stations, each with one examiner. The evidence is stronger for face validity, with more research needed to explore content validity and predictive validity. In published studies, MMIs do not appear biased against applicants on the basis of age, gender, or socio-economic status. However, applicants of certain ethnic and social backgrounds did less well in a very small number of published studies. Performance on MMIs does not correlate strongly with other measures of noncognitive attributes, such as personality inventories and measures of emotional intelligence. DISCUSSION: MMI does not automatically mean a more reliable selection process but it can do, if carefully designed. Effective MMIs require careful identification of the noncognitive attributes sought by the program and institution. Attention needs to be given to the number of stations, the blueprint and examiner training. CONCLUSION: More work is required on MMIs as they may disadvantage groups of certain ethnic or social backgrounds. There is a compelling argument for multi-institutional studies to investigate areas such as the relationship of MMI content to curriculum domains, graduate outcomes, and social missions; relationships of applicants' performance on different MMIs; bias in selecting applicants of minority groups; and the long-term outcomes appropriate for studies of predictive validity.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The multiple mini interview (MMI) format has been widely adopted in medical school admissions and can assess a range of nonacademic skills, such as communication, ethical reasoning, teamwork, and empathy [1-5].”
From AI-Generated Multiple Mini Interview (MMI) Stations for Medical School Admissions: Psychometric Evaluation (2026) · cites this paper as [4]
In the citing paper
“The latest article included in the systemic review was in 2014”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as [7]
In the citing paper
“but applicants of certain ethnic and social backgrounds demonstrated low performance in a very small number of published studies.7”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as [7]
In the citing paper
“it only focused on the utility of MMI in health profession education.”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as [7]
In the citing paper
“A recent systematic review revealed that MMIs used for the selection of undergraduate health programs appear to have reasonable feasibility, acceptability, validity, and reliability.7”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as [7]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Salovey, P., & Mayer, J. D. (1990) · Imagination, Cognition and Personality · doi:10.2190/dugg-p24e-52wk-6cdg · Cited by 10,484
Messick, S. (1995) · American Psychologist · doi:10.1037/0003-066x.50.9.741 · Cited by 4,198
The Intraclass Correlation Coefficient as a Measure of Reliability
Bartko, J. J. (1966) · Psychological Reports · doi:10.2466/pr0.1966.19.1.3 · Cited by 2,299
Standards for Educational and Psychological Testing.
García, E. J. (2017) · Revista Española de Pedagogía · doi:10.22550/2174-0909.3051 · Cited by 2,217
Diversity and Higher Education: Theory and Impact on Educational Outcomes
Gurin, P., Dey, E., Hurtado, S., & Gurin, G. (2002) · Harvard Educational Review · doi:10.17763/haer.72.3.01151786u134n051 · Cited by 1,975 · cited 1×
“This diversity has been shown to increase learning, change attitudes, and increase health care provision over time to underserved populations (Gurin et al. 2002; Whitla et al. 2003).”
BackgroundThe assessment of professional competence: Developments, research and practical implications
Van Der Vleuten, C. P. M. (1996) · Advances in Health Sciences Education · doi:10.1007/bf00596229 · Cited by 1,247
A contemplated revision of the NEO Five-Factor Inventory
McCrae, R. R., & Costa, P. T. (2004) · Personality and Individual Differences · doi:10.1016/s0191-8869(03)00118-1 · Cited by 1,153
TRAINING AND DEVELOPMENT HANDBOOK.
Craig, R. L., & Bittel, L. R. (1967) · CERN Document Server (European Organization for Nuclear Research) · Cited by 926
Liaison Committee on Medical Education: Past Successes, Future Challenges
Schwarz, M. R. (1992) · JAMA · doi:10.1001/jama.1992.03490090033010 · Cited by 916
Reeves, M. (1993) · Cited by 903
Q Methodology and Qualitative Research
Brown, S. R. (1996) · Qualitative Health Research · doi:10.1177/104973239600600408 · Cited by 830 · cited 1×
“For example, Harris and Owen (2007) utilized Q methodology (Brown 1996) to determine the attributes most valued by stakeholders at the Australian National University, and identified six factors: love of medicine and learning, groundedness, self-confidence, balanced approach, mature social skills,…”
MethodAn admissions OSCE: the multiple mini-interview
Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004) · Medical Education · doi:10.1046/j.1365-2923.2004.01776.x · Cited by 617 · cited 25×
“Generalizability analyses have repeatedly indicated that MMIs with greater numbers of stations will have greater reliability (Eva et al. 2004c; Roberts et al. 2008; Sebok et al. 2013).”
BackgroundMethodResultInfluentialAssessing Personal Qualities in Medical School Admissions
Albanese, M. A., Snow, M. H., Skochelak, S. E., Huggett, K. N., & Farrell, P. M. (2003) · Academic Medicine · doi:10.1097/00001888-200303000-00016 · Cited by 335 · cited 1×
“Clearly such professional programs should select for a combination of cognitive and non-cognitive attributes (Albanese et al. 2003).”
BackgroundReliability and Validity of Admissions Tools Used to Select Students for the Health Professions
Salvatori, P. (2001) · Advances in Health Sciences Education · doi:10.1023/a:1011489618208 · Cited by 311 · cited 2×
“2012) have all been tried, to no avail; each of these approaches is fraught with low reliability (Salvatori 2001; Kulatunga-Moruzi & Norman 2002).”
BackgroundSocial accountability and accreditation: a new frontier for educational institutions
Boelen, C., & Woollard, B. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03413.x · Cited by 307 · cited 1×
“These requirements for admission processes are embedded in accreditation standards across the Western world (Boelen & Woollard 2009; Liaison Committee on Medical Education 2012).”
BackgroundEducational Benefits of Diversity in Medical School: A Survey of Students
Whitla, D. K., Orfield, G., Silen, W., Teperow, C., Howard, C., & Reede, J. (2003) · Academic Medicine · doi:10.1097/00001888-200305000-00007 · Cited by 299 · cited 1×
“This diversity has been shown to increase learning, change attitudes, and increase health care provision over time to underserved populations (Gurin et al. 2002; Whitla et al. 2003).”
BackgroundDonnon, T., Paolucci, E. O., & Violato, C. (2007) · Academic Medicine · doi:10.1097/01.acm.0000249878.25186.b7 · Cited by 297 · cited 2×
“In the countries where graduate entry to health programs is more usual, such as the USA and Australia, national admissions tests such as the Medical College Admission Test (MCAT) and Graduate Australian Medical Schools Admissions Test (GAMSAT) provide a common assessment of cognitive ability irrespective of the underg…”
MethodResultPredictive validity of the multiple mini-interview for selecting medical trainees
Eva, K. W., Reiter, H. I., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G. R. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03407.x · Cited by 267 · cited 2×
“Further, MMI scores from this population were correlated with number of stations passed on the Medical Council of Canada Qualifying Examination (MCCQE) part II (r¼ 0.35, p< 0.05; Eva et al. 2009).”
ResultInfluentialGeneralizability theory for the perplexed: A practical introduction and guide: AMEE Guide No. 68
Bloch, R., & Norman, G. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.703791 · Cited by 248 · cited 1×
“Generalizability (G) refers to the contribution to the overall variance in scores that can be attributed to the variable under investigation (Bloch & Norman 2012).”
BackgroundMultiple mini-interviews predict clerkship and licensing examination performance
Reiter, H. I., Eva, K. W., Rosenfeld, J., & Norman, G. R. (2007) · Medical Education · doi:10.1111/j.1365-2929.2007.02709.x · Cited by 235 · cited 2×
“…(b¼ 0.44, p< 0.01; Eva et al. 2004a), clinical OSCE performance (b¼ 0.4, p< 0.05), clerkship performance, measured by end of clerkship ratings assigned by clerkship directors (b¼ 0.7, p< 0.001) and clinical encounter cards provided by clinical supervisors (b¼ 0.5, p< 0.01; Reiter et al. 2007).”
BackgroundMethodInfluentialThe ability of the multiple mini-interview to predict preclerkship performance in medical school
Eva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004) · Academic Medicine · doi:10.1097/00001888-200410001-00012 · Cited by 225 · cited 20×
“Generalizability analyses have repeatedly indicated that MMIs with greater numbers of stations will have greater reliability (Eva et al. 2004c; Roberts et al. 2008; Sebok et al. 2013).”
BackgroundMethodResultInfluentialPrideaux, D., Roberts, C., Eva, K., Centeno, A., Mccrorie, P., Mcmanus, C., Patterson, F., Powis, D., Tekian, A., & Wilkinson, D. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.551560 · Cited by 221 · cited 2×
“Though usually associated with testing applicants enrolled in a program, the principles of assessments are equally important in making decisions as to who should be admitted to the program in the first place (Prideaux et al. 2011).”
BackgroundEmotional intelligence in medical education: a critical review
Cherry, M. G., Fletcher, I., O'Sullivan, H., & Dornan, T. (2014) · Medical Education · doi:10.1111/medu.12406 · Cited by 203 · cited 1×
“Cherry et al. (2014) warn against using assessments of emotional intelligence in admissions, and suggest, rather, that attention should be paid to developing students’ emotional intelligence within the curriculum.”
BackgroundPau, A., Jeevaratnam, K., Chen, Y. S., Fall, A. A., Khoo, C., & Nadarajah, V. D. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.829912 · Cited by 195 · cited 1×
“In a recent systematic review that sought to explore the evidence for the reliability, validity, acceptability, and feasibility of the MMI in the selection of health profession students, Pau et al. (2013) concluded that MMIs were feasible and acceptable.”
BackgroundEngel, B. T. (1967) · Archives of General Psychiatry · doi:10.1001/archpsyc.1967.01730230058008 · Cited by 187
Investigating the Reliability of the Medical School Admissions Interview
Kreiter, C. D., Yin, P., Solow, C. M., & Brennan, R. L. (2004) · Advances in Health Sciences Education · doi:10.1023/b:ahse.0000027464.22411.0f · Cited by 184 · cited 1×
“Panel interviews, however, do not offer sufficient reliability to ensure the correct applicants are selected (Kreiter et al. 2004).”
BackgroundKulatunga Moruzi, C., & Norman, G. R. (2002) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1401_9 · Cited by 179
On the generality of specificity
Eva, K. W. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01563.x · Cited by 178 · cited 1×
“One problem inherent in panel interviews is context specificity: how an applicant behaves in one situation is not predictive of how they behave in others (Eva 2003; van der Vleuten 2014).”
BackgroundGeneral overview of the theories used in assessment: AMEE Guide No. 57
Schuwirth, L. W. T., & van der Vleuten, C. P. M. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.611022 · Cited by 176
Characteristics of first-year students in Canadian medical schools.
Dhalla, I. A., Kwong, J., Streiner, D. L., Baddour, R. E., Waddell, A., & Johnson, I. L. (2002) · PubMed Central · Cited by 157
Assessment of non-cognitive traits through the admissions multiple mini-interview
Lemay, J., Lockyer, J. M., Collin, V. T., & Brownell, A. K. W. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02767.x · Cited by 156 · cited 2×
“Finally, Lemay et al. (2007) report the analysis of their MMI to reveal a 10-factor solution, with each station forming a single factor.”
BackgroundMethodInfluentialSTORIES statement: Publication standards for healthcare education evidence synthesis
Gordon, M., & Gibbs, T. (2014) · BMC Medicine · doi:10.1186/s12916-014-0143-0 · Cited by 146 · cited 1×
“This is a systematic review reported in accordance with the STORIES statement (Gordon & Gibbs 2014).”
Eva, K. W., Reiter, H. I., Rosenfeld, J., Trinh, K., Wood, T. J., & Norman, G. R. (2012) · JAMA · doi:10.1001/jama.2012.36914 · Cited by 130 · cited 10×
“MMI scores have been reported to have low correlations with measures of past academic performance such as GPA (r¼ 0.006–0.06; Kulasegaram et al. 2010; Eva et al. 2012), prepharmacy average (r ¼ 0.025; Cameron & MacKeigan 2012) or Universities Admission Index (r ¼ 0.03 to 0.11; Griffin & Wilson 2012).”
BackgroundMethodInfluentialRoberts, C., Walton, M., Rothnie, I., Crossley, J., Lyon, P., Kumar, K., & Tiller, D. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03018.x · Cited by 121 · cited 5×
“Generalizability analyses have repeatedly indicated that MMIs with greater numbers of stations will have greater reliability (Eva et al. 2004c; Roberts et al. 2008; Sebok et al. 2013).”
BackgroundResultInfluentialDo Racial and Ethnic Group Differences in Performance on the MCAT Exam Reflect Test Bias?
Davis, D., Dorsey, J. K., Franks, R. D., Sackett, P. R., Searcy, C. A., & Zhao, X. (2013) · Academic Medicine · doi:10.1097/acm.0b013e318286803a · Cited by 120 · cited 3×
“This differential performance did not result in smaller proportions of Black or Hispanic applicants receiving offers of study, as other criteria were also considered in the selection process (Davis et al. 2013).”
BackgroundResultEva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004) · Academic Medicine · doi:10.1097/00001888-200406000-00021 · Cited by 112 · cited 18×
“Generalizability analyses have repeatedly indicated that MMIs with greater numbers of stations will have greater reliability (Eva et al. 2004c; Roberts et al. 2008; Sebok et al. 2013).”
BackgroundMethodResultInfluentialMultiple mini-interviews: same concept, different approaches
Knorr, M., & Hissbach, J. (2014) · Medical Education · doi:10.1111/medu.12535 · Cited by 110
Young, M. E., Razack, S., Hanson, M. D., Slade, S., Varpio, L., Dore, K. L., & McKnight, D. (2012) · Academic Medicine · doi:10.1097/acm.0b013e31826daf74 · Cited by 110 · cited 1×
“The finding that aboriginal applicants and rural applicants attain significantly lower scores on MMI is worrying given that these cohorts are already underrepresented in health professions (Dhalla et al. 2002; Young et al. 2012).”
BackgroundAdmissions processes for five year medical courses at English schools: review
Parry, J., Mathers, J., Stevens, A., Parsons, A., Lilford, R., Spurgeon, P., & Thomas, H. (2006) · BMJ · doi:10.1136/bmj.38768.590174.55 · Cited by 105 · cited 1×
“Within the UK the majority of schools have traditionally used a combination of academic ability, a personal statement, and a reference to shortlist applicants for a panel interview (Parry et al. 2006).”
BackgroundHarris, S., & Owen, C. (2007) · Medical Education · doi:10.1111/j.1365-2929.2007.02682.x · Cited by 101 · cited 2×
“…other interview methods such as panel interviews as they required fewer interviewers and less time commitment per interviewer (Brownell et al. 2007; Harris & Owen 2007), though it should be noted that many MMIs incur the extra logistics of organizing simulated patients for communication…”
MethodRosenfeld, J. M., Reiter, H. I., Trinh, K., & Eva, K. W. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9029-z · Cited by 88
Predictive validity of the Dundee multiple mini-interview
Husbands, A., & Dowell, J. (2013) · Medical Education · doi:10.1111/medu.12193 · Cited by 83 · cited 1×
“The MMI used to select medical students at Dundee in 2009 (n¼ 128) significantly predicted OSCE performance in both semesters of year 1 (semester 1 b¼ 0.18, p¼ 0.034; semester 2 b¼ 0.34, p< 0.001), and in year 2 (b¼ 0.30, p< 0.001; Husbands & Dowell 2013).”
MethodInfluentialMultiple mini-interviews versus traditional interviews: stakeholder acceptability comparison
Razack, S., Faremo, S., Drolet, F., Snell, L., Wiseman, J., & Pickering, J. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03447.x · Cited by 82 · cited 5×
“Razack et al. (2009) reported that applicants found the MMI more stressful than the traditional interview (3.78 versus 3.39 on six-point Likert scale, F¼ 6.04, p¼ 0.016).”
BackgroundInfluentialHow to Design a useful Test: The Principles of Assessment
Schuwirth, L., & van der Vleuten, C. P. M. (2010) · Wiley eBooks · doi:10.1002/9781444320282.ch14 · Cited by 82
McManus, I. C., Dewberry, C., Nicholson, S., Dowell, J. S., Woolf, K., & POTTS, H. W. W. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-243 · Cited by 75 · cited 1×
“A levels), has been demonstrated to be highly predictive of academic performance in medical school, accounting for 65% of variance in undergraduate and postgraduate examination performance (McManus et al. 2013).”
BackgroundGriffin, B., Harding, D. W., Wilson, I. G., & Yeomans, N. D. (2008) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2008.tb02024.x · Cited by 74 · cited 1×
“Griffin et al. (2008) investigated the effect of previous interview experience and coaching on MMI performance in a sample of 287 applicants.”
BackgroundThe multiple mini-interview in the UK context: 3 years of experience at Dundee
Dowell, J., Lynch, B., Till, H., Kumwenda, B., & Husbands, A. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652706 · Cited by 74 · cited 14×
“Challenges posed through the introduction of MMIs include: recruiting sufficient interviewers, space, organization and station development (Dowell et al. 2012).”
BackgroundInfluentialBrownell, K., Lockyer, J., Collin, T., & Lemay, J.-F. (2007) · Medical Teacher · doi:10.1080/01421590701311713 · Cited by 72 · cited 16×
“MMIs have been reported to be feasible (Brownell et al. 2007), and some schools have even found them to be ‘‘logistically simpler’’ than other interview methods such as panel interviews as they required fewer interviewers and less time commitment per interviewer (Brownell et al.”
BackgroundMethodResultInfluentialJerant, A., Griffin, E., Rainwater, J., Henderson, M., Sousa, F., Bertakis, K. D., Fenton, J. J., & Franks, P. (2012) · Academic Medicine · doi:10.1097/acm.0b013e31826102ad · Cited by 70 · cited 5×
“Jerant et al. (2012) reported a positive association between MMI score and female sex (p< 0.01), though no significant differences were found in five other studies (Eva et al. 2004c; Hecker et al. 2009; O’Brien et al. 2011; Uijtdehaage et al. 2011; Reiter et al. 2012).”
BackgroundMethodResultInfluentialExperiences of the multiple mini-interview: a qualitative analysis
Kumar, K., Roberts, C., Rothnie, I., Du Fresne, C., & Walton, M. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03291.x · Cited by 68 · cited 6×
“Interviewers indicated that they perceive the MMI to be a fair selection tool (Brownell et al. 2007; Kumar et al. 2009; Razack et al. 2009; Dowell et al. 2012), but some have concerns regarding how stressful it may be for applicants (Kumar et al. 2009; Dowell et al. 2012).”
BackgroundInfluentialPredicting academic outcomes in an Australian graduate entry medical programme
Puddey, I. B., & Mercer, A. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-31 · Cited by 65 · cited 1×
“…Medical Schools Admissions Test (GAMSAT) provide a common assessment of cognitive ability irrespective of the undergraduate program of study, and deliver similar results: they are highly predictive of academic achievement throughout medical school (Donnon et al. 2007; Puddey & Mercer 2014).”
ResultCameron, A. J., & MacKeigan, L. D. (2012) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe76110 · Cited by 63 · cited 12×
“…r¼ 0.13–0.22), and 3 (non-verbal reasoning; r¼ 0.11 to 0.06; Griffin & Wilson 2012), UK Clinical Aptitude Test (b¼ 0.00, p¼ 0.28; O’Brien et al. 2011), Medical College Admissions Test (r¼ 0.10; Kulasegaram et al. 2010), and Pharmacy College Admission Test (r¼ 0.042; Cameron & MacKeigan 2012).”
BackgroundInfluentialReiter, H. I., Lockyer, J., Ziola, B., Courneya, C.-A., & Eva, K. (2012) · Academic Medicine · doi:10.1097/acm.0b013e318248f7f3 · Cited by 61 · cited 5×
“Uijtdehaage et al. (2011) found no difference in MMI score for (self-reported) economically disadvantaged applicants; findings from Reiter et al. (2012) corroborate this.”
BackgroundResultInfluentialInternet-based multiple mini-interviews for candidate selection for graduate entry programmes
Tiller, D., O'Mara, D., Rothnie, I., Dunn, S., Lee, L., & Roberts, C. (2013) · Medical Education · doi:10.1111/medu.12224 · Cited by 61 · cited 2×
“In an effort to reduce costs associated with mounting an MMI at an international site for international applicants, Tiller et al. (2013) introduced an internet-based iMMI that utilized Skype.”
BackgroundAdapting health care institutions and medical schools to societiesʼ needs
Boelen, C. (1999) · Academic Medicine · doi:10.1097/00001888-199908000-00024 · Cited by 61 · cited 1×
“Other medical schools espouse a social accountability framework, defined by the WHO as ‘‘the obligation to direct their education, research, and service activities towards addressing the priority health concerns of the community, region, and/or nation they have a mandate to serve’’ (Boelen 1999).”
BackgroundRoberts, C., Rothnie, I., Zoanetti, N., & Crossley, J. (2010) · Medical Education · doi:10.1111/j.1365-2923.2010.03689.x · Cited by 59 · cited 1×
“Three studies have reported using multi-faceted Rasch modeling (MFRM) to adjust for rater stringency or leniency within MMIs (Roberts et al. 2009, 2010; Till et al. 2013).”
Uijtdehaage, S., Doyle, L. “., & Parker, N. (2011) · Academic Medicine · doi:10.1097/acm.0b013e3182223ab7 · Cited by 57 · cited 13×
“Uijtdehaage et al. (2011) found no difference in MMI score for (self-reported) economically disadvantaged applicants; findings from Reiter et al. (2012) corroborate this.”
BackgroundResultInfluentialA comparison of multiple mini-interviews and structured interviews in a UK setting
O’Brien, A., Harvey, J., Shannon, M., Lewis, K., & Valencia, O. (2011) · Medical Teacher · doi:10.3109/0142159x.2010.541532 · Cited by 55 · cited 8×
“O’Brien et al. (2011) found no correlation between age and MMI score on either the undergraduate or graduate entry medicine programs at St George’s University of London.”
BackgroundResultInfluentialSubjective and objective admissions factors as predictors of clinical clerkship performance
Meredith, K. E., Dunlap, M. R., & Baker, H. H. (1982) · Academic Medicine · doi:10.1097/00001888-198210000-00002 · Cited by 53 · cited 1×
“In an attempt to demonstrate the predictive validity of cognitive and non-cognitive admissions measures, Meredith et al. (1982) arguably developed the precursor of the MMI.”
BackgroundAssessment of Applicants to the Veterinary Curriculum Using a Multiple Mini-Interview Method
Hecker, K., Donnon, T., Fuentealba, C., Hall, D., Illanes, O., Morck, D. W., & Muelling, C. (2009) · Journal of Veterinary Medical Education · doi:10.3138/jvme.36.2.166 · Cited by 50 · cited 8×
“Jerant et al. (2012) reported a positive association between MMI score and female sex (p< 0.01), though no significant differences were found in five other studies (Eva et al. 2004c; Hecker et al. 2009; O’Brien et al. 2011; Uijtdehaage et al. 2011; Reiter et al. 2012).”
BackgroundMethodResultInfluentialYates, J. (2012) · BMC Medical Education · doi:10.1186/1472-6920-12-43 · Cited by 49 · cited 1×
“Many of these programs have low attrition rates (Yates 2012; Fortin et al. 2015) and for some, their graduates are automatically entitled to registration with regulatory bodies; the vast majority of applicants selected will proceed to practise in the field.”
BackgroundRoberts, C., Zoanetti, N., & Rothnie, I. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03292.x · Cited by 49 · cited 2×
“Three studies have reported using multi-faceted Rasch modeling (MFRM) to adjust for rater stringency or leniency within MMIs (Roberts et al. 2009, 2010; Till et al. 2013).”
BackgroundAssociations between the big five personality factors and multiple mini-interviews
Griffin, B., & Wilson, I. (2012) · Advances in Health Sciences Education · doi:10.1007/s10459-011-9316-1 · Cited by 48 · cited 4×
“…parts 1 (logical reasoning ability; r¼ 0.11 to 0.01), 2 (interpersonal understanding; r¼ 0.13–0.22), and 3 (non-verbal reasoning; r¼ 0.11 to 0.06; Griffin & Wilson 2012), UK Clinical Aptitude Test (b¼ 0.00, p¼ 0.28; O’Brien et al. 2011), Medical College Admissions Test (r¼ 0.10;…”
BackgroundMethodInfluentialSebok‐Syer, S. S., Luu, K., & Klinger, D. A. (2013) · Advances in Health Sciences Education · doi:10.1007/s10459-013-9463-7 · Cited by 48 · cited 3×
“Generalizability analyses have repeatedly indicated that MMIs with greater numbers of stations will have greater reliability (Eva et al. 2004c; Roberts et al. 2008; Sebok et al. 2013).”
BackgroundInfluentialReiter, H. I., Salvatori, P., Rosenfeld, J., Trinh, K., & Eva, K. W. (2006) · Medical Education · doi:10.1111/j.1365-2929.2005.02348.x · Cited by 47 · cited 1×
“Reiter et al. (2006) observed that access to station details (for one of two pilot stations) in advance of an MMI did not result in improved performance over those who did not have access (4.92 versus 4.94, t(383)¼ 0.24, p> 0.8).”
BackgroundInfluentialPerkins, A., Burton, L., Dray, B., & Elcock, K. (2013) · Nurse Education Today · doi:10.1016/j.nedt.2012.04.023 · Cited by 46 · cited 1×
“Seventyone percent of interviewers at the School of Nursing, Kingston University and St George’s University of London stated a preference for MMIs over traditional interviews (Perkins et al. 2013).”
BackgroundValidating MMI scores: are we measuring multiple attributes?
Oliver, T., Hecker, K., Hausdorf, P. A., & Conlon, P. (2014) · Advances in Health Sciences Education · doi:10.1007/s10459-013-9480-6 · Cited by 45 · cited 5×
“Oliver et al. (2014) report that a two-factor model (oral communication and problem evaluation) best explains the variance in their MMI, though they note the two constructs were highly correlated at 0.87.”
BackgroundMethodInfluentialWhen I say … context specificity
van der Vleuten, C. (2014) · Medical Education · doi:10.1111/medu.12263 · Cited by 45 · cited 1×
“One problem inherent in panel interviews is context specificity: how an applicant behaves in one situation is not predictive of how they behave in others (Eva 2003; van der Vleuten 2014).”
BackgroundThe multiple mini‐interview: how long is long enough?
Dodson, M., Crotty, B., Prideaux, D., Carne, R., Ward, A., & De Leeuw, E. (2009) · Medical Education · doi:10.1111/j.1365-2923.2008.03260.x · Cited by 44 · cited 1×
“Dodson et al. (2009) studied 175 applicants for entry to Deakin University School of Medicine.”
BackgroundNon-association between Neo-5 personality tests and multiple mini-interview
Kulasegaram, K., Reiter, H. I., Wiesner, W., Hackett, R. D., & Norman, G. R. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-009-9209-8 · Cited by 41
An evaluation of the multiple mini-interview as a selection tool for dental students
McAndrew, R., & Ellis, J. (2012) · British Dental Journal · doi:10.1038/sj.bdj.2012.267 · Cited by 39 · cited 4×
“…their performance on one would not affect their performance on the next, each station offered a ‘‘clean slate’’ (Eva et al. 2004c; Kumar et al. 2009), although others felt that poor performance could not be forgotten and affected their performance on subsequent stations (McAndrew & Ellis 2012).”
BackgroundInfluentialReliability estimates: behavioural stations and questionnaires in medical school admissions
Gafni, N., Moshinsky, A., Eisenberg, O., Zeigler, D., & Ziv, A. (2012) · Medical Education · doi:10.1111/j.1365-2923.2011.04155.x · Cited by 33 · cited 6×
“2012) and strong correlations with a biographical questionnaire (MOR: r1⁄4 0.72, MIRKAM: r1⁄4 0.72; Gafni et al. 2012).”
BackgroundInfluentialImproving student selection using multiple mini-interviews with multifaceted Rasch modeling
Till, H., Myford, C., & Dowell, J. (2013) · Academic Medicine · doi:10.1097/acm.0b013e31827c0c5d · Cited by 31 · cited 2×
“Till et al. (2013) found that using ‘‘fair scores’’ (those adjusted for rater stringency) would alter the admissions decision for between 3.1% and 4.2% of applicants for undergraduate medicine at Dundee.”
InfluentialInterviewer bias in medical student selection
Griffin, B. N., & Wilson, I. G. (2010) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2010.tb03946.x · Cited by 31
The admissions process in a graduate-entry dental school: can we predict academic performance?
Foley, J., & Hijazi, K. (2013) · BDJ · doi:10.1038/sj.bdj.2013.56 · Cited by 30 · cited 1×
“Foley and Hijazi (2013) reported a correlation between the MMI scores of students at Aberdeen dental school (n¼ 75) and end of year assessments consisting of short answer questions and OSCES (r¼ 0.18, p¼ 0.001).”
BackgroundInfluentialHecker, K., & Violato, C. (2011) · Teaching and Learning in Medicine · doi:10.1080/10401334.2011.611769 · Cited by 26 · cited 3×
“This finding is contradicted by that of Hecker and Violato (2011) who found a nonsignificant main effect of interviewer type, faculty member (mean score: 10.33/15) versus community veterinarian (mean score: 10.06/15), on MMI scores (F1,1428¼ 3.18, p¼ 0.075).”
BackgroundResultInfluentialInterviewer bias in medical student selection
Griffin, B. N., & Wilson, I. G. (2010) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2010.tb04015.x · Cited by 25
Moreau, K., Reiter, H., & Eva, K. W. (2006) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1801_12 · Cited by 23 · cited 3×
“Moreau et al. (2006) reported that aboriginal interviewers or interviewees made no difference to MMI scores, although Reiter et al. (2012) found a negative correlation between aboriginal status and MMI score in both the 2008 (aboriginal: n¼ 45, z ¼ 0.69; other applicants: n¼ 1635, z¼ 0.02; F¼ 20.8,…”
BackgroundInfluentialYen, W., Hovey, R., Hodwitz, K., & Zhang, S. (2011) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9241-8 · Cited by 22 · cited 1×
“Yen et al. (2011) reported no correlation between a validated self-report measure of emotional intelligence, defined as ‘‘a type of social intelligence that involves the ability to monitor one’s own thinking and actions’’ (Salovey & Mayer 1990), and a total MMI score for 196 applicants for…”
BackgroundMultiple mini-interview scores of medical school applicants with and without rural attributes
Raghavan, M., Martin, B., Burnett, M., Aoki, F., Christensen, H., MacKalski, B., Young, D., & Ripstein, I. (2013) · Rural and Remote Health · doi:10.22605/rrh2362 · Cited by 21 · cited 6×
“Applicants who had graduated from rural high schools achieved significantly lower MMI scores than those from urban high schools in a study of applicants to the University of Manitoba, Canada (4.4 versus 4.6, t¼ 2.96, p¼ 0.003; Raghavan et al. 2013).”
BackgroundInfluentialFortin, Y., Kealey, L., Slade, S., & Hanson, M. D. (2015) · Medical Teacher · doi:10.3109/0142159x.2015.1045847 · Cited by 14 · cited 1×
“Many of these programs have low attrition rates (Yates 2012; Fortin et al. 2015) and for some, their graduates are automatically entitled to registration with regulatory bodies; the vast majority of applicants selected will proceed to practise in the field.”
BackgroundMcAndrew, R., & Ellis, J. (2013) · British Dental Journal · doi:10.1038/sj.bdj.2013.1142 · Cited by 6
Research works printed but not linked: 4
- Dhalla IA, 2002, Can Med Assoc J, 166, 1029
- Meredith KE, 1982, J Med Educ, 57, 743
- Nayer M., 1992, Physiother Can, 44, 41
- Raghavan M, 2013, Rural Remote Health., 13, 2362
Cited by
Medical student wellbeing – a consensus statement from Australia and New Zealand
Kemp, S., Hu, W., Bishop, J., Forrest, K., Hudson, J. N., Wilson, I., Teodorczuk, A., Rogers, G. D., Roberts, C., & Wearn, A. (2019) · BMC Medical Education · doi:10.1186/s12909-019-1505-2 · Cited by 87
What we measure … and what we should measure in medical education
Boulet, J. R., & Durning, S. J. (2019) · Medical Education · doi:10.1111/medu.13652 · Cited by 77
Knowledge syntheses in medical education: A bibliometric analysis
Maggio, L. A., Costello, J. A., Norton, C. J., Driessen, E. W., & Artino, A. R. (2020) · Perspectives on Medical Education · doi:10.1007/s40037-020-00626-9 · Cited by 96
The UK medical education database (UKMED) what is it? Why and how might you use it?
Dowell, J. S., Cleland, J., Fitzpatrick, S., McManus, I. C., Nicholson, S., Oppé, T. E., Petty-Saphon, K., King, O. S., Smith, D. T., Thornton, S., et al. (2018) · BMC Medical Education · doi:10.1186/s12909-017-1115-9 · Cited by 65
Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review
Yusoff, M. S. B. (2019) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2019.03.006 · Cited by 57
Metadata
- Authors
- Rees, E. L., Hawarden, A. W., Dent, G., Hays, R., Bates, J., & Hassell, A. B.
- Affiliations
- University Hospitals of North Midlands NHS Trust (GB)
- Keele University (GB)
- University of Tasmania (AU)
- University of British Columbia (CA)
- Corresponding author
- Rees, Eliot L.
- Venue
- Medical Teacher
- Publisher
- Informa UK Limited
- Published
- 2016-04-06
- Volume
- 38
- Issue
- 5
- Pages
- 443-455
- ISSN
- 0142-159X (ISSN-L)
- 1466-187X
- OpenAlex
- W2329538042
- Cited by
- 139
- References
- 82
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 7 · 2023: 12 · 2022: 16 · 2021: 14 · 2020: 13 · 2019: 20 · 2018: 27 · 2017: 12 · 2016: 4
- FWCI
- 3.10
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Licence
- cc-by
- Locations
- Medical Teacher (publishedVersion)
- PubMed (publishedVersion)
- ResearchOnline at James Cook University (James Cook University) (acceptedVersion)
- eCite Digital Repository (University of Tasmania) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Topics
- Social Sciences › Education › Medical Education and Admissions
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Gender Studies › Diversity and Career in Medicine
- Keywords
- multiple mini-interviews, postgraduate admission, medical student selection, pharmacy, predictive validity, face validity, content validity, reliability statistics, examiner training, ethnic prejudice, socioeconomic status, emotional intelligence, personality tests
- MeSH terms
- Choice Behavior, Education, Medical, Undergraduate, Humans, Interviews as Topic, School Admission Criteria, Databases, Factual, Guidelines as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.