Multiple mini‐interview test characteristics: ‘tis better to ask candidates to recall than to imagine
Abstract
Context: The multiple mini‐interview ( MMI ), used to facilitate the selection of applicants in health professional programmes, has been shown to be capable of generating reliable data predictive of success. It is a process rather than a single instrument and therefore its psychometric properties can be expected to vary according to the stations generated, the alignment between the stations and the qualities an institution prioritises, and the outcomes used. The purpose of this study was to explore the MMI 's test characteristics when station type is manipulated. Methods: A 12‐station MMI was established in which four stations were presented in three different ways. These included: situational judgement ( SJ ) stations, in which applicants were asked to imagine what they would do in specific situations; behavioural interview ( BI ) stations, in which applicants were asked to recall what they did in experienced situations, and free form ( FF ) stations, which were unstructured in that the examiner was simply given a brief explanation of the intent of the station without further guidance on how to conduct the discussion. Four circuits of the 12 stations were run with one examiner within each station. Candidates and examiners were surveyed regarding their experience. The reliability of the scores derived from the assessment was analysed separately for each station type. Results: A total of 41 medical school candidates participated after completing the regular admission process. Although the score assigned did not differ across station type, BI stations more reliably differentiated between candidates ( g = 0.77) than did the other station types ( SJ , g = 0.69; FF , g = 0.66). The correlation between actual MMI scores and BI stations was also greatest ( BI , r = 0.57; SJ , r = 0.45; FF , r = 0.42). Candidates’ opinions indicated that FF stations were more anxiety‐provoking, less clear, and more difficult than structured stations ( SJ and BI stations). Examiner opinions indicated equivalence on these measures. Conclusions: The results suggest that structuring stations has value, although that value was gained only through the use of BI stations, in which candidates were asked to recall and discuss a specific experience of relevance to the purpose of the interview station.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In Eva & Macala (2014), candidates considered FF stations to be more challenging and more anxiety-provoking than either SJ or BJ stations.
“Candidates considered the FF stations to be more challenging and more anxiety-provoking than either the SJ or BJ stations”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as Eva K. W. and Macala C., (51) 2014
In Eva & Macala (2014), the correlation between actual MMI scores and BI stations was greatest (BI, r=0.57; SJ, r=0.45; FF, r=0.42).
“The correlation between actual MMI scores and BI stations was also greatest (BI, r = 0.57; SJ, r = 0.45; FF, r = 0.42)”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as Eva K. W. and Macala C., (51) 2014
In Eva & Macala (2014), behavioral interview stations more reliably differentiated between candidates (G=0.77) than did other station types (SJ, G=0.69; FF, G=0.66).
“Behavioral interview (BI) stations more reliably differentiated between candidates (G = 0.77) than did the other station types (SJ, G – 0.69; FF, G = 0.66)”
From Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review (2019) · cites this paper as Eva K. W. and Macala C., (51) 2014
Cites
The person and the situation: perspectives of social psychology
(1992) · Choice Reviews Online · doi:10.5860/choice.29-3604 · Cited by 2,205 · see citing works
Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.256 · Cited by 1,925 · see citing works
A model for programmatic assessment fit for purpose.
van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652239 · Cited by 892 · see citing works
Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream
Ambady, N., Bernieri, F. J., & Richeson, J. A. (2000) · Advances in experimental social psychology · doi:10.1016/s0065-2601(00)80006-4 · Cited by 716 · see citing works
The CanMEDS 2005 Physician Competency Framework: Better standards. Better physicians. Better care.
Frank, J. R. (2005) · Medical Entomology and Zoology · Cited by 643 · see citing works
An admissions OSCE: the multiple mini-interview
Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004) · Medical Education · doi:10.1046/j.1365-2923.2004.01776.x · Cited by 617 · see citing works
Self-citationNorcini, J. J., Anderson, M. B., Bóllela, V. R., Burch, V. C., Costa, M. J., Duvivier, R. J., Galbraith, R. M., Hays, R., Kent, A., Perrott, V., et al. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.551559 · Cited by 587 · see citing works
Barrick, M. R., Shaffer, J. A., & DeGrassi, S. W. (2009) · Journal of Applied Psychology · doi:10.1037/a0016532 · Cited by 431 · see citing works
Reliability and Validity of Admissions Tools Used to Select Students for the Health Professions
Salvatori, P. S. (2001) · Advances in Health Sciences Education · doi:10.1023/a:1011489618208 · Cited by 311 · see citing works
The use of impression management tactics in structured interviews: A function of question type?
Ellis, A. P. J., West, B. J., Ryan, A. M., & DeShon, R. P. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.6.1200 · Cited by 293 · see citing works
Predictive validity of the multiple mini-interview for selecting medical trainees
Eva, K. W., Reiter, H. I., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G. R. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03407.x · Cited by 267 · see citing works
Self-citationMultiple mini-interviews predict clerkship and licensing examination performance
Reiter, H. I., Eva, K. W., Rosenfeld, J., & Norman, G. R. (2007) · Medical Education · doi:10.1111/j.1365-2929.2007.02709.x · Cited by 235 · see citing works
Self-citationElstein, A. S., Shulman, L. S., & Sprafka, S. A. (1990) · Evaluation & the Health Professions · doi:10.1177/016327879001300102 · Cited by 216 · see citing works
Engel, B. T. (1967) · Archives of General Psychiatry · doi:10.1001/archpsyc.1967.01730230058008 · Cited by 185 · see citing works
On the generality of specificity
Eva, K. W. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01563.x · Cited by 182 · see citing works
Self-citationEXPERIENCE‐BASED AND SITUATIONAL INTERVIEW QUESTIONS: STUDIES OF VALIDITY
Pulakos, E. D., & Schmitt, N. (1995) · Personnel Psychology · doi:10.1111/j.1744-6570.1995.tb01758.x · Cited by 163 · see citing works
Taylor, P. J., & Small, B. (2002) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317902320369712 · Cited by 145 · see citing works
Eva, K. W., Reiter, H. I., Rosenfeld, J., Trinh, K., Wood, T. J., & Norman, G. R. (2012) · JAMA · doi:10.1001/jama.2012.36914 · Cited by 130 · see citing works
Self-citationEva, K. W., Reiter, H., Rosenfeld, J. M., & Norman, G. R. (2004) · Academic Medicine · doi:10.1097/00001888-200406000-00021 · Cited by 112 · see citing works
Self-citationRosenfeld, J. M., Reiter, H. I., Trinh, K., & Eva, K. W. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9029-z · Cited by 88 · see citing works
Self-citationGriffin, B., Harding, D. W., Wilson, I. G., & Yeomans, N. D. (2008) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2008.tb02024.x · Cited by 74 · see citing works
Scylla or Charybdis? Can we navigate between objectification and judgement in assessment?
Eva, K. W., & Hodges, B. D. (2012) · Medical Education · doi:10.1111/j.1365-2923.2012.04310.x · Cited by 69 · see citing works
Self-citationKlehe, U., & Latham, G. P. (2006) · Human Performance · doi:10.1207/s15327043hup1904_3 · Cited by 58 · see citing works
Reiter, H. I., Salvatori, P., Rosenfeld, J., Trinh, K., & Eva, K. W. (2006) · Medical Education · doi:10.1111/j.1365-2929.2005.02348.x · Cited by 47 · see citing works
Self-citationThe multiple mini‐interview: how long is long enough?
Michael, D., Crotty, B. J., Prideaux, D. J., Carne, R. P., Ward, A. C., & de Leeuw, E. (2009) · Medical Education · doi:10.1111/j.1365-2923.2008.03260.x · Cited by 45 · see citing works
Rater and occasion impacts on the reliability of pre‐admission assessments
Axelson, R. D., & Kreiter, C. D. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03537.x · Cited by 35 · see citing works
Axelson, R. D., Kreiter, C. D., Ferguson, K. J., Solow, C. M., & Huebner, K. (2010) · Teaching and Learning in Medicine · doi:10.1080/10401334.2010.511978 · Cited by 18 · see citing works
Kreiter, C. D., Solow, C. M., Brennan, R. L., Yin, P., Ferguson, K. J., & Huebner, K. (2005) · Teaching and Learning in Medicine · doi:10.1207/s15328015tlm1801_2 · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
How effective are selection methods in medical education? A systematic review
Patterson, F., Knight, A., Dowell, J. S., Nicholson, S., Cousans, F., & Cleland, J. (2015) · Medical Education · doi:10.1111/medu.12817 · Cited by 489 · see citing works
Validity: one word with a plurality of meanings
St‐Onge, C., Young, M. E., Eva, K. W., & Hodges, B. D. (2016) · Advances in Health Sciences Education · doi:10.1007/s10459-016-9716-3 · Cited by 91 · see citing works
Self-citationDore, K. L., Reiter, H., Kreuger, S., & Norman, G. R. (2016) · Advances in Health Sciences Education · doi:10.1007/s10459-016-9739-9 · Cited by 91 · see citing works
Roberts, C., Khanna, P., Rigby, L., Bartle, E. K., Llewellyn, A., Gustavs, J., Newton, L., Newcombe, J. P., Davies, M. W., Thistlethwaite, J. E., et al. (2017) · Medical Teacher · doi:10.1080/0142159x.2017.1367375 · Cited by 64 · see citing works
Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review
Yusoff, M. S. B. (2019) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2019.03.006 · Cited by 57 · see citing works
“, (35) 2012 Cross-sectional Study design* Australian School of Medicine 2006: N 1⁄4 364 2007: N 1⁄4 336 2008: N 1⁄4 337 Medical School Assess how MMI scores relate to both the five factors and their associated facets.”
BackgroundA systematic review of stakeholder views of selection methods for medical schools admission
Kelly, M. E., Patterson, F., O’Flynn, S., Mulligan, J., & Murphy, A. W. (2018) · BMC Medical Education · doi:10.1186/s12909-018-1235-x · Cited by 49 · see citing works
Eva, K. W. (2018) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2018.03.008 · Cited by 39 · see citing works
Self-citationHow Different Medical School Selection Processes Call upon Different Personality Characteristics
Schripsema, N. R., van Trigt, A. M., van der Wal, M. A., & Cohen‐Schotanus, J. (2016) · PLoS ONE · doi:10.1371/journal.pone.0150645 · Cited by 33 · see citing works
<p>Multiple Mini-Interviews: Current Perspectives on Utility and Limitations</p>
Ali, S., Sadiq Hashmi, M. S., Umair, M., Beg, M. A., & Huda, N. (2019) · Advances in Medical Education and Practice · doi:10.2147/amep.s181332 · Cited by 32 · see citing works
Anglim, J., Bozic, S., Little, J., & Lievens, F. (2017) · Advances in Health Sciences Education · doi:10.1007/s10459-017-9796-8 · Cited by 33 · see citing works
Impact of sociodemographic characteristics of applicants in multiple mini-interviews
Leduc, J., Rioux, R., Gagnon, R. J., Bourdy, C., & Dennis, A. A. (2016) · Medical Teacher · doi:10.1080/0142159x.2017.1270431 · Cited by 32 · see citing works
Callwood, A., Jeevaratnam, K., Kotronoulas, G., Schneider, A., Lewis, L. S., & Nadarajah, V. D. (2018) · Nurse Education Today · doi:10.1016/j.nedt.2018.01.016 · Cited by 174 · see citing works
Twelve tips for constructing a multiple mini-interview
Eva, K. W., Macala, C., & Fleming, B. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1429586 · Cited by 25 · see citing works
Self-citationBest practices for interviewing applicants for medical school admissions: a systematic review
Lin, J. C. H., Lokhande, A., Margo, C. E., & Greenberg, P. B. (2022) · Perspectives on Medical Education · doi:10.1007/s40037-022-00726-8 · Cited by 27 · see citing works
Yoshimura, H., Kitazono, H., Fujitani, S., Machi, J., Saiki, T., Suzuki, Y., & Ponnamperuma, G. G. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0361-y · Cited by 24 · see citing works
The reliability of a portfolio of workplace-based assessments in anesthesia training
Castanelli, D. J., van Loon, J., Jolly, B. C., & Weller, J. M. (2018) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-018-1251-7 · Cited by 22 · see citing works
Yamada, T., Sato, J., Yoshimura, H., Okubo, T., Hiraoka, E., Shiga, T., Kubota, T., Fujitani, S., Machi, J., & Ban, N. (2017) · BMC Medical Education · doi:10.1186/s12909-017-0898-z · Cited by 19 · see citing works
Breil, S. M., Forthmann, B., Hertel-Waszak, A., Ahrens, H., Brouwer, B., Schönefeld, E., Marschall, B., & Back, M. D. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1670337 · Cited by 19 · see citing works
Validity of a new assessment rubric for a short-answer test of clinical reasoning
Yeung, E., Kulasagarem, K., Woods, N. N., Dubrowski, A., Hodges, B. D., & Carnahan, H. (2016) · BMC Medical Education · doi:10.1186/s12909-016-0714-1 · Cited by 17 · see citing works
Multiple mini-interviews as a selection tool for initial teacher education admissions
Metsäpelto, R., Utriainen, J., Poikkeus, A., Muotka, J. S., Tolvanen, A. J., & Warinowski, A. (2022) · Teaching and Teacher Education · doi:10.1016/j.tate.2022.103660 · Cited by 11 · see citing works
Abdul Rahim, A. F., & Yusoff, M. S. B. (2016) · Education in Medicine Journal · doi:10.5959/eimj.v8i2.437 · Cited by 10 · see citing works
Does the Multiple Mini-Interview Address Stakeholder Needs? An Applicant's Perspective
Phillips, A. W., & Garmel, G. M. (2014) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2014.01.021 · Cited by 10 · see citing works
Rater perceptions of bias using the multiple mini-interview format: a qualitative study
Alweis, R. L., Fitzpatrick, C., & Donato, A. A. (2015) · Journal of Education and Training Studies · doi:10.11114/jets.v3i5.818 · Cited by 9 · see citing works
The factors that count in selecting future dentists: sensorimotor and soft skills
Mirghani, I., Mushtaq, F., Balkhoyor, A. M., Al‐Saud, L. M., Osnes, C. A., Keeling, A. J., Mon-Williams, M. A., & Manogue, M. R. (2019) · BDJ · doi:10.1038/s41415-019-0030-3 · Cited by 9 · see citing works
Intended and unintended test constructs in a Multiple-Mini admission Interview. A validity study.
O’Neill, L. D., Lykkegaard, E., & Kulasageram, K. (2019) · Dansk Universitetspædagogisk Tidsskrift · doi:10.7146/dut.v14i26.106217 · Cited by 5 · see citing works
Interviews, Multiple Mini-Interviews, and Selection Centers
Knorr, M., Hissbach, J. C., & Hampe, W. (2018) · Springer eBooks · doi:10.1007/978-3-319-94971-0_5 · Cited by 4 · see citing works
Classroom Context Influence on Pre-Service Teacher Pupil Control Ideology
Herron, J. P., & Hennessey, M. N. (2021) · The Teacher Educator · doi:10.1080/08878730.2021.1974627 · Cited by 2 · see citing works
Eva, K. W., Martin, S., Macala, C., & Shirzad, S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02256-z · Cited by 2 · see citing works
Self-citationYoshimura, H., Ganaha, H., Sakumoto, K., & Ponnamperuma, G. G. (2025) · Journal of St Marianna University · doi:10.17264/stmarieng.16.33 · Cited by 2 · see citing works
Yoshimura, H., Ganaha, H., Sakumoto, K., & Ponnamperuma, G. G. (2025) · Journal of St Marianna University · doi:10.17264/stmarieng.16.53 · Cited by 1 · see citing works
The multiple mini‐interview and dental hygiene admissions: A feasible option?
Nenad, M. W. (2020) · Journal of Dental Education · doi:10.1002/jdd.12114 · Cited by 1 · see citing works
Leduc, J., Béland, S., Renaud, J., Bégin, P., Gagnon, R. J., Ouellet, A., Bourdy, C., & Loye, N. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03681-4 · Cited by 1 · see citing works
Qualitative Analysis of Multiple Mini Interview Interviewer Comments
Manuel, R. S., Dickens, L., & Young, K. M. (2019) · Medical Science Educator · doi:10.1007/s40670-019-00778-2 · Cited by 1 · see citing works
Effect of station format on the psychometric properties of Multiple Mini Interviews
Renaud, J., Bourget, M., St‐Onge, C., Eva, K. W., Tavares, W., Loye, A. S., Leduc, J., & Homer, M. (2022) · Medical Education · doi:10.1111/medu.14855 · Cited by 1 · see citing works
Self-citationKim, D.-H., Kim, E. J., Hwang, J., Shin, J., & Lee, S. H. (2016) · Korean Medical Education Review · doi:10.17496/kmer.2016.18.2.90
Selection Practices and Methods in Other Fields
Kim, L. E., & Klassen, R. M. (2021) · Springer eBooks · doi:10.1007/978-3-030-76188-2_5
Elçin, M. (2022) · Akademisyen Kitabevi eBooks · doi:10.37609/akya.2282
Mini-entretiens multiples : un outil pour la réforme de l’accès aux études en santé en France
Villain, C., Passirani, C., Gouel, Y. C., Lerolle, N., Benhaïm, A., & Touzé, E. (2023) · Pédagogie médicale · doi:10.1051/pmed/2022033
Yoshimura, H., & Ponnamperuma, G. G. (2026) · Medical Teacher · doi:10.1080/0142159x.2026.2752868
- Authors
- Eva, K. W., & Macala, C.
- Affiliations
- University of British Columbia (CA)
- Corresponding author
- Eva, Kevin W
- ORCID
- Eva, Kevin W
- Venue
- Medical Education
- Publisher
- Wiley
- Published
- 2014-05-08
- Volume
- 48
- Issue
- 6
- Pages
- 604-613
- ISSN
- 0308-0110 (ISSN-L)
- 1365-2923
- OpenAlex
- W2039876160
- Cited by
- 39
- References
- 28
- Citations by year
- 2026: 1 · 2025: 3 · 2023: 1 · 2022: 5 · 2021: 2 · 2020: 1 · 2019: 6 · 2018: 6 · 2017: 3 · 2016: 7 · 2015: 3 · 2014: 1
- FWCI
- 4.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Education (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Medical Education and Admissions
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Gender Studies › Diversity and Career in Medicine
- Keywords
- multiple mini-interviews, medical school admissions, test-retest reliability, prediction of academic success, examiner perceptions
- MeSH terms
- Adult, Analysis of Variance, College Admission Test, Faculty, Medical, Female, Humans, Interviews as Topic, Male, Psychometrics, Mental Recall, School Admission Criteria, Students, Medical, Reproducibility of Results, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
