Rater perceptions of bias using the multiple mini-interview format: a qualitative study
Open accessAbstract
Introduction The Multiple Mini-Interview (MMI) format appears to mitigate individual rater biases. However, the format itself may introduce structural systematic bias, favoring extroverted personality types. This study aimed to gain a better understanding of these biases from the perspective of the interviewer. Methods A sample of MMI interviewers participated in a series of primary and follow-up one-on-one semi-structured interviews. Interviews pursued subjects of perception of biases (including norming; applicant personality, appearance and behavior; and interviewer personality) associated with the MMI process. Emergent qualitative data analysis was performed using the constant-comparative method. Results A number of perceived biases were identified by subjects, sub-grouped into cultural factors, personality factors, perception of prior preparation, concerns with norming, and biases associated with specific applicant characteristics. Discussion While the MMI appears to help mitigate individual rater biases, our analysis suggests that raters perceive structural systematic biases may be introduced by the question type and format of the MMI itself. Whether rater awareness of these biases mitigates them, and whether these herald other unconscious biases is unknown.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Qualitative Data Analysis: An Expanded Sourcebook
(1994) · Journal of Environmental Psychology · doi:10.1016/s0272-4944(05)80231-2 · Cited by 48,420 · see citing works
Wagner, H. R., Glaser, B. G., & Strauss, A. L. (1968) · Social Forces · doi:10.2307/2575405 · Cited by 46,517 · see citing works
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449 · see citing works
Standards for Reporting Qualitative Research: A Synthesis of Recommendations
O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014) · Academic Medicine · doi:10.1097/acm.0000000000000388 · Cited by 12,211 · see citing works
(2008) · Cited by 6,727 · see citing works
Interviews – Learning the Craft of Qualitative Research Interviewing
Vaivio, J. (2012) · European Accounting Review · doi:10.1080/09638180.2012.675165 · Cited by 4,558 · see citing works
Assessment of clinical competence
Wass, V., van der Vleuten, C. P. M., Shatzer, J. H., & Jones, R. F. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04221-5 · Cited by 1,023 · see citing works
Landy, F. J., & Farr, J. L. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.87.1.72 · Cited by 904 · see citing works
An admissions OSCE: the multiple mini-interview
Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004) · Medical Education · doi:10.1046/j.1365-2923.2004.01776.x · Cited by 617 · see citing works
Assessment in Health Professions Education
Downing, S. M., & Yudkowsky, R. (2009) · Routledge eBooks · doi:10.4324/9780203880135 · Cited by 328 · see citing works
Predictive validity of the multiple mini-interview for selecting medical trainees
Eva, K. W., Reiter, H. I., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G. R. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03407.x · Cited by 267 · see citing works
I DEAS AND O PINIONS Avoiding Pitfalls While Implementing New Guidelines on
Cassese, T., Sharkey, M. S., Pincavage, A. T., et al. (2019) · doi:10.1097/acm · Cited by 253 · see citing works
Multiple mini-interviews predict clerkship and licensing examination performance
Reiter, H. I., Eva, K. W., Rosenfeld, J., & Norman, G. R. (2007) · Medical Education · doi:10.1111/j.1365-2929.2007.02709.x · Cited by 235 · see citing works
Personality scale validities increase throughout medical school.
Lievens, F., Ones, D. S., & Dilchert, S. (2009) · Journal of Applied Psychology · doi:10.1037/a0016137 · Cited by 213 · see citing works
The interview in the admission process
Edwards, J. C., Johnson, E. K., & Molidor, J. B. (1990) · Academic Medicine · doi:10.1097/00001888-199003000-00008 · Cited by 192 · see citing works
Assessment of non-cognitive traits through the admissions multiple mini-interview
Lemay, J., Lockyer, J. M., Collin, V. T., & Brownell, A. K. W. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02767.x · Cited by 156 · see citing works
Structured interviewing: A note on incremental validity and alternative question types.
Campion, M. A., Campion, J. E., & Hudson, J. P. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.6.998 · Cited by 155 · see citing works
Investigating the Reliability of the Medical School Admissions Interview
Kreiter, C. D., Yin, P., Solow, C. M., & Brennan, R. L. (2004) · Advances in Health Sciences Education · doi:10.1023/b:ahse.0000027464.22411.0f · Cited by 153 · see citing works
Roberts, C., Walton, M., Rothnie, I., Crossley, J., Lyon, P., Kumar, K., & Tiller, D. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03018.x · Cited by 121 · see citing works
Comparing Structured Interview Question Types: Construct Validity and Applicant Reactions
Conway, J. M., & Peneno, G. M. (1999) · Journal of Business and Psychology · doi:10.1023/a:1022914803347 · Cited by 112 · see citing works
Harris, S., & Owen, C. (2007) · Medical Education · doi:10.1111/j.1365-2929.2007.02682.x · Cited by 101 · see citing works
McHarg, J., Bradley, P., Chamberlain, S., Ricketts, C., Searle, J., & McLachlan, J. C. (2005) · Medical Education · doi:10.1111/j.1365-2929.2004.02060.x · Cited by 97 · see citing works
Rosenfeld, J. M., Reiter, H. I., Trinh, K., & Eva, K. W. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9029-z · Cited by 88 · see citing works
Multiple mini-interviews versus traditional interviews: stakeholder acceptability comparison
Razack, S., Faremo, S., Drolet, F., Snell, L., Wiseman, J., & Pickering, J. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03447.x · Cited by 82 · see citing works
Multiple mini-interviews: opinions of candidates and interviewers
Humphrey, S., Dowson, S., Wall, D., Diwakar, V., & Goodyear, H. M. (2008) · Medical Education · doi:10.1111/j.1365-2923.2007.02972.x · Cited by 74 · see citing works
Hofmeister, M., Lockyer, J., & Crutcher, R. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03380.x · Cited by 73 · see citing works
Jerant, A., Griffin, E., Rainwater, J., Henderson, M., Sousa, F., Bertakis, K. D., Fenton, J. J., & Franks, P. (2012) · Academic Medicine · doi:10.1097/acm.0b013e31826102ad · Cited by 70 · see citing works
Experiences of the multiple mini-interview: a qualitative analysis
Kumar, K., Roberts, C., Rothnie, I., Du Fresne, C., & Walton, M. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03291.x · Cited by 68 · see citing works
Reliability and validity of interviewersʼ judgments of medical school candidates
Harasym, P. H., Woloschuk, W., Mandin, H., & Brundin‐Mather, R. (1996) · Academic Medicine · doi:10.1097/00001888-199601000-00038 · Cited by 57 · see citing works
Assessment of Applicants to the Veterinary Curriculum Using a Multiple Mini-Interview Method
Hecker, K. G., Donnon, T., Fuentealba, C., Hall, D. C., Illanes, O., Morck, D. W., & Muelling, C. (2009) · Journal of Veterinary Medical Education · doi:10.3138/jvme.36.2.166 · Cited by 50 · see citing works
Associations between the big five personality factors and multiple mini-interviews
Griffin, B., & Wilson, I. (2012) · Advances in Health Sciences Education · doi:10.1007/s10459-011-9316-1 · Cited by 48 · see citing works
Reiter, H. I., Salvatori, P., Rosenfeld, J., Trinh, K., & Eva, K. W. (2006) · Medical Education · doi:10.1111/j.1365-2929.2005.02348.x · Cited by 47 · see citing works
Validating MMI scores: are we measuring multiple attributes?
Oliver, T., Hecker, K., Hausdorf, P. A., & Conlon, P. (2014) · Advances in Health Sciences Education · doi:10.1007/s10459-013-9480-6 · Cited by 45 · see citing works
Non-association between Neo-5 personality tests and multiple mini-interview
Kulasegaram, K., Reiter, H. I., Wiesner, W., Hackett, R. D., & Norman, G. R. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-009-9209-8 · Cited by 41 · see citing works
Eva, K. W., & Macala, C. (2014) · Medical Education · doi:10.1111/medu.12402 · Cited by 39 · see citing works
Rater and occasion impacts on the reliability of pre‐admission assessments
Axelson, R. D., & Kreiter, C. D. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03537.x · Cited by 35 · see citing works
Interviewer bias in medical student selection
Griffin, B., & Wilson, I. G. (2010) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2010.tb03946.x · Cited by 31 · see citing works
Interviewer bias in medical student selection
Griffin, B., & Wilson, I. G. (2010) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2010.tb04015.x · Cited by 25 · see citing works
Fraga, J. D., Oluwasanjo, A., Wasser, T., Donato, A., & Alweis, R. (2013) · Journal of Community Hospital Internal Medicine Perspectives · doi:10.3402/jchimp.v3i3-4.21362 · Cited by 23 · see citing works
Self-citationMcManus, I. C., Mollon, J., Duke, O., & Vale, J. (2005) · BMC Medicine · doi:10.1186/1741-7015-3-13 · Cited by 13 · see citing works
The Mediating Processes Linking Applicant Personality Traits and Interviewer Evaluation
Chen, C., Huang, Y., Huang, T., & Liu, I. (2011) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2011.00558.x · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review
Yusoff, M. S. B. (2019) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2019.03.006 · Cited by 57 · see citing works
Callwood, A., Jeevaratnam, K., Kotronoulas, G., Schneider, A., Lewis, L. S., & Nadarajah, V. D. (2018) · Nurse Education Today · doi:10.1016/j.nedt.2018.01.016 · Cited by 174 · see citing works
Multiple Mini Interviews as Part of Holistic Admissions Review for Nursing Schools
Thompson, T., & Sonke, A. (2021) · Journal of Professional Nursing · doi:10.1016/j.profnurs.2021.08.009 · Cited by 13 · see citing works
Trust Me, I Know Them: Assessing Interpersonal Bias in Surgery Residency Interviews
Towaij, C., Gawad, N., Alibhai, K. M., Doan, D., & Raîche, I. (2022) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-21-00882.1 · Cited by 11 · see citing works
Fung, B. S. C., Gawad, N., Rosenzveig, A., & Raîche, I. (2022) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2022.05.013 · Cited by 6 · see citing works
Roberts, C., Burgess, A. W., Mossman, K., & Kumar, K. (2025) · BMC Medical Education · doi:10.1186/s12909-024-06535-3 · Cited by 3 · see citing works
Hadie, S. N. H., Kasim, F., Ismail, Z. I. M., Asari, M. A., Yusof, N. A. M., Simok, A. A., Shamsuddin, S. A., & Amin, M. S. I. M. (2022) · Education in Medicine Journal · doi:10.21315/eimj2022.14.4.9 · Cited by 2 · see citing works
Influencing factors of examiner: Examples from certificated orthodontists board examination
Chen, T.-C., Hung, Y.-J., Chen, T.-C., Lo, F.-Y., Wu, H.-Y., & Lai, E. H. (2023) · Journal of the Formosan Medical Association · doi:10.1016/j.jfma.2023.09.013 · Cited by 1 · see citing works
Qualitative Analysis of Multiple Mini Interview Interviewer Comments
Manuel, R. S., Dickens, L., & Young, K. M. (2019) · Medical Science Educator · doi:10.1007/s40670-019-00778-2 · Cited by 1 · see citing works
- Authors
- Alweis, R. L., Fitzpatrick, C., & Donato, A. A.
- Affiliations
- Thomas Jefferson University (US)
- Alvernia University (US)
- Venue
- Journal of Education and Training Studies
- Publisher
- Redfame Publishing
- Published
- 2015-06-26
- Volume
- 3
- Issue
- 5
- ISSN
- 2324-805X (ISSN-L)
- 2324-8068
- OpenAlex
- W1702222569
- Cited by
- 9
- References
- 40
- Citations by year
- 2025: 1 · 2023: 1 · 2022: 3 · 2021: 1 · 2019: 2 · 2018: 1
- FWCI
- 0.36
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Journal of Education and Training Studies (publishedVersion)
- Topics
- Social Sciences › Education › Medical Education and Admissions
- Social Sciences › Education › Innovations in Medical Education
- Social Sciences › Gender Studies › Diversity and Career in Medicine
- Keywords
- multiple mini-interviews, rater bias, structural bias, systematic bias, rater perceptions, extraversion, norming, implicit bias, selection interviews
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
