The effect of defined violations of test security on admissions outcomes using multiple mini-interviews
Open accessAbstract
Introduction Heterogeneous results exist regarding the impact of security violations on student performances in objective structured clinical examinations (OSCEs). Three separate studies investigate whether anticipated security violations result in undesirable enhancement of MMI performance ratings. Methods Study 1: low‐stakes: MMI station stems provided to a random half of 57 medical school applicants 2 weeks in advance of participation in a research study. Study 2: high‐stakes: 384 medical school applicants sat a 12‐station MMI to determine admission. Each half received 1 of 2 pilot MMI station stems 2 weeks in advance. Study 3: high‐stakes: 38 interviewees with dual applications to occupational therapy and physiotherapy experienced the same 7‐station MMI twice on the same date. Results No statistically significant differences in MMI performances were detected. Conclusions Predictable violations of MMI security do not unduly influence applicant performance ratings.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
An admissions OSCE: the multiple mini-interview
Eva, K. W., Rosenfeld, J., Reiter, H. I., & Norman, G. R. (2004) · Medical Education · doi:10.1046/j.1365-2923.2004.01776.x · Cited by 617 · see citing works
Self-citationThe ability of the multiple mini-interview to predict preclerkship performance in medical school
Eva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004) · Academic Medicine · doi:10.1097/00001888-200410001-00012 · Cited by 225 · see citing works
Self-citationEva, K. W., Reiter, H., Rosenfeld, J. M., & Norman, G. R. (2004) · Academic Medicine · doi:10.1097/00001888-200406000-00021 · Cited by 112 · see citing works
Self-citationReliability and learning from the objective structured clinical examination
Roberts, J. G., & Norman, G. R. (1990) · Medical Education · doi:10.1111/j.1365-2923.1990.tb00004.x · Cited by 85 · see citing works
Direct, standardized assessment of clinical competence
Williams, R. G., Barrows, H. S., Vu, N. V., Verhulst, S. J., Colliver, J. A., Marcy, M. L., & Steward, D. E. (1987) · Medical Education · doi:10.1111/j.1365-2923.1987.tb01407.x · Cited by 75 · see citing works
Swanson, D. B., Clauser, B. E., & Case, S. M. (1999) · Advances in Health Sciences Education · doi:10.1023/a:1009862220473 · Cited by 61 · see citing works
Does studying for an objective structured clinical examination make a difference?
Mavis, B. (2000) · Medical Education · doi:10.1046/j.1365-2923.2000.00687.x · Cited by 43 · see citing works
Is test security a concern when OSCE stations are repeated across clerkship rotations?
Niehaus, A. H., DaRosa, D. A., Markwell, S. J., & Folse, R. (1996) · Academic Medicine · doi:10.1097/00001888-199603000-00023 · Cited by 39 · see citing works
Sharing of Information by Students in an Objective Structured Clinical Examination
Rutala, P. J. (1991) · Archives of Internal Medicine · doi:10.1001/archinte.1991.00400030089016 · Cited by 37 · see citing works
Test security in examinations that use standardized-patient cases at one medical school
Colliver, J. A., Barrows, H. S., Vu, N. V., Verhulst, S. J., Mast, T. A., & Travis, T. A. (1991) · Academic Medicine · doi:10.1097/00001888-199105000-00011 · Cited by 36 · see citing works
Positive effects of a clinical performance assessment program
Stillman, P. L., Haley, H.-L. A., Regan, M. B., & Philbin, M. M. (1991) · Academic Medicine · doi:10.1097/00001888-199108000-00015 · Cited by 32 · see citing works
Swartz, M. H., Colliver, J. A., Cohen, D., & Barrows, H. S. (1993) · Academic Medicine · doi:10.1097/00001888-199310000-00052 · Cited by 29 · see citing works
Impact of repeated use of objective structured clinical examination stations
Cohen, R., Rothman, A. I., Ross, J., & Poldre, P. A. (1993) · Academic Medicine · doi:10.1097/00001888-199310000-00051 · Cited by 23 · see citing works
Is Test Security an Issue in a Multistation Clinical Assessment?—A Preliminary Study
Dawson-Saunders, B. K., Stillman, P. L., Haley, H.-L. A., Sutnick, A. I., PHILBIN, M. M., Smith, R. S., OʼDONNELL, J., & Pohl, H. S. (1991) · Academic Medicine · doi:10.1097/00001888-199109001-00010 · Cited by 21 · see citing works
Test security in standardized-patient examinations
Colliver, J. A., Travis, T. A., ROBBS, R. S., Barnhart, A., Shirar, L. E., & Vu, N. V. (1992) · Academic Medicine · doi:10.1097/00001888-199210000-00022 · Cited by 21 · see citing works
Gledhill, R. F., & Capatos, D. (1985) · PubMed · Cited by 19 · see citing works
Learning effect of reusing stations in an objective structured clinical examination
Jolly, B. C., Newble, D. I., & Chinner, T. L. (1993) · Teaching and Learning in Medicine · doi:10.1080/10401339309539593 · Cited by 18 · see citing works
Analyzing and adjusting for variables in a large-scale standardized-patient examination
Battles, J. B., Carpenter, J. L., McIntire, D. D., & Wagner, J. M. (1994) · Academic Medicine · doi:10.1097/00001888-199405000-00010 · Cited by 18 · see citing works
Champlain, A. F. D., MACMILLAN, M. K., Margolis, M. J., KLASS, D. J., Nungester, R. J., Schimpfiauser, F., & Zinnerstrom, K. H. (1999) · Academic Medicine · doi:10.1097/00001888-199910000-00037 · Cited by 16 · see citing works
WILKINSON, T. J., Fontaine, S., & Egan, T. G. (2003) · Medical Teacher · doi:10.1080/0142159021000061413 · Cited by 15 · see citing works
Sharing of information by students in an objective structured clinical examination
Pj, R. (1991) · Archives of Internal Medicine · doi:10.1001/archinte.151.3.541 · Cited by 9 · see citing works
Furman, G. E., Colliver, J. A., Galofré, A., Reaka, M. A., ROBBS, R. S., & King, A. (1997) · Advances in Health Sciences Education · doi:10.1023/a:1009773709752 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [14]14. Colliver JA, Barrows HS, Vu NV, Verhulst SJ, Mast TA, Travis TA. Test security in examinations that use stan- dardized-patient cases. Acad Med 1991;66:279-82.
Cited by
Sorted by citations
Prideaux, D. J., Roberts, C., Eva, K. W., Centeno, A., McCrorie, P., McManus, I. C., Patterson, F., POWIS, D. A., Tekian, A. S., & Wilkinson, D. (2011) · Medical Teacher · doi:10.3109/0142159x.2011.551560 · Cited by 221 · see citing works
Self-citationAssessment of non-cognitive traits through the admissions multiple mini-interview
Lemay, J., Lockyer, J. M., Collin, V. T., & Brownell, A. K. W. (2007) · Medical Education · doi:10.1111/j.1365-2923.2007.02767.x · Cited by 156 · see citing works
Rees, E. L., Hawarden, A. W., Dent, G., Hays, R., Bates, J., & Hassell, A. B. (2016) · Medical Teacher · doi:10.3109/0142159x.2016.1158799 · Cited by 139 · see citing works
Roberts, C., Walton, M., Rothnie, I., Crossley, J., Lyon, P., Kumar, K., & Tiller, D. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03018.x · Cited by 121 · see citing works
Multiple mini-interviews: same concept, different approaches
Knorr, M., & Hissbach, J. (2014) · Medical Education · doi:10.1111/medu.12535 · Cited by 110 · see citing works
Rosenfeld, J. M., Reiter, H. I., Trinh, K., & Eva, K. W. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9029-z · Cited by 88 · see citing works
Self-citationPredictive validity of the Dundee multiple mini-interview
Husbands, A., & Dowell, J. (2013) · Medical Education · doi:10.1111/medu.12193 · Cited by 83 · see citing works
The multiple mini-interview in the UK context: 3 years of experience at Dundee
Dowell, J., Lynch, B., Till, H., Kumwenda, B., & Husbands, A. (2012) · Medical Teacher · doi:10.3109/0142159x.2012.652706 · Cited by 74 · see citing works
Griffin, B., Harding, D. W., Wilson, I. G., & Yeomans, N. D. (2008) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2008.tb02024.x · Cited by 74 · see citing works
Cameron, A. J., & MacKeigan, L. D. (2012) · American Journal of Pharmaceutical Education · doi:10.5688/ajpe76110 · Cited by 63 · see citing works
Internet-based multiple mini-interviews for candidate selection for graduate entry programmes
Tiller, D., O'Mara, D., Rothnie, I., Dunn, S., Lee, L., & Roberts, C. (2013) · Medical Education · doi:10.1111/medu.12224 · Cited by 61 · see citing works
Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review
Yusoff, M. S. B. (2019) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2019.03.006 · Cited by 57 · see citing works
A comparison of multiple mini-interviews and structured interviews in a UK setting
O’Brien, A., Harvey, J., Shannon, M., Lewis, K., & Valencia, O. (2011) · Medical Teacher · doi:10.3109/0142159x.2010.541532 · Cited by 55 · see citing works
Husbands, A., Rodgerson, M. J., Dowell, J. S., & Patterson, F. (2015) · BMC Medical Education · doi:10.1186/s12909-015-0424-0 · Cited by 55 · see citing works
Roberts, C., Clark, T., Burgess, A., Frommer, M., Grant, M., & Mossman, K. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-169 · Cited by 49 · see citing works
The acceptability of the multiple mini interview for resident selection.
(2008) · PubMed · Cited by 49 · see citing works
Perkins, A., Burton, L., Dray, B., & Elcock, K. (2013) · Nurse Education Today · doi:10.1016/j.nedt.2012.04.023 · Cited by 46 · see citing works
The multiple mini-interview for emergency medicine resident selection
Hopson, L. R., Burkhardt, J. C., Stansfield, R. B., Vohra, T., Turner-Lawrence, D., & Losman, E. D. (2014) · The Journal of Emergency Medicine · doi:10.1016/j.jemermed.2013.08.119 · Cited by 43 · see citing works
Eva, K. W., & Macala, C. (2014) · Medical Education · doi:10.1111/medu.12402 · Cited by 39 · see citing works
Self-citationTwelve tips for constructing a multiple mini-interview
Eva, K. W., Macala, C., & Fleming, B. (2018) · Medical Teacher · doi:10.1080/0142159x.2018.1429586 · Cited by 25 · see citing works
Self-citationBest practices for interviewing applicants for medical school admissions: a systematic review
Lin, J. C. H., Lokhande, A., Margo, C. E., & Greenberg, P. B. (2022) · Perspectives on Medical Education · doi:10.1007/s40037-022-00726-8 · Cited by 27 · see citing works
How do people think about later life when making workplace pension saving decisions?
James, H., Price, D., & Buffel, T. (2020) · Journal of Aging Studies · doi:10.1016/j.jaging.2020.100869 · Cited by 21 · see citing works
Baird, J., Caro, D. H., & Hopfenbeck, T. N. (2016) · Irish Educational Studies · doi:10.1080/03323315.2016.1227719 · Cited by 12 · see citing works
Multiple mini-interviews as a selection tool for initial teacher education admissions
Metsäpelto, R., Utriainen, J., Poikkeus, A., Muotka, J. S., Tolvanen, A. J., & Warinowski, A. (2022) · Teaching and Teacher Education · doi:10.1016/j.tate.2022.103660 · Cited by 11 · see citing works
Development and implementation of the multiple mini-interview in pharmacy admissions
Stowe, C. D., Castleberry, A. N., O’Brien, C. E., Flowers, S. K., Warmack, T. S., & Gardner, S. F. (2014) · Currents in Pharmacy Teaching and Learning · doi:10.1016/j.cptl.2014.07.007 · Cited by 11 · see citing works
Abdul Rahim, A. F., & Yusoff, M. S. B. (2016) · Education in Medicine Journal · doi:10.5959/eimj.v8i2.437 · Cited by 10 · see citing works
Steele, K. (2011) · Europe PMC (PubMed Central) · Cited by 10 · see citing works
Reliability of a regional junior doctor recruitment process
Goodyear, H. M., Jyothish, D., Diwakar, V., & Wall, D. (2007) · Medical Teacher · doi:10.1080/01421590701526823 · Cited by 10 · see citing works
Rater perceptions of bias using the multiple mini-interview format: a qualitative study
Alweis, R. L., Fitzpatrick, C., & Donato, A. A. (2015) · Journal of Education and Training Studies · doi:10.11114/jets.v3i5.818 · Cited by 9 · see citing works
Moshinsky, A., Ziegler, D., & Gafni, N. (2017) · International Journal of Testing · doi:10.1080/15305058.2016.1263638 · Cited by 9 · see citing works
Selecting medical students: considering qualities other than academic ability
Griffin, B. (2017) · Medical Education · doi:10.1111/medu.13450 · Cited by 9 · see citing works
Multiple Mini-Interview in Selecting Medical Students
Roh, H., Lee, H. J., Park, S.-B., Yang, J. H., Kim, D.-J., Kim, S. H., Lee, S.-J., & Chae, G. (2009) · Korean journal of medical education · doi:10.3946/kjme.2009.21.2.103 · Cited by 9 · see citing works
Using Multiple Mini-Interviews for Students’ Admissions in Pakistan: A Pilot Study
Haider, S. I., Bari, M. F., & ijaz, S. (2020) · Advances in Medical Education and Practice · doi:10.2147/amep.s246285 · Cited by 8 · see citing works
Thomas, A., Young, M. E., Mazer, B. L., Lubarsky, S., & Razack, S. (2017) · British Journal of Occupational Therapy · doi:10.1177/0308022617713980 · Cited by 8 · see citing works
From Hippocrates to the Francis Report--Reflections on empathy.
Weir, J. M., Aicken, M. D., Cupples, M., & Steele, K. (2015) · Europe PMC (PubMed Central) · Cited by 7 · see citing works
Support for Interview Preparation with Deep Learning Based Language Model
Czejdo, C., & Bhattacharya, S. (2021) · International Conference on Emerging Techniques in Computational Intelligence (ICETCI) · doi:10.1109/icetci51973.2021.9574063 · Cited by 5 · see citing works
Lambe, P., Waters, C. M., & Bristow, D. R. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.801941 · Cited by 4 · see citing works
Interviews, Multiple Mini-Interviews, and Selection Centers
Knorr, M., Hissbach, J. C., & Hampe, W. (2018) · Springer eBooks · doi:10.1007/978-3-319-94971-0_5 · Cited by 4 · see citing works
A novel student‐selected component in medical admissions
O’Brien, A. A., Young, C. A., & Lomax, A. (2012) · Medical Education · doi:10.1111/j.1365-2923.2012.04224.x · Cited by 3 · see citing works
Eva, K. W., Martin, S., Macala, C., & Shirzad, S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02256-z · Cited by 2 · see citing works
Self-citationMultiple mini interview performance of repeat applicants to medical school admission
Raghavan, M., & Martin, B. D. (2014) · The Medical Journal of Australia · doi:10.5694/mja14.00728 · Cited by 2 · see citing works
Multiple Mini Interview as a Selection Tool for Initial Teacher Education AdmissionsPreprint
Metsäpelto, R., Utriainen, J., Poikkeus, A., Muotka, J. S., Tolvanen, A. J., & Warinowski, A. (2020) · doi:10.31234/osf.io/ph5t8 · Cited by 2 · see citing works
Developing a Customized Multiple Interview for Dental School Admissions
Gardner, K. M. (2014) · Journal of Dental Education · doi:10.1002/j.0022-0337.2014.78.4.tb05711.x · Cited by 1 · see citing works
Tahir, M., & Wakeel, A. (2019) · Kardan Journal of Social Sciences and Humanities · doi:10.31841/kjssh.2021.27
Mini-entretiens multiples : un outil pour la réforme de l’accès aux études en santé en France
Villain, C., Passirani, C., Gouel, Y. C., Lerolle, N., Benhaïm, A., & Touzé, E. (2023) · Pédagogie médicale · doi:10.1051/pmed/2022033
Paschoalini, Z. F. B., & Paschoalini, M. D. S. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09858-5
- Authors
- Reiter, H. I., Salvatori, P., Rosenfeld, J., Trinh, K., & Eva, K. W.
- Affiliations
- McMaster University (CA)
- Corresponding author
- Eva, Kevin W
- Venue
- Medical Education
- Publisher
- Wiley
- Published
- 2005-12-21
- Volume
- 40
- Issue
- 1
- Pages
- 36-42
- ISSN
- 0308-0110 (ISSN-L)
- 1365-2923
- OpenAlex
- W2140015094
- Cited by
- 47
- References
- 22
- Citations by year
- 2026: 1 · 2025: 1 · 2023: 1 · 2022: 2 · 2021: 1 · 2020: 3 · 2019: 2 · 2018: 2 · 2017: 3 · 2016: 3 · 2015: 3 · 2014: 8 · 2013: 3 · 2012: 4
- FWCI
- 2.09
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Medical Education (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Board of Medical Examiners
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Medical Education and Admissions
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Keywords
- multiple mini-interviews, test security, security breaches, medical school admissions, high-stakes testing, low-stakes assessment
- MeSH terms
- Interviews as Topic, Ontario, School Admission Criteria, Schools, Medical, Security Measures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
