Much ado about differences: why expert-novice comparisons add little to the validity argument
Abstract
One approach to validating assessment scores involves evaluating the ability of scores to discriminate among groups who differ in a specific characteristic, such as training status (in education) or disease state (in clinical applications). Such known-groups comparison studies provide validity evidence of "relationships with other variables." The typical education research study might compare scores between staff physicians and postgraduate trainees with the hypothesis that those with more advanced training (the "experts") will have higher scores than those less advanced (the "novices"). However, such comparisons are too nonspecific to support clear conclusions, and expert-novice comparisons (and known-groups comparisons in general) thus contribute little to the validity argument. The major flaw is the problem of confounding: there are multiple plausible explanations for any observed between-group differences. The absence of hypothesized differences would suggest a serious flaw in the validity argument, but the confirmation of such differences adds little. As such, accurate known-groups discrimination may be necessary, but will never be sufficient, to support the validity of scores. This article elaborates on this and other problems with the known-groups comparison that limit its utility as a source of validity evidence.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Known-groups comparisons on their own are non-specific and inconclusive because of possible confounding effects.
“Though necessary, this type of comparison (on its own) is known to be non-specific and inconclusive because of possible confounding effects (Cook 2015).”
From Measuring physician cognitive load: validity evidence for a physiologic and a psychometric tool (2017) · cites this paper as Cook 2015
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · see citing works
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · see citing works
The standards for educational and psychological testing.
Eignor, D. R. (2013) · American Psychological Association eBooks · doi:10.1037/14047-013 · Cited by 6,975 · see citing works
Standards for Educational and Psychological Testing
(2007) · Sage eBooks · doi:10.4135/9781412952644.n432 · Cited by 3,137 · see citing works
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,858 · see citing works
Standards for Educational and Psychological Testing.
Jiménez-García, E. (2017) · Revista Española de Pedagogía · doi:10.22550/2174-0909.3051 · Cited by 2,227 · see citing works
Empirical evidence of design-related bias in studies of diagnostic tests.
Lijmer, J. G. (1999) · JAMA · doi:10.1001/jama.282.11.1061 · Cited by 1,861 · see citing works
Current Concepts in Validity and Reliability for Psychometric Instruments: Theory and Application
Cook, D. A., & Beckman, T. J. (2006) · The American Journal of Medicine · doi:10.1016/j.amjmed.2005.10.036 · Cited by 1,745 · see citing works
Self-citationValidity: on the meaningful interpretation of assessment data
Downing, S. M. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01594.x · Cited by 1,434 · see citing works
Fleming, C. M., & Travers, R. M. W. (1955) · British Journal of Educational Studies · doi:10.2307/3119507 · Cited by 915 · see citing works
Educational Measurement (4th ed.)
Borsboom, D. (2009) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705510903206097 · Cited by 387 · see citing works
OSCE checklists do not capture increasing levels of expertise
Hodges, B. D., Regehr, G., McNaughton, N., Tiberius, R. G., & Hanson, M. D. (1999) · Academic Medicine · doi:10.1097/00001888-199910000-00017 · Cited by 361 · see citing works
Cook, D. A., Zendejas, B., Hamstra, S. J., Hatala, R. A., & Brydges, R. (2013) · Advances in Health Sciences Education · doi:10.1007/s10459-013-9458-4 · Cited by 318 · see citing works
Self-citationTechnology-Enhanced Simulation to Assess Health Professionals
Cook, D. A., Brydges, R., Zendejas, B., Hamstra, S. J., & Hatala, R. A. (2013) · Academic Medicine · doi:10.1097/acm.0b013e31828ffdcf · Cited by 269 · see citing works
Self-citationClinical problem‐solving by medical students: a cross‐sectional and longitudinal analysis
Neufeld, V. R., Norman, G. R., Feightner, J., & Barrows, H. S. (1981) · Medical Education · doi:10.1111/j.1365-2923.1981.tb02495.x · Cited by 216 · see citing works
Cook, D. A., & West, C. P. (2012) · Academic Medicine · doi:10.1097/acm.0b013e31827c3d78 · Cited by 160 · see citing works
Self-citationPrystowsky, J. B., & Bordage, G. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00910.x · Cited by 147 · see citing works
Competency-Based Education and Training in Internal Medicine
Weinberger, S. E., Pereira, A. G., Iobst, W. F., Mechaber, A. J., Bronze, M. S., & and the Alliance for Academic Internal Medicine Education Redesign Task Force II* (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-153-11-201012070-00009 · Cited by 115 · see citing works
Albanese, M. A., Mejicano, G. C., & Gruppen, L. D. (2008) · Academic Medicine · doi:10.1097/acm.0b013e31818c6638 · Cited by 52 · see citing works
Cook, D. A. (2014) · Medical Education · doi:10.1111/medu.12401 · Cited by 43 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
A contemporary approach to validity arguments: a practical guide to K ane's framework
Cook, D. A., Brydges, R., Ginsburg, S., & Hatala, R. A. (2015) · Medical Education · doi:10.1111/medu.12678 · Cited by 644 · see citing works
Self-citationValidation of educational assessments: a primer for simulation and beyond
Cook, D. A., & Hatala, R. A. (2016) · Advances in Simulation · doi:10.1186/s41077-016-0033-y · Cited by 382 · see citing works
Self-citationHatala, R. A., Cook, D. A., Brydges, R., & Hawkins, R. E. (2015) · Advances in Health Sciences Education · doi:10.1007/s10459-015-9593-1 · Cited by 177 · see citing works
Self-citationGathering Validity Evidence for Surgical Simulation
Borgersen, N. J., Naur, T. M. H., Sørensen, S. M. D., Bjerrum, F., Konge, L., Subhi, Y., & Thomsen, A. S. S. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002652 · Cited by 158 · see citing works
Zendejas, B., Ruparel, R. K., & Cook, D. A. (2015) · Surgical Endoscopy · doi:10.1007/s00464-015-4233-7 · Cited by 126 · see citing works
Self-citationMeasuring physician cognitive load: validity evidence for a physiologic and a psychometric tool
Szulewski, A., Gegenfurtner, A., Howes, D. W., Sivilotti, M. L. A., & van Merriënboer, J. J. G. (2017) · Advances in Health Sciences Education · doi:10.1007/s10459-016-9725-2 · Cited by 94 · see citing works
High correlation between performance on a virtual‐reality simulator and real‐life cataract surgery
Thomsen, A. S. S., Smith, P., Subhi, Y., la Cour, M., Tang, L., Saleh, G. M., & Konge, L. (2016) · Acta Ophthalmologica · doi:10.1111/aos.13275 · Cited by 84 · see citing works
Quantifying Heuristic Bias: Anchoring, Availability, and Representativeness
Richie, M. B., & Josephson, S. A. (2017) · Teaching and Learning in Medicine · doi:10.1080/10401334.2017.1332631 · Cited by 56 · see citing works
Hatala, R. A., Gutman, J., Lineberry, M. C., Triola, M. M., & Pusic, M. V. (2018) · Advances in Health Sciences Education · doi:10.1007/s10459-018-9846-x · Cited by 49 · see citing works
Development and Assessment of a Distal Radial Fracture Model as a Clinical Teaching Tool
Mayne, I. P., Brydges, R., Moktar, J., & Murnaghan, M. L. (2016) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.o.00565 · Cited by 42 · see citing works
Is there inter‐procedural transfer of skills in intraocular surgery? A randomized controlled trial
Thomsen, A. S. S., Kiilgaard, J. F., la Cour, M., Brydges, R., & Konge, L. (2017) · Acta Ophthalmologica · doi:10.1111/aos.13434 · Cited by 41 · see citing works
Hip-fracture osteosynthesis training: exploring learning curves and setting proficiency standards
Gustafsson, A., Pedersen, P., Rømer, T. B., Viberg, B., Palm, H., & Konge, L. (2019) · Acta Orthopaedica · doi:10.1080/17453674.2019.1607111 · Cited by 41 · see citing works
Simulation training in vitreoretinal surgery: a systematic review
Rasmussen, R. C., Grauslund, J., & Vergmann, A. S. (2019) · BMC Ophthalmology · doi:10.1186/s12886-019-1098-x · Cited by 32 · see citing works
Evaluating structured assessment of anaesthesiologists' non‐technical skills
Jepsen, R. M. H. G., Dieckmann, P., Spanager, L., Lyk–Jensen, H. T., Konge, L., Ringsted, C. V., & Østergaard, D. (2016) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.12709 · Cited by 29 · see citing works
Cognitive task analysis-based training in surgery: a meta-analysis
Edwards, T. C., Coombs, A. W., Szyszka, B., Logishetty, K., & Cobb, J. P. (2021) · BJS Open · doi:10.1093/bjsopen/zrab122 · Cited by 28 · see citing works
Bjerrum, F., Strandbygaard, J. J. S., Rosthøj, S., Grantcharov, T., Ottesen, B., & Sørensen, J. L. (2016) · Journal of surgical education · doi:10.1016/j.jsurg.2016.08.006 · Cited by 28 · see citing works
Assessment of Residents Readiness to Perform Lumbar Puncture: A Validation Study
Henriksen, M. J. V., Wienecke, T., Thagesen, H., Jacobsen, R. B., Subhi, Y., Ringsted, C. V., & Konge, L. (2017) · Journal of General Internal Medicine · doi:10.1007/s11606-016-3981-y · Cited by 28 · see citing works
Assessment of EVAR Competence: Validity of a Novel Rating Scale (EVARATE) in a Simulated Setting
Strøm, M., Lönn, L., Konge, L., Schroeder, T. V., Lindgren, H. I. V., Nyheim, T., Venermo, M. A., & Bech, B. (2018) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2018.02.035 · Cited by 25 · see citing works
Azari, D. P., Miller, B. L., Le, B. V., Greenberg, C. C., & Radwin, R. G. (2020) · Applied Ergonomics · doi:10.1016/j.apergo.2020.103136 · Cited by 24 · see citing works
Intraocular surgery – assessment and transfer of skills using a virtual‐reality simulator
Thomsen, A. S. S. (2017) · Acta Ophthalmologica · doi:10.1111/aos.13505 · Cited by 24 · see citing works
Vela, J., Caterina Contreras, B., Jarry, C., Varas, J., & Corvetto, M. (2020) · Revista Latinoamericana de Simulación Clínica · doi:10.35366/92936 · Cited by 23 · see citing works
Lavery, M. R., Bostic, J. D., Kruse, L., Krupa, E. E., & Carney, M. B. (2020) · Educational Measurement Issues and Practice · doi:10.1111/emip.12378 · Cited by 23 · see citing works
Development and Validation of a Feline Medial Saphenous Venipuncture Model and Rubric
Hunt, J., Hughes, C., Asciutto, M., & Johnson, J. T. (2019) · Journal of Veterinary Medical Education · doi:10.3138/jvme.0718-085 · Cited by 22 · see citing works
Psychometric properties of the Fundamentals of Endoscopic Surgery (FES) skills examination
Lineberry, M. C., & Ritter, E. M. (2017) · Surgical Endoscopy · doi:10.1007/s00464-017-5590-1 · Cited by 21 · see citing works
An Inquiry Into the Early Careers of Master Clinicians
Murthy, V. K., OʼBrien, B. C., & Dhaliwal, G. (2018) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-18-00208.1 · Cited by 21 · see citing works
Lerner, V. T., DeStephano, C. C., Ulrich, A., Han, E. S., LeClaire, E. L., & Chen, C. C. G. (2021) · Journal of Minimally Invasive Gynecology · doi:10.1016/j.jmig.2021.04.010 · Cited by 18 · see citing works
Xu, J., Campisi, P., Forte, V., Carrillo, B., Vescan, A. D., & Brydges, R. (2018) · Journal of Otolaryngology - Head and Neck Surgery · doi:10.1186/s40463-018-0317-4 · Cited by 19 · see citing works
Examining validity evidence for a simulation-based assessment tool for basic robotic surgical skills
Havemann, M. C., Dalsgaard, T., Sørensen, J. L., Røssaak, K., Brisling, S. K., Mosgaard, B. J., Høgdall, C. K., & Bjerrum, F. (2018) · Journal of Robotic Surgery · doi:10.1007/s11701-018-0811-8 · Cited by 17 · see citing works
Essentials in Minimally Invasive Gynecology Manual Skills Construct Validation Trial
Munro, M. G., Advincula, A. P., Banks, E. H., Auguste, T., Chahine, E. B., Chen, C. C. G., Curlin, H. L., Jorgensen, E. M., Kim, J. H., King, C. R., et al. (2020) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000003936 · Cited by 17 · see citing works
Learning to diagnose collaboratively: validating a simulation for medical students
Radkowitsch, A., Fischer, M. R., Schmidmaier, R., & Fischer, F. (2020) · PubMed Central · doi:10.3205/zma001344 · Cited by 17 · see citing works
Alvarez-López, F., Maina, M. F., Arango, F., & Saigí-Rubió, F. (2020) · JMIR Serious Games · doi:10.2196/19723 · Cited by 16 · see citing works
Ensuring Basic Competence in Thoracentesis
Rasmussen, K. M. B., Hertz, P., Laursen, C. B., Arshad, A., Saghir, Z., Clementsen, P. F., & Konge, L. (2019) · Respiration · doi:10.1159/000495686 · Cited by 14 · see citing works
Using Immersive Virtual Reality Simulation to Ensure Competence in Contrast-Enhanced Ultrasound
Jacobsen, N., Larsen, J. D., Falster, C., Nolsøe, C., Konge, L., Graumann, O., & Laursen, C. B. (2022) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2022.01.015 · Cited by 14 · see citing works
Establishing Validity for a Vaginal Hysterectomy Simulation Model for Surgical Skills Assessment
Chen, C. C. G., Lockrow, E. G., DeStephano, C. C., Nihira, M. A., Matthews, C. A., Kammire, L. D., Landrum, L. M., Anderson, B. D., & Miyazaki, D. (2020) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000004085 · Cited by 12 · see citing works
Comparing Surgical Experience and Skill Using a High-Fidelity, Total Laparoscopic Hysterectomy Model
Arora, C., Menzies, A. V., Han, E., Lee, M., Lauer, J. K., Hur, H.-C., Kim, J. H., & Advincula, A. P. (2020) · Obstetrics and Gynecology · doi:10.1097/aog.0000000000003897 · Cited by 13 · see citing works
Development and validation of a feline abdominal palpation model and scoring rubric
Williamson, J. A., Hecker, K. G., Yvorchuk, K., Artemiou, E., French, H. M., & Fuentealba, C. (2015) · Veterinary Record · doi:10.1136/vr.103212 · Cited by 12 · see citing works
Simulation for High-Stakes Assessment in Pediatric Emergency Medicine
Calhoun, A. W., Bhanji, F., Sherbino, J., & Hatala, R. A. (2016) · Clinical Pediatric Emergency Medicine · doi:10.1016/j.cpem.2016.05.001 · Cited by 11 · see citing works
Distinguishing Laparoscopic Surgery Experts from Novices Using EEG Topographic Features
Manabe, T., Rahul, F., Fu, Y., Intes, X., Schwaitzberg, S., De, S., Cavuoto, L., & Dutta, A. (2023) · Brain Science · doi:10.3390/brainsci13121706 · Cited by 10 · see citing works
Darici, D., Reißner, C., & Missler, M. (2023) · PubMed Central · doi:10.3205/zma001642 · Cited by 10 · see citing works
Assessment of Competence in EVAR Stent Graft Sizing and Selection
Strøm, M., Lönn, L., Bech, B., Schroeder, T. V., & Konge, L. (2017) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2017.03.007 · Cited by 8 · see citing works
Use of the Limbs and Things Hysterectomy Model to Describe the Process for Establishing Validity
DeStephano, C. C., Chen, A. H., Heckman, M. G., Chimato, N. T., Guha, P., Espinal, M., & Dinh, T. A. (2018) · Journal of Minimally Invasive Gynecology · doi:10.1016/j.jmig.2018.02.003 · Cited by 8 · see citing works
Hasselager, A. B., Østergaard, D., Kristensen, T. D., Sønderskov, C., Bohnstedt, C., Lauritsen, T. L. B., Konge, L., & Tolsgaard, M. G. (2018) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-018-0544-8 · Cited by 8 · see citing works
Sequential dependencies in categorical judgments of radiographic images
Beckstead, J. W., Boutis, K., Pecaric, M. R., & Pusic, M. V. (2016) · Advances in Health Sciences Education · doi:10.1007/s10459-016-9692-7 · Cited by 7 · see citing works
A Comparison of Expert Ratings and Marker-Less Hand Tracking Along OSATS-Derived Motion Scales
Azari, D. P., Miller, B. L., Le, B. V., Greenberg, J. A., Bruskewitz, R. C., Long, K. L., Chen, G., & Radwin, R. G. (2020) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2020.3035763 · Cited by 7 · see citing works
Spencer, M., Sherbino, J., & Hatala, R. A. (2022) · Advances in Health Sciences Education · doi:10.1007/s10459-022-10114-w · Cited by 7 · see citing works
Sheppard, G., Williams, K.-L., Metcalfe, B., Clark, M., Bromley, M., Pageau, P., Woo, M. Y., Yi, Y., Devasahayam, A. J., & Dubrowski, A. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04030-9 · Cited by 7 · see citing works
The Psychometric Properties of the Outcome Rating Scale Used in Practice: A Narrative Review
Harris, R. M., Murphy, M. G., & Rakes, S. (2019) · Journal of Evidence-Based Social Work · doi:10.1080/26408066.2019.1645071 · Cited by 7 · see citing works
The negative consequences of consequential validity
Norman, G. R. (2015) · Advances in Health Sciences Education · doi:10.1007/s10459-015-9615-z · Cited by 7 · see citing works
Decision-making in midwifery: A tripartite clinical decision
Smith, J. (2016) · British Journal of Midwifery · doi:10.12968/bjom.2016.24.8.574 · Cited by 5 · see citing works
Hatala, R. A., & Cook, D. A. (2019) · Springer eBooks · doi:10.1007/978-3-030-26837-4_26 · Cited by 5 · see citing works
Self-citationAssessing the utility of a novel entrustment‐supervision assessment tool
Dewhirst, S., Wood, T. J., Cheung, W. J., & Frank, J. R. (2023) · Medical Education · doi:10.1111/medu.15156 · Cited by 4 · see citing works
Validity evidence of a task trainer for normal and difficult lumbar puncture
Akaishi, Y., Okada, Y., Lee-Jayaram, J. J., Seo, J. S., Yamada, T., & Berg, B. W. (2020) · Medicine · doi:10.1097/md.0000000000022622 · Cited by 3 · see citing works
Zhou, J. H., Liao, H., Lin, Z., Lyu, M., Chong, W., Chen, N., Hai, Y., Wang, Z., Lu, T., & Liu, L. (2021) · Surgical Endoscopy · doi:10.1007/s00464-021-08606-8 · Cited by 3 · see citing works
Goderstad, J. M., Sandvik, L., Fosse, E. T., & Lieng, M. (2019) · European Journal of Obstetrics & Gynecology and Reproductive Biology X · doi:10.1016/j.eurox.2019.100092 · Cited by 3 · see citing works
Comparing Simulator Metrics and Rater Assessment of Laparoscopic Suturing Skills
Jørgensen, R. J., Olsen, R. G., Svendsen, M. B. S., Stadeager, M. W., Konge, L., & Bjerrum, F. (2022) · Journal of surgical education · doi:10.1016/j.jsurg.2022.09.020 · Cited by 3 · see citing works
Gomes, S. H., Trindade, M., Petrișor, C., Costa, D., Correia‐Pinto, J., Costa, P., & Pêgo, J. M. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04146-y · Cited by 2 · see citing works
Kinnear, B., Schumacher, D. J., Varpio, L., Driessen, E. W., & Konopasky, A. (2024) · Perspectives on Medical Education · doi:10.5334/pme.1404 · Cited by 2 · see citing works
Miller, L. M. J., Scully, C. M., Morris, V., Bonnema, H., Trantham, N., & Hunt, J. (2025) · Journal of Veterinary Medical Education · doi:10.3138/jvme-2024-0139 · Cited by 2 · see citing works
Samaratunga, R., Johnson, L., Gatzidis, C., Swain, I., Wainwright, T. W., & Middleton, R. (2021) · Journal of Medical Engineering & Technology · doi:10.1080/03091902.2021.1921868 · Cited by 2 · see citing works
A Cooperative Model of Development and Validation of a Curriculum-Based Scientific Competence Test
Lin, S., & Shie, W.-C. (2023) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-023-10366-x · Cited by 2 · see citing works
St‐Onge, C., & Lineberry, M. C. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-815657-5.00058-9 · Cited by 2 · see citing works
Evaluating the strength of evidence in research and education: The theory of anchored narratives
Leppink, J. (2017) · Journal of Taibah University Medical Sciences · doi:10.1016/j.jtumed.2017.01.002 · Cited by 2 · see citing works
Laypersons versus experienced surgeons in assessing simulated robot-assisted radical prostatectomy
Olsen, R. G., Konge, L., Hayatzaki, K., Mortensen, M. A., Bube, S. H., Røder, M. A., Azawi, N., & Bjerrum, F. (2023) · World Journal of Urology · doi:10.1007/s00345-023-04664-w · Cited by 1 · see citing works
Vamadevan, A., Carstensen, S. M. D., Konge, L., & Bjerrum, F. (2024) · Current Problems in Surgery · doi:10.1016/j.cpsurg.2024.101509 · Cited by 1 · see citing works
Development and Validation of a Bovine Left Displaced Abomasum Reduction Model and Rubric
Schank, N., Hunt, J., Marcum, M., & Brockman, R. (2025) · Journal of Veterinary Medical Education · doi:10.3138/jvme-2024-0145 · Cited by 1 · see citing works
Gazit, N., Ben‐Gal, G., & Eliashar, R. (2026) · JMIR Medical Education · doi:10.2196/82515 · Cited by 1 · see citing works
Lee, M. M. Y., Desy, J. R., Tonelli, A. C., Walsh, M. H., & Ma, I. W. Y. (2023) · The Ultrasound Journal · doi:10.1186/s13089-023-00333-6 · Cited by 1 · see citing works
Alvarez-López, F., Saigí-Rubió, F., & Maina, M. F. (2024) · Revista de Educación a Distancia (RED) · doi:10.6018/red.608201 · Cited by 1 · see citing works
Jeyalingam, T. (2020) · TSpace (University of Toronto) · Cited by 1 · see citing works
Pandian, T. K., Abbott, E., Farley, D. R., & Cook, D. A. (2016) · Annals of Surgery · doi:10.1097/sla.0000000000001940
Self-citationJohnston, M. J., Arora, S., & Darzi, A. W. (2017) · Annals of Surgery · doi:10.1097/sla.0000000000001942
Through the eyes of the physician
Szulewski (2019) · doi:10.26481/dis.20190620as
Leppink, J. (2020) · Springer texts in education · doi:10.1007/978-3-030-43082-5_9
Gathering Validity Evidence for a Simulation-Based Test of Otoscopy Skills
von Buchwald, J. H., Frendø, M., Frithioff, A., Britze, A., Frederiksen, T. W., Melchiors, J., & Andersen, S. A. W. (2024) · Annals of Otology Rhinology & Laryngology · doi:10.1177/00034894241288434
Measuring Residents' Competence in Chest Tube Insertion on Thiel-Embalmed Bodies
Mol, L. D., HERZEELE, I. V., de Voorde, P. V., Vanommeslaeghe, H., Konge, L., Desender, L. M., & Willaert, W. (2024) · Simulation in Healthcare The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0000000000000842
Bonnema, H., Kelly, C. J., Hunt, J., Trantham, N., & Miller, L. M. J. (2025) · Journal of Veterinary Medical Education · doi:10.3138/jvme-2024-0103
Lee, M., Desy, J. R., Tonelli, A. C., Walsh, M. H., & Ma, I. W. Y. (2024) · Figshare · doi:10.6084/m9.figshare.26616748
Portes, K. P., Oliveira, G. C. D., Maia, R. B. S., Porto, B. C., Artifon, E. L., Otoch, J. P., & da Cruz, J. A. S. (2026) · Journal of surgical education · doi:10.1016/j.jsurg.2026.104129
Andersen, A. G., Olsen, R. G., Haidari, T. A., Konge, L., Svendsen, M. B. S., Bjerrum, F., & Petersen, R. H. (2026) · Surgical Endoscopy · doi:10.1007/s00464-026-13375-3
Czaja, A. S., Schwartz, A., Boyer, D. L., Green, M. L., Zurca, A., Maa, T., Werner, J. A., Turner, D. A., Moffatt, M. E., McFadden, V. C., et al. (2026) · Academic Pediatrics · doi:10.1016/j.acap.2026.103486
- Authors
- Cook, D. A.
- Affiliations
- Mayo Clinic (US)
- Corresponding author
- Cook, David A.
- ORCID
- Cook, David A.
- Venue
- Advances in Health Sciences Education
- Published
- 2014-09-27
- Volume
- 20
- Issue
- 3
- Pages
- 829-834
- ISSN
- 1382-4996 (ISSN-L)
- 1573-1677
- OpenAlex
- W2002440178
- Cited by
- 81
- References
- 20
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 6 · 2023: 8 · 2022: 3 · 2021: 4 · 2020: 12 · 2019: 9 · 2018: 8 · 2017: 9 · 2016: 10 · 2015: 5
- FWCI
- 7.92
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Advances in Health Sciences Education (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Education › Health Education and Validation
- Keywords
- known-groups comparison, expert-novice comparison, validity evidence, confounding, postgraduate trainees, group differences
- MeSH terms
- Clinical Competence, Competency-Based Education, Data Interpretation, Statistical, Education, Medical, Expert Testimony, Humans, Reproducibility of Results, Validation Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
