The Use of Task-Evoked Pupillary Response as an Objective Measure of Cognitive Load in Novices and Trained Physicians: A New Tool for the Assessment of Expertise
Abstract
PURPOSE: Task-evoked pupillary responses (TEPRs), or changes in pupil size, correlate with changes in cognitive processing demands. The magnitude of this change is a reliable marker of cognitive load. The authors used TEPRs to compare cognitive load between novices and trained physicians as they answered clinical knowledge questions. METHOD: In 2013, 20 emergency medicine trainees were recruited and divided into novice (n = 10) and trained physician (n = 10) groups. The authors used mobile eye-tracking glasses to assess changes in pupil diameter as participants answered arithmetic questions, general knowledge questions, and clinical emergency medicine questions in a controlled setting. Questions were categorized by difficulty a priori. RESULTS: Difficult arithmetic questions caused greater changes in TEPRs than easy ones (P = .024). TEPRs were similar between groups when answering general knowledge questions (P = .383) but were significantly greater for novices than trained physicians when answering clinical questions (P < .001). TEPRs in trained physicians were significantly greater when answering difficult clinical questions than easy ones (P < .001), whereas TEPRs in novices were similar (P = .291). For those clinical questions answered correctly by both groups, TEPRs in novices were greater than those in trained physicians despite all participants answering correctly (P < .001). CONCLUSIONS: Novices require more mental effort to answer clinical questions than trained physicians, even when both respond correctly. Measuring TEPRs has the potential to be a valuable assessment tool by providing objective measures of expertise and is worthy of further study.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Measuring changes in pupil size as a surrogate marker for cognitive load has shown that experienced physicians expend less cognitive load when answering domain-specific multiple choice questions than novices.
“With respect to resuscitation medicine content and resuscitation medicine expertise, measuring changes in pupil size as a surrogate marker for cognitive load has shown that experienced physicians expend less cognitive load when answering domain-specific multiple choice questions than novices (Szulewski et al. 2015).”
From Measuring physician cognitive load: validity evidence for a physiologic and a psychometric tool (2017) · cites this paper as Szulewski et al. 2015
Previous studies utilizing pupillometry as a physiologic measure of cognitive load have concluded that measuring cognitive load accurately requires an analysis of both the magnitude of the change in pupil size as well as the duration of time between question presentation and verbalization of a response.
“Previous studies utilizing pupillometry as a physiologic measure of cognitive load have concluded that measuring cognitive load accurately requires an analysis of both the magnitude of the change in pupil size as well as the duration of time between question presentation and verbalization of a response (i.e. the time that a participant is thinking about an answer) (Szulewski et al. 2015).”
From Measuring physician cognitive load: validity evidence for a physiologic and a psychometric tool (2017) · cites this paper as Szulewski et al. 2015
This new technology has also been successfully utilized in cognitive load measurement experiments in medical testing where questions posed to participants were shown to affect pupil size in predictable ways based on question and participant characteristics.
“This new technology has also been successfully utilized in cognitive load measurement experiments in medical testing where questions posed to participants were shown to affect pupil size in predictable ways based on question and participant characteristics (Szulewski et al. 2015).”
From Measuring physician cognitive load: validity evidence for a physiologic and a psychometric tool (2017) · cites this paper as Szulewski et al. 2015
Cites
Cognitive Architecture and Instructional Design
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998) · Educational Psychology Review · doi:10.1023/a:1022193728205 · Cited by 6,023 · see citing works
Ericsson, K. A., & Kintsch, W. (1995) · Psychological Review · doi:10.1037/0033-295x.102.2.211 · Cited by 3,498 · see citing works
Cognitive Load Measurement as a Means to Advance Cognitive Load Theory
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_8 · Cited by 2,527 · see citing works
Task-evoked pupillary responses, processing load, and the structure of processing resources.
Beatty, J. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.91.2.276 · Cited by 1,959 · see citing works
Pupil Diameter and Load on Memory
Kahneman, D., & Beatty, J. (1966) · Science · doi:10.1126/science.154.3756.1583 · Cited by 1,617 · see citing works
Cognitive load theory, educational research, and instructional design: some food for thought
de Jong, T. (2010) · Instructional Science · doi:10.1007/s11251-009-9110-0 · Cited by 1,180 · see citing works
Pupil Size in Relation to Mental Activity during Simple Problem-Solving
Hess, E. H., & Polt, J. M. (1964) · Science · doi:10.1126/science.143.3611.1190 · Cited by 1,169 · see citing works
Decision making and the avoidance of cognitive demand
Kool, W., McGuire, J. T., Rosen, Z. B., & Botvinick, M. M. (2010) · Journal of Experimental Psychology: General · doi:10.1037/a0020198 · Cited by 1,141 · see citing works
Gegenfurtner, A., Lehtinen, E., & Säljö, R. (2011) · Educational Psychology Review · doi:10.1007/s10648-011-9174-7 · Cited by 715 · see citing works
DeLeeuw, K. E., & Mayer, R. E. (2008) · Journal of Educational Psychology · doi:10.1037/0022-0663.100.1.223 · Cited by 558 · see citing works
Hess, E. H. (1965) · Scientific American · doi:10.1038/scientificamerican0465-46 · Cited by 515 · see citing works
Klingner, J., Tversky, B., & Hanrahan, P. (2011) · Psychophysiology · doi:10.1111/j.1469-8986.2010.01069.x · Cited by 255 · see citing works
The nature of expertise: A review
Farrington-Darby, T., & Wilson, J. R. (2005) · Applied Ergonomics · doi:10.1016/j.apergo.2005.09.001 · Cited by 212 · see citing works
Pupil diameter measurement errors as a function of gaze direction in corneal reflection eyetrackers
Brisson, J., Mainville, M., Mailloux, D., Beaulieu, C., Serres, J., & Sirois, S. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0327-0 · Cited by 149 · see citing works
Schubert, C. C., Denmark, T. K., Crandall, B., Grome, A., & Pappas, J. (2013) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2012.08.034 · Cited by 92 · see citing works
The Use Of Item Analysis For The Improvement Of Objective Examinations
Siri, A., & Freddano, M. (2011) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2011.11.224 · Cited by 42 · see citing works
USING ITEM ANALYSIS TO IMPROVE TESTS
Lange, A. L., Lehmann, I. J., & Mehrens, W. A. (1967) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1967.tb00572.x · Cited by 27 · see citing works
Combining First-Person Video and Gaze-Tracking in Medical Simulation: A Technical Feasibility Study
Szulewski, A., & Howes, D. W. (2014) · The Scientific World JOURNAL · doi:10.1155/2014/975752 · Cited by 25 · see citing works
Self-citationSzulewski, A., Fernando, S. M., Baylis, J., & Howes, D. (2014) · Open Journal of Emergency Medicine · doi:10.4236/ojem.2014.21002 · Cited by 17 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
From Theory to Practice: The Application of Cognitive Load Theory to the Practice of Medicine
Szulewski, A., Howes, D., van Merriënboer, J. J. G., & Sweller, J. (2021) · Academic Medicine · doi:10.1097/acm.0000000000003524 · Cited by 171 · see citing works
Self-citationMeasuring physician cognitive load: validity evidence for a physiologic and a psychometric tool
Szulewski, A., Gegenfurtner, A., Howes, D. W., Sivilotti, M. L. A., & van Merriënboer, J. J. G. (2017) · Advances in Health Sciences Education · doi:10.1007/s10459-016-9725-2 · Cited by 94 · see citing works
Self-citationRuberto, A. J., Rodenburg, D., Ross, K., Sarkar, P., Hungler, P. C., Etemad, A., Howes, D., Clarke, D., McLellan, J., Wilson, D., et al. (2021) · AEM Education and Training · doi:10.1002/aet2.10605 · Cited by 22 · see citing works
Self-citation
- Authors
- Szulewski, A., Roth, N., & Howes, D.
- Affiliations
- Queen's University (CA)
- Corresponding author
- Howes, Daniel
- ORCID
- Szulewski, Adam
- Venue
- Academic Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-07
- Volume
- 90
- Issue
- 7
- Pages
- 981-987
- ISSN
- 1040-2446 (ISSN-L)
- OpenAlex
- W2048144107
- Cited by
- 84
- References
- 19
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 5 · 2023: 6 · 2022: 9 · 2021: 9 · 2020: 9 · 2019: 12 · 2018: 12 · 2017: 5 · 2016: 5 · 2015: 1 · 2014: 1
- FWCI
- 7.36
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Academic Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Keywords
- task-evoked pupillary response, cognitive load, pupil size, eye tracking, emergency medicine trainees, physician expertise, question difficulty, cognitive effort
- MeSH terms
- Clinical Competence, Cognition, Diagnostic Techniques, Ophthalmological, Emergency Medicine, Humans, Ontario, Physicians, Pupil, Students, Medical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
