The DOI matches its Crossref record; 95 of the 95 references it cites were found in a registry or an index.
Measuring mental workload using physiological measures: A systematic review
Open accessAbstract
Technological advances have led to physiological measurement being increasingly used to measure and predict operator states. Mental workload (MWL) in particular has been characterised using a variety of physiological sensor data. This systematic review contributes a synthesis of the literature summarising key findings to assist practitioners to select measures for use in evaluation of MWL. We also describe limitations of the methods to assist selection when being deployed in applied or laboratory settings. We detail fifty-eight peer reviewed journal articles which present original data using physiological measures to include electrocardiographic, respiratory, dermal, blood pressure and ocular. Electroencephalographic measures have been included if they are presented with another measure to constrain scope. The literature reviewed covers a range of applied and experimental studies across various domains, safety-critical applications being highly represented in the sample of applied literature reviewed. We present a summary of the six measures and provide an evidence base which includes how to deploy each measure, and characteristics that can affect or preclude the use of a measure in research. Measures can be used to discriminate differences in MWL caused by task type, task load, and in some cases task difficulty. Varying ranges of sensitivity to sudden or gradual changes in taskload are also evident across the six measures. We conclude that there is no single measure that clearly discriminates mental workload but there is a growing empirical basis with which to inform both science and practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Heart rate variability: Standards of measurement, physiological interpretation, and clinical use
Malik, M., Bigger, J. T., Camm, A. J., Kleiger, R. E., Malliani, A., Moss, A. J., & Schwartz, P. J. (1996) · European Heart Journal · doi:10.1093/oxfordjournals.eurheartj.a014868 · Cited by 15,289
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
Multiple Resources and Mental Workload
Wickens, C. D. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288394 · Cited by 2,164
Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting
Kluge, A., Badura, B., Urbas, L., & Burkolter, D. (2010) · Cited by 1,127
State of science: mental workload in ergonomics
Young, M. S., Brookhuis, K. A., Wickens, C. D., & Hancock, P. A. (2015) · Ergonomics · doi:10.1080/00140139.2014.956151 · Cited by 948
The Subjective Workload Assessment Technique: A Scaling Procedure for Measuring Mental Workload
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62387-0 · Cited by 821
The effect of mental stress on heart rate variability and blood pressure during computer work
Hjortskov, N., Riss�n, D., Blangsted, A. K., Fallentin, N., Lundberg, U., & S�gaard, K. (2004) · European Journal of Applied Physiology · doi:10.1007/s00421-004-1055-z · Cited by 766
EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks.
Berka, C., Levendowski, D. J., Lumicao, M. N., Yau, A., Davis, G., Zivkovic, V. T., Olmstead, R. E., Tremoulet, P. D., & Craven, P. L. (2007) · PubMed · Cited by 765
EEG-alpha rhythms and memory processes
Klimesch, W. (1997) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(97)00773-3 · Cited by 732
Mental workload while driving: Effects on visual search, discrimination, and decision making.
Recarte, M. A., & Nunes, L. M. (2003) · Journal of Experimental Psychology: Applied · doi:10.1037/1076-898x.9.2.119 · Cited by 682
Biocybernetic system evaluates indices of operator engagement in automated task
Pope, A. T., Bogart, E. H., & Bartolome, D. S. (1995) · Biological Psychology · doi:10.1016/0301-0511(95)05116-3 · Cited by 663
Bernardi, L., Wdowczyk-Szulc, J., Valenti, C., Castoldi, S., Passino, C., Spadacini, G., & Sleight, P. (2000) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(00)00595-7 · Cited by 516
An Analysis of Mental Workload in Pilots During Flight Using Multiple Psychophysiological Measures
Wilson, G. F. (2002) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap1201_2 · Cited by 488
Psychophysiological responses to changes in workload during simulated air traffic control
Brookings, J. B., Wilson, G. F., & Swain, C. R. (1996) · Biological Psychology · doi:10.1016/0301-0511(95)05167-8 · Cited by 475
Physiological metrics of mental workload: A review of recent progress
Kramer, A. F. (2020) · Multiple-task performance · doi:10.1201/9781003069447-14 · Cited by 459
Physiological workload reactions to increasing levels of task difficulty
VELTMAN, J. A., & GAILLARD, A. W. K. (1998) · Ergonomics · doi:10.1080/001401398186829 · Cited by 433 · cited 10×
“Veltman and Gaillard (1998) observed a systematic decrease in NN intervals as a task became more difficult during a simulated flight task.”
BackgroundInfluentialMental workload : its theory and measurement
Moray, N. P. (1979) · Medical Entomology and Zoology · Cited by 414
Ryu, K., & Myung, R. (2005) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2005.04.005 · Cited by 399
Mehler, B., Reimer, B., Coughlin, J. F., & Dusek, J. A. (2009) · Transportation Research Record: Journal of the Transportation Research Board · doi:10.3141/2138-02 · Cited by 392
Wilson, G. F., & Russell, C. A. (2003) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.45.4.635.27088 · Cited by 375
Wickens, C. D. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288420 · Cited by 334
Physiological indices of workload in a simulated flight task
Veltman, J. A., & Gaillard, A. W. K. (1996) · Biological Psychology · doi:10.1016/0301-0511(95)05165-1 · Cited by 332
Delaney, J. P. A., & Brodie, D. A. (2000) · Perceptual and Motor Skills · doi:10.2466/pms.2000.91.2.515 · Cited by 298
Hogervorst, M. A., Brouwer, A.-M., & van Erp, J. B. F. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00322 · Cited by 292
The influence of task demand and learning on the psychophysiological response
Fairclough, S. H., Venables, L., & Tattersall, A. (2005) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2004.11.003 · Cited by 290
The Psychometrics of Mental Workload: Multiple Measures Are Sensitive but Divergent
Matthews, G., Reinerman-Jones, L. E., Barber, D. J., & Abich, J. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814539505 · Cited by 265
Jorna, P. G. A. M. (1992) · Biological Psychology · doi:10.1016/0301-0511(92)90017-o · Cited by 259
Using near infrared spectroscopy and heart rate variability to detect mental overload
Durantin, G., Gagnon, J.-F., Tremblay, S., & Dehais, F. (2014) · Behavioural Brain Research · doi:10.1016/j.bbr.2013.10.042 · Cited by 259
Hankins, T., & Wilson, G. F. (1998) · PubMed · Cited by 251
Xie, B., & Salvendy, G. (2000) · Work & Stress · doi:10.1080/026783700417249 · Cited by 247
Heart rate variability and cognitive processing: The autonomic response to task demands
Luque-Casado, A., Perales, J. C., Cárdenas, D., & Sanabria, D. (2016) · Biological Psychology · doi:10.1016/j.biopsycho.2015.11.013 · Cited by 243
Respiratory Changes in Response to Cognitive Load: A Systematic Review
Grassmann, M., Vlemincx, E., von Leupoldt, A., Mittelstädt, J. M., & Van den Bergh, O. (2016) · Neural Plasticity · doi:10.1155/2016/8146809 · Cited by 219
Assessing pilot workload. Why measure heart rate, HRV and respiration?
Roscoe, A. H. (1992) · Biological Psychology · doi:10.1016/0301-0511(92)90018-p · Cited by 213
A Closed-Loop System for Examining Psychophysiological Measures for Adaptive Task Allocation
Prinzel, L. J., Freeman, F. G., Scerbo, M. W., Mikulka, P. J., & Pope, A. T. (2000) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap1004_6 · Cited by 211
Backs, R. W., & Seljos, K. A. (1994) · International Journal of Psychophysiology · doi:10.1016/0167-8760(94)90042-6 · Cited by 206
De Rivecourt, M., Kuperus, M. N., Post, W. J., & Mulder, L. J. M. (2008) · Ergonomics · doi:10.1080/00140130802120267 · Cited by 204
Prediction of subjective states from psychophysiology: A multivariate approach
Fairclough, S. H., & Venables, L. (2006) · Biological Psychology · doi:10.1016/j.biopsycho.2005.03.007 · Cited by 199
Wilson, G. F., & Russell, C. A. (2003) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.45.3.381.27252 · Cited by 195
Information complexity-mental workload and performance in combat aircraft
Svensson, E., Angelborg-Thanderz, M., Sjöberg, L., & Olsson, S. (1997) · Ergonomics · doi:10.1080/001401397188206 · Cited by 184
Engineering Psychophysiology : Issues and Applications
Boucsein, W. (2009) · Medical Entomology and Zoology · Cited by 173
Holland, M. K., & Tarlow, G. (1972) · Psychological Reports · doi:10.2466/pr0.1972.31.1.119 · Cited by 173
Fallahi, M., Motamedzade, M., Heidarimoghadam, R., Soltanian, A. R., & Miyake, S. (2016) · Applied Ergonomics · doi:10.1016/j.apergo.2015.07.009 · Cited by 172
Mental workload measurement for emergency operating procedures in digital nuclear power plants
Gao, Q., Wang, Y., Song, F., Li, Z., & Dong, X. (2013) · Ergonomics · doi:10.1080/00140139.2013.790483 · Cited by 168
Nickel, P., & Nachreiner, F. (2003) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.45.4.575.27094 · Cited by 167
Physiological Metrics of Mental Workload: A Review of Recent Progress
Kramer, A. F. (1990) · Defense Technical Information Center (DTIC) · doi:10.21236/ada223701 · Cited by 163
A Subjective Rating Scale for Assessing Pilot Workload in Flight: A decade of Practical Use
Roscoe, A. H., & Ellis, G. A. (1990) · Defense Technical Information Center (DTIC) · Cited by 161
Multivariate workload evaluation combining physiological and subjective measures
Miyake, S. (2001) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(00)00191-4 · Cited by 159
Fournier, L. R., Wilson, G. F., & Swain, C. R. (1999) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(98)00049-x · Cited by 158
Inflight workload assessment: comparison of subjective and physiological measurements.
Lee, Y.-H., & Liu, B. (2003) · PubMed · Cited by 152
Assessment of pilot performance and mental workload in rotary wing aircraft
Sirevaag, E. J., Kramer, A. F., Wickens, C. D., Strayer, D. L., & Grenell, J. F. (1993) · Ergonomics · doi:10.1080/00140139308967983 · Cited by 151
The Relationship Between Cardiovascular Responses in the Laboratory and in the Field
Johnston, D. W., Anastasiades, P., & Wood, C. (1990) · Psychophysiology · doi:10.1111/j.1469-8986.1990.tb02175.x · Cited by 141
Level of operator control and changes in heart rate variability during simulated flight maintenance.
Tattersall, A. J., & Hockey, G. R. J. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095778995517 · Cited by 126
Predicting work performance in nuclear power plants
Hwang, S., Yau, Y., Lin, Y., Chen, J., Huang, T.-H., Yenn, T., & Hsu, C. (2007) · Safety Science · doi:10.1016/j.ssci.2007.06.005 · Cited by 124
Brief review on physiological and biochemical evaluations of human mental workload
Le-an, Y., & Fu, S. (2011) · Human Factors and Ergonomics in Manufacturing & Service Industries · doi:10.1002/hfm.20269 · Cited by 124
Fighter pilots’ heart rate, heart rate variation and performance during instrument approaches
Mansikka, H., Simola, P. E., Virtanen, K., Harris, D., & Oksama, L. (2016) · Ergonomics · doi:10.1080/00140139.2015.1136699 · Cited by 104
Applications of Human Performance Models to System Design
(1989) · Springer eBooks · doi:10.1007/978-1-4757-9244-7 · Cited by 98
Cardiovascular response to Stroop: Effect of verbal response and task difficulty
Boutcher, Y. N., & Boutcher, S. H. (2006) · Biological Psychology · doi:10.1016/j.biopsycho.2006.04.005 · Cited by 95
Guzik, P., & Malík, M. (2016) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2016.07.030 · Cited by 93
Air-to-ground training missions: a psychophysiological workload analysis
Wilson, G. F. (1993) · Ergonomics · doi:10.1080/00140139308967979 · Cited by 91
Applied use of cardiac and respiration measures: Practical considerations and precautions
Wilson, G. F. (1992) · Biological Psychology · doi:10.1016/0301-0511(92)90014-l · Cited by 87
Physiological responses to workload change. A test/retest examination
Miyake, S., Yamada, S., Shoji, T., Takae, Y., Kuge, N., & Yamamura, T. (2009) · Applied Ergonomics · doi:10.1016/j.apergo.2009.02.005 · Cited by 86
Lehrer, P. M., Karavidas, M. K., Lu, S., Vaschillo, E. G., Vaschillo, B., & Cheng, A. (2010) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2010.02.006 · Cited by 85
A Comparative Study of Psychophysiological Reactions During Simulator and Real Flight
Veltman, J. A. (2002) · International Journal of Aviation Psychology · doi:10.1207/s15327108ijap1201_4 · Cited by 85
The Development and Application of Models to Predict Operator Workload During System Design
Aldrich, T. B., Szabo, S. M., & Bierbaum, C. R. (1989) · Springer eBooks · doi:10.1007/978-1-4757-9244-7_5 · Cited by 78
Mental workload classification using heart rate metrics
Henelius, A., Hirvonen, K., Holm, A., Korpela, J., & Müller, K. (2009) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2009.5332602 · Cited by 78
Finsen, L., Søgaard, K., Jensen, C., Borg, V., & Christensen, H. (2001) · Ergonomics · doi:10.1080/00140130110099065 · Cited by 77
Physical, mental, emotional, and subjective workload components in train drivers
Myrtek, M., DEUTSCHMANN-JANICKE, E., STROHMAIER, H., et al. (1994) · Ergonomics · doi:10.1080/00140139408964897 · Cited by 76
Causse, M., Sénard, J., Démonet, J., & Pastor, J. (2009) · Applied Psychophysiology and Biofeedback · doi:10.1007/s10484-009-9115-0 · Cited by 75
Mansikka, H., Virtanen, K., Harris, D., & Simola, P. E. (2016) · Applied Ergonomics · doi:10.1016/j.apergo.2016.04.006 · Cited by 74
Designing automation for complex work environments under different levels of stress
Sauer, J., Nickel, P., & Wastell, D. G. (2012) · Applied Ergonomics · doi:10.1016/j.apergo.2012.05.008 · Cited by 71
Svensson, E., & Wilson, G. F. (2002) · International Journal of Aviation Psychology · doi:10.1207/s15327108ijap1201_8 · Cited by 70
Effective Indices for Monitoring Mental Workload While Performing Multiple Tasks
Hsu, B.-W., Wang, M. J., Chen, C., & Chen, F. (2015) · Perceptual and Motor Skills · doi:10.2466/22.pms.121c12x5 · Cited by 67
Psychophysiological Measures of Workload during Continuous Manual Performance
Backs, R. W., Ryan, A. M., & Wilson, G. F. (1994) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872089403600308 · Cited by 65
Measuring workload with electrodermal activity during common braking actions
Collet, C., Salvia, É., & Petit-Boulanger, C. (2014) · Ergonomics · doi:10.1080/00140139.2014.899627 · Cited by 63
Improving pilot mental workload evaluation with combined measures
Wanyan, X., Zhuang, D., & Zhang, H. (2014) · Bio-Medical Materials and Engineering · doi:10.3233/bme-141041 · Cited by 60
Inflight application of three pilot workload measurement techniques.
Hart, S. G., & Hauser, J. R. (1987) · PubMed · Cited by 60
Fundamental examination of mental workload in the rail industry
Pickup, L., Wilson, J. R., Sharpies, S., NORRIS, B. J., Clarke, T., & Young, M. S. (2005) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500078021 · Cited by 60
Reiner, M., & Gelfeld, T. M. (2013) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2013.11.002 · Cited by 59
Effects of mental tasks on the cardiorespiratory synchronization
Zhang, J., Yu, X., & Xie, D. (2009) · Respiratory Physiology & Neurobiology · doi:10.1016/j.resp.2009.11.003 · Cited by 55
Auditory Discrimination and the Eyeblink
Bauer, L. O., Strock, B. D., Goldstein, R., Stern, J. A., & Walrath, L. C. (1985) · Psychophysiology · doi:10.1111/j.1469-8986.1985.tb01660.x · Cited by 53
Immediate and delayed after-effects of long lasting mentally demanding work
Schellekens, J. M. H., Sijtsma, G. J., Vegter, E., & Meijman, T. F. (2000) · Biological Psychology · doi:10.1016/s0301-0511(00)00039-9 · Cited by 52
A Psychophysiological Approach to Working Conditions
Mulder, L. J. M., & van Roon, A. M. (2000) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 51
The Impact of Workload on Heart Rate and Blood Pressure in En-Route and Tower Air Traffic Control
Vogt, J., Hagemann, T., & Kastner, M. (2006) · Journal of Psychophysiology · doi:10.1027/0269-8803.20.4.297 · Cited by 50
Context Effects in Subjective Mental Workload Ratings
Colle, H. A., & Reid, G. B. (1998) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872098779649283 · Cited by 50
Heart Rate Variability: A New Index of Driver Alertness/Fatigue
O’Hanlon, J. F. (1972) · SAE technical papers on CD-ROM/SAE technical paper series · doi:10.4271/720141 · Cited by 46
Grassmann, M., Vlemincx, E., von Leupoldt, A., & Van den Bergh, O. (2017) · Applied Ergonomics · doi:10.1016/j.apergo.2016.09.006 · Cited by 45
Hohnsbein, J., Falkenstein, M., & Hoormann, J. (1995) · Biological Psychology · doi:10.1016/0301-0511(95)05109-0 · Cited by 40
(2007) · PubMed · Cited by 38
Heart rate responses to real and simulated BA Hawk MK 51 flight.
Ylönen, H., Lyytinen, H., Leino, T. K., Leppäluoto, J., & Kuronen, P. (1997) · PubMed · Cited by 34
A psychophysiological approach to the assessment of work underload
Braby, C. D., Harris, D., & Muir, H. C. (1993) · Ergonomics · doi:10.1080/00140139308967975 · Cited by 33
Nixon, J., & Charles, R. L. (2017) · Cognition Technology & Work · doi:10.1007/s10111-017-0431-5 · Cited by 26
Self-citationCardiorespiratory Indices of Mental Workload during Simulated Air Traffic Control
Backs, R. W., Navidzadeh, H. T., & Xu, X. (2000) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120004401323 · Cited by 25
Coadaptive Aiding and Automation Enhance Operator Performance
Christensen, J. C., & Estepp, J. R. (2013) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720813476883 · Cited by 20
Environmental and Behavioral Factors That Can Affect Blood Pressure
Adams, C. E., & Leverland, M. B. (1985) · The Nurse Practitioner · doi:10.1097/00006205-198511000-00005 · Cited by 19
Wu, B., Hou, F., Zhi, Y., Niu, J., & Huang, W. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21799-9_48 · Cited by 9
Cited by
Outlook on human-centric manufacturing towards Industry 5.0
Lu, Y., Zheng, H., Chand, S., Xia, W., Liu, Z., Xu, X., Wang, L., Qin, Z., & Bao, J. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.02.001 · Cited by 671
Human Digital Twin in the context of Industry 5.0
Wang, B., Zhou, H., Li, X., Yang, G., Zheng, P., Song, C., Yuan, Y., Wuest, T., Yang, H., & Wang, L. (2024) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2023.102626 · Cited by 424
Review of Eye Tracking Metrics Involved in Emotional and Cognitive Processes
Skaramagkas, V., Giannakakis, G., Ktistakis, E., Manousos, D., Karatzanis, I., Tachos, N., Tripoliti, E., Marias, K., Fotiadis, D. I., & Tsiknakis, M. (2023) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2021.3066072 · Cited by 419
A Systematic Review of Physiological Measures of Mental Workload
Tao, D., Tan, H., Wang, H., Zhang, X., Qu, X., & Zhang, T. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16152716 · Cited by 303
EEG power spectral measures of cognitive workload: A meta‐analysis
Chikhi, S., Matton, N., & Blanchet, S. (2022) · Psychophysiology · doi:10.1111/psyp.14009 · Cited by 258
Metadata
- Authors
- Charles, R. L., & Nixon, J.
- Venue
- Applied Ergonomics
- Publisher
- Elsevier BV
- Published
- 2019-01
- Volume
- 74
- Pages
- 221-232
- ISSN
- 0003-6870 (ISSN-L)
- OpenAlex
- W2890072046
- Cited by
- 704
- References
- 95
- FWCI
- 9.20
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.