Classifying driver workload using physiological and driving performance data: two field studies
The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Understanding the driver's cognitive load is important for evaluating in-vehicle user interfaces. This paper describes experiments to assess machine learning classification algorithms on their ability to automatically identify elevated cognitive workload levels in drivers, leading towards the development of robust tools for automobile user interface evaluation. We look at using both driver performance as well as physiological data. These measures can be collected in real-time and do not interfere with the primary task of driving the vehicle. We report classification accuracies of up to 90% for detecting elevated levels of cognitive load, and show that the inclusion of physiological data leads to higher classification accuracy than vehicle sensor data evaluated alone. Finally, we show results suggesting that models can be built to classify cognitive load across individuals, instead of building individual models for each per-son. By collecting data from drivers in two large field studies on the highway (20 drivers and 99 drivers), this work extends prior work and demonstrates feasibility and potential of such measures for HCI research in vehicles.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The relation of strength of stimulus to rapidity of habit formation
Yerkes, R. M., & Dodson, J. D. (1908) · Journal of Comparative Neurology and Psychology · doi:10.1002/cne.920180503 · Cited by 6,911
Handbook of perception and human performance
Boff, K. R., Kaufman, L., & Thomas, J. P. (1986) · Wiley eBooks · Cited by 4,128
Detecting Stress During Real-World Driving Tasks Using Physiological Sensors
Healey, J. A., & Picard, R. W. (2005) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2005.848368 · Cited by 2,207 · cited 3×
“[14] used physiological data (EKG, EMG, EDA, and respiration) from nine drivers in three real-world settings: rest, highway and urban.”
BackgroundMethodA Critical View of Driver Behavior Models: What Do We Know, What Should We Do?
Michon, J. A. (1985) · Human Behavior and Traffic Safety · doi:10.1007/978-1-4613-2173-6_19 · Cited by 1,230 · cited 1×
“Driving itself is a dynamic, complex activity involving visual, cognitive and manual tasks: the driver has to form strategic goals, monitor the roadway environment and the vehicle systems, process information and make tactical action plans as well as execute control level activities [23].”
BackgroundZander, T. O., & Kothe, C. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/8/2/025005 · Cited by 887 · cited 1×
“In addition, the methods we described would apply to classifying workload in other contexts, such as game user experience evaluation or passive, adaptive user interfaces (similar to passive BCI work [1,9,35,44]), and our sample size is larger than most papers in those areas.”
MethodHandbook of Human Factors and Ergonomics Methods
(2004) · CRC Press eBooks · doi:10.1201/9780203489925 · Cited by 836
Effects of visual and cognitive load in real and simulated motorway driving
Engström, J., Johansson, E., & Östlund, J. (2005) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2005.04.012 · Cited by 834 · cited 1×
“The European HASTE study [8] produced mixed results on the relative sensitivity of heart rate and skin conductance in response to the demands surrogate secondary tasks in a sample of 24 drivers in the field.”
ResultToward Brain-Computer Interfacing
(2007) · The MIT Press eBooks · doi:10.7551/mitpress/7493.001.0001 · Cited by 785
Ck, F. (1970) · PubMed · Cited by 641
Real-time detection of driver cognitive distraction using support vector machines
Liang, Y., Reyes, M. L., & Lee, J. D. (2007) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2007.895298 · Cited by 482 · cited 3×
“Measuring Physiological Signals in the Car Literature on measuring physiological signals in the car can be divided into simulator studies [13,18,19,24,36] and field studies [14,15,18,28,30,33,40].”
BackgroundWorkload: An examination of the concept.
Gopher, D., & Donchin, E. (1986) · Digital Commons - University of South Florida (University of South Florida) · Cited by 470
Physiological workload reactions to increasing levels of task difficulty
VELTMAN, J. A., & GAILLARD, A. W. K. (1998) · Ergonomics · doi:10.1080/001401398186829 · Cited by 433 · cited 1×
“Cardiovascular measures have been reported to be sensitive to mental workload changes (e.g. [5,12,25,38,39]).”
BackgroundYoung, M. S., & Stanton, N. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024497709 · Cited by 404 · cited 1×
“Advances in technology have led to a shift in the demands in many working environments from the largely physical to more supervisory oversight of automation, more cognitive demand, and increased frequency of vocal command interaction [34,42].”
BackgroundMehler, B., Reimer, B., Coughlin, J. F., & Dusek, J. A. (2009) · Transportation Research Record: Journal of the Transportation Research Board · doi:10.3141/2138-02 · Cited by 392 · cited 4×
“Measuring Physiological Signals in the Car Literature on measuring physiological signals in the car can be divided into simulator studies [13,18,19,24,36] and field studies [14,15,18,28,30,33,40].”
BackgroundMethodInfluentialSelf-citationMeasurement and analysis methods of heart rate and respiration for use in applied environments
Mulder, L. J. M. (1992) · Biological Psychology · doi:10.1016/0301-0511(92)90016-n · Cited by 316 · cited 2×
“In contrast, [25] claims HRV to be a selective measure for mental load.”
BackgroundHaigney, D. E., Taylor, R. G., & Westerman, S. J. (2000) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/s1369-8478(00)00020-6 · Cited by 298 · cited 1×
“Cardiovascular measures have been reported to be sensitive to mental workload changes (e.g. [5,12,25,38,39]).”
BackgroundA decrease in brain activation associated with driving when listening to someone speak
Just, M. A., Keller, T. A., & Cynkar, J. (2008) · Brain Research · doi:10.1016/j.brainres.2007.12.075 · Cited by 296 · cited 1×
“This development can be observed in the field of driving as well [16,37].”
BackgroundMehler, B., Reimer, B., & Coughlin, J. F. (2012) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720812442086 · Cited by 294 · cited 3×
“This has been shown in previous work to increase cognitive demand [20,29] and is being utilized as a calibration task that provides drivers with a consistent, and validated, dose of high cognitive demand that has been used in prior studies.”
BackgroundMethodSelf-citationKantowitz, B. H. (1987) · Advances in psychology · Cited by 290
Feasibility and pragmatics of classifying working memory load with an electroencephalograph
Grimes, D., Tan, D. S., Hudson, S. E., Shenoy, P., & Rao, R. P. N. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357187 · Cited by 192 · cited 2×
“as window size can have an impact on classification results in EEG data [11].”
BackgroundReimer, B., & Mehler, B. (2011) · Ergonomics · doi:10.1080/00140139.2011.604431 · Cited by 183 · cited 1×
“Measuring Physiological Signals in the Car Literature on measuring physiological signals in the car can be divided into simulator studies [13,18,19,24,36] and field studies [14,15,18,28,30,33,40].”
BackgroundSelf-citationJahn, G., Oehme, A., Krems, J. F., & Gelau, C. (2005) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2005.04.009 · Cited by 180 · cited 2×
“Measuring Physiological Signals in the Car Literature on measuring physiological signals in the car can be divided into simulator studies [13,18,19,24,36] and field studies [14,15,18,28,30,33,40].”
BackgroundKohlmorgen, J., Dornhege, G., Braun, M. L., Blankertz, B., Müller, K.-R., Curio, G., Hagemann, K., Bruns, A., Schrauf, M., & Kincses, W. E. (2007) · Toward Brain-Computer Interfacing · doi:10.7551/mitpress/7493.003.0031 · Cited by 180 · cited 1×
“In addition, there has been interest in classifying workload from brain data during driving (e.g. [ 17]).”
BackgroundVehicle automation and driving performance
STANTON, N. A., & YOUNG, M. S. (1998) · Ergonomics · doi:10.1080/001401398186568 · Cited by 169 · cited 1×
“This development can be observed in the field of driving as well [16,37].”
BackgroundReimer, B., Mehler, B. L., Wang, Y., & Coughlin, J. F. (2012) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720812437274 · Cited by 153 · cited 4×
“The skin conductance recordings were filtered using a wavelet transform to remove high frequency noise ([29]).”
BackgroundMethodInfluentialSelf-citationDynamic difficulty using brain metrics of workload
Afergan, D., Peck, E. M., Solovey, E. T., Jenkins, A., Hincks, S. W., Brown, E. T., Chang, R., & Jacob, R. J. K. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557230 · Cited by 148 · cited 1×
“In addition, the methods we described would apply to classifying workload in other contexts, such as game user experience evaluation or passive, adaptive user interfaces (similar to passive BCI work [1,9,35,44]), and our sample size is larger than most papers in those areas.”
MethodSelf-citationA data set of real world driving to assess driver workload
Schneegass, S., Pfleging, B., Broy, N., Heinrich, F., & Schmidt, A. (2013) · Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/2516540.2516561 · Cited by 135 · cited 2×
“[33] present a field study with ten participants in which they collected EKG, skin conductance, and skin temperature data while participants drove in road environments presumed to induce differing levels of demand.”
BackgroundWAARD, D. D., JESSURUN, M., STEYVERS, F. J. J. M., REGGATT, P. T. F., & BROOKHUIS, K. A. (1995) · Ergonomics · doi:10.1080/00140139508925197 · Cited by 120 · cited 2×
“Measuring Physiological Signals in the Car Literature on measuring physiological signals in the car can be divided into simulator studies [13,18,19,24,36] and field studies [14,15,18,28,30,33,40].”
BackgroundMonitoring, managing, and motivating driver safety and well-being
Coughlin, J. F., Reimer, B., & Mehler, B. L. (2011) · IEEE Pervasive Computing · doi:10.1109/mprv.2011.54 · Cited by 119 · cited 1×
“These vehicle and physiological measures provide the potential to monitor the dynamic state of the driver while actually driving and provide inputs to make adjustments in the characteristics of the vehicle or interfaces to improve performance [6].”
BackgroundSelf-citationElectrodermal Activity in the 1980s: A Review
Christie, M. J. (1981) · Journal of the Royal Society of Medicine · doi:10.1177/014107688107400812 · Cited by 111 · cited 1×
“Electrodermal activity (EDA) refers to the electrical activity from (eccrine) sweat glands and their associated dermal and epidermal tissues [4].”
BackgroundPhysiological and behavioural changes associated to the management of secondary tasks while driving
Collet, C., Clarion, A., Morel, M., Chapon, A., & Petit, C. (2009) · Applied Ergonomics · doi:10.1016/j.apergo.2009.01.007 · Cited by 109 · cited 1×
“[5] collected heart rate and skin resistance (inverse of SCL) data in 10 drivers on a closed driving track and found that both measures increased during secondary tasks requiring their attention (a phone conversation, a radio broadcast with content they would be quizzed on, and a conversation).”
BackgroundSalvucci, D. D., Zuber, M., Beregovaia, E., & Markley, D. (2005) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055052 · Cited by 103 · cited 1×
“It may be useful to compare physiological sensing with cognitive modeling tools such as Distract-R [32] that also can be used for in-vehicle interface evaluation.”
MethodHandbook of Human Factors and Ergonomics Methods.
Mulder, L. J. M. (2004) · Cited by 100
Brainput: enhancing interactive systems with streaming fnirs brain input
Solovey, E., Schermerhorn, P., Scheutz, M., Sassaroli, A., Fantini, S., & Jacob, R. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208372 · Cited by 86 · cited 1×
“In addition, the methods we described would apply to classifying work-load in other contexts, such as game user experience evaluation or passive, adaptive user interfaces (similar to passive BCI work [1,9,35,44]), and our sample size is larger than most papers in those areas.”
MethodSelf-citationSaga, K. (2001) · Journal of Investigative Dermatology Symposium Proceedings · doi:10.1046/j.0022-202x.2001.00005.x · Cited by 77 · cited 1×
“Though related, eccrine and apocrine sweat glands are distinct in size, structure, function, distribution and nervous control [31].”
BackgroundPETERSON, F., & JUNG, C. G. (1907) · Brain · doi:10.1093/brain/30.2.153 · Cited by 76
Workload-adaptive cruise control – A new generation of advanced driver assistance systems
Hajek, W., Gaponova, I., Fleischer, K. H., & Krems, J. F. (2013) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2013.06.001 · Cited by 73 · cited 1×
“Measuring Physiological Signals in the Car Literature on measuring physiological signals in the car can be divided into simulator studies [13,18,19,24,36] and field studies [14,15,18,28,30,33,40].”
BackgroundCardiac Measures of Driver Workload during Simulated Driving with and without Visual Occlusion
Backs, R. W., Lenneman, J. K., Wetzel, J. M., & Green, P. (2003) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.45.4.525.27089 · Cited by 67 · cited 1×
“Heart rate (HR) and heart rate variability (HRV) are widely adopted mental workload measures because they are easy to use and provide fundamental information about the autonomic nervous system [3,25].”
BackgroundMehler, B., Reimer, B., & Wang, Y. (2011) · Driving Assessment Conference 2011 · doi:10.17077/drivingassessment.1451 · Cited by 60
Self-citationSon, J., Reimer, B., Mehler, B. L., Pohlmeyer, A. E., Godfrey, K. M., Orszulak, J., Long, J., Kim, M. H., Lee, Y. T., & Coughlin, J. F. (2010) · International Journal of Automotive Technology · doi:10.1007/s12239-010-0065-6 · Cited by 51 · cited 2×
“EDA measures such as skin conductance level (SCL) and skin conductance response (SCR) have been reported to be sensitive to arousal and mental workload in driving as well [14,36].”
BackgroundSelf-citationSingleton, W. T. (1989) · Cambridge University Press eBooks · doi:10.1017/cbo9780511525407 · Cited by 50
Girouard, A., Solovey, E. T., & Jacob, R. J. K. (2013) · International Journal of Autonomous and Adaptive Communications Systems · doi:10.1504/ijaacs.2013.050689 · Cited by 39 · cited 1×
“In addition, the methods we described would apply to classifying workload in other contexts, such as game user experience evaluation or passive, adaptive user interfaces (similar to passive BCI work [1,9,35,44]), and our sample size is larger than most papers in those areas.”
MethodSelf-citationDriver’s cognitive distraction detection using AdaBoost on pattern recognition basis
Miyaji, M., Danno, M., Kawanaka, H., & Oguri, K. (2008) · IEEE International Conference on Vehicular Electronics and Safety (ICVES) · doi:10.1109/icves.2008.4640853 · Cited by 38 · cited 1×
“Measuring Physiological Signals in the Car Literature on measuring physiological signals in the car can be divided into simulator studies [13,18,19,24,36] and field studies [14,15,18,28,30,33,40].”
BackgroundSingleton, W. T. (1989) · Medical Entomology and Zoology · Cited by 36
Szabó, A., Péronnet, F., Gauvin, L. I., & Furedy, J. J. (1994) · International Journal of Psychophysiology · doi:10.1016/0167-8760(94)90063-9 · Cited by 35 · cited 1×
“Cardiovascular measures have been reported to be sensitive to mental workload changes (e.g. [5,12,25,38,39]).”
BackgroundMedford, R. (2011) · PsycEXTRA Dataset · doi:10.1037/e733492011-001 · Cited by 14
Multirate processing of the ECG using filter banks
Afonso, V. X., Tompkins, W. J., Nguyen, T. Q., & Luo, S. (2002) · Computers in cardiology · doi:10.1109/cic.1996.542519 · Cited by 10 · cited 1×
“We employed a QRS detection algorithm [2,7] to identify heart beats in the signal.”
MethodFilter bank interpolation and reconstruction from generalized and recurrent nonuniform samples
Eldar, Y. C., & Oppenheim, A. V. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2000.861961 · Cited by 2 · cited 1×
“We employed a QRS detection algorithm [2,7] to identify heart beats in the signal.”
Method
Research works printed but not linked: 2
- National Highway Traffic Safety Admin. Traffic Safety Facts: Distracted Driving . Washington, DC , ( 2011 ). National Highway Traffic Safety Admin. Traffic Safety Facts: Distracted Driving. Washington, DC, (2011).
- Östlund , J. , Adaptive Integrated Driver-Vehicle Interface (AIDE): Driving performance assessment methods and metrics . Gothenburg , Sweden , ( 2005 ). Östlund, J., et al. Adaptive Integrated Driver-Vehicle Interface (AIDE): Driving performance assessment methods and metrics. Gothenburg, Sweden, (2005).
Cited by
A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving
Lohani, M., Payne, B. R., & Strayer, D. L. (2019) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2019.00057 · Cited by 348
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304
“…Cardiovascular Heart Rate [4, 10, 25, 27, 40, 41, 49, 50, 76, 78, 82, 85, 87, 99, 106, 146, 152, 160, 169, 184, 185, 194, 201, 202, 214, 216, 217, 224, 225, 227, 240, 244, 264, 264, 266, 268] Respiration [87, 99, 106, 184, 185, 194] Temperature [1, 2, 8, 87, 184, 185, 194, 216, 217, 268] Dermal…”
HeartPy: A novel heart rate algorithm for the analysis of noisy signals
van Gent, P., Farah, H., van Nes, N., & van Arem, B. (2019) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2019.09.015 · Cited by 234
Faure, V., Lobjois, R., & Benguigui, N. (2016) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2016.04.007 · Cited by 187
RelaWorld: Neuroadaptive and Immersive Virtual Reality Meditation System
Kosunen, I., Salminen, M., Järvelä, S., Ruonala, A., Ravaja, N., & Jacucci, G. (2016) · Proceedings of the 21st International Conference on Intelligent User Interfaces · doi:10.1145/2856767.2856796 · Cited by 157
“Recent work, for example, includes identifying workload for safety in supervisory tasks or driving [5] [43], detecting relevance for implicit feedback in information retrieval[11] [4] [39], research and adaptation in computer games [28] [45] [46], interactive storytelling [16], training cognitive performance [32], and…”
Background
Metadata
- Authors
- Solovey, E. T., Zec, M., Garcia Perez, E. A., Reimer, B., & Mehler, B.
- Affiliations
- Drexel University (US)
- Technical University of Munich (DE)
- Massachusetts Institute of Technology, Cambridge, MA, USA
- Massachusetts Institute of Technology (US)
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2014-04-26
- Pages
- 4057-4066
- OpenAlex
- W1968299435
- Cited by
- 182
- References
- 48
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 16 · 2023: 12 · 2022: 20 · 2021: 19 · 2020: 18 · 2019: 23 · 2018: 16 · 2017: 14 · 2016: 11 · 2015: 12 · 2014: 5
- FWCI
- 27.48
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- National Science Foundation
- Computing Research Association
- Consejo Nacional de Ciencia y Tecnología
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and Work-Related Fatigue
- Keywords
- cognitive load detection, physiological signals, driving performance, in-vehicle user interfaces, user interface evaluation, machine learning classification, highway driving
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.