Cognitive Workload Recognition Using EEG Signals and Machine Learning: A Review
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Machine learning and its subfield deep learning techniques provide opportunities for the development of operator mental state monitoring, especially for cognitive workload recognition using electroencephalogram (EEG) signals. Although a variety of machine learning methods have been proposed for recognizing cognitive workload via EEG recently, there does not yet exist a review that covers in-depth the application of machine learning methods. To alleviate this gap, in this article, we survey cognitive workload and machine learning literature to identify the approaches and highlight the primary advances. To be specific, we first introduce the concepts of cognitive workload and machine learning. Then, we discuss the steps of classical machine learning for cognitive workload recognition from the following aspects, i.e., EEG data preprocessing, feature extraction and selection, classification method, and evaluation methods. Further, we review the commonly used deep learning methods for this domain. Finally, we expound on the open problem and future outlooks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · cited 4×
“It addresses the question of how to build computer systems that learn and improve automatically through experience and data [39].”
BackgroundInfluentialDEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093
Multiple resources and performance prediction
Wickens, C. D. (2002) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220210123806 · Cited by 2,621 · cited 3×
“C URRENTLY, with the advanced development of detection devices and machine learning techniques, human mental states monitoring [1], [2], such as fatigue [3], emotion [4], and cognitive workload [5], has been widely used in the fields of human–robot interaction and passive brain– computer interface.”
BackgroundMethodOn the utility of P3 amplitude as a measure of processing capacity
Kok, A. (2001) · Psychophysiology · doi:10.1017/s0048577201990559 · Cited by 1,766
Deep learning for electroencephalogram (EEG) classification tasks: a review
Craik, A., He, Y., & Contreras-Vidal, J. L. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab0ab5 · Cited by 1,758 · cited 3×
“the input of CNN, then spectrograms or topographic maps may be the most prevalent choice [44], [123].”
BackgroundInfluentialDeep learning-based electroencephalography analysis: a systematic review
Roy, Y., Banville, H., Albuquerque, I., Gramfort, A., Falk, T. H., & Faubert, J. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab260c · Cited by 1,532 · cited 1×
“Also, for code reproducibility, Roy et al. [140] recommended that the published studies should clearly describe the architecture of models, the data used, or existing data sets used, whenever possible.”
MethodLearning Representations from EEG with Deep Recurrent-Convolutional Neural Networks
Bashivan, P., Rish, I., Yeasin, M., & Codella, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06448 · Cited by 836
Emotional state classification from EEG data using machine learning approach
Wang, X.-W., Nie, D., & Lu, B.-L. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.06.046 · Cited by 755 · cited 1×
“CURRENTLY, with the advanced development of detection devices and machine learning techniques, human mental states monitoring [1], [2], such as fatigue [3], emotion [4], and cognitive workload [5], has been widely used in the fields of human–robot interaction and passive brain– computer interface.”
BackgroundReview of advanced techniques for the estimation of brain connectivity measured with EEG/MEG
Sakkalis, V. (2011) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2011.06.020 · Cited by 637 · cited 1×
“EEG-related functional connectivity networks can be constructed by some methods [86], such as Pearson correlation coefficient [16], phase-locking value [85] [56], phase-locking index [87], and partial directed coherence.”
MethodTransfer Learning for Brain–Computer Interfaces: A Euclidean Space Data Alignment Approach
He, H., & Wu, D. (2020) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2019.2913914 · Cited by 569 · cited 1×
“…we can also learn lessons from other EEG-based classification tasks, such as motor imagery that adopting the data alignment method as a procedure [150], which aligns EEG trials from different subjects in the Euclidean space to make them more similar; and emotion recognition that using transfer…”
MethodA Review of Feature Reduction Techniques in Neuroimaging
Mwangi, B., Tian, T. S., & Soares, J. C. (2014) · Neuroinformatics · doi:10.1007/s12021-013-9204-3 · Cited by 559
Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods
Gevins, A., Smith, M. E., Leong, H., McEvoy, L., Whitfield, S., Du, R., & Rush, G. (1998) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872098779480578 · Cited by 535
Estimating workload using EEG spectral power and ERPs in the n-back task
Brouwer, A.-M., Hogervorst, M. A., van Erp, J. B. F., Heffelaar, T., Zimmerman, P. H., & Oostenveld, R. (2012) · Journal of Neural Engineering · doi:10.1088/1741-2560/9/4/045008 · Cited by 407
EEG-based emotion classification using deep belief networks
Zheng, W.-L., Zhu, J.-Y., Peng, Y., & Lu, B.-L. (2014) · 2014 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2014.6890166 · Cited by 367 · cited 13×
“(c) is the deep learning method that can learn features automatically, e.g., convolution neural network (CNN), recurrent neural network (RNN), and deep belief network (DBN).”
MethodInfluentialAn electroencephalographic signature predicts antidepressant response in major depression
Wu, W., Zhang, Y., Jiang, J., Lucas, M. V., Fonzo, G. A., Rolle, C. E., Cooper, C., Chin-Fatt, C., Krepel, N., Cornelssen, C. A., et al. (2020) · Nature Biotechnology · doi:10.1038/s41587-019-0397-3 · Cited by 302
Cross-session classification of mental workload levels using EEG and an adaptive deep learning model
Yin, Z., & Zhang, J. (2017) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2016.11.013 · Cited by 163
Fournier, L. R., Wilson, G. F., & Swain, C. R. (1999) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(98)00049-x · Cited by 158 · cited 2×
“The MATB task is a platform designed to evaluate human operator performance and workload in multitask conditions [36].”
BackgroundMethodAricò, P., Borghini, G., Di Flumeri, G., Colosimo, A., Pozzi, S., & Babiloni, F. (2016) · Progress in Brain Research · doi:10.1016/bs.pbr.2016.04.021 · Cited by 145 · cited 4×
“extracting PSD features, stepwise LDA [22], and automatic stop stepwise LDA [113] are adopted as classifiers since these methods have the advantage of automatic feature selection.”
BackgroundMethodInfluential
Cited by
Neural Decoding of EEG Signals with Machine Learning: A Systematic Review
Saeidi, M., Karwowski, W., Farahani, F. V., Fiok, K., Taiar, R., Hancock, P. A., & Al-Juaid, A. (2021) · Brain Sciences · doi:10.3390/brainsci11111525 · Cited by 258
Metadata
- Authors
- Zhou, Y., Huang, S., Xu, Z., Wang, P., Wu, X., & Zhang, D.
- Venue
- IEEE Transactions on Cognitive and Developmental Systems
- Published
- 2022-09
- Volume
- 14
- Issue
- 3
- Pages
- 799-818
- ISSN
- 2379-8920 (ISSN-L)
- 2379-8939
- OpenAlex
- W3175508462
- Cited by
- 221
- References
- 21
- FWCI
- 13.18
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.