The DOI matches its Crossref record; 76 of the 77 references it cites were found in a registry or an index.
A multimodal approach to estimating vigilance using EEG and forehead EOG
Open accessAbstract
Objective . Covert aspects of ongoing user mental states provide key context information for user-aware human computer interactions. In this paper, we focus on the problem of estimating the vigilance of users using EEG and EOG signals. Approach . The PERCLOS index as vigilance annotation is obtained from eye tracking glasses. To improve the feasibility and wearability of vigilance estimation devices for real-world applications, we adopt a novel electrode placement for forehead EOG and extract various eye movement features, which contain the principal information of traditional EOG. We explore the effects of EEG from different brain areas and combine EEG and forehead EOG to leverage their complementary characteristics for vigilance estimation. Considering that the vigilance of users is a dynamic changing process because the intrinsic mental states of users involve temporal evolution, we introduce continuous conditional neural field and continuous conditional random field models to capture dynamic temporal dependency. Main results . We propose a multimodal approach to estimating vigilance by combining EEG and forehead EOG and incorporating the temporal dependency of vigilance into model training. The experimental results demonstrate that modality fusion can improve the performance compared with a single modality, EOG and EEG contain complementary information for vigilance estimation, and the temporal dependency-based models can enhance the performance of vigilance estimation. From the experimental results, we observe that theta and alpha frequency activities are increased, while gamma frequency activities are decreased in drowsy states in contrast to awake states. Significance . The forehead setup allows for the simultaneous collection of EEG and EOG and achieves comparative performance using only four shared electrodes in comparison with the temporal and posterior sites.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The claim comes from somewhere else, not from this source.
In the citing paper
“Zheng and his colleges presented an emotion recognition method combining EEG signals and pupillary response collected from eye tracker and achieved average accuracies of 73.59% and 72.98% for three emotion states using feature level fusion strategy and decision-level fusion strategy, respectively.”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [32]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Most commercial biometric systems do not provide access to the feature sets nor the raw data which they use in their products [22].”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [22]
Experimental — accuracy not yet measuredIn the citing paper
“Superior performance was obtained compared to the results in the state-of-the-art results [3, 11, 32].”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371 · cited 3×
“ICA is a blind source separation method proposed to decompose a multivariate signal into independent non-Gaussian signals (Delorme & Makeig 2004).”
MethodPan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
(1992) · Applied Ergonomics · doi:10.1016/0003-6870(92)90318-p · Cited by 5,774
Removing electroencephalographic artifacts by blind source separation
Jung, T., Makeig, S., Humphries, C., Lee, T.-W., McKeown, M. J., Iragui, V. J., & B, T. J. S. A. (2000) · Psychophysiology · doi:10.1111/1469-8986.3720163 · Cited by 3,342
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,701
Zheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,477 · cited 6×
“In this study, we adopt an automatic continuous vigilance annotation method using eye tracking glasses, which was proposed in our previous work (Gao, Zhang, Zheng & Lu 2015).”
BackgroundMethodInfluentialSelf-citationAffect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893
Decoding mental states from brain activity in humans
Haynes, J., & Rees, G. E. (2006) · Nature reviews. Neuroscience · doi:10.1038/nrn1931 · Cited by 1,868
Differential entropy feature for EEG-based emotion classification
Duan, R.-N., Zhu, J., & Lu, B. (2013) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2013.6695876 · Cited by 1,225
Self-citation
Research works printed but not linked: 1
- Dinges, 1998, PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance, US Department of Transportation, Federal Highway Administration, Publication Number FHWA-MCRT-98-006
Cited by
Sorted by citations
A review of emotion recognition using physiological signals
Shu, L., Xie, J., Yang, M., Li, Z., Li, Z., Liao, D., Xu, X., & Yang, X. (2018) · Sensors · doi:10.3390/s18072074 · Cited by 923
“The SEED database [135] contains EEG and eye movement of three different emotions (positive, neutral and negative).”
BackgroundEEG-Based Spatio–Temporal Convolutional Neural Network for Driver Fatigue Evaluation
Gao, Z., Wang, X., Yang, Y., Mu, C., Cai, Q., Dang, W., & Zuo, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2886414 · Cited by 486
“In [17], a multimodal approach was proposed to estimate driver fatigue by combin-”
Transfer Learning for EEG-Based Brain–Computer Interfaces: A Review of Progress Made Since 2016
Wu, D., Xu, Y., & Lu, B.-L. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3007453 · Cited by 434
“There are many important BCI regression problems, e.g., driver drowsiness estimation [38]–[40], vigilance estimation [11], [12], [113], and user reaction time estimation [41], which were not adequately addressed in previous reviews.”
Self-citationTian, L., Zimmerman, B., Akhtar, A., Yu, K. J., Moore, M., Wu, J. C., Larsen, R. J., Lee, J. W., Li, J., Liu, Y., et al. (2019) · Nature Biomedical Engineering · doi:10.1038/s41551-019-0347-x · Cited by 411
EEG-Based Emotion Recognition: A State-of-the-Art Review of Current Trends and Opportunities
Suhaimi, N. S., Mountstephens, J., & Teo, J. (2020) · Computational Intelligence and Neuroscience · doi:10.1155/2020/8875426 · Cited by 395
“As for video-related contents, SEED [124–126], DREAMER [127], and ASCERTAIN [107] do provide their video database either openly or upon request.”
BackgroundSelf-Supervised Learning for Electroencephalography
Rafiei, M. H., Gauthier, L. V., Adeli, H., & Takabi, D. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3190448 · Cited by 333
“SEED is divided into three subdatasets: 1) SEED I [138]; 2) SEED-IV [140]; and 3) SEED-VIG [141].”
MethodCross-Subject Emotion Recognition Using Flexible Analytic Wavelet Transform From EEG Signals
Gupta, V., Chopda, M. D., & Pachori, R. B. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2883497 · Cited by 295
“of operator [5], [6].”
Brain-Computer Interface: Advancement and Challenges
Mridha, M. F., Das, S. C., Kabir, M. M., Lima, A. A., Islam, M. R., & Watanobe, Y. (2021) · Sensors · doi:10.3390/s21175746 · Cited by 288
A Review of Recent Developments in Driver Drowsiness Detection Systems
Albadawi, Y., Takruri, M., & Awad, M. (2022) · Sensors · doi:10.3390/s22052069 · Cited by 259
“02% Dataset1: Fpz-Cz channel dataset [89,90] Dataset2:SVDD dataset [91]”
MethodRecognition of human emotions using EEG signals: A review
Rahman, M. M., Sarkar, A. K., Hossain, M. A., Hossain, M. S., Islam, M. R., Hossain, M. B., Quinn, J. M. W., & Moni, M. A. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104696 · Cited by 232
EEG-based emotion recognition: Review of commercial EEG devices and machine learning techniques
Dadebayev, D., Goh, W. W., & Tan, E. X. (2021) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.03.009 · Cited by 226
Learning CNN features from DE features for EEG-based emotion recognition
Hwang, S., Hong, K., Son, G., & Byun, H. (2019) · Pattern Analysis and Applications · doi:10.1007/s10044-019-00860-w · Cited by 167
“Table 11 Vigilance estimation results on VIG dataset [39]”
BackgroundMethodInfluentialDriver sleepiness detection from EEG and EOG signals using GAN and LSTM networks
Jiao, Y., Deng, Y., Luo, Y., & Lu, B. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.05.108 · Cited by 168
“Zheng and Lu combined EEG and forehead EOG to leverage their complementary characteristics for vigilance estimation [37] .”
BackgroundSelf-citationReview on Emotion Recognition Based on Electroencephalography
Liu, H., Zhang, Y., Li, Y., & Kong, X. (2021) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2021.758212 · Cited by 178
Understanding mental fatigue and its detection: a comparative analysis of assessments and tools
Kunasegaran, K., Ismail, A. M. H., Ramasamy, S., Gnanou, J. V., Caszo, B. A., & Chen, P. L. (2023) · PeerJ · doi:10.7717/peerj.15744 · Cited by 188
Machine learning and deep learning techniques for driver fatigue and drowsiness detection: a review
El‐Nabi, S. A., El‐Shafai, W., El‐Rabaie, E. M., Ramadan, K. F., El‐Samie, F. E. A., & Mohsen, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15054-0 · Cited by 150
“Additionally, the simulated virtual driving driver (SVDD) dataset from [149] was used.”
A Review on EEG-Based Automatic Sleepiness Detection Systems for Driver
Balandong, R. P., Ahmad, R. F., Saad, M. N. M., & Malik, A. S. (2018) · IEEE Access · doi:10.1109/access.2018.2811723 · Cited by 138
“Numerous physiological indicators are used to bridge the physiological-sleepiness relationship such as electrocardiogram (ECG) [72]–[75], electrooculography (EOG) [76]–[78], functional Near Infrared Spectroscopy (fNIR) [79] and electroencephalography (EEG) [7], [70], [73], [79]–[86].”
MethodInfluentialAffective Brain–Computer Interfaces (aBCIs): A Tutorial
Wu, D., Lu, B., Hu, B., & Zeng, Z. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3277471 · Cited by 135
“…applications in neural rehabilitation [12], text input [13], gaming [14], emotion recognition [15], mental fatigue evaluation [16], vigilance estimation [17], speech synthesis [18], sentence decoding [19], movement and touch functionality recovery [20], [21], robot control [22], [23], and so on.”
BackgroundSelf-citationCai, Y., Li, X., & Li, J. (2023) · Sensors · doi:10.3390/s23052455 · Cited by 149
“SEED * [210] Induced 15 participants; average age is 23.”
BackgroundCNN and LSTM-Based Emotion Charting Using Physiological Signals
Dar, M. N., Akram, M. U., Khawaja, S. G., & Pujari, A. N. (2020) · Sensors · doi:10.3390/s20164551 · Cited by 137
“They incorporated spectral features from EEG with the GELM classifier for SEED [26] and DEAP datasets, respectively.”
MethodA new fractal pattern feature generation function based emotion recognition method using EEG
Tuncer, T., Doğan, Ş., & Subaşı, A. (2021) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2021.110671 · Cited by 126
Accurate Markov Boundary Discovery for Causal Feature Selection
Wu, X., Jiang, B., Yu, K., Miao, C., & Chen, H. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2940509 · Cited by 120
“formed on a newly developed electroencephalography (EEG) dataset, SEED [32], [33], to compare our algorithms with other state-of-the-art methods in Section VI-C.”
MethodFusion of Facial Expressions and EEG for Multimodal Emotion Recognition
Huang, Y., Yang, J., Liao, P., & Pan, J. (2017) · Computational Intelligence and Neuroscience · doi:10.1155/2017/2107451 · Cited by 115
“Superior performance was obtained compared to the results in the state-of-the-art results [3, 11, 32].”
ResultVigilance Estimation Using a Wearable EOG Device in Real Driving Environment
Zheng, W., Gao, K., Li, G., Liu, W., Chao, L., Liu, J., Wang, G., & Lu, B. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2889962 · Cited by 118
“In this paper, we applied two temporal dependency models, namely, continuous conditional neural field (CCNF) and continuous conditional random field (CCRF), to capture the vigilance dynamics [31] and compared their performance with those of the conventional SVR model and the dynamic model using the Linear Dynamic Syst…”
BackgroundMethodResultInfluentialSelf-citationMusic mood and human emotion recognition based on physiological signals: a systematic review
Chaturvedi, V., Kaur, A. B., Varshney, V., Garg, A., Chhabra, G. S., & Kumar, M. (2021) · Multimedia Systems · doi:10.1007/s00530-021-00786-6 · Cited by 119
Multi-Scale Neural Network for EEG Representation Learning in BCI
Ko, W., Jeon, E., Jeong, S., & Suk, H. (2021) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2021.3061875 · Cited by 110
“nner for extracting spatio-spectral features and classified the learned features using an SVM with a radial basis function (RBF) kernel ( = 1=d input where d input denotes the input feature dimension) [10]. 4) Linear Models - Seizure: In addition, we also reimplemented Shoeb and Guttag [14]’s method for the seizure ons…”
MethodInfluentialBalam, V. P., & Chinara, S. (2020) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2020.108927 · Cited by 115
Reducing the Subject Variability of EEG Signals with Adversarial Domain Generalization
Ma, B.-Q., Li, H., Zheng, W., & Lu, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-36708-4_3 · Cited by 104
Self-citationUsing EEG for Mental Fatigue Assessment: A Comprehensive Look Into the Current State of the Art
Monteiro, T. G., Skourup, C., & Zhang, H. (2019) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2019.2938156 · Cited by 103
“We can identify the studies using three [59], [76], [89] or four [67], [77], [91] different MF states.”
BackgroundMethodDriving Fatigue Classification Based on Fusion Entropy Analysis Combining EOG and EEG
Wang, H., Wu, C., Li, T., He, Y., Chen, P., & Bezerianos, A. (2019) · IEEE Access · doi:10.1109/access.2019.2915533 · Cited by 103
“, Zheng and Lu proposed different electrode placements on the forehead and extracted various eye movement features (EOG) for fatigue detection [38], [39].”
BackgroundConvolutional Neural Network-Based EEG Signal Analysis: A Systematic Review
Rajwal, S., & Aggarwal, S. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09920-1 · Cited by 97
“There is one openly available dataset called SEED-VIG [97] which consists of EEG signals collected for a simulated driving experiment.”
MethodA systematic review of physiological signals based driver drowsiness detection systems
Saleem, A. A., Siddiqui, H. U. R., Raza, M. A., Rustam, F., Dudley, S., & Ashraf, I. (2022) · Cognitive Neurodynamics · doi:10.1007/s11571-022-09898-9 · Cited by 102
Multi-method Fusion of Cross-Subject Emotion Recognition Based on High-Dimensional EEG Features
Yang, F. S., Zhao, X., Jiang, W., Gao, P., & Liu, G. (2019) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2019.00053 · Cited by 94
“Emotional research has a preliminary development trend in the field of computer science, such as task workload assessments and operator vigilance (Shi and Lu, 2013; Zheng and Lu, 2017b).”
BackgroundDriving Fatigue Detection from EEG Using a Modified PCANet Method
Ma, Y., Chen, B., Li, R., et al. (2019) · Computational Intelligence and Neuroscience · doi:10.1155/2019/4721863 · Cited by 93
Abbas, Q., & Alsheddy, A. (2020) · Sensors · doi:10.3390/s21010056 · Cited by 92
“Afterward, the authors performed machine learning classification algorithms, such as multilayer deep learning (DL), PSO, SVM, ANN, and LDA [55–73].”
BackgroundMethodZhang, G., & Etemad, A. (2021) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2021.3089594 · Cited by 102
“Blink and saccade movements can be detected from EOG recordings by applying peak detection on wavelet coefficients calculated with continuous wavelet transform [1].”
BackgroundMethodInfluentialKim, N., Kim, J., & Ahn, C. R. (2021) · Advanced Engineering Informatics · doi:10.1016/j.aei.2021.101359 · Cited by 90
Peng, Y. K., Qin, F., Kong, W., Ge, Y., Nie, F., & Cichocki, A. S. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3082803 · Cited by 87
Lazarou, E., & Exarchos, T. P. (2024) · AIMS neuroscience · doi:10.3934/neuroscience.2024006 · Cited by 107
Zhang, X., Pan, J., Shen, J. G., Din, Z. U., Li, J., Lu, D., Wu, M., & Hu, B. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981440 · Cited by 84
TFormer: A time–frequency Transformer with batch normalization for driver fatigue recognition
Li, R., Hu, M., Gao, R., Wang, L., Suganthan, P. N., & Sourina, O. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102575 · Cited by 87
Xu, Y., Yang, W., Yu, X., Li, H., Cheng, T., Lu, X., & Wang, Z. L. (2021) · ACS Nano · doi:10.1021/acsnano.1c00536 · Cited by 81
A Systematic Survey of Driving Fatigue Monitoring
Zhang, Z., Ning, H., & Zhou, F. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3189346 · Cited by 82
“Public datasets such as NTHU-DDD [25], YawDD [26], seed-vig [27], and E-pilots [28] are used to research driver mental states.”
MethodCorrelation between Situational Awareness and EEG signals
Kästle, J. L., Anvari, B., Król, J., & Wurdemann, H. A. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.12.026 · Cited by 84
A recurrence network-based convolutional neural network for fatigue driving detection from EEG
Gao, Z., Li, Y., Yang, Y., & Ma, C. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120538 · Cited by 81
Shahbakhti, M., Beiramvand, M., Rejer, I., et al. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3096984 · Cited by 78
Driving Fatigue Detection Based on Hybrid Electroencephalography and Eye Tracking
Lian, Z., Xu, T., Yuan, Z., et al. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3446952 · Cited by 78
“Zheng et al. proposed a hybrid approach for estimating vigilance by combining EEG and forehead EOG data, with their data labeled using the percentage of eye closure (PERCLOS) index calculated from eye tracking data [24].”
MethodInfluentialEfficacy of transformer networks for classification of EEG data
Siddhad, G., Gupta, A., Dogra, D. P., & Roy, P. P. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105488 · Cited by 91
“EEG has long been used for di ff erent purposes like disease diagnosis [8], BCI [9], neuro-feedback [10], workload estimation [11], motor imagery classification [12], and even brain-to-braincommunication [13].”
MethodChen, L., Zhang, A., & Xiao-guang, L. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.02.005 · Cited by 78
Eye-Tracking Feature Extraction for Biometric Machine Learning
Lim, J. Z., Mountstephens, J., & Teo, J. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2021.796895 · Cited by 74
“The level of driver alertness SEED-VIG (Zheng and Lu, 2017) is used as the dataset for the simulation experiment.”
MethodDynamic entropy-based pattern learning to identify emotions from EEG signals across individuals
Lu, Y., Wang, M., Wu, W., Han, Y., Zhang, Q., & Chen, S. (2020) · Measurement · doi:10.1016/j.measurement.2019.107003 · Cited by 73
Koay, H. V., Chuah, J. H., Chow, C., & Chang, Y.-L. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105309 · Cited by 72
Multimodal System to Detect Driver Fatigue Using EEG, Gyroscope, and Image Processing
Karuppusamy, N. S., & Kang, B. (2020) · IEEE Access · doi:10.1109/access.2020.3009226 · Cited by 69
“In another method forehead EOG was used to extract various information on eye movement features and combined with EEG as EOG contains complementary characteristics for vigilance estimation [34].”
MethodYaacob, H. S., Hossain, F., Shari, S., Khare, S. K., Ooi, C. P., & Acharya, U. R. (2023) · IEEE Access · doi:10.1109/access.2023.3296382 · Cited by 70
A Regression Method With Subnetwork Neurons for Vigilance Estimation Using EOG and EEG
Wu, W., Wu, Q. M. J., Sun, W. J., Yang, Y., Yuan, X., Zheng, W., & Lu, B. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2889223 · Cited by 68
“Huo et al. [21] and Zheng and Lu [22] found the complementarity of forehead-EOG and EEG signals, and using the feature fusion for vigilance estimation, improved the accuracy rate.”
MethodResultInfluentialSelf-citationGao, D., Wang, K., Wang, M., Zhou, J., & Zhang, Y. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3285268 · Cited by 66
“Research shows combining five frequency bands is better than using a single frequency band [18].”
BackgroundChen, J., Cui, Y., Wei, C., et al. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.114097 · Cited by 65
Kaklauskas, A., Abraham, A., Ubartė, I., Klıukas, R., Luksaite, V., Binkytė-Vėlienė, A., Vetloviene, I., & Kaklauskiene, L. (2022) · Sensors · doi:10.3390/s22207824 · Cited by 77
EEG-based neural networks approaches for fatigue and drowsiness detection: A survey
Othmani, A., Sabri, A. Q. M., Aslan, S., Chaieb, F., Rameh, H., Alfred, R., & Cohen, D. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126709 · Cited by 70
VIGNet: A Deep Convolutional Neural Network for EEG-based Driver Vigilance Estimation
Ko, W., Oh, K., Jeon, E., & Suk, H. (2020) · International Winter Conference on Brain-Computer Interface (BCI) · doi:10.1109/bci48061.2020.9061668 · Cited by 62
“, a DE calculation as like as Zheng and Lu [2], a CNN-based latent feature learning, and a decision making, i.”
BackgroundMethodInfluentialPeng, Y. K., Jin, F., Kong, W., Nie, F., Lu, B., & Cichocki, A. S. (2022) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2022.3175464 · Cited by 61
Self-citationEOG-Based Human–Computer Interface: 2000–2020 Review
Belkhiria, C., Boudir, A., Hurter, C., & Peysakhovich, V. (2022) · Sensors · doi:10.3390/s22134914 · Cited by 61
“Kumar [84] 2016 • • • • Dhuliawala [85] 2016 • • • • Shimizu [86] 2016 • • • • • Shimizu bis [87] 2016 • • • • • Bissoli [88] 2016 • • • • Wilaiprasitporn [89] 2016 • • • • Fang [90] 2016 • • • • • Tamura [91] 2016 • • • • • • • Barbara [92] 2016 • • • • • Atique [93] 2016 • • • • • Naijian [94] 2016 • • • Ogai [95] 2…”
EEG Driving Fatigue Detection With PDC-Based Brain Functional Network
Wang, F., Wu, S., Ping, J., et al. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3058658 · Cited by 59
“These results are consistent with previous findings in the literature [31], [51] and support the previous evidence that the increasing trend for the ratio of slow and fast waves of EEG activities reflects decreasing attentional demands [36].”
ResultJia, H., Xiao, Z., & Ji, P. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106431 · Cited by 59
Electroencephalogram-Based Approaches for Driver Drowsiness Detection and Management: A Review
Li, G., & Chung, W. (2022) · Sensors · doi:10.3390/s22031100 · Cited by 78
“Li et [86,87].”
Ahmed, M. Z. I., Sinha, N., Ghaderpour, E., Phadikar, S., & Ghosh, R. (2023) · Bioengineering · doi:10.3390/bioengineering10010054 · Cited by 58
“The two major techniques used for emotion elicitation are using external stimuli, such as audio–visual [8,12,13] or memory recall [14].”
MethodA Comprehensive Survey on Tiny Machine Learning for Human Behavior Analysis
Lamaakal, I., Essahraui, S., Maleh, Y., Makkaoui, K. E., Ouahbi, I., Bouami, M. F., El‐Latif, A. A. A., Almousa, M., Peng, J., & Niyato, D. T. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3565688 · Cited by 67
“The SEED-VIG dataset [225] includes EEG recordings for vigilance studies.”
Fatigue driving detection method based on Time-Space-Frequency features of multimodal signals
Shi, J., & Wang, K. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.104744 · Cited by 55
A Hybrid Deep Neural Network Approach to Recognize Driving Fatigue Based on EEG Signals
Alghanim, M., Attar, H. H., Rezaee, K., et al. (2024) · International Journal of Intelligent Systems · doi:10.1155/2024/9898333 · Cited by 54
Torkamani‐Azar, M., Kanık, S. D., Aydın, S., & Çetin, M. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2980056 · Cited by 53
“In addition, in a 2-hour simulated experiment, Zheng and Lu [18] observed that, in transition to the drowsy states, alpha and theta significantly increased in the temporal and parietal cortex.”
BackgroundVideo‐triggered EEG‐emotion public databases and current methods: A survey
Hu, W., Huang, G., Li, L., Zhang, L., Zhang, Z., & Liang, Z. (2020) · Brain Science Advances · doi:10.26599/bsa.2020.9050026 · Cited by 54
“(3) SEED‐VIG database: This experiment recruited 23 participants and asked them to monotonously repeat straight road driving tasks in a simulated driving system, which could easily induct fatigue and change their vigilance [84].”
BackgroundAl-Shargie, F., Hassanin, O., Tariq, U., & Al-Nashash, H. A. (2020) · IEEE Access · doi:10.1109/access.2020.3004504 · Cited by 51
Transformers in biosignal analysis: A review
Anwar, A., Khalifa, Y., Coyle, J. L., & Sejdić, E. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102697 · Cited by 60
Detecting driving fatigue with multimodal deep learning
Du, L., Liu, W., Zheng, W., et al. (2017) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2017.8008295 · Cited by 49
“To incorporate the temporal dependency into driving fatigue detection, continuous conditional random field (CCNF) has been applied to detect driving fatigue [10].”
BackgroundMethodSelf-citationLee, C., Chen, C., Lin, C., et al. (2019) · Applied Sciences · doi:10.3390/app9235004 · Cited by 48
“[31] proposed a personalized product configuration process to determine design attributes in a cloud-based adaptable product configuration.”
BackgroundThe Auto-Correlation Function Aided Sparse Support Matrix Machine for EEG-Based Fatigue Detection
Li, Y., Wang, D., & Liu, F. (2022) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2022.3211931 · Cited by 47
“The dataset used in this experiment is the SEED-VIG fatigue dataset [27], describing as follows.”
MethodPhyDAA: Physiological Dataset Assessing Attention
Delvigne, V., Wannous, H., Dutoit, T., Ris, L., & Vandeborre, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3061719 · Cited by 47
“In another domain, Zheng and Lu [53] and Cao et al. [11] both propose a dataset composed of EEG records assessing attention during a driving task.”
BackgroundGalea: A physiological sensing system for behavioral research in Virtual Environments
Bernal, G., Hidalgo, N., Russomanno, C., & Maes, P. (2022) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr51125.2022.00024 · Cited by 46
“While understanding a single physiological signal can be insightful, it is even more useful to be able to examine multiple modalities so as to understand human behavior and the individual’s responses to their environment and gain insight into their cognitive processes [41,74].”
BackgroundMultiple nonlinear features fusion based driving fatigue detection
Wang, F., Wu, S., Zhang, W., et al. (2020) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2020.102075 · Cited by 45
Multimodal Vigilance Estimation Using Deep Learning
Wu, W., Sun, W. J., Wu, Q. M. J., Yang, Y., Zhang, H., Zheng, W., & Lu, B. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3022647 · Cited by 45
“Unlike the traditional EOG (EOGt)-based method, a simple but more robust method proposed by Zheng and Lu [21] utilizes the placement of wearable electrodes on the forehead area.”
MethodResultSelf-citationSiddiqui, M. F. H., Dhakal, P., Yang, X., & Javaid, A. Y. (2022) · Multimodal Technologies and Interaction · doi:10.3390/mti6060047 · Cited by 46
Review and Challenges of Technologies for Real-Time Human Behavior Monitoring
Dávila-Montero, S., Le, J. A., Bente, G., Hall, A. T., & Mason, A. J. (2021) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2021.3060617 · Cited by 48
“On the other hand, Zheng and Lu [170] employed an SVR with radial basis function to estimate the level of vigilance based on data from EEG and EOG.”
MethodGao, D., Tang, X., Wan, M., Guo, H., & Zhang, Y. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1136609 · Cited by 41
Spatio-Temporal EEG Representation Learning on Riemannian Manifold and Euclidean Space
Zhang, G., & Etemad, A. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3332549 · Cited by 47
“EEG signals are split into EEG segments of T = 8 seconds with no overlap, as presented in Table I. 2) Seed-Vig: The SEED-VIG dataset has been collected by [6] to estimate driver vigilance.”
BackgroundMethodInfluentialFatigue State Detection for Tired Persons in Presence of Driving Periods
Alharbey, R., Dessouky, M. M., Sedik, A., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3185251 · Cited by 41
“Used Dataset DROZY EEG signals SEED-VIG [33] Self-collected EEG”
MethodFeature Selection Model based on EEG Signals for Assessing the Cognitive Workload in Drivers
Becerra-Sánchez, P., Reyes, A., & Ibáñez, J. A. G. (2020) · Sensors · doi:10.3390/s20205881 · Cited by 40
Liu, Z., Jiang, X., Luo, H., et al. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.11.019 · Cited by 38
Drowsiness detection using portable wireless EEG
Sagila Gangadharan, K., & Vinod, A. P. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106535 · Cited by 38
A Product Fuzzy Convolutional Network for Detecting Driving Fatigue
Du, G., Long, S., Li, C., et al. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3123842 · Cited by 38
“Zheng and Lu [7] proposed a multimodal method to evaluate vigilance by integrating EEG and EOG and introducing the temporal dependency-based vigilance into model training.”
MethodCheng, X., Wei, W., Du, C., Qiu, S., Tian, S., Ma, X., & He, H. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548367 · Cited by 37
“[37] extracted features from eye movements such as blinks and saccades based on the wavelet transform method from forehead EOG to realize vigilance detection.”
MethodInfluentialPeng, B., Zhang, Y., Wang, M., Chen, J., & Gao, D. (2023) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2023.107863 · Cited by 38
Drowsiness Detection in Drivers: A Systematic Review of Deep Learning-Based Models
Fonseca, T., & Ferreira, S. (2025) · Applied Sciences · doi:10.3390/app15169018 · Cited by 37
Balam, V. P., & Chinara, S. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3094619 · Cited by 40
Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System
Riquelme, F., Olivares, R., Muñoz, F., Molinero, X., & Serna, M. (2021) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.022553 · Cited by 37
Zheng, R., Wang, Z., He, Y., et al. (2021) · Cognitive Neurodynamics · doi:10.1007/s11571-021-09714-w · Cited by 36
Multimodal Vigilance Estimation with Adversarial Domain Adaptation Networks
Li, H., Zheng, W., & Lu, B. (2018) · doi:10.1109/ijcnn.2018.8489212 · Cited by 34
“In this paper, we integrate the multimodal approach [11] and adversarial domain adaptation”
MethodInfluentialSelf-citationDepersonalized Cross-Subject Vigilance Estimation with Adversarial Domain Generalization
Ma, B.-Q., Li, H., Luo, Y., & Lu, B. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852347 · Cited by 34
“further improve the performance of the vigilance estimation models [6] [7].”
BackgroundSelf-citationAn enhanced ensemble deep random vector functional link network for driver fatigue recognition
Li, R., Gao, R., Yuan, L., Suganthan, P. N., Wang, L., & Sourina, O. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106237 · Cited by 36
EEG-based emotion recognition for road accidents in a simulated driving environment
Chen, J., Lin, X., Ma, W., Wang, Y., & Tang, W. H. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105411 · Cited by 33
Decoding Neural Correlates of Cognitive States to Enhance Driving Experience
Chavarriaga, R., Ušćumlić, M., Zhang, H., Khaliliardali, Z., Aydarkhanov, R., Saeedi, S., Gheorghe, L., & Millán, J. D. R. (2018) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2018.2848289 · Cited by 33
Metadata
- Authors
- Zheng, W.-L., & Lu, B.-L.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Corresponding author
- Zheng, Wei-Long
- Lu, Bao-Liang
- Venue
- Journal of Neural Engineering
- Publisher
- IOP Publishing
- Published
- 2017-02-22
- Volume
- 14
- Issue
- 2
- Pages
- 026017
- ISSN
- 1741-2552 (ISSN-L)
- 1741-2560
- OpenAlex
- W2558193840
- Semantic Scholar
- babe42b42bffd9029d7fef3d6c97ec0985fa6528
- Cited by
- 424
- References
- 77
- Citations by year
- 2026: 60 · 2025: 66 · 2024: 72 · 2023: 55 · 2022: 46 · 2021: 50 · 2020: 37 · 2019: 21 · 2018: 14 · 2017: 3
- FWCI
- 11.78
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Journal of Neural Engineering (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Technology Research and Development Program of China Railway Corporation (2016Z003-B)
- National Basic Research Program of China (2013CB329401)
- National Natural Science Foundation of China (61272248)
- Major Basic Research Program of Shanghai Science and Technology Committee (15JC1400103)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and Work-Related Fatigue
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- EEG, multimodal fusion, PERCLOS, eye movement features, temporal dependencies, drowsiness detection, gamma oscillations, wearable EEG
- MeSH terms
- Adult, Algorithms, Arousal, Electroencephalography, Electrooculography, Female, Humans, Male, Pattern Recognition, Automated, Psychomotor Performance, Sensitivity and Specificity, Reproducibility of Results, Brain Waves
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
