A multimodal approach to estimating vigilance using EEG and forehead EOG
Open accessAbstract
Objective . Covert aspects of ongoing user mental states provide key context information for user-aware human computer interactions. In this paper, we focus on the problem of estimating the vigilance of users using EEG and EOG signals. Approach . The PERCLOS index as vigilance annotation is obtained from eye tracking glasses. To improve the feasibility and wearability of vigilance estimation devices for real-world applications, we adopt a novel electrode placement for forehead EOG and extract various eye movement features, which contain the principal information of traditional EOG. We explore the effects of EEG from different brain areas and combine EEG and forehead EOG to leverage their complementary characteristics for vigilance estimation. Considering that the vigilance of users is a dynamic changing process because the intrinsic mental states of users involve temporal evolution, we introduce continuous conditional neural field and continuous conditional random field models to capture dynamic temporal dependency. Main results . We propose a multimodal approach to estimating vigilance by combining EEG and forehead EOG and incorporating the temporal dependency of vigilance into model training. The experimental results demonstrate that modality fusion can improve the performance compared with a single modality, EOG and EEG contain complementary information for vigilance estimation, and the temporal dependency-based models can enhance the performance of vigilance estimation. From the experimental results, we observe that theta and alpha frequency activities are increased, while gamma frequency activities are decreased in drowsy states in contrast to awake states. Significance . The forehead setup allows for the simultaneous collection of EEG and EOG and achieves comparative performance using only four shared electrodes in comparison with the temporal and posterior sites.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most commercial biometric systems do not provide access to the feature sets nor the raw data which they use in their products.
“Most commercial biometric systems do not provide access to the feature sets nor the raw data which they use in their products [22].”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [22]
The Discussion mentions commercial EEG systems but says nothing about whether they provide access to raw data or feature sets. The Experiment Setup describes access to this study’s dataset, not the access policies of most commercial biometric systems. Searches for commercial-system data and feature access found no passage addressing the claim; page numbers were not available in our copy. (LLM, conf=0.74)
Experimental — accuracy not yet measuredZheng and his colleagues presented an emotion recognition method combining EEG signals and pupillary response collected from eye tracker and achieved average accuracies of 73.59% and 72.98% for three emotion states using feature level fusion strategy and decision-level fusion strategy, respectively.
“Zheng and his colleges presented an emotion recognition method combining EEG signals and pupillary response collected from eye tracker and achieved average accuracies of 73.59% and 72.98% for three emotion states using feature level fusion strategy and decision-level fusion strategy, respectively.”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371 · cited 3× · see citing works
“ICA is a blind source separation method proposed to decompose a multivariate signal into independent non-Gaussian signals (Delorme & Makeig 2004).”
MethodPan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · see citing works
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,953 · see citing works
(1992) · Applied Ergonomics · doi:10.1016/0003-6870(92)90318-p · Cited by 5,774 · see citing works
Removing electroencephalographic artifacts by blind source separation
Jung, T., Makeig, S., Humphries, C., Lee, T.-W., McKeown, M. J., Iragui, V. J., & B, T. J. S. A. (2000) · Psychophysiology · doi:10.1111/1469-8986.3720163 · Cited by 3,342 · see citing works
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,701 · see citing works
Zheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,477 · cited 6× · see citing works
“In this study, we adopt an automatic continuous vigilance annotation method using eye tracking glasses, which was proposed in our previous work (Gao, Zhang, Zheng & Lu 2015).”
BackgroundMethodInfluentialSelf-citationAffect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893 · see citing works
Decoding mental states from brain activity in humans
Haynes, J., & Rees, G. E. (2006) · Nature reviews. Neuroscience · doi:10.1038/nrn1931 · Cited by 1,804 · see citing works
Ray, W. J., & Cole, H. W. (1985) · Science · doi:10.1126/science.3992243 · Cited by 1,146 · cited 2× · see citing works
“These results are consistent with previous findings in the literature (Ray & Cole 1985, Davidson et al. 2007, Huang et 2009, O’Connell et al. 2009, Peiris et al. 2011, Lin et al. 2013, Martel et al. 2014) and support the previous evidence that the increasing trend for the ratio of slow and fast…”
BackgroundResultDifferential entropy feature for EEG-based emotion classification
Duan, R.-N., Zhu, J., & Lu, B. (2013) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2013.6695876 · Cited by 1,108 · see citing works
Self-citationA neuromarker of sustained attention from whole-brain functional connectivity
Rosenberg, M. D., Finn, E. S., Scheinost, D., Papademetris, X., Shen, X., Constable, R. T., & Chun, M. M. (2015) · Nature Neuroscience · doi:10.1038/nn.4179 · Cited by 1,004 · cited 1× · see citing works
“Rosenberg and colleagues recently presented a neuromarker for sustained attention from whole-brain functional connectivity (Rosenberg, Finn, Scheinost, Papademetris, Shen, Constable & Chun 2016).”
BackgroundEEG artifact removal—state-of-the-art and guidelines
Urigüen, J. A., & García-Zapirain, B. (2015) · Journal of Neural Engineering · doi:10.1088/1741-2560/12/3/031001 · Cited by 973 · see citing works
Zander, T. O., & Kothe, C. A. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/8/2/025005 · Cited by 906 · cited 1× · see citing works
“Zander and Kothe proposed the concept of a passive brain-computer interface (BCI) to fuse conventional BCI systems with cognitive monitoring (Zander & Kothe 2011).”
BackgroundReal-time neuroimaging and cognitive monitoring using wearable dry EEG
Mullen, T. R., Kothe, C. A., Chi, Y. M., Ojeda, A., Kerth, T., Makeig, S., Jung, T., & Cauwenberghs, G. (2015) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2481482 · Cited by 801 · cited 1× · see citing works
“Several commercial EEG systems have emerged for increasing the usability in real-world applications (Grozea, Voinescu & Fazli 2011, Hairston, Whitaker, Ries, Vettel, Bradford, Kerick & McDowell 2014, Mullen, Kothe, Chi, Ojeda, Kerth, Makeig, Jung & Cauwenberghs 2015).”
BackgroundDetecting Driver Drowsiness Based on Sensors: A Review
Sahayadhas, A., Sundaraj, K., & Murugappan, M. (2012) · Sensors · doi:10.3390/s121216937 · Cited by 786 · cited 1× · see citing works
“Various approaches for estimating vigilance levels have been proposed in the literature (Ji, Zhu & Lan 2004, Dong, Hu, Uchimura & Murayama 2011, Sahayadhas, Sundaraj & Murugappan 2012).”
BackgroundEEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks.
Berka, C., Levendowski, D. J., Lumicao, M. N., Yau, A., Davis, G., Zivkovic, V. T., Olmstead, R. E., Tremoulet, P. D., & Craven, P. L. (2007) · PubMed · Cited by 765 · see citing works
Real-Time Nonintrusive Monitoring and Prediction of Driver Fatigue
Ji, Q., Zhu, Z., & Lan, P. (2004) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2004.830974 · Cited by 754 · cited 1× · see citing works
“Various approaches for estimating vigilance levels have been proposed in the literature (Ji, Zhu & Lan 2004, Dong, Hu, Uchimura & Murayama 2011, Sahayadhas, Sundaraj & Murugappan 2012).”
BackgroundUsing EEG spectral components to assess algorithms for detecting fatigue
Jap, B. T., Lal, S., Fischer, P., & Bekiaris, E. D. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.12.043 · Cited by 737 · see citing works
Driver Inattention Monitoring System for Intelligent Vehicles: A Review
Dong, Y., Hu, Z., Uchimura, K., & Murayama, N. (2011) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2010.2092770 · Cited by 732 · cited 3× · see citing works
“Moreover, drivers’ vigilance levels cannot be simply classified into several discrete categories but should be quantified in the same way as the blood alcohol level (Dong et al. 2011, Ranney 2008).”
BackgroundEye Movement Analysis for Activity Recognition Using Electrooculography
Bulling, A., Ward, J. A., Gellersen, H. W., & Tröster, G. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.86 · Cited by 720 · see citing works
Real-Time System for Monitoring Driver Vigilance
Bergasa, L. M., Nuevo, J., Sotelo, M. Á., Barea, R., & López-Guillén, E. (2006) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2006.869598 · Cited by 715 · see citing works
Pfurtscheller, G. (2010) · Frontiers in Neuroscience · doi:10.3389/fnpro.2010.00003 · Cited by 673 · see citing works
A Review and Meta-Analysis of Multimodal Affect Detection Systems
D’Mello, S. K., & Kory-Westlund, J. M. (2015) · ACM Computing Surveys · doi:10.1145/2682899 · Cited by 654 · see citing works
Driver Drowsiness Classification Using Fuzzy Wavelet-Packet-Based Feature-Extraction Algorithm
Khushaba, R. N., Kodagoda, S., Lal, S., et al. (2010) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2010.2077291 · Cited by 525 · cited 2× · see citing works
“For the single EEG modality, the posterior EEG contains the most critical information for vigilance estimation, which is consistent with previous findings (Khushaba et al. 2011, Shi & Lu 2013).”
MethodResultNicolaou, M. A., Gunes, H., & Pantic, M. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.9 · Cited by 450 · see citing works
Differential entropy feature for EEG-based vigilance estimation
Shi, L.-C., Jiao, Y.-Y., & Lu, B.-L. (2013) · 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6611075 · Cited by 396 · see citing works
Self-citationThe Vigilance Decrement Reflects Limitations in Effortful Attention, Not Mindlessness
Grier, R. A., Warm, J. S., Dember, W. N., Matthews, G., Galinsky, T. L., Szalma, J. L., & Parasuraman, R. (2003) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.45.3.349.27253 · Cited by 353 · see citing works
Ferrara, M., & Gennaro, L. D. (2001) · Sleep Medicine Reviews · doi:10.1053/smrv.2000.0138 · Cited by 344 · cited 1× · see citing works
“Most experiments were performed in the early afternoon (approximately 13:30) after lunch to induce fatigue easily when the circadian rhythm of sleepiness reached its peak (Ferrara & De Gennaro 2001).”
BackgroundLapse in alertness: coherence of fluctuations in performance and EEG spectrum
Makeig, S., & Inlow, M. H. (1993) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(93)90064-3 · Cited by 344 · cited 1× · see citing works
“To date, researchers have proposed various vigilance annotation methods in the literature, such as lane departure and local error rates (Wang, Jung & Lin 2015, Makeig & Inlow 1993).”
MethodO’Connell, R. G., Dockree, P. M., Robertson, I. H., Bellgrove, M. A., Foxe, J. J., & Kelly, S. P. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.5967-08.2009 · Cited by 273 · cited 1× · see citing works
“These results are consistent with previous findings in the literature (Ray & Cole 1985, Davidson et al. 2007, Huang et 2009, O’Connell et al. 2009, Peiris et al. 2011, Lin et al. 2013, Martel et al. 2014) and support the previous evidence that the increasing trend for the ratio of slow and fast…”
ResultPapadelis, C. L., Chen, Z. S., Kourtidou‐Papadeli, C., Bamidis, P. D., Chouvarda, I., Bekiaris, E. D., & Maglaveras, N. (2007) · Clinical Neurophysiology · doi:10.1016/j.clinph.2007.04.031 · Cited by 268 · cited 2× · see citing works
“Papadelis et al. designed an on-board system to assess a driver’s alertness level in real driving conditions (Papadelis et al. 2007).”
BackgroundHor, H., & Tafti, M. (2009) · Science · doi:10.1126/science.1178713 · Cited by 258 · see citing works
Bristle-sensors—low-cost flexible passive dry EEG electrodes for neurofeedback and BCI applications
Grozea, C., Voinescu, C., & Fazli, S. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/8/2/025008 · Cited by 256 · see citing works
Wireless and Wearable EEG System for Evaluating Driver Vigilance
Lin, C., Chuang, C., Huang, C.-S., Tsai, S.-F., Lu, S.-W., Chen, Y.-H., & Ko, L. (2014) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2014.2316224 · Cited by 251 · cited 2× · see citing works
“Lin et al. presented a wireless and wearable EEG system for evaluating drivers’ vigilance levels, and they tested their system in a virtual driving environment (Lin et al. 2014).”
BackgroundMultimodal emotion recognition using EEG and eye tracking data
Zheng, W.-L., Dong, B.-N., & Lu, B.-L. (2014) · 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/embc.2014.6944757 · Cited by 230 · see citing works
Self-citationUsability of four commercially-oriented EEG systems
Hairston, W. D., Whitaker, K. W., Ries, A. J., Vettel, J. M., Bradford, J. C., Kerick, S. E., & McDowell, K. (2014) · Journal of Neural Engineering · doi:10.1088/1741-2560/11/4/046018 · Cited by 216 · cited 1× · see citing works
“Several commercial EEG systems have emerged for increasing the usability in real-world applications (Grozea, Voinescu & Fazli 2011, Hairston, Whitaker, Ries, Vettel, Bradford, Kerick & McDowell 2014, Mullen, Kothe, Chi, Ojeda, Kerth, Makeig, Jung & Cauwenberghs 2015).”
BackgroundCognitive fatigue influences students’ performance on standardized tests
Sievertsen, H. H., Gino, F., & Piovesan, M. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516947113 · Cited by 194 · see citing works
McMullen, D. P., Hotson, G., Katyal, K. D., Wester, B. A., Fifer, M. S., McGee, T. G., Harris, A., Johannes, M. S., Vogelstein, R. J., Ravitz, A., et al. (2014) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2013.2294685 · Cited by 189 · see citing works
Combining eye movements and EEG to enhance emotion recognition
Lu, Y., Zheng, W., Li, B., & Lu, B. (2015) · Cited by 185 · see citing works
Self-citationFORCe: Fully online and automated artifact removal for brain-computer interfacing
Daly, I., Scherer, R., Billinger, M., & Muller-Putz, G. (2015) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2014.2346621 · Cited by 174 · see citing works
EEG-based workload estimation across affective contexts
hl, C. M. ⁄., Jeunet, C., & Lotte, F. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00114 · Cited by 161 · see citing works
EEG-Based Lapse Detection With High Temporal Resolution
Davidson, P. R., Jones, R. D., & Peiris, M. T. R. (2007) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2007.893452 · Cited by 158 · cited 4× · see citing works
“In these cases, a decrease in vigilance (Grier, Warm, Dember, Matthews, Galinsky, Szalma & Parasuraman 2003) or a momentary lapse of attention (Peiris, Davidson, Bones & Jones 2011, Davidson, Jones & Peiris 2007) might severely endanger public transportation safety.”
BackgroundResultInfluentialKim, I.-H., Kim, J.-W., Haufe, S., & Lee, S.-W. (2015) · Journal of Neural Engineering · doi:10.1088/1741-2560/12/1/016001 · Cited by 158 · see citing works
Wu, D., Courtney, C. G., Lance, B. J., Narayanan, S. S., Dawson, M. E., Oie, K. S., & Parsons, T. D. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.12 · Cited by 148 · see citing works
EEG-Based Attention Tracking During Distracted Driving
Wang, Y., Jung, T., & Lin, C. (2015) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2015.2415520 · Cited by 147 · cited 2× · see citing works
“To date, researchers have proposed various vigilance annotation methods in the literature, such as lane departure and local error rates (Wang, Jung & Lin 2015, Makeig & Inlow 1993).”
MethodElectrophysiology-based detection of emergency braking intention in real-world driving
Haufe, S., Kim, J.-W., Kim, I.-H., Sonnleitner, A., Schrauf, M., Curio, G., & Blankertz, B. (2014) · Journal of Neural Engineering · doi:10.1088/1741-2560/11/5/056011 · Cited by 139 · cited 1× · see citing works
“Haufe et al. performed a study to assess the real-world feasibility of EEG-based detection of emergency braking intention (Haufe, Kim, Kim, Sonnleitner, Schrauf, Curio & Blankertz 2014).”
BackgroundDriver Distraction: A Review of the Current State-of-Knowledge
(2008) · PsycEXTRA Dataset · doi:10.1037/e729212011-001 · Cited by 136 · see citing works
EEG-based vigilance estimation using extreme learning machines
Shi, L.-C., & Lu, B. (2012) · Neurocomputing · doi:10.1016/j.neucom.2012.02.041 · Cited by 126 · see citing works
Self-citationPeng, J., Bo, L., & Xu, J. (2009) · Cited by 120 · see citing works
Personalizing EEG-based affective models with transfer learning
Zheng, W., & Lu, B. (2016) · International Joint Conference on Artificial Intelligence · Cited by 118 · see citing works
Self-citationLearning a common dictionary for subject-transfer decoding with resting calibration
Morioka, H., Kanemura, A., Hirayama, J.-I., Shikauchi, M., Ogawa, T., Ikeda, S., Kawanabe, M., & Ishii, S. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.02.015 · Cited by 113 · see citing works
Dimensional affect recognition using Continuous Conditional Random Fields
Baltrušaitis, T., Banda, N., & Robinson, P. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553785 · Cited by 110 · see citing works
Driving fatigue detection with fusion of EEG and forehead EOG
Huo, X.-Q., Zheng, W., & Lu, B. (2016) · doi:10.1109/ijcnn.2016.7727294 · Cited by 96 · cited 2× · see citing works
“…existing labelled data from a group of subjects and generalize the models to the new subjects with transfer learning techniques (Pan & Yang 2010, Wronkiewicz, Larson & Lee 2015, Morioka, Kanemura, Hirayama, Shikauchi, Ogawa, Ikeda, Kawanabe & Ishii 2015, Zhang, Zheng & Lu 2015, Zheng & Lu 2016).”
BackgroundMethodSelf-citationOff-line and on-line vigilance estimation based on linear dynamical system and manifold learning
Shi, L.-C., & Lu, B. (2010) · Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/iembs.2010.5627125 · Cited by 87 · see citing works
Self-citationZander, T., & Jatzev, S. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/9/1/016003 · Cited by 83 · cited 1× · see citing works
“Covert aspects of ongoing user mental states provide key context information in user-aware human computer interactions (Zander & Jatzev 2012), which can help systems react adaptively in a proper manner.”
BackgroundTonic Changes in EEG Power Spectra during Simulated Driving
Huang, R., Jung, T., & Makeig, S. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02812-0_47 · Cited by 82 · see citing works
Tonic and phasic EEG and behavioral changes induced by arousing feedback
Lin, C., Huang, K.-C., Chao, C.-F., Chen, J.-A., Chiu, T.-W., Ko, L., & Jung, T. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.04.250 · Cited by 79 · cited 1× · see citing works
“9 The training of parameters in CCRF and CCNF is based on the conditional loglikelihood ( ∣ ) P y x as a multivariate Gaussian. For more details regarding the learning and infer ence of CCRF and CCNF, please refer to Baltrušaitis et al (2014) and Imbrasaite et al (2014). The outputs of support vector regression are…”
Method15th Int. Conf on Machine Learning
Cristianini, N., Shawe-Taylor, J. S., & Sybacek, P. (1998) · Cited by 76 · see citing works
A brain–computer interface method combined with eye tracking for 3D interaction
Lee, E. C., Woo, J. C., Kim, J. H., et al. (2010) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2010.05.008 · Cited by 76 · see citing works
PERCLOS: An Alertness Measure of the Past
Trutschel, U. E., Sirois, B., Sommer, D., et al. (2011) · International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design (Driving Assessment Conference) · doi:10.17077/drivingassessment.1394 · Cited by 74 · see citing works
A novel approach to driving fatigue detection using forehead EOG
Zhang, Y.-F., Gao, X., Zhu, J., et al. (2015) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2015.7146721 · Cited by 70 · cited 2× · see citing works
“The electrode placement of the forehead EOG (Zhang, Gao, Zhu, Zheng & Lu 2015) is shown in Figure 2.”
MethodSelf-citationEEG predictors of covert vigilant attention
Martel, A., Dähne, S., & Blankertz, B. (2014) · Journal of Neural Engineering · doi:10.1088/1741-2560/11/3/035009 · Cited by 65 · cited 3× · see citing works
“…previous findings in the literature (Ray & Cole 1985, Davidson et al. 2007, Huang et 2009, O’Connell et al. 2009, Peiris et al. 2011, Lin et al. 2013, Martel et al. 2014) and support the previous evidence that the increasing trend for the ratio of slow and fast waves of EEG activities reflects…”
ResultDetection of lapses in responsiveness from the EEG
Peiris, M. T. R., Davidson, P. R., Bones, P. J., & Jones, R. D. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/8/1/016003 · Cited by 56 · cited 2× · see citing works
“In these cases, a decrease in vigilance (Grier, Warm, Dember, Matthews, Galinsky, Szalma & Parasuraman 2003) or a momentary lapse of attention (Peiris, Davidson, Bones & Jones 2011, Davidson, Jones & Peiris 2007) might severely endanger public transportation safety.”
BackgroundResultEvaluating driving fatigue detection algorithms using eye tracking glasses
Gao, X., Zhang, Y.-F., Zheng, W., et al. (2015) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2015.7146736 · Cited by 53 · cited 2× · see citing works
“…existing labelled data from a group of subjects and generalize the models to the new subjects with transfer learning techniques (Pan & Yang 2010, Wronkiewicz, Larson & Lee 2015, Morioka, Kanemura, Hirayama, Shikauchi, Ogawa, Ikeda, Kawanabe & Ishii 2015, Zhang, Zheng & Lu 2015, Zheng & Lu 2016).”
BackgroundMethodSelf-citationFoundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience
Schmorrow, D., & Fidopiastis, C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-39952-2 · Cited by 45 · see citing works
Can arousing feedback rectify lapses in driving? Prediction from EEG power spectra
Lin, C., Huang, K.-C., Chuang, C., Ko, L., & Jung, T. (2013) · Journal of Neural Engineering · doi:10.1088/1741-2560/10/5/056024 · Cited by 43 · cited 3× · see citing works
“By monitoring the changes of EEG patterns, they were able to detect driving performance and estimate the efficacy of arousing warning feedback delivered to drowsy subjects (Lin, Huang, Chuang, Ko & Jung 2013).”
BackgroundResultInfluentialFuzzy Fusion of Eyelid Activity Indicators for Hypovigilance-Related Accident Prediction
Damousis, I. G., & Tzovaras, D. K. (2008) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2008.928241 · Cited by 42 · cited 1× · see citing works
“…information on various eye movements, which are often utilized to estimate vigilance because of its easy setup and high signal-noise ratio (Papadelis, Chen, Kourtidou-Papadeli, Bamidis, Chouvarda, Bekiaris & Maglaveras 2007, Damousis & Tzovaras 2008, Ma, Shi & Lu 2010, Ma, Shi & Lu 2014).”
BackgroundDelayed verbal reactions are specifically impaired in patients with schizophrenia
Pinelli, P., Ceriani, F., Colombo, R., Pasetti, C., Terazzi, M., & Castignoli, G. (2000) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(00)00088-x · Cited by 38 · see citing works
An EOG-based Vigilance Estimation Method Applied for Driver Fatigue Detection
Ma, J., Shi, L.-C., & Lu, B. (2014) · Neuroscience and Biomedical Engineering · doi:10.2174/2213385202666141218104855 · Cited by 38 · cited 2× · see citing works
“…information on various eye movements, which are often utilized to estimate vigilance because of its easy setup and high signal-noise ratio (Papadelis, Chen, Kourtidou-Papadeli, Bamidis, Chouvarda, Bekiaris & Maglaveras 2007, Damousis & Tzovaras 2008, Ma, Shi & Lu 2010, Ma, Shi & Lu 2014).”
BackgroundMethodSelf-citationTransfer Components Between Subjects for EEG-based Driving Fatigue Detection
Zhang, Y., Zheng, W., & Lu, B. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-26561-2_8 · Cited by 36 · see citing works
Self-citationLeveraging anatomical information to improve transfer learning in brain–computer interfaces
Wronkiewicz, M., Larson, E. B., & Lee, A. K. C. (2015) · Journal of Neural Engineering · doi:10.1088/1741-2560/12/4/046027 · Cited by 36 · cited 1× · see citing works
“…existing labelled data from a group of subjects and generalize the models to the new subjects with transfer learning techniques (Pan & Yang 2010, Wronkiewicz, Larson & Lee 2015, Morioka, Kanemura, Hirayama, Shikauchi, Ogawa, Ikeda, Kawanabe & Ishii 2015, Zhang, Zheng & Lu 2015, Zheng & Lu 2016).”
BackgroundHuman implicit intent recognition based on the phase synchrony of EEG signals
Kang, J.-S., Park, U., Gonuguntla, V., et al. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2015.06.013 · Cited by 35 · cited 1× · see citing works
“Various studies have introduced the assessment of the mental states of users, such as intention, emotion, and workload, to promote active interactions between users and machines (M¨uhl, Jeunet & Lotte 2014, Lu, Zheng, Li & Lu 2015, Kang, Park, Gonuguntla, Veluvolu & Lee 2015, Zheng & Lu 2015).”
BackgroundVigilance estimation by using electrooculographic features
Ma, J., Shi, L.-C., & Lu, B. (2010) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2010.5627122 · Cited by 31 · cited 2× · see citing works
“…information on various eye movements, which are often utilized to estimate vigilance because of its easy setup and high signal-noise ratio (Papadelis, Chen, Kourtidou-Papadeli, Bamidis, Chouvarda, Bekiaris & Maglaveras 2007, Damousis & Tzovaras 2008, Ma, Shi & Lu 2010, Ma, Shi & Lu 2014).”
BackgroundMethodSelf-citationImbrasaitė, V., Baltrušaitis, T., & Robinson, P. (2014) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew.2014.6890697 · Cited by 14 · cited 1× · see citing works
“For more details regarding the learning and inference of CCRF and CCNF, please refer to (Baltruˇsaitis et al. 2014, Imbrasaite et al. 2014).”
MethodSleeping and driving: Not a safe dual-task
Oken, B., & Salinsky, M. (2007) · Clinical Neurophysiology · doi:10.1016/j.clinph.2007.06.004 · Cited by 11 · see citing works
Flagged first, then by citations
1 of the 77 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Dinges, 1998, PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance, US Department of Transportation, Federal Highway Administration, Publication Number FHWA-MCRT-98-006
Cited by
Sorted by citations
A review of emotion recognition using physiological signals
Shu, L., Xie, J., Yang, M., Li, Z., Li, Z., Liao, D., Xu, X., & Yang, X. (2018) · Sensors · doi:10.3390/s18072074 · Cited by 923 · see citing works
“The SEED database [135] contains EEG and eye movement of three different emotions (positive, neutral and negative).”
BackgroundEEG-Based Spatio–Temporal Convolutional Neural Network for Driver Fatigue Evaluation
Gao, Z., Wang, X., Yang, Y., Mu, C., Cai, Q., Dang, W., & Zuo, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2886414 · Cited by 484 · see citing works
“In [17], a multimodal approach was proposed to estimate driver fatigue by combin-”
Transfer Learning for EEG-Based Brain–Computer Interfaces: A Review of Progress Made Since 2016
Wu, D., Xu, Y., & Lu, B.-L. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3007453 · Cited by 434 · see citing works
“There are many important BCI regression problems, e.g., driver drowsiness estimation [38]–[40], vigilance estimation [11], [12], [113], and user reaction time estimation [41], which were not adequately addressed in previous reviews.”
Self-citationTian, L., Zimmerman, B., Akhtar, A., Yu, K. J., Moore, M., Wu, J. C., Larsen, R. J., Lee, J. W., Li, J., Liu, Y., et al. (2019) · Nature Biomedical Engineering · doi:10.1038/s41551-019-0347-x · Cited by 395 · see citing works
EEG-Based Emotion Recognition: A State-of-the-Art Review of Current Trends and Opportunities
Suhaimi, N. S., Mountstephens, J., & Teo, J. (2020) · Computational Intelligence and Neuroscience · doi:10.1155/2020/8875426 · Cited by 361 · see citing works
“As for video-related contents, SEED [124–126], DREAMER [127], and ASCERTAIN [107] do provide their video database either openly or upon request.”
BackgroundSelf-Supervised Learning for Electroencephalography
Rafiei, M. H., Gauthier, L. V., Adeli, H., & Takabi, D. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3190448 · Cited by 324 · see citing works
“SEED is divided into three subdatasets: 1) SEED I [138]; 2) SEED-IV [140]; and 3) SEED-VIG [141].”
MethodCross-Subject Emotion Recognition Using Flexible Analytic Wavelet Transform From EEG Signals
Gupta, V., Chopda, M. D., & Pachori, R. B. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2883497 · Cited by 292 · see citing works
“of operator [5], [6].”
Brain-Computer Interface: Advancement and Challenges
Mridha, M. F., Das, S. C., Kabir, M. M., Lima, A. A., Islam, M. R., & Watanobe, Y. (2021) · Sensors · doi:10.3390/s21175746 · Cited by 272 · see citing works
A Review of Recent Developments in Driver Drowsiness Detection Systems
Albadawi, Y., Takruri, M., & Awad, M. (2022) · Sensors · doi:10.3390/s22052069 · Cited by 259 · see citing works
“02% Dataset1: Fpz-Cz channel dataset [89,90] Dataset2:SVDD dataset [91]”
MethodRecognition of human emotions using EEG signals: A review
Rahman, M. M., Sarkar, A. K., Hossain, M. A., Hossain, M. S., Islam, M. R., Hossain, M. B., Quinn, J. M. W., & Moni, M. A. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104696 · Cited by 226 · see citing works
EEG-based emotion recognition: Review of commercial EEG devices and machine learning techniques
Dadebayev, D., Goh, W. W., & Tan, E. X. (2021) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.03.009 · Cited by 214 · see citing works
Learning CNN features from DE features for EEG-based emotion recognition
Hwang, S., Hong, K., Son, G., & Byun, H. (2019) · Pattern Analysis and Applications · doi:10.1007/s10044-019-00860-w · Cited by 165 · see citing works
“Table 11 Vigilance estimation results on VIG dataset [39]”
BackgroundMethodInfluentialDriver sleepiness detection from EEG and EOG signals using GAN and LSTM networks
Jiao, Y., Deng, Y., Luo, Y., & Lu, B. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.05.108 · Cited by 165 · see citing works
“Zheng and Lu combined EEG and forehead EOG to leverage their complementary characteristics for vigilance estimation [37] .”
BackgroundSelf-citationReview on Emotion Recognition Based on Electroencephalography
Liu, H., Zhang, Y., Li, Y., & Kong, X. (2021) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2021.758212 · Cited by 161 · see citing works
Understanding mental fatigue and its detection: a comparative analysis of assessments and tools
Kunasegaran, K., Ismail, A. M. H., Ramasamy, S., Gnanou, J. V., Caszo, B. A., & Chen, P. L. (2023) · PeerJ · doi:10.7717/peerj.15744 · Cited by 171 · see citing works
Machine learning and deep learning techniques for driver fatigue and drowsiness detection: a review
El‐Nabi, S. A., El‐Shafai, W., El‐Rabaie, E. M., Ramadan, K. F., El‐Samie, F. E. A., & Mohsen, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15054-0 · Cited by 150 · see citing works
“Additionally, the simulated virtual driving driver (SVDD) dataset from [149] was used.”
A Review on EEG-Based Automatic Sleepiness Detection Systems for Driver
Balandong, R. P., Ahmad, R. F., Saad, M. N. M., & Malik, A. S. (2018) · IEEE Access · doi:10.1109/access.2018.2811723 · Cited by 138 · see citing works
“Numerous physiological indicators are used to bridge the physiological-sleepiness relationship such as electrocardiogram (ECG) [72]–[75], electrooculography (EOG) [76]–[78], functional Near Infrared Spectroscopy (fNIR) [79] and electroencephalography (EEG) [7], [70], [73], [79]–[86].”
MethodInfluentialAffective Brain–Computer Interfaces (aBCIs): A Tutorial
Wu, D., Lu, B., Hu, B., & Zeng, Z. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3277471 · Cited by 133 · see citing works
“…applications in neural rehabilitation [12], text input [13], gaming [14], emotion recognition [15], mental fatigue evaluation [16], vigilance estimation [17], speech synthesis [18], sentence decoding [19], movement and touch functionality recovery [20], [21], robot control [22], [23], and so on.”
BackgroundSelf-citationCai, Y., Li, X., & Li, J. (2023) · Sensors · doi:10.3390/s23052455 · Cited by 143 · see citing works
“SEED * [210] Induced 15 participants; average age is 23.”
BackgroundCNN and LSTM-Based Emotion Charting Using Physiological Signals
Dar, M. N., Akram, M., Khawaja, S. G., & Pujari, A. N. (2020) · Italian National Conference on Sensors · doi:10.3390/s20164551 · Cited by 119 · see citing works
“They incorporated spectral features from EEG with the GELM classifier for SEED [26] and DEAP datasets, respectively.”
MethodA new fractal pattern feature generation function based emotion recognition method using EEG
Tuncer, T., Doğan, Ş., & Subaşı, A. (2021) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2021.110671 · Cited by 126 · see citing works
Accurate Markov Boundary Discovery for Causal Feature Selection
Wu, X., Jiang, B., Yu, K., Miao, C., & Chen, H. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2940509 · Cited by 120 · see citing works
“formed on a newly developed electroencephalography (EEG) dataset, SEED [32], [33], to compare our algorithms with other state-of-the-art methods in Section VI-C.”
MethodFusion of Facial Expressions and EEG for Multimodal Emotion Recognition
Huang, Y., Yang, J., Liao, P., & Pan, J. (2017) · Computational Intelligence and Neuroscience · doi:10.1155/2017/2107451 · Cited by 115 · see citing works
“Superior performance was obtained compared to the results in the state-of-the-art results [3, 11, 32].”
ResultVigilance Estimation Using a Wearable EOG Device in Real Driving Environment
Zheng, W., Gao, K., Li, G., Liu, W., Chao, L., Liu, J., Wang, G., & Lu, B. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2889962 · Cited by 118 · see citing works
“In this paper, we applied two temporal dependency models, namely, continuous conditional neural field (CCNF) and continuous conditional random field (CCRF), to capture the vigilance dynamics [31] and compared their performance with those of the conventional SVR model and the dynamic model using the Linear Dynamic Syst…”
BackgroundMethodResultInfluentialSelf-citationMusic mood and human emotion recognition based on physiological signals: a systematic review
Chaturvedi, V., Kaur, A. B., Varshney, V., Garg, A., Chhabra, G. S., & Kumar, M. (2021) · Multimedia Systems · doi:10.1007/s00530-021-00786-6 · Cited by 115 · see citing works
Multi-Scale Neural Network for EEG Representation Learning in BCI
Ko, W., Jeon, E., Jeong, S., & Suk, H. (2021) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2021.3061875 · Cited by 111 · see citing works
“nner for extracting spatio-spectral features and classified the learned features using an SVM with a radial basis function (RBF) kernel ( = 1=d input where d input denotes the input feature dimension) [10]. 4) Linear Models - Seizure: In addition, we also reimplemented Shoeb and Guttag [14]’s method for the seizure ons…”
MethodInfluentialBalam, V. P., & Chinara, S. (2020) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2020.108927 · Cited by 107 · see citing works
Reducing the Subject Variability of EEG Signals with Adversarial Domain Generalization
Ma, B.-Q., Li, H., Zheng, W., & Lu, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-36708-4_3 · Cited by 104 · see citing works
Self-citationUsing EEG for Mental Fatigue Assessment: A Comprehensive Look Into the Current State of the Art
Monteiro, T. G., Skourup, C., & Zhang, H. (2019) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2019.2938156 · Cited by 103 · see citing works
“We can identify the studies using three [59], [76], [89] or four [67], [77], [91] different MF states.”
BackgroundMethodDriving Fatigue Classification Based on Fusion Entropy Analysis Combining EOG and EEG
Wang, H., Wu, C., Li, T., He, Y., Chen, P., & Bezerianos, A. (2019) · IEEE Access · doi:10.1109/access.2019.2915533 · Cited by 103 · see citing works
“, Zheng and Lu proposed different electrode placements on the forehead and extracted various eye movement features (EOG) for fatigue detection [38], [39].”
BackgroundConvolutional Neural Network-Based EEG Signal Analysis: A Systematic Review
Rajwal, S., & Aggarwal, S. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09920-1 · Cited by 97 · see citing works
“There is one openly available dataset called SEED-VIG [97] which consists of EEG signals collected for a simulated driving experiment.”
MethodA systematic review of physiological signals based driver drowsiness detection systems
Saleem, A. A., Siddiqui, H. U. R., Raza, M. A., Rustam, F., Dudley, S., & Ashraf, I. (2022) · Cognitive Neurodynamics · doi:10.1007/s11571-022-09898-9 · Cited by 102 · see citing works
Multi-method Fusion of Cross-Subject Emotion Recognition Based on High-Dimensional EEG Features
Yang, F. S., Zhao, X., Jiang, W., Gao, P., & Liu, G. (2019) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2019.00053 · Cited by 94 · see citing works
“Emotional research has a preliminary development trend in the field of computer science, such as task workload assessments and operator vigilance (Shi and Lu, 2013; Zheng and Lu, 2017b).”
BackgroundDriving Fatigue Detection from EEG Using a Modified PCANet Method
Ma, Y., Chen, B., Li, R., et al. (2019) · Computational Intelligence and Neuroscience · doi:10.1155/2019/4721863 · Cited by 93 · see citing works
Abbas, Q., & Alsheddy, A. (2020) · Sensors · doi:10.3390/s21010056 · Cited by 92 · see citing works
“Afterward, the authors performed machine learning classification algorithms, such as multilayer deep learning (DL), PSO, SVM, ANN, and LDA [55–73].”
BackgroundMethodZhang, G., & Etemad, A. (2021) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2021.3089594 · Cited by 102 · see citing works
“Blink and saccade movements can be detected from EOG recordings by applying peak detection on wavelet coefficients calculated with continuous wavelet transform [1].”
BackgroundMethodInfluentialKim, N., Kim, J., & Ahn, C. R. (2021) · Advanced Engineering Informatics · doi:10.1016/j.aei.2021.101359 · Cited by 90 · see citing works
Peng, Y. K., Qin, F., Kong, W., Ge, Y., Nie, F., & Cichocki, A. S. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3082803 · Cited by 87 · see citing works
Lazarou, E., & Exarchos, T. P. (2024) · AIMS neuroscience · doi:10.3934/neuroscience.2024006 · Cited by 90 · see citing works
Zhang, X., Pan, J., Shen, J. G., Din, Z. U., Li, J., Lu, D., Wu, M., & Hu, B. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981440 · Cited by 84 · see citing works
TFormer: A time–frequency Transformer with batch normalization for driver fatigue recognition
Li, R., Hu, M., Gao, R., Wang, L., Suganthan, P. N., & Sourina, O. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102575 · Cited by 85 · see citing works
Xu, Y., Yang, W., Yu, X., Li, H., Cheng, T., Lu, X., & Wang, Z. L. (2021) · ACS Nano · doi:10.1021/acsnano.1c00536 · Cited by 81 · see citing works
A Systematic Survey of Driving Fatigue Monitoring
Zhang, Z., Ning, H., & Zhou, F. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3189346 · Cited by 82 · see citing works
“Public datasets such as NTHU-DDD [25], YawDD [26], seed-vig [27], and E-pilots [28] are used to research driver mental states.”
MethodCorrelation between Situational Awareness and EEG signals
Kästle, J. L., Anvari, B., Król, J., & Wurdemann, H. A. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.12.026 · Cited by 84 · see citing works
A recurrence network-based convolutional neural network for fatigue driving detection from EEG
Gao, Z., Li, Y., Yang, Y., & Ma, C. (2019) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120538 · Cited by 79 · see citing works
Shahbakhti, M., Beiramvand, M., Rejer, I., et al. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3096984 · Cited by 78 · see citing works
Driving Fatigue Detection Based on Hybrid Electroencephalography and Eye Tracking
Lian, Z., Xu, T., Yuan, Z., et al. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3446952 · Cited by 78 · see citing works
“Zheng et al. proposed a hybrid approach for estimating vigilance by combining EEG and forehead EOG data, with their data labeled using the percentage of eye closure (PERCLOS) index calculated from eye tracking data [24].”
MethodInfluentialEfficacy of transformer networks for classification of EEG data
Siddhad, G., Gupta, A., Dogra, D. P., & Roy, P. P. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105488 · Cited by 86 · see citing works
“EEG has long been used for di ff erent purposes like disease diagnosis [8], BCI [9], neuro-feedback [10], workload estimation [11], motor imagery classification [12], and even brain-to-braincommunication [13].”
MethodChen, L., Zhang, A., & Xiao-guang, L. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.02.005 · Cited by 76 · see citing works
Eye-Tracking Feature Extraction for Biometric Machine Learning
Lim, J. Z., Mountstephens, J., & Teo, J. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2021.796895 · Cited by 74 · see citing works
“The level of driver alertness SEED-VIG (Zheng and Lu, 2017) is used as the dataset for the simulation experiment.”
MethodDynamic entropy-based pattern learning to identify emotions from EEG signals across individuals
Lu, Y., Wang, M., Wu, W., Han, Y., Zhang, Q., & Chen, S. (2020) · Measurement · doi:10.1016/j.measurement.2019.107003 · Cited by 73 · see citing works
Koay, H. V., Chuah, J. H., Chow, C., & Chang, Y.-L. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105309 · Cited by 70 · see citing works
Multimodal System to Detect Driver Fatigue Using EEG, Gyroscope, and Image Processing
Karuppusamy, N. S., & Kang, B. (2020) · IEEE Access · doi:10.1109/access.2020.3009226 · Cited by 69 · see citing works
“In another method forehead EOG was used to extract various information on eye movement features and combined with EEG as EOG contains complementary characteristics for vigilance estimation [34].”
MethodYaacob, H. S., Hossain, F., Shari, S., Khare, S. K., Ooi, C. P., & Acharya, U. R. (2023) · IEEE Access · doi:10.1109/access.2023.3296382 · Cited by 68 · see citing works
A Regression Method With Subnetwork Neurons for Vigilance Estimation Using EOG and EEG
Wu, W., Wu, Q. M. J., Sun, W. J., Yang, Y., Yuan, X., Zheng, W., & Lu, B. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2889223 · Cited by 68 · see citing works
“Huo et al. [21] and Zheng and Lu [22] found the complementarity of forehead-EOG and EEG signals, and using the feature fusion for vigilance estimation, improved the accuracy rate.”
MethodResultInfluentialSelf-citationGao, D., Wang, K., Wang, M., Zhou, J., & Zhang, Y. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3285268 · Cited by 66 · see citing works
“Research shows combining five frequency bands is better than using a single frequency band [18].”
BackgroundChen, J., Cui, Y., Wei, C., et al. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.114097 · Cited by 65 · see citing works
Kaklauskas, A., Abraham, A., Ubartė, I., Klıukas, R., Luksaite, V., Binkytė-Vėlienė, A., Vetloviene, I., & Kaklauskiene, L. (2022) · Sensors · doi:10.3390/s22207824 · Cited by 68 · see citing works
EEG-based neural networks approaches for fatigue and drowsiness detection: A survey
Othmani, A., Sabri, A. Q. M., Aslan, S., Chaieb, F., Rameh, H., Alfred, R., & Cohen, D. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126709 · Cited by 63 · see citing works
VIGNet: A Deep Convolutional Neural Network for EEG-based Driver Vigilance Estimation
Ko, W., Oh, K., Jeon, E., & Suk, H. (2020) · International Winter Conference on Brain-Computer Interface (BCI) · doi:10.1109/bci48061.2020.9061668 · Cited by 62 · see citing works
“, a DE calculation as like as Zheng and Lu [2], a CNN-based latent feature learning, and a decision making, i.”
BackgroundMethodInfluentialPeng, Y. K., Jin, F., Kong, W., Nie, F., Lu, B., & Cichocki, A. S. (2022) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2022.3175464 · Cited by 61 · see citing works
Self-citationEOG-Based Human–Computer Interface: 2000–2020 Review
Belkhiria, C., Boudir, A., Hurter, C., & Peysakhovich, V. (2022) · Sensors · doi:10.3390/s22134914 · Cited by 61 · see citing works
“Kumar [84] 2016 • • • • Dhuliawala [85] 2016 • • • • Shimizu [86] 2016 • • • • • Shimizu bis [87] 2016 • • • • • Bissoli [88] 2016 • • • • Wilaiprasitporn [89] 2016 • • • • Fang [90] 2016 • • • • • Tamura [91] 2016 • • • • • • • Barbara [92] 2016 • • • • • Atique [93] 2016 • • • • • Naijian [94] 2016 • • • Ogai [95] 2…”
EEG Driving Fatigue Detection With PDC-Based Brain Functional Network
Wang, F., Wu, S., Ping, J., et al. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3058658 · Cited by 59 · see citing works
“These results are consistent with previous findings in the literature [31], [51] and support the previous evidence that the increasing trend for the ratio of slow and fast waves of EEG activities reflects decreasing attentional demands [36].”
ResultJia, H., Xiao, Z., & Ji, P. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106431 · Cited by 59 · see citing works
Electroencephalogram-Based Approaches for Driver Drowsiness Detection and Management: A Review
Li, G., & Chung, W. (2022) · Sensors · doi:10.3390/s22031100 · Cited by 78 · see citing works
“Li et [86,87].”
Ahmed, M. Z. I., Sinha, N., Ghaderpour, E., Phadikar, S., & Ghosh, R. (2023) · Bioengineering · doi:10.3390/bioengineering10010054 · Cited by 56 · see citing works
“The two major techniques used for emotion elicitation are using external stimuli, such as audio–visual [8,12,13] or memory recall [14].”
MethodA Comprehensive Survey on Tiny Machine Learning for Human Behavior Analysis
Lamaakal, I., Essahraui, S., Maleh, Y., Makkaoui, K. E., Ouahbi, I., Bouami, M. F., El‐Latif, A. A. A., Almousa, M., Peng, J., & Niyato, D. T. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3565688 · Cited by 61 · see citing works
“The SEED-VIG dataset [225] includes EEG recordings for vigilance studies.”
Fatigue driving detection method based on Time-Space-Frequency features of multimodal signals
Shi, J., & Wang, K. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.104744 · Cited by 54 · see citing works
A Hybrid Deep Neural Network Approach to Recognize Driving Fatigue Based on EEG Signals
Alghanim, M., Attar, H. H., Rezaee, K., et al. (2024) · International Journal of Intelligent Systems · doi:10.1155/2024/9898333 · Cited by 54 · see citing works
Torkamani‐Azar, M., Kanık, S. D., Aydın, S., & Çetin, M. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2980056 · Cited by 53 · see citing works
“In addition, in a 2-hour simulated experiment, Zheng and Lu [18] observed that, in transition to the drowsy states, alpha and theta significantly increased in the temporal and parietal cortex.”
BackgroundVideo‐triggered EEG‐emotion public databases and current methods: A survey
Hu, W., Huang, G., Li, L., Zhang, L., Zhang, Z., & Liang, Z. (2020) · Brain Science Advances · doi:10.26599/bsa.2020.9050026 · Cited by 53 · see citing works
“(3) SEED‐VIG database: This experiment recruited 23 participants and asked them to monotonously repeat straight road driving tasks in a simulated driving system, which could easily induct fatigue and change their vigilance [84].”
BackgroundAl-Shargie, F., Hassanin, O., Tariq, U., & Al-Nashash, H. A. (2020) · IEEE Access · doi:10.1109/access.2020.3004504 · Cited by 51 · see citing works
Transformers in biosignal analysis: A review
Anwar, A., Khalifa, Y., Coyle, J. L., & Sejdić, E. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102697 · Cited by 55 · see citing works
Detecting driving fatigue with multimodal deep learning
Du, L., Liu, W., Zheng, W., et al. (2017) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2017.8008295 · Cited by 49 · see citing works
“To incorporate the temporal dependency into driving fatigue detection, continuous conditional random field (CCNF) has been applied to detect driving fatigue [10].”
BackgroundMethodSelf-citationLee, C., Chen, C., Lin, C., et al. (2019) · Applied Sciences · doi:10.3390/app9235004 · Cited by 48 · see citing works
“[31] proposed a personalized product configuration process to determine design attributes in a cloud-based adaptable product configuration.”
BackgroundThe Auto-Correlation Function Aided Sparse Support Matrix Machine for EEG-Based Fatigue Detection
Li, Y., Wang, D., & Liu, F. (2022) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2022.3211931 · Cited by 47 · see citing works
“The dataset used in this experiment is the SEED-VIG fatigue dataset [27], describing as follows.”
MethodPhyDAA: Physiological Dataset Assessing Attention
Delvigne, V., Wannous, H., Dutoit, T., Ris, L., & Vandeborre, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3061719 · Cited by 47 · see citing works
“In another domain, Zheng and Lu [53] and Cao et al. [11] both propose a dataset composed of EEG records assessing attention during a driving task.”
BackgroundGalea: A physiological sensing system for behavioral research in Virtual Environments
Bernal, G., Hidalgo, N., Russomanno, C., & Maes, P. (2022) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr51125.2022.00024 · Cited by 46 · see citing works
“While understanding a single physiological signal can be insightful, it is even more useful to be able to examine multiple modalities so as to understand human behavior and the individual’s responses to their environment and gain insight into their cognitive processes [41,74].”
BackgroundMultiple nonlinear features fusion based driving fatigue detection
Wang, F., Wu, S., Zhang, W., et al. (2020) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2020.102075 · Cited by 45 · see citing works
Multimodal Vigilance Estimation Using Deep Learning
Wu, W., Sun, W. J., Wu, Q. M. J., Yang, Y., Zhang, H., Zheng, W., & Lu, B. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3022647 · Cited by 45 · see citing works
“Unlike the traditional EOG (EOGt)-based method, a simple but more robust method proposed by Zheng and Lu [21] utilizes the placement of wearable electrodes on the forehead area.”
MethodResultSelf-citationSiddiqui, M. F. H., Dhakal, P., Yang, X., & Javaid, A. Y. (2022) · Multimodal Technologies and Interaction · doi:10.3390/mti6060047 · Cited by 46 · see citing works
Review and Challenges of Technologies for Real-Time Human Behavior Monitoring
Dávila-Montero, S., Dana-Lê, J. A., Bente, G., Hall, A., & Mason, A. J. (2021) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2021.3060617 · Cited by 34 · see citing works
“On the other hand, Zheng and Lu [170] employed an SVR with radial basis function to estimate the level of vigilance based on data from EEG and EOG.”
MethodGao, D., Tang, X., Wan, M., Guo, H., & Zhang, Y. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2023.1136609 · Cited by 41 · see citing works
Spatio-Temporal EEG Representation Learning on Riemannian Manifold and Euclidean Space
Zhang, G., & Etemad, A. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3332549 · Cited by 42 · see citing works
“EEG signals are split into EEG segments of T = 8 seconds with no overlap, as presented in Table I. 2) Seed-Vig: The SEED-VIG dataset has been collected by [6] to estimate driver vigilance.”
BackgroundMethodInfluentialFatigue State Detection for Tired Persons in Presence of Driving Periods
Alharbey, R., Dessouky, M. M., Sedik, A., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3185251 · Cited by 41 · see citing works
“Used Dataset DROZY EEG signals SEED-VIG [33] Self-collected EEG”
MethodFeature Selection Model based on EEG Signals for Assessing the Cognitive Workload in Drivers
Becerra-Sánchez, P., Reyes, A., & Ibáñez, J. A. G. (2020) · Sensors · doi:10.3390/s20205881 · Cited by 40 · see citing works
Liu, Z., Jiang, X., Luo, H., et al. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.11.019 · Cited by 38 · see citing works
Drowsiness detection using portable wireless EEG
Sagila Gangadharan, K., & Vinod, A. P. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106535 · Cited by 38 · see citing works
A Product Fuzzy Convolutional Network for Detecting Driving Fatigue
Du, G., Long, S., Li, C., et al. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3123842 · Cited by 38 · see citing works
“Zheng and Lu [7] proposed a multimodal method to evaluate vigilance by integrating EEG and EOG and introducing the temporal dependency-based vigilance into model training.”
MethodCheng, X., Wei, W., Du, C., Qiu, S., Tian, S., Ma, X., & He, H. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548367 · Cited by 37 · see citing works
“[37] extracted features from eye movements such as blinks and saccades based on the wavelet transform method from forehead EOG to realize vigilance detection.”
MethodInfluentialPeng, B., Zhang, Y., Wang, M., et al. (2023) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2023.107863 · Cited by 37 · see citing works
Drowsiness Detection in Drivers: A Systematic Review of Deep Learning-Based Models
Fonseca, T., & Ferreira, S. (2025) · Applied Sciences · doi:10.3390/app15169018 · Cited by 37 · see citing works
Balam, V. P., & Chinara, S. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3094619 · Cited by 40 · see citing works
Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System
Riquelme, F., Olivares, R., Muñoz, F., Molinero, X., & Serna, M. (2021) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.022553 · Cited by 37 · see citing works
Zheng, R., Wang, Z., He, Y., et al. (2021) · Cognitive Neurodynamics · doi:10.1007/s11571-021-09714-w · Cited by 36 · see citing works
Multimodal Vigilance Estimation with Adversarial Domain Adaptation Networks
Li, H., Zheng, W., & Lu, B. (2018) · doi:10.1109/ijcnn.2018.8489212 · Cited by 34 · see citing works
“In this paper, we integrate the multimodal approach [11] and adversarial domain adaptation”
MethodInfluentialSelf-citationDepersonalized Cross-Subject Vigilance Estimation with Adversarial Domain Generalization
Ma, B.-Q., Li, H., Luo, Y., et al. (2019) · doi:10.1109/ijcnn.2019.8852347 · Cited by 34 · see citing works
“further improve the performance of the vigilance estimation models [6] [7].”
BackgroundSelf-citationAn enhanced ensemble deep random vector functional link network for driver fatigue recognition
Li, R., Gao, R., Yuan, L., Suganthan, P. N., Wang, L., & Sourina, O. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106237 · Cited by 36 · see citing works
EEG-based emotion recognition for road accidents in a simulated driving environment
Chen, J., Lin, X., Ma, W., Wang, Y., & Tang, W. H. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105411 · Cited by 33 · see citing works
Decoding Neural Correlates of Cognitive States to Enhance Driving Experience
Chavarriaga, R., Ušćumlić, M., Zhang, H., Khaliliardali, Z., Aydarkhanov, R., Saeedi, S., Gheorghe, L., & Millán, J. D. R. (2018) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2018.2848289 · Cited by 33 · see citing works
- Authors
- Zheng, W.-L., & Lu, B.-L.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Corresponding author
- Zheng, Wei-Long
- Lu, Bao-Liang
- Venue
- Journal of Neural Engineering
- Publisher
- IOP Publishing
- Published
- 2017-02-22
- Volume
- 14
- Issue
- 2
- Pages
- 026017
- ISSN
- 1741-2552 (ISSN-L)
- 1741-2560
- OpenAlex
- W2558193840
- Semantic Scholar
- babe42b42bffd9029d7fef3d6c97ec0985fa6528
- Cited by
- 424
- References
- 77
- Citations by year
- 2026: 60 · 2025: 66 · 2024: 72 · 2023: 55 · 2022: 46 · 2021: 50 · 2020: 37 · 2019: 21 · 2018: 14 · 2017: 3
- FWCI
- 11.78
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Journal of Neural Engineering (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Technology Research and Development Program of China Railway Corporation (2016Z003-B)
- National Basic Research Program of China (2013CB329401)
- National Natural Science Foundation of China (61272248)
- Major Basic Research Program of Shanghai Science and Technology Committee (15JC1400103)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and Work-Related Fatigue
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Keywords
- EEG, multimodal fusion, PERCLOS, eye movement features, temporal dependencies, drowsiness detection, gamma oscillations, wearable EEG
- MeSH terms
- Adult, Algorithms, Arousal, Electroencephalography, Electrooculography, Female, Humans, Male, Pattern Recognition, Automated, Psychomotor Performance, Sensitivity and Specificity, Reproducibility of Results, Brain Waves
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
