The DOI matches its Crossref record; 76 of the 77 references it cites were found in a registry or an index.
A multimodal approach to estimating vigilance using EEG and forehead EOG
Open accessAbstract
Objective . Covert aspects of ongoing user mental states provide key context information for user-aware human computer interactions. In this paper, we focus on the problem of estimating the vigilance of users using EEG and EOG signals. Approach . The PERCLOS index as vigilance annotation is obtained from eye tracking glasses. To improve the feasibility and wearability of vigilance estimation devices for real-world applications, we adopt a novel electrode placement for forehead EOG and extract various eye movement features, which contain the principal information of traditional EOG. We explore the effects of EEG from different brain areas and combine EEG and forehead EOG to leverage their complementary characteristics for vigilance estimation. Considering that the vigilance of users is a dynamic changing process because the intrinsic mental states of users involve temporal evolution, we introduce continuous conditional neural field and continuous conditional random field models to capture dynamic temporal dependency. Main results . We propose a multimodal approach to estimating vigilance by combining EEG and forehead EOG and incorporating the temporal dependency of vigilance into model training. The experimental results demonstrate that modality fusion can improve the performance compared with a single modality, EOG and EEG contain complementary information for vigilance estimation, and the temporal dependency-based models can enhance the performance of vigilance estimation. From the experimental results, we observe that theta and alpha frequency activities are increased, while gamma frequency activities are decreased in drowsy states in contrast to awake states. Significance . The forehead setup allows for the simultaneous collection of EEG and EOG and achieves comparative performance using only four shared electrodes in comparison with the temporal and posterior sites.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The claim comes from somewhere else, not from this source.
In the citing paper
“Zheng and his colleges presented an emotion recognition method combining EEG signals and pupillary response collected from eye tracker and achieved average accuracies of 73.59% and 72.98% for three emotion states using feature level fusion strategy and decision-level fusion strategy, respectively.”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [32]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Most commercial biometric systems do not provide access to the feature sets nor the raw data which they use in their products [22].”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [22]
Experimental — accuracy not yet measuredIn the citing paper
“Superior performance was obtained compared to the results in the state-of-the-art results [3, 11, 32].”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371 · cited 3×
“ICA is a blind source separation method proposed to decompose a multivariate signal into independent non-Gaussian signals (Delorme & Makeig 2004).”
MethodPan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
(1992) · Applied Ergonomics · doi:10.1016/0003-6870(92)90318-p · Cited by 5,774
Removing electroencephalographic artifacts by blind source separation
Jung, T., Makeig, S., Humphries, C., Lee, T.-W., McKeown, M. J., Iragui, V. J., & B, T. J. S. A. (2000) · Psychophysiology · doi:10.1111/1469-8986.3720163 · Cited by 3,342
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,701
Zheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,477 · cited 6×
“In this study, we adopt an automatic continuous vigilance annotation method using eye tracking glasses, which was proposed in our previous work (Gao, Zhang, Zheng & Lu 2015).”
BackgroundMethodInfluentialSelf-citationAffect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893
Decoding mental states from brain activity in humans
Haynes, J., & Rees, G. E. (2006) · Nature reviews. Neuroscience · doi:10.1038/nrn1931 · Cited by 1,868
Differential entropy feature for EEG-based emotion classification
Duan, R.-N., Zhu, J., & Lu, B. (2013) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2013.6695876 · Cited by 1,225
Self-citation
Research works printed but not linked: 1
- Dinges, 1998, PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance, US Department of Transportation, Federal Highway Administration, Publication Number FHWA-MCRT-98-006
Cited by
Sorted by citations
A review of emotion recognition using physiological signals
Shu, L., Xie, J., Yang, M., Li, Z., Li, Z., Liao, D., Xu, X., & Yang, X. (2018) · Sensors · doi:10.3390/s18072074 · Cited by 923
“The SEED database [135] contains EEG and eye movement of three different emotions (positive, neutral and negative).”
BackgroundEEG-Based Spatio–Temporal Convolutional Neural Network for Driver Fatigue Evaluation
Gao, Z., Wang, X., Yang, Y., Mu, C., Cai, Q., Dang, W., & Zuo, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2886414 · Cited by 486
“In [17], a multimodal approach was proposed to estimate driver fatigue by combin-”
Transfer Learning for EEG-Based Brain–Computer Interfaces: A Review of Progress Made Since 2016
Wu, D., Xu, Y., & Lu, B.-L. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3007453 · Cited by 434
“There are many important BCI regression problems, e.g., driver drowsiness estimation [38]–[40], vigilance estimation [11], [12], [113], and user reaction time estimation [41], which were not adequately addressed in previous reviews.”
Self-citationTian, L., Zimmerman, B., Akhtar, A., Yu, K. J., Moore, M., Wu, J. C., Larsen, R. J., Lee, J. W., Li, J., Liu, Y., et al. (2019) · Nature Biomedical Engineering · doi:10.1038/s41551-019-0347-x · Cited by 411
EEG-Based Emotion Recognition: A State-of-the-Art Review of Current Trends and Opportunities
Suhaimi, N. S., Mountstephens, J., & Teo, J. (2020) · Computational Intelligence and Neuroscience · doi:10.1155/2020/8875426 · Cited by 395
“As for video-related contents, SEED [124–126], DREAMER [127], and ASCERTAIN [107] do provide their video database either openly or upon request.”
Background
Metadata
- Authors
- Zheng, W.-L., & Lu, B.-L.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Corresponding author
- Zheng, Wei-Long
- Lu, Bao-Liang
- Venue
- Journal of Neural Engineering
- Publisher
- IOP Publishing
- Published
- 2017-02-22
- Volume
- 14
- Issue
- 2
- Pages
- 026017
- ISSN
- 1741-2552 (ISSN-L)
- 1741-2560
- OpenAlex
- W2558193840
- Semantic Scholar
- babe42b42bffd9029d7fef3d6c97ec0985fa6528
- Cited by
- 424
- References
- 77
- Citations by year
- 2026: 60 · 2025: 66 · 2024: 72 · 2023: 55 · 2022: 46 · 2021: 50 · 2020: 37 · 2019: 21 · 2018: 14 · 2017: 3
- FWCI
- 11.78
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Journal of Neural Engineering (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Technology Research and Development Program of China Railway Corporation (2016Z003-B)
- National Basic Research Program of China (2013CB329401)
- National Natural Science Foundation of China (61272248)
- Major Basic Research Program of Shanghai Science and Technology Committee (15JC1400103)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and Work-Related Fatigue
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- EEG, multimodal fusion, PERCLOS, eye movement features, temporal dependencies, drowsiness detection, gamma oscillations, wearable EEG
- MeSH terms
- Adult, Algorithms, Arousal, Electroencephalography, Electrooculography, Female, Humans, Male, Pattern Recognition, Automated, Psychomotor Performance, Sensitivity and Specificity, Reproducibility of Results, Brain Waves
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
