Convolutional Neural Network for Multi-Category Rapid Serial Visual Presentation BCI
Open accessAbstract
Brain computer interfaces rely on machine learning (ML) algorithms to decode the brain's electrical activity into decisions. For example, in rapid serial visual presentation (RSVP) tasks, the subject is presented with a continuous stream of images containing rare target images among standard images, while the algorithm has to detect brain activity associated with target images. Here, we continue our previous work, presenting a deep neural network model for the use of single trial EEG classification in RSVP tasks. Deep neural networks have shown state of the art performance in computer vision and speech recognition and thus have great promise for other learning tasks, like classification of EEG samples. In our model, we introduce a novel spatio-temporal regularization for EEG data to reduce overfitting. We show improved classification performance compared to our earlier work on a five categories RSVP experiment. In addition, we compare performance on data from different sessions and validate the model on a public benchmark data set of a P300 speller task. Finally, we discuss the advantages of using neural network models compared to manually designing feature extraction algorithms.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior P300 CNNs do not address the challenge of reducing the number of trainable parameters.
“they do not address the challenge of reducing the number of trainable parameters.”
From A Lightweight Multi-Scale Convolutional Neural Network for P300 Decoding: Analysis of Training Strategies and Uncovering of Network Decision (2021) · cites this paper as Manor and Geva, 2015
The Introduction (page unavailable) says CNN weight sharing reduces the number of network parameters. Section 2.5 (page unavailable) says this paper’s P300 CNN shares filter weights across time, contrary to the claim that it does not address parameter reduction. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredShan et al. (2018) proved that an architecture with the first layer performing a mixed spatio-temporal convolution (OCLNN) improved decoding performance compared with the architecture proposed by Cecotti and Graser (2011) and other variants.
“The authors proved that an architecture with the first layer performing a mixed spatio-temporal convolution (OCLNN) improved the decoding performance compared with the architecture proposed by Cecotti and Graser (2011) and other variants (Manor and Geva, 2015; Liu et al., 2018).”
From A Lightweight Multi-Scale Convolutional Neural Network for P300 Decoding: Analysis of Training Strategies and Uncovering of Network Decision (2021) · cites this paper as Manor and Geva, 2015
Results §3.3 reports slightly better P300 detection performance than Cecotti and Gräser (2011). But Methods §2.5 describes the first layer as spatial convolution, with temporal convolutions in later layers—not a mixed spatio-temporal first layer or an OCLNN comparison with other variants. The 2015 paper also cannot report a result by Shan et al. (2018). (LLM, conf=0.88)
Experimental — accuracy not yet measuredThe Borra et al. (2020a) solution outperformed significantly a CNN derived from Manor and Geva (2015) with a spatial convolutional layer as the first layer, long short-term memories, and traditional ML approaches.
“The solution of the authors outperformed significantly a CNN derived from Manor and Geva (2015) with a spatial convolutional layer as the first layer, long short-term memories, and traditional ML approaches (Simões et al., 2020).”
From A Lightweight Multi-Scale Convolutional Neural Network for P300 Decoding: Analysis of Training Strategies and Uncovering of Network Decision (2021) · cites this paper as Manor and Geva, 2015
Prior P300 CNNs learn spatial features (spatial convolution across electrodes) and then temporal features in the next layers (temporal convolution across time samples).
“they learn spatial features (i.e., spatial convolution, performed across electrodes) and then temporal features in the next layers (i.e., temporal convolution, performed across time samples)”
From A Lightweight Multi-Scale Convolutional Neural Network for P300 Decoding: Analysis of Training Strategies and Uncovering of Network Decision (2021) · cites this paper as Manor and Geva, 2015
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371 · see citing works
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480 · see citing works
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
Brain–computer interfaces for communication and control
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00057-3 · Cited by 7,927 · see citing works
Updating P300: An integrative theory of P3a and P3b
Polich, J. (2007) · Clinical Neurophysiology · doi:10.1016/j.clinph.2007.04.019 · Cited by 7,919 · see citing works
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,865 · see citing works
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · see citing works
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,222 · see citing works
Measures of Multivariate Skewness and Kurtosis with Applications
Mardia, K. V. (1970) · Biometrika · doi:10.1093/biomet/57.3.519 · Cited by 4,859 · see citing works
Caffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 4,749 · see citing works
(1989) · neural information processing systems · Cited by 3,736 · see citing works
Electrophysiological Studies of Face Perception in Humans
Shlomo Bentin, A., Allison, T., Puce, A., Perez, E., & McCarthy, G. (1996) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1996.8.6.551 · Cited by 3,081 · see citing works
Advances in Neural Information Processing Systems 37
Touretzky, D. S. (2024) · doi:10.52202/079017 · Cited by 2,874 · see citing works
Some methods of speeding up the convergence of iteration methods
Polyak, B. T. (1964) · USSR Computational Mathematics and Mathematical Physics · doi:10.1016/0041-5553(64)90137-5 · Cited by 2,867 · see citing works
A review of classification algorithms for EEG-based brain–computer interfaces
Lotte, F., Congedo, M., Lécuyer, A., Lamarche, F., & Arnaldi, B. (2007) · Journal of Neural Engineering · doi:10.1088/1741-2560/4/2/r01 · Cited by 2,665 · see citing works
Convolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213 · see citing works
Optimizing Spatial filters for Robust EEG Single-Trial Analysis
Blankertz, B., Tomioka, R., Lemm, S., Kawanabe, M., & Muller, K.-R. (2008) · IEEE Signal Processing Magazine · doi:10.1109/msp.2008.4408441 · Cited by 2,080 · see citing works
Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans
Wolpaw, J. R., & McFarland, D. J. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0403504101 · Cited by 1,512 · see citing works
The mental prosthesis: assessing the speed of a P300-based brain-computer interface
Donchin, E., Spencer, K., & Wijesinghe, R. S. (2000) · IEEE Transactions on Rehabilitation Engineering · doi:10.1109/86.847808 · Cited by 1,217 · see citing works
Event-Related Brain Potentials in Man
Donchin, E., Ritter, W., & McCallum, C. (1978) · Elsevier eBooks · doi:10.1016/b978-0-12-155150-6.x5001-9 · Cited by 956 · see citing works
Designing optimal spatial filters for single-trial EEG classification in a movement task
Müller-Gerking, J., Pfurtscheller, G., & Flyvbjerg, H. (1999) · Clinical Neurophysiology · doi:10.1016/s1388-2457(98)00038-8 · Cited by 846 · see citing works
Convolutional Neural Networks for P300 Detection with Application to Brain-Computer Interfaces
Cecotti, H., & Graser, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.125 · Cited by 778 · see citing works
A P300-based brain–computer interface: Initial tests by ALS patients
Sellers, E. W., & Donchin, E. (2006) · Clinical Neurophysiology · doi:10.1016/j.clinph.2005.06.027 · Cited by 713 · see citing works
COGNITIVE PSYCHOPHYSIOLOGY: THE ENDOGENOUS COMPONENTS OF THE ERP
Donchin, E., Ritter, W., & McCallum, W. C. (1978) · Elsevier eBooks · doi:10.1016/b978-0-12-155150-6.50019-5 · Cited by 681 · see citing works
Blankertz, B., Muller, K.-R., Curio, G., Vaughan, T. M., Schalk, G., Wolpaw, J. R., Schlogl, A., Neuper, C., Pfurtscheller, G., Hinterberger, T., et al. (2004) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2004.826692 · Cited by 670 · see citing works
Pfurtscheller, G., Neuper, C., Schlögl, A., & Lugger, K. (1998) · IEEE Transactions on Rehabilitation Engineering · doi:10.1109/86.712230 · Cited by 553 · see citing works
BCI Competition 2003—Data Set IIb: Support Vector Machines for the P300 Speller Paradigm
Kaper, M., Meinicke, P., Großekathöfer, U., Lingner, T., & Ritter, H. (2004) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2004.826698 · Cited by 496 · see citing works
Müller, K., Tangermann, M., Dornhege, G., Krauledat, M., Curio, G., & Blankertz, B. (2007) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.09.022 · Cited by 475 · see citing works
Linear and nonlinear methods for brain-computer interfaces
Müller, K. R., Anderson, C. W., & Birch, G. E. (2003) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2003.814484 · Cited by 361 · see citing works
Cortically coupled computer vision for rapid image search
Gerson, A. D., Parra, L. C., & Sajda, P. (2006) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2006.875550 · Cited by 258 · see citing works
Invariant Common Spatial Patterns: Alleviating Nonstationarities in Brain-Computer Interfacing
Blankertz, B., Kawanabe, M., Tomioka, R., Hohlefeld, F. U., Nikulin, V. V., & Müller, K. R. (2008) · Cited by 214 · see citing works
Brain Activity-Based Image Classification From Rapid Serial Visual Presentation
Bigdely-Shamlo, N., Vankov, A., Ramirez, R. R., & Makeig, S. (2008) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2008.2003381 · Cited by 177 · see citing works
In a Blink of an Eye and a Switch of a Transistor: Cortically Coupled Computer Vision
Sajda, P., Pohlmeyer, E. A., Wang, J., Parra, L. C., Christoforou, C., Dmochowski, J., Hanna, B. P., Bahlmann, C., Singh, M. K., & Chang, S.-F. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2009.2038406 · Cited by 151 · see citing works
Time–frequency analysis reveals multiple functional components during oddball P300
Kolev, V., Demıralp, T., Yordanova, J., Ademoğlu, A., & İşoĝlu-Alkaç, Ü. (1997) · Neuroreport · doi:10.1097/00001756-199705260-00050 · Cited by 141 · see citing works
Spatiotemporal Linear Decoding of Brain State
Parra, L. C., Christoforou, C., Gerson, A. D., Dyrholm, M., Luo, A., Wagner, M., Philiastides, M. G., & Sajda, P. (2008) · IEEE Signal Processing Magazine · doi:10.1109/msp.2008.4408447 · Cited by 135 · see citing works
Fuhrmann Alpert, G., Manor, R., Spanier, A. B., Deouell, L. Y., & Geva, A. B. (2014) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2013.2289898 · Cited by 74 · see citing works
Self-citationBilinear Discriminant Component Analysis
Dyrholm, M., Christoforou, C., & Parra, L. C. (2007) · doi:10.5555/1248659.1248696 · Cited by 62 · see citing works
Analyzing EEG signals using the probability estimating guarded neural classifier
Felzer, T., & Freisieben, B. (2003) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2003.819785 · Cited by 55 · see citing works
Learning discrimination trajectories in EEG sensor space: application to inferring task difficulty
Luo, A., & Sajda, P. (2005) · Journal of Neural Engineering · doi:10.1088/1741-2560/3/1/l01 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces
Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., & Lance, B. J. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aace8c · Cited by 5,115 · see citing works
“For all ERP-based analyses we compare EEGNet to the convolutional network model by Manor and Geva [32] (see Table 1 of [32]).”
BackgroundMethodInfluentialDeep learning with convolutional neural networks for EEG decoding and visualization
Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., Hutter, F., Burgard, W., & Ball, T. (2017) · Human Brain Mapping · doi:10.1002/hbm.23730 · Cited by 3,751 · see citing works
A survey of deep neural network architectures and their applications
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239 · see citing works
A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update
Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., & Yger, F. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aab2f2 · Cited by 2,239 · see citing works
A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers
Zhang, X., Yao, L., Wang, X., Monaghan, J., McAlpine, D., & Zhang, Y. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/abc902 · Cited by 296 · see citing works
A review of rapid serial visual presentation-based brain–computer interfaces
Lees, S., Dayan, N., Cecotti, H., McCullagh, P., Maguire, L., Lotte, F., & Coyle, D. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aa9817 · Cited by 149 · see citing works
Data Augmentation for Deep Neural Networks Model in EEG Classification Task: A Review
He, C., Liu, J., Zhu, Y., & Du, W. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.765525 · Cited by 105 · see citing works
Simões, M., Borra, D., Santamaría-Vázquez, E., GBT-UPM, Bittencourt-Villalpando, M., Krzemiński, D., Miladinović, A., Neural_Engineering_Group, Schmid, T., Zhao, H., et al. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.568104 · Cited by 72 · see citing works
Borra, D., Fantozzi, S., & Magosso, E. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.655840 · Cited by 36 · see citing works
Single-trial EEG RSVP classification using convolutional neural networks
Shamwell, J., Lee, H., Kwon, H., Marathe, A. R., Lawhern, V., & Nothwang, W. (2016) · SPIE Proceedings · doi:10.1117/12.2224172 · Cited by 34 · see citing works
Santamaría-Vázquez, E., Martínez-Cagigal, V., Gomez-Pilar, J., & Hornero, R. (2020) · IFMBE Proceedings · doi:10.1007/978-3-030-31635-8_224 · Cited by 15 · see citing works
de Arancibia, L., Sánchez-González, P., Gómez, E. J., Hernando, M. E., & Oropesa, I. (2020) · IFMBE Proceedings · doi:10.1007/978-3-030-31635-8_227 · Cited by 11 · see citing works
Wu, H., & Wu, D. (2022) · Brain Science Advances · doi:10.26599/bsa.2022.9050001 · Cited by 7 · see citing works
Zhang, B., Xu, M., Zhang, Y., Ye, S., & Chen, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11040347 · Cited by 4 · see citing works
- Authors
- Manor, R., & Geva, A. B.
- Affiliations
- Ben-Gurion University of the Negev (IL)
- Corresponding author
- Manor, Ran
- ORCID
- Geva, Amir B.
- Venue
- Frontiers in Computational Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2015-12-02
- Volume
- 9
- Pages
- 146-146
- ISSN
- 1662-5188 (ISSN-L)
- PMC
- PMC4667102
- OpenAlex
- W2238628508
- Cited by
- 144
- References
- 43
- Citations by year
- 2026: 7 · 2025: 7 · 2024: 15 · 2023: 19 · 2022: 21 · 2021: 19 · 2020: 14 · 2019: 16 · 2018: 14 · 2017: 9 · 2016: 3
- FWCI
- 5.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Frontiers in Computational Neuroscience (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Keywords
- brain-computer interface, rapid serial visual presentation, single-trial EEG classification, convolutional neural networks, spatio-temporal regularization, overfitting, P300 speller, multi-class classification, feature extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
