Toward a compact hybrid brain-computer interface (BCI): Performance evaluation of multi-class hybrid EEG-fNIRS BCIs with limited number of channels
Open accessThe DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Abstract
It has been demonstrated that the performance of typical unimodal brain-computer interfaces (BCIs) can be noticeably improved by combining two different BCI modalities. This so-called "hybrid BCI" technology has been studied for decades; however, hybrid BCIs that particularly combine electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) (hereafter referred to as hBCIs) have not been widely used in practical settings. One of the main reasons why hBCI systems are so unpopular is that their hardware is generally too bulky and complex. Therefore, to make hBCIs more appealing, it is necessary to implement a lightweight and compact hBCI system with minimal performance degradation. In this study, we investigated the feasibility of implementing a compact hBCI system with significantly less EEG channels and fNIRS source-detector (SD) pairs, but that can achieve a classification accuracy high enough to be used in practical BCI applications. EEG and fNIRS data were acquired while participants performed three different mental tasks consisting of mental arithmetic, right-hand motor imagery, and an idle state. Our analysis results showed that the three mental states could be classified with a fairly high classification accuracy of 77.6 ± 12.1% using an hBCI system with only two EEG channels and two fNIRS SD pairs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig 1 Arrangement of EEG channels (blue) and fNIRS optodes (red: Sources, green: Detectors) on the frontal (left) and motor (right) areas. Channels were numbered in the same manner as in a previous study [20].](https://figures.cochat.ai/paper-figures/81515620525741e0996158567bd74420/toward-a-compact-hybrid-brain-computer-interface-bci-performance-evaluation-of-figure-1.jpg)




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Separately, the feasibility of non-invasive methods for communication brain computer interfaces is being actively investigated77–80, with the goal of improving signal-to-noise and spatial resolution.”
From The speech neuroprosthesis (2024) · cites this paper as [78]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371 · cited 1×
“com/bbci/bbci_public) were employed [22, 23].”
Brain–computer interfaces for communication and control
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00057-3 · Cited by 7,927
A well-conditioned estimator for large-dimensional covariance matrices
Ledoit, O., & Wolf, M. (2004) · Journal of Multivariate Analysis · doi:10.1016/s0047-259x(03)00096-4 · Cited by 3,046 · cited 1×
“The fNIRS data were segmented into epochs from −1 to 15 s considering that the hemodynamic delay is in the order of several seconds [27].”
MethodBrain-computer interface technology: a review of the first international meeting
Wolpaw, J. R., Birbaumer, N., Heetderks, W. J., McFarland, D. J., Peckham, P. H., Schalk, G., Donchin, E., Quatrano, L. A., Robinson, C. J., & Vaughan, T. M. (2000) · IEEE Transactions on Rehabilitation Engineering · doi:10.1109/tre.2000.847807 · Cited by 2,324
Brain Computer Interfaces, a Review
Nicolas-Alonso, L. F., & Gomez-Gil, J. (2012) · Sensors · doi:10.3390/s120201211 · Cited by 2,143
Motor imagery and direct brain-computer communication
Pfurtscheller, G., & Neuper, C. (2001) · Proceedings of the IEEE · doi:10.1109/5.939829 · Cited by 1,990
Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks
Pfurtscheller, G., Brunner, C., Schlögl, A., & Lopes da Silva, F. H. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2005.12.003 · Cited by 1,587
Schäfer, J., & Strimmer, K. (2005) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1175 · Cited by 1,561
Brain–computer interfaces in neurological rehabilitation
Daly, J. J., & Wolpaw, J. R. (2008) · The Lancet Neurology · doi:10.1016/s1474-4422(08)70223-0 · Cited by 1,149
Single-trial analysis and classification of ERP components — A tutorial
Blankertz, B., Lemm, S., Treder, M., Haufe, S., & Müller, K.-R. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.048 · Cited by 1,138
Neuper, C., Scherer, R., Reiner, M., & Pfurtscheller, G. (2005) · Cognitive Brain Research · doi:10.1016/j.cogbrainres.2005.08.014 · Cited by 721 · cited 1×
“Additionally, while the participants were performing kinesthetic motor imagery (not visual motor imagery [21]), they were instructed to try to make the band powers match as closely as possible those observed during actual finger tapping execution.”
BackgroundGalán, F., Nuttin, M., Lew, E., Ferrez, P. W., Vanacker, G., Philips, J., & Millán, J. D. R. (2008) · Clinical Neurophysiology · doi:10.1016/j.clinph.2008.06.001 · Cited by 660
Introduction to machine learning for brain imaging
Lemm, S., Blankertz, B., Dickhaus, T., & Müller, K.-R. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.11.004 · Cited by 630
Enhanced performance by a hybrid NIRS–EEG brain computer interface
Fazli, S., Mehnert, J., Steinbrink, J., Curio, G., Villringer, A., Müller, K.-R., & Blankertz, B. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.07.084 · Cited by 599
Performance Comparison of Several Published Tissue Near-Infrared Spectroscopy Algorithms
Matcher, S. J., Elwell, C. E., Cooper, C. E., Cope, M., & Delpy, D. T. (1995) · Analytical Biochemistry · doi:10.1006/abio.1995.1252 · Cited by 588 · cited 1×
“As for the fNIRS data, the raw optical densities (ODs) were converted to concentration changes of deoxy/oxy-hemoglobin (ΔHbR/ΔHbO) using the following formula [24]:”
MethodTowards a cure for BCI illiteracy
Vidaurre, C., & Blankertz, B. (2010) · Brain Topography · doi:10.1007/s10548-009-0121-6 · Cited by 501 · cited 1×
“(Step 1) We first determined the minimum number of EEG channels (denoted by m) required to achieve an average classification accuracy for MI and IS tasks surpassing the threshold of an effective BCI (70%) [35].”
MethodControl of a humanoid robot by a noninvasive brain–computer interface in humans
Bell, C. J., Shenoy, P., Chalodhorn, R., & Rao, R. P. N. (2008) · Journal of Neural Engineering · doi:10.1088/1741-2560/5/2/012 · Cited by 387
Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface
Wang, Y., Gao, S., & Gao, X. (2005) · 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference · doi:10.1109/iembs.2005.1615701 · Cited by 379 · cited 1×
“[41] reported an average classification accuracy of 92.”
BackgroundOn the suitability of near-infrared (NIR) systems for next-generation brain–computer interfaces
Coyle, S., Ward, T., Markham, C., & McDarby, G. (2004) · Physiological Measurement · doi:10.1088/0967-3334/25/4/003 · Cited by 361
Open Access Dataset for EEG+NIRS Single-Trial Classification
Shin, J., von Luhmann, A., Blankertz, B., Kim, D.-W., Jeong, J., Hwang, H.-J., & Muller, K.-R. (2017) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2016.2628057 · Cited by 287
Self-citationDecoding of four movement directions using hybrid NIRS-EEG brain-computer interface
Khan, M. J., Hong, M. J., & Hong, K.-S. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00244 · Cited by 280 · cited 1×
“[11, 12] demonstrated that hBCIs could achieve a better overall performance compared with unimodal BCIs in terms of classification accuracy and information transfer rate (ITR).”
BackgroundEEG-Based Brain-Computer Interfaces: A Thorough Literature Survey
Hwang, H.-J., Kim, S., Choi, S., & Im, C.-H. (2013) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2013.780869 · Cited by 257
Self-citationHybrid EEG–fNIRS-Based Eight-Command Decoding for BCI: Application to Quadcopter Control
Khan, M. J., & Hong, K.-S. (2017) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2017.00006 · Cited by 227 · cited 1×
“[11, 12] demonstrated that hBCIs could achieve a better overall performance compared with unimodal BCIs in terms of classification accuracy and information transfer rate (ITR).”
BackgroundSimultaneous acquisition of EEG and NIRS during cognitive tasks for an open access dataset
Shin, J., von Lühmann, A., Kim, D.-W., Mehnert, J., Hwang, H.-J., & Müller, K.-R. (2018) · Scientific Data · doi:10.1038/sdata.2018.3 · Cited by 218
Self-citationA Review of Hybrid Brain-Computer Interface Systems
Amiri, S., Fazel-Rezai, R., & Asadpour, V. (2013) · Advances in Human-Computer Interaction · doi:10.1155/2013/187024 · Cited by 209 · cited 1×
“These improvements in performance are thought to originate from the synergetic effect caused by the high temporal resolution of EEG and the relatively lower variability of fNIRS [13, 14].”
BackgroundThe Berlin Brain-Computer Interface: Progress Beyond Communication and Control
Blankertz, B., Acqualagna, L., Dähne, S., Haufe, S., Schultze-Kraft, M., Sturm, I., Ušćumlic, M., Wenzel, M. A., Curio, G., & Müller, K.-R. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00530 · Cited by 204 · cited 1×
“com/bbci/bbci_public) were employed [22, 23].”
Kaiser, V., Bauernfeind, G., Kreilinger, A., Kaufmann, T., Kübler, A., Neuper, C., & Müller-Putz, G. R. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.04.097 · Cited by 192 · cited 1×
“[38] reported an average two-class classification accuracy of 76.”
BackgroundThe MindGame: A P300-based brain–computer interface game
Finke, A., Lenhardt, A., & Ritter, H. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.07.003 · Cited by 175
Moghimi, S., Kushki, A., Marie Guerguerian, A., & Chau, T. (2013) · Assistive Technology · doi:10.1080/10400435.2012.723298 · Cited by 166
Yin, X., Xu, B., Jiang, C., Fu, Y., Wang, Z., Li, H., & Shi, G. (2015) · Journal of Neural Engineering · doi:10.1088/1741-2560/12/3/036004 · Cited by 157 · cited 2×
“[14] achieved an average classification accuracy of 89.”
BackgroundTanida, M., Katsuyama, M., & Sakatani, K. (2007) · Brain Research · doi:10.1016/j.brainres.2007.09.058 · Cited by 144 · cited 1×
“The optimal SD pairs were mostly located in the left prefrontal area, which is in line with the important role of the dorsolateral prefrontal cortex in cognitive task performance and previous studies reporting that brain activity in the left prefrontal area is closely associated with mental arithmetic tasks [37].”
ResultExperiencing BCI Control in a Popular Computer Game
van de Laar, B., Gürkök, H., Bos, D. P.-O., Poel, M., & Nijholt, A. (2013) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2013.2253778 · Cited by 143
Bimodal BCI Using Simultaneously NIRS and EEG
Tomita, Y., Vialatte, F.-B., Dreyfus, G., Mitsukura, Y., Bakardjian, H., & Cichocki, A. (2014) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2014.2300492 · Cited by 111
Shin, J., Kwon, J., & Im, C. (2018) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2018.00005 · Cited by 102 · cited 7×
“This approach using multiple windows is known to result in higher classification accuracy than using a single temporal window [20, 26].”
BackgroundMethodResultInfluentialSelf-citationA Brain-Computer Interface Based on a Few-Channel EEG-fNIRS Bimodal System
Ge, S., Yang, Q., Wang, R., Lin, P., Gao, J., Leng, Y., Yang, Y., & Wang, H. (2017) · IEEE Access · doi:10.1109/access.2016.2637409 · Cited by 89 · cited 1×
“In a recent study, an hBCI system with reduced numbers of sensors, namely three EEG channels and 12 fNIRS optodes, was implemented [39].”
MethodKhan, B. W., Wildey, C., Francis, R., Tian, F., Delgado, M. R., Liu, H., MacFarlane, D. L., & Alexandrakis, G. (2012) · Biomedical Optics Express · doi:10.1364/boe.3.000878 · Cited by 81 · cited 1×
“inevitable for acquiring high-quality signals free from interference originating from hairs (especially dark dense hair) unless certain specialized equipment is employed [40].”
BackgroundHan, C. H., Kim, Y.-W., Kim, D. Y., Kim, S. H., Nenadić, Z., & Im, C. (2019) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-019-0493-0 · Cited by 78 · cited 1×
“[4] implemented an electroencephalography (EEG)-based BCI system for online binary communication with a patient in completely locked-in state, and demonstrated high online classification accuracy of 87.”
MethodSelf-citationHwang, H., Choi, H. N., Kim, J.-Y., Chang, W.-D., Kim, D.-W., Kim, K., Jo, S., & Im, C. (2016) · Journal of Biomedical Optics · doi:10.1117/1.jbo.21.9.091303 · Cited by 64
Self-citationFriedrich, E. V. C., Scherer, R., & Neuper, C. (2012) · Clinical Neurophysiology · doi:10.1016/j.clinph.2012.05.020 · Cited by 47 · cited 1×
“The multiple passbands were chosen considering the participantdependency of the task-relevant frequency band, as reported by a previous study [25].”
Shin, J., Kwon, J., Choi, J., & Im, C. (2018) · IEEE Access · doi:10.1109/access.2018.2822238 · Cited by 24
Self-citation
Research works printed but not linked: 1
- M Tomczak, 2014, The need to report effect size estimates revisited. An overview of some recommended measures of effect size, Trends Sport Sci, 21
Cited by
Concurrent fNIRS and EEG for Brain Function Investigation: A Systematic, Methodology-Focused Review
Li, R., Yang, D., Fang, F., Hong, K.-S., Reiss, A. L., & Zhang, Y. (2022) · Sensors · doi:10.3390/s22155865 · Cited by 236
A systematic review on hybrid EEG/fNIRS in brain-computer interface
Liu, Z., Shore, J., Wang, M., Yuan, F., Buss, A. T., & Zhao, X. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.102595 · Cited by 167
Silva, A. B., Littlejohn, K. T., Liu, J. R., Moses, D. A., & Chang, E. F. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-024-00819-9 · Cited by 103
Analysis of Human Gait Using Hybrid EEG-fNIRS-Based BCI System: A Review
Khan, H., Naseer, N., Yazidi, A., Eide, P. K., Hassan, H. W., & Mirtaheri, P. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2020.613254 · Cited by 106
Yang, D., Shin, Y., & Hong, K. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.629323 · Cited by 94
“In addition, hybrid neuroimaging modalities have been suggested to evaluate the neuromodulation effect to seek reliable results (Kwon et al., 2020; Lee S. et al., 2020).”
Background
Metadata
- Authors
- Kwon, J., Shin, J., & Im, C.-H.
- Affiliations
- Hanyang University (KR)
- Wonkwang University (KR)
- Corresponding author
- Im, Chang-Hwan
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-03-18
- Volume
- 15
- Issue
- 3
- Pages
- e0230491
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC7080269
- OpenAlex
- W3010955800
- Cited by
- 53
- References
- 41
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 11 · 2023: 7 · 2022: 5 · 2021: 15 · 2020: 4
- FWCI
- 4.17
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- National Research Foundation
- National Research Foundation of Korea (2017-0-00432, 2015M3C7A1031969, NRF‐2015M3C7A1031969, 2019M3C7A1031278, NRF-2019M3C7A1031278)
- Ministry of Science and ICT, South Korea (NRF-2019M3C7A1031278, 2017-0-00432)
- Institute for Information and Communications Technology Promotion (2017-0-00432)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Optical Imaging and Spectroscopy Techniques
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- hybrid brain-computer interface, functional near-infrared spectroscopy, EEG, multi-class classification, mental arithmetic, motor imagery, mental state classification
- MeSH terms
- Adult, Electroencephalography, Female, Humans, Male, Models, Theoretical, Motor Cortex, Psychomotor Performance, Spectroscopy, Near-Infrared, Young Adult, Brain-Computer Interfaces
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.