Multimodal BCIs: Target Detection, Multidimensional Control, and Awareness Evaluation in Patients With Disorder of Consciousness
The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Abstract
Despite rapid advances in the study of brain–computer interfaces (BCIs) in recent decades, two fundamental challenges, namely, improvement of target detection performance and multidimensional control, continue to be major barriers for further development and applications. In this paper, we review the recent progress in multimodal BCIs (also called hybrid BCIs), which may provide potential solutions for addressing these challenges. In particular, improved target detection can be achieved by developing multimodal BCIs that utilize multiple brain patterns, multimodal signals, or multisensory stimuli. Furthermore, multidimensional object control can be accomplished by generating multiple control signals from different brain patterns or signal modalities. Here, we highlight several representative multimodal BCI systems by analyzing their paradigm designs, detection/control methods, and experimental results. To demonstrate their practicality, we report several initial clinical applications of these multimodal BCI systems, including awareness evaluation/detection in patients with disorder of consciousness (DOC). As an evolving research area, the study of multimodal BCIs is increasingly requiring more synergetic efforts from multiple disciplines for the exploration of the underlying brain mechanisms, the design of new effective paradigms and means of neurofeedback, and the expansion of the clinical applications of these systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It should be noted that SVM classifier was widely used in the EEG-based brain computer interface in practice [26–29].”
From Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition (2017) · cites this paper as [27]
Citations
Cites
Brain–computer interfaces for communication and control
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00057-3 · Cited by 7,927 · cited 1×
“A brain–computer interface (BCI) system directly translates the signals produced by brain activities into control signals and uses these signals to control external devices without the participation of peripheral nerves and muscles [1].”
BackgroundEvent-related EEG/MEG synchronization and desynchronization: basic principles
Pfurtscheller, G., & Lopes da Silva, F. H. (1999) · Clinical Neurophysiology · doi:10.1016/s1388-2457(99)00141-8 · Cited by 7,328 · cited 1×
“The brain patterns that are utilized for EEG-based BCIs mainly include the P300 potential [5]; steady-state evoked potentials (SSEP), particularly steady-state visual evoked potentials (SSVEPs) [6]; and event-related desynchronization/synchronization (ERD/ERS) produced by motor imagery (MI) [7].”
BackgroundFarwell, L. A., & Donchin, E. (1988) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(88)90149-6 · Cited by 3,228 · cited 1×
“The brain patterns that are utilized for EEG-based BCIs mainly include the P300 potential [5]; steady-state evoked potentials (SSEP), particularly steady-state visual evoked potentials (SSVEPs) [6]; and event-related desynchronization/synchronization (ERD/ERS) produced by motor imagery (MI) [7].”
BackgroundFeigenson, L., Dehaene, S., & Spelke, E. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.05.002 · Cited by 2,466
The minimally conscious state: Definition and diagnostic criteria
Giacino, J. T., Ashwal, S., Childs, N., Cranford, R., Jennett, B., Katz, D. I., Kelly, J. P., Rosenberg, J. H., Whyte, J., Zafonte, R. D., et al. (2002) · Neurology · doi:10.1212/wnl.58.3.349 · Cited by 2,422
A spelling device for the paralysed
Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Taub, E., & Flor, H. (1999) · Nature · doi:10.1038/18581 · Cited by 1,577 · cited 1×
“For instance, BCIs can offer augmented or repaired sensory-motor functions and may appeal to individuals with severe motor disabilities [2].”
BackgroundControl of a two-dimensional movement signal by a noninvasive brain-computer interface in humans
Wolpaw, J. R., & McFarland, D. J. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0403504101 · Cited by 1,512 · cited 3×
“Similar to the system presented in [18], the target in this system is also confined to one of eight candidates.”
BackgroundMethodResultWillful Modulation of Brain Activity in Disorders of Consciousness
Monti, M. M., Vanhaudenhuyse, A., Coleman, M. R., Boly, M., Pickard, J. D., Tshibanda, L., Owen, A. M., & Laureys, S. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa0905370 · Cited by 1,461
Schnakers, C., Vanhaudenhuyse, A., Giacino, J., Ventura, M., Boly, M., Majerus, S., Moonen, G., & Laureys, S. (2009) · BMC Neurology · doi:10.1186/1471-2377-9-35 · Cited by 1,348 · cited 1×
“For instance, the misdiagnosis rates for VS/UWS and MCS patients are rather high, ranging from 37% to 43% [64].”
BackgroundPERSISTENT VEGETATIVE STATE AFTER BRAIN DAMAGE
Jennett, B., & Plum, F. (1972) · The Lancet · doi:10.1016/s0140-6736(72)90242-5 · Cited by 1,227 · cited 1×
“be awake but show no awareness of themselves or their environment [62].”
Background
Cited by
A Benchmark Dataset for SSVEP-Based Brain–Computer Interfaces
Wang, Y., Chen, X., Gao, X., & Gao, S. (2017) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2016.2627556 · Cited by 501
“, P300) have become more and more popular in recent studies [10]–[13].”
BackgroundEEG Based Emotion Recognition by Combining Functional Connectivity Network and Local Activations
Li, P., Liu, H., Si, Y., Li, C., Li, F., Zhu, X., Huang, X., Zeng, Y., Yao, D., Zhang, Y., et al. (2019) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2019.2897651 · Cited by 396
“tion methods may lead to some differences in optimal feature subsets because of their different design purposes [34], [38], [39], [46], [64], which may finally account for the different”
MethodMulti-kernel extreme learning machine for EEG classification in brain-computer interfaces
Zhang, Y., Wang, Y., Zhou, G., Jin, J., Wang, B., Wang, X., & Cichocki, A. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.12.015 · Cited by 255
Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces
He, Z., Li, Z., Yang, F., Wang, L., Li, J., Zhou, C., & Pan, J. (2020) · Brain Sciences · doi:10.3390/brainsci10100687 · Cited by 184
“[9] proposed that in addition to combining different input signals, emotion recognition should include a variety of heterogeneous sensory stimuli (such as audio-visual stimulation) to induce emotions.”
BackgroundSelf-citationLi, F., Wang, J., Liao, Y., Yi, C., Jiang, Y., Si, Y., Peng, W., Yao, D., Zhang, Y., Dong, W., et al. (2019) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2019.2900725 · Cited by 124
Metadata
- Authors
- Li, Y., Pan, J., Long, J., Yu, T., Wang, F., Yu, Z., & Wu, W.
- Affiliations
- South China Normal University (CN)
- Venue
- Proceedings of the IEEE
- Published
- 2016-02
- Volume
- 104
- Issue
- 2
- Pages
- 332-352
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W2247338144
- Cited by
- 117
- References
- 81
- Citations by year
- 2026: 6 · 2025: 7 · 2024: 8 · 2023: 8 · 2022: 16 · 2021: 11 · 2020: 10 · 2019: 17 · 2018: 14 · 2017: 14 · 2016: 6
- FWCI
- 6.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the IEEE (publishedVersion)
- Funders
- National High-Tech R&D Program of China (2012AA011601)
- National Key Basic Research Program of China (2015CB351703)
- National Natural Science Foundation of China (91120305, 91420302, 61175114, 61503143)
- Guangdong Natural Science Foundation (2014A030312005, 2014A030310244)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Social Sciences › Arts and Humanities › Philosophy › Free Will and Agency
- Keywords
- hybrid BCI, target detection, multidimensional control, disorder of consciousness, multisensory stimuli, neurofeedback
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multimodal BCIs: Target Detection, Multidimensional Control, and Awareness Evaluation in Patients With Disorder of Consciousness. https://app.cochat.ai/papers/10.1109/jproc.2015.2469106
- The paper
- Li, Y., Pan, J., Long, J., Yu, T., Wang, F., Yu, Z., & Wu, W. (2016). Multimodal BCIs: Target Detection, Multidimensional Control, and Awareness Evaluation in Patients With Disorder of Consciousness. Proceedings of the IEEE. https://doi.org/10.1109/jproc.2015.2469106
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.