Performance measurement for brain–computer or brain–machine interfaces: a tutorial
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Brain-computer interfaces (BCIs) have the potential to be valuable clinical tools. However, the varied nature of BCIs, combined with the large number of laboratories participating in BCI research, makes uniform performance reporting difficult. To address this situation, we present a tutorial on performance measurement in BCI research. APPROACH: A workshop on this topic was held at the 2013 International BCI Meeting at Asilomar Conference Center in Pacific Grove, California. This paper contains the consensus opinion of the workshop members, refined through discussion in the following months and the input of authors who were unable to attend the workshop. MAIN RESULTS: Checklists for methods reporting were developed for both discrete and continuous BCIs. Relevant metrics are reviewed for different types of BCI research, with notes on their use to encourage uniform application between laboratories. SIGNIFICANCE: Graduate students and other researchers new to BCI research may find this tutorial a helpful introduction to performance measurement in the field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Brain–computer interfaces for communication and control
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00057-3 · Cited by 7,927 · cited 1×
“movement [1].”
Neuronal ensemble control of prosthetic devices by a human with tetraplegia
Hochberg, L. R., Serruya, M. D., Friehs, G. M., Mukand, J. A., Saleh, M., Caplan, A. H., Branner, A., Chen, D., Penn, R. D., & Donoghue, J. P. (2006) · Nature · doi:10.1038/nature04970 · Cited by 3,295
Reach and grasp by people with tetraplegia using a neurally controlled robotic arm
Hochberg, L. R., Bacher, D., Jarosiewicz, B., Masse, N. Y., Simeral, J. D., Vogel, J., Haddadin, S., Liu, J., Cash, S. S., van der Smagt, P., et al. (2012) · Nature · doi:10.1038/nature11076 · Cited by 2,719
Self-citationLearning to control a brain–machine interface for reaching and grasping by primates
Carmena, J. M., Lebedev, M. A., Crist, R. E., O'Doherty, J. E., Santucci, D. M., Dimitrov, D. F., Patil, P. G., Henriquez, C. S., & Nicolelis, M. A. L. (2003) · PLoS Biology · doi:10.1371/journal.pbio.0000042 · Cited by 1,761
Cortical control of a prosthetic arm for self-feeding
Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., & Schwartz, A. B. (2008) · Nature · doi:10.1038/nature06996 · Cited by 1,591 · cited 1×
“An emerging task is control of a prosthetic arm with multiple degrees of freedom [3, 56, 57].”
BackgroundControl of a two-dimensional movement signal by a noninvasive brain-computer interface in humans
Wolpaw, J. R., & McFarland, D. J. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0403504101 · Cited by 1,512
Emergence of a stable cortical map for neuroprosthetic control
Ganguly, K., & Carmena, J. M. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000153 · Cited by 578 · cited 1×
“A typical task is ‘center out’ cursor movement acquiring radial targets or targets distributed through the workspace [4, 31, 32, 49–54].”
BackgroundSimeral, J. D., Kim, S.-P., Black, M. J., Donoghue, J. P., & Hochberg, L. R. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/8/2/025027 · Cited by 496
Self-citationAccuracy measures for evaluating computer pointing devices
MacKenzie, I. S., Kauppinen, T., & Silfverberg, M. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365028 · Cited by 448
A Hybrid BCI System Combining P300 and SSVEP and Its Application to Wheelchair Control
Li, Y., Pan, J., Wang, F., & Yu, Z. (2013) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2013.2270283 · Cited by 421 · cited 1×
“The applications are similarly varied, including both communication and control of devices such as virtual keyboard [18–20], prostheses [21, 22], wheelchairs [23–26], or environmental controls [27, 28].”
Background
Cited by
Brain computer interface: control signals review
Ramadan, R. A., & Vasilakos, A. V. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.10.024 · Cited by 551
Jarosiewicz, B., Sarma, A. A., Bacher, D., Masse, N. Y., Simeral, J. D., Sorice, B., Oakley, E. M., Blabe, C., Pandarinath, C., Gilja, V., et al. (2015) · Science Translational Medicine · doi:10.1126/scitranslmed.aac7328 · Cited by 340
“In the radial keyboard, in which all of the possible targets have the same size, CSPM was also translated into eBR, the number of bits of information conveyed per second (17, 32) extrapolated to a virtual keyboard.”
BackgroundMethodSelf-citationErrare machinale est: the use of error-related potentials in brain-machine interfaces
Chavarriaga, R., Sobolewski, A., & Millán, J. D. R. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00208 · Cited by 311
Brain-Computer Interface: Advancement and Challenges
Mridha, M. F., Das, S. C., Kabir, M. M., Lima, A. A., Islam, M. R., & Watanobe, Y. (2021) · Sensors · doi:10.3390/s21175746 · Cited by 263
Nason, S. R., Vaskov, A. K., Willsey, M. S., Welle, E. J., An, H., Vu, P. P., Bullard, A. J., Nu, C. S., Kao, J. C., Shenoy, K. V., et al. (2020) · Nature Biomedical Engineering · doi:10.1038/s41551-020-0591-0 · Cited by 113
Self-citation
Metadata
- Authors
- Thompson, D. E., Quitadamo, L. R., Mainardi, L., Laghari, K. U. R., Gao, S., Kindermans, P.-J., Simeral, J. D., Fazel-Rezai, R., Matteucci, M., Falk, T. H., Bianchi, L., Chestek, C. A., & Huggins, J. E.
- Venue
- Journal of Neural Engineering
- Publisher
- IOP Publishing
- Published
- 2014-05-19
- Volume
- 11
- Issue
- 3
- Pages
- 035001
- ISSN
- 1741-2552 (ISSN-L)
- 1741-2560
- OpenAlex
- W1980047894
- Cited by
- 87
- References
- 13
- FWCI
- 4.94
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Performance measurement for brain–computer or brain–machine interfaces: a tutorial. https://app.cochat.ai/papers/10.1088/1741-2560/11/3/035001
- The paper
- Thompson, D. E., Quitadamo, L. R., Mainardi, L., Laghari, K. U. R., Gao, S., Kindermans, P.-J., Simeral, J. D., Fazel-Rezai, R., Matteucci, M., Falk, T. H., Bianchi, L., Chestek, C. A., & Huggins, J. E. (2014). Performance measurement for brain–computer or brain–machine interfaces: a tutorial. Journal of Neural Engineering. https://doi.org/10.1088/1741-2560/11/3/035001
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.