An accurate and rapidly calibrating speech neuroprosthesis
Open accessPreprint — published as 10.1056/nejmoa2314132 · Paper page
The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Abstract
Brain-computer interfaces can enable rapid, intuitive communication for people with paralysis by transforming the cortical activity associated with attempted speech into text on a computer screen. Despite recent advances, communication with brain-computer interfaces has been restricted by extensive training data requirements and inaccurate word output. A man in his 40’s with ALS with tetraparesis and severe dysarthria (ALSFRS-R = 23) was enrolled into the BrainGate2 clinical trial. He underwent surgical implantation of four microelectrode arrays into his left precentral gyrus, which recorded neural activity from 256 intracortical electrodes. We report a speech neuroprosthesis that decoded his neural activity as he attempted to speak in both prompted and unstructured conversational settings. Decoded words were displayed on a screen, then vocalized using text-to-speech software designed to sound like his pre-ALS voice. On the first day of system use, following 30 minutes of attempted speech training data, the neuroprosthesis achieved 99.6% accuracy with a 50-word vocabulary. On the second day, the size of the possible output vocabulary increased to 125,000 words, and, after 1.4 additional hours of training data, the neuroprosthesis achieved 90.2% accuracy. With further training data, the neuroprosthesis sustained 97.5% accuracy beyond eight months after surgical implantation. The participant has used the neuroprosthesis to communicate in self-paced conversations for over 248 hours. In an individual with ALS and severe dysarthria, an intracortical speech neuroprosthesis reached a level of performance suitable to restore naturalistic communication after a brief training period.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Subsequent work has expanded this initial demonstration by directly decoding cortical activity into audible speech13,14 and allowing more generalizable and rapid text decoding14–16 in individuals with vocal-tract paralysis (Fig. 1).”
From The speech neuroprosthesis (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current speech-decoding applications from brain activity are primarily aimed at restoring communication to individuals with both limb and vocal-tract paralysis14–16.”
From The speech neuroprosthesis (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent studies have also achieved high-performance speech-decoding results15,16 by measuring relevant single-unit and multi-unit neuronal activity from multiple small (4 × 4 mm2 (ref. 83)) cortical regions on the anterior precentral gyrus15,16,52,98.”
From The speech neuroprosthesis (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent works have achieved large-vocabulary decoding while maintaining decoding speeds closer to natural speech, by using CTC loss.”
From The speech neuroprosthesis (2024) · cites this paper as Card et al. 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A second approach, specific to text decoding, involves leveraging large language models to automatically recalibrate MEA-based text neuroprostheses.”
From The speech neuroprosthesis (2024) · cites this paper as [16]
Experimental — accuracy not yet measured
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations
Valiev, M., Bylaska, E. J., Govind, N., Kowalski, K., Straatsma, T. P., Van Dam, H. J. J., Wang, D., Nieplocha, J., Apra, E., Windus, T. L., et al. (2010) · Computer Physics Communications · doi:10.1016/j.cpc.2010.04.018 · Cited by 5,779
A multi-modal parcellation of human cerebral cortex
Glasser, M. F., Coalson, T. S., Robinson, E. C., Hacker, C. D., Harwell, J., Yacoub, E., Ugurbil, K., Andersson, J., Beckmann, C. F., Jenkinson, M., et al. (2016) · Nature · doi:10.1038/nature18933 · Cited by 5,111 · cited 2×
“…four 64-electrode Utah arrays chronically implanted in the precentral gyrus, targeted to brain areas that contributed most to speech decoding from recent studies 16,18,20 using the Human Connectome Project’s multi-modal MRI-derived cortical parcellation precisely mapped to SP2’s brain 21 (Fig.”
BackgroundMethodSelf-citationUnbiased average age-appropriate atlases for pediatric studies
Fonov, V., Evans, A. C., Botteron, K., Almli, C. R., McKinstry, R. C., & Collins, D. L. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.07.033 · Cited by 2,290
SWITCHBOARD: telephone speech corpus for research and development
Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992) · [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1992.225858 · Cited by 2,134 · cited 4×
“We used neural activity collected during the speech tasks to train a recurrent neural network (RNN, Section S2) to predict the probability of each English phoneme being spoken.”
MethodInfluentialUnbiased nonlinear average age-appropriate brain templates from birth to adulthood
Fonov, V., Evans, A., McKinstry, R., Almli, C., & Collins, D. (2009) · NeuroImage · doi:10.1016/s1053-8119(09)70884-5 · Cited by 2,102
Salimi-Khorshidi, G., Douaud, G., Beckmann, C. F., Glasser, M. F., Griffanti, L., & Smith, S. M. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.11.046 · Cited by 1,770
Self-citationHigh-performance brain-to-text communication via handwriting
Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M., & Shenoy, K. V. (2021) · Nature · doi:10.1038/s41586-021-03506-2 · Cited by 1,024
Self-citationSpeech synthesis from neural decoding of spoken sentences
Anumanchipalli, G. K., Chartier, J., & Chang, E. F. (2019) · Nature · doi:10.1038/s41586-019-1119-1 · Cited by 883
A brain-computer interface that evokes tactile sensations improves robotic arm control
Flesher, S. N., Downey, J. E., Weiss, J. M., Hughes, C. L., Herrera, A. J., Tyler-Kabara, E. C., Boninger, M. L., Collinger, J. L., & Gaunt, R. A. (2021) · Science · doi:10.1126/science.abd0380 · Cited by 617
Harms, M. P., Somerville, L. H., Ances, B. M., Andersson, J., Barch, D. M., Bastiani, M., Bookheimer, S. Y., Brown, T. B., Buckner, R. L., Burgess, G. C., et al. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.09.060 · Cited by 584
Self-citationNeuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria
Moses, D. A., Metzger, S. L., Liu, J. R., Anumanchipalli, G. K., Makin, J. G., Sun, P. F., Chartier, J., Dougherty, M. E., Liu, P. M., Abrams, G. M., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2027540 · Cited by 582 · cited 9×
“We used two metrics to analyze the speech decoding performance: phoneme error rate (PER) and word error rate (WER), consistent with previous speech decoding studies 16,18,20 .”
BackgroundMethodInfluentialA high-performance speech neuroprosthesis
Willett, F. R., Kunz, E. M., Fan, C., Avansino, D. T., Wilson, G. H., Choi, E. Y., Kamdar, F., Glasser, M. F., Hochberg, L. R., Druckmann, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06377-x · Cited by 581
Self-citationPandarinath, C., Nuyujukian, P., Blabe, C. H., Sorice, B. L., Saab, J., Willett, F. R., Hochberg, L. R., Shenoy, K. V., & Henderson, J. M. (2017) · eLife · doi:10.7554/elife.18554 · Cited by 566
Self-citationWodlinger, B., Downey, J. E., Tyler-Kabara, E. C., Schwartz, A. B., Boninger, M. L., & Collinger, J. L. (2015) · Journal of Neural Engineering · doi:10.1088/1741-2560/12/1/016011 · Cited by 536
A high-performance neuroprosthesis for speech decoding and avatar control
Metzger, S. L., Littlejohn, K. T., Silva, A. B., Moses, D. A., Seaton, M. P., Wang, R., Dougherty, M. E., Liu, J. R., Wu, P., Berger, M. A., et al. (2023) · Nature · doi:10.1038/s41586-023-06443-4 · Cited by 522 · cited 10×
“Two recent studies have established the state-of-the-art for ‘brain-to-text’ speech BCIs 18,20 by decoding the neural underpinnings of attempted speech into phonemes (the building blocks of words), which are then assembled into words and sentences using a language model and displayed on a computer…”
BackgroundMethodInfluentialSimeral, J. D., Kim, S.-P., Black, M. J., Donoghue, J. P., & Hochberg, L. R. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/8/2/025027 · Cited by 496
Self-citationThe Pile: An 800GB Dataset of Diverse Text for Language Modeling
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492
Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01116 · Cited by 478
Chestek, C. A., Gilja, V., Nuyujukian, P., Foster, J. D., Fan, J. M., Kaufman, M. T., Churchland, M. M., Rivera-Alvidrez, Z., Cunningham, J. P., Ryu, S. I., et al. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/8/4/045005 · Cited by 350
Jarosiewicz, B., Sarma, A. A., Bacher, D., Masse, N. Y., Simeral, J. D., Sorice, B., Oakley, E. M., Blabe, C., Pandarinath, C., Gilja, V., et al. (2015) · Science Translational Medicine · doi:10.1126/scitranslmed.aac7328 · Cited by 340
Self-citationMultimodal surface matching with higher-order smoothness constraints
Robinson, E. C., Garcia, K., Glasser, M. F., Chen, Z., Coalson, T. S., Makropoulos, A., Bozek, J., Wright, R., Schuh, A., Webster, M., et al. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.10.037 · Cited by 324
Self-citationA Wireless Brain-Machine Interface for Real-Time Speech Synthesis
Guenther, F. H., Brumberg, J. S., Wright, E. J., Nieto-Castanon, A., Tourville, J. A., Panko, M., Law, R., Siebert, S. A., Bartels, J. L., Andreasen, D. S., et al. (2009) · PLoS ONE · doi:10.1371/journal.pone.0008218 · Cited by 306 · cited 2×
“We reduced the WER to 2.5% by session 15, and 1% by session 17.”
BackgroundInfluentialBrain-to-text: decoding spoken phrases from phone representations in the brain
Herff, C., Heger, D., de Pesters, A., Telaar, D., Brunner, P., Schalk, G., & Schultz, T. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00217 · Cited by 297
A high-performance speech neuroprosthesisPreprint
Willett, F. R., Kunz, E. M., Fan, C., et al. (2023) · bioRxiv · doi:10.1101/2023.01.21.524489 · Cited by 284 · cited 10×
“Two recent studies have established the state-of-the-art for ‘brain-to-text’ speech BCIs 18,20 by decoding the neural underpinnings of attempted speech into phonemes (the building blocks of words), which are then assembled into words and sentences using a language model and displayed on a computer…”
BackgroundMethodResultInfluentialSelf-citationGlasser, M. F., Coalson, T. S., Bijsterbosch, J. D., Harrison, S. J., Harms, M. P., Anticevic, A., Van Essen, D. C., & Smith, S. M. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.04.076 · Cited by 257
Self-citationAccurate Estimation of Neural Population Dynamics without Spike Sorting
Trautmann, E. M., Stavisky, S. D., Lahiri, S., Ames, K. C., Kaufman, M. T., O’Shea, D. J., Vyas, S., Sun, X., Ryu, S. I., Ganguli, S., et al. (2019) · Neuron · doi:10.1016/j.neuron.2019.05.003 · Cited by 255
Self-citationPerge, J. A., Homer, M. L., Malik, W. Q., Cash, S., Eskandar, E., Friehs, G., Donoghue, J. P., & Hochberg, L. R. (2013) · Journal of Neural Engineering · doi:10.1088/1741-2560/10/3/036004 · Cited by 243 · cited 1×
“Previous studies have reported that intracortical devices often require recalibration due to signal nonstationarities 28–30 .”
BackgroundSelf-citationA Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement
Valin, J.-M. (2018) · 2018 IEEE 20th International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2018.8547084 · Cited by 241
Decoding spoken words using local field potentials recorded from the cortical surface
Kellis, S., Miller, K., Thomson, K., Brown, R., House, P., & Greger, B. (2010) · Journal of Neural Engineering · doi:10.1088/1741-2560/7/5/056007 · Cited by 235
Real-time decoding of question-and-answer speech dialogue using human cortical activity
Moses, D. A., Leonard, M. K., Makin, J. G., & Chang, E. F. (2019) · Nature Communications · doi:10.1038/s41467-019-10994-4 · Cited by 224
Life can be worth living in locked-in syndrome
Lulé, D., Zickler, C., Häcker, S., Bruno, M. A., Demertzi, A., Pellas, F., Laureys, S., & Kübler, A. (2009) · Progress in Brain Research · doi:10.1016/s0079-6123(09)17723-3 · Cited by 217 · cited 1×
“As communication fails, people report increased rates of isolation, depression, and decreased quality of life 2,3 ; losing communication often determines if a person will pursue or withdraw life-sustaining care in advanced ALS.”
BackgroundMaking brain–machine interfaces robust to future neural variability
Sussillo, D., Stavisky, S. D., Kao, J. C., Ryu, S. I., & Shenoy, K. V. (2016) · Nature Communications · doi:10.1038/ncomms13749 · Cited by 213 · cited 1×
“Here, adopting the recent recognition that recent days' neural data can be used to calibrate an effective neural decoder for a new day 31,32 , we demonstrated that a speech decoder could similarly be used to provide >95% accuracy at the start of each session.”
MethodSelf-citationMugler, E. M., Patton, J. L., Flint, R. D., Wright, Z. A., Schuele, S. U., Rosenow, J., Shih, J. J., Krusienski, D. J., & Slutzky, M. W. (2014) · Journal of Neural Engineering · doi:10.1088/1741-2560/11/3/035015 · Cited by 192
Hand Knob Area of Premotor Cortex Represents the Whole Body in a Compositional Way
Willett, F. R., Deo, D. R., Avansino, D. T., Rezaii, P., Hochberg, L. R., Henderson, J. M., & Shenoy, K. V. (2020) · Cell · doi:10.1016/j.cell.2020.02.043 · Cited by 171
Self-citationRapid calibration of an intracortical brain–computer interface for people with tetraplegia
Brandman, D. M., Hosman, T., Saab, J., Burkhart, M. C., Shanahan, B. E., Ciancibello, J. G., Sarma, A. A., Milstein, D. J., Vargas-Irwin, C. E., Franco, B., et al. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aa9ee7 · Cited by 153 · cited 1×
“Rapid communication with an intracortical speech BCI builds on our previous demonstration of rapid point-and-click communication with first-time BCI users 27 .”
BackgroundSelf-citationMetzger, S. L., Liu, J. R., Moses, D. A., Dougherty, M. E., Seaton, M. P., Littlejohn, K. T., Chartier, J., Anumanchipalli, G. K., Tu-Chan, A., Ganguly, K., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33611-3 · Cited by 153 · cited 1×
“We reduced the WER to 2.5% by session 15, and 1% by session 17.”
BackgroundInfluentialIntracortical recording stability in human brain–computer interface users
Downey, J. E., Schwed, N., Chase, S. M., Schwartz, A. B., & Collinger, J. L. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aab7a0 · Cited by 149 · cited 1×
“Previous studies have reported that intracortical devices often require recalibration due to signal nonstationarities 28–30 .”
BackgroundCommunication Status and Survival with Ventilatory Support
Bach, J. R. (1993) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/00002060-199312000-00002 · Cited by 130
Neural Point-and-Click Communication by a Person With Incomplete Locked-In Syndrome
Bacher, D., Jarosiewicz, B., Masse, N. Y., Stavisky, S. D., Simeral, J. D., Newell, K., Oakley, E. M., Cash, S. S., Friehs, G., & Hochberg, L. R. (2015) · Neurorehabilitation and Neural Repair · doi:10.1177/1545968314554624 · Cited by 124
Self-citationDecoding spoken phonemes from sensorimotor cortex with high-density ECoG grids
Ramsey, N. F., Salari, E., Aarnoutse, E. J., Vansteensel, M. J., Bleichner, M. G., & Freudenburg, Z. V. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.10.011 · Cited by 124
A protocol for identification of early bulbar signs in amyotrophic lateral sclerosis
Ball, L. J., Willis, A., Beukelman, D. R., & Pattee, G. L. (2001) · Journal of the Neurological Sciences · doi:10.1016/s0022-510x(01)00623-2 · Cited by 115
A Neurosurgical Functional Dissection of the Middle Precentral Gyrus during Speech Production
Silva, A. B., Liu, J. R., Zhao, L., Levy, D. F., Scott, T. L., & Chang, E. F. (2022) · The Journal of Neuroscience · doi:10.1523/jneurosci.1614-22.2022 · Cited by 111 · cited 1×
“The array in area 4 also showed high phoneme encoding, as did the array in area 55b, which has recently been proposed as an important node in the wider speech production network 33 .”
BackgroundKoch Fager, S., Fried-Oken, M., Jakobs, T., & Beukelman, D. R. (2019) · Augmentative and Alternative Communication · doi:10.1080/07434618.2018.1556730 · Cited by 107 · cited 1×
“Existing augmentative and assistive communication technologies such as eye trackers suffer from low information transfer rates and become increasingly less reliable and more onerous for patients as they lose voluntary motor control 4 .”
BackgroundBrain-Computer Interface: Applications to Speech Decoding and Synthesis to Augment Communication
Luo, S., Rabbani, Q., & Crone, N. E. (2022) · Neurotherapeutics · doi:10.1007/s13311-022-01190-2 · Cited by 106 · cited 1×
“Brain-computer interfaces (BCIs) are a promising assistive technology to meet patients’ fundamental need for fast and effortless communication by bypassing the damaged parts of the nervous system and directly decoding their intended speech from neural measurements (reviewed in 5 ).”
BackgroundNeural ensemble dynamics in dorsal motor cortex during speech in people with paralysis
Stavisky, S. D., Willett, F. R., Wilson, G. H., Murphy, B. A., Rezaii, P., Avansino, D. T., Memberg, W. D., Miller, J. P., Kirsch, R. F., Hochberg, L. R., et al. (2019) · eLife · doi:10.7554/elife.46015 · Cited by 105
Self-citationChristie, B. P., Tat, D. M., Irwin, Z. T., Gilja, V., Nuyujukian, P., Foster, J. D., Ryu, S. I., Shenoy, K. V., Thompson, D. E., & Chestek, C. A. (2015) · Journal of Neural Engineering · doi:10.1088/1741-2560/12/1/016009 · Cited by 103
Angrick, M., Luo, S., Rabbani, Q., Candrea, D. N., Shah, S., Milsap, G. W., Anderson, W. S., Gordon, C. R., Rosenblatt, K. R., Clawson, L., et al. (2023) · doi:10.1101/2023.06.30.23291352 · Cited by 95 · cited 2×
“While the demonstrated brain-to-text capabilities can provide widely useful communication, they do not capture the full expressive richness of voice; the more difficult challenge of closed-loop brain-to-voice synthesis remains an active area of speech BCI research 18,34,40 .”
BackgroundLuo, S., Angrick, M., Coogan, C., Candrea, D. N., Wyse‐Sookoo, K., Shah, S., Rabbani, Q., Milsap, G. W., Weiss, A. R., Anderson, W. S., et al. (2023) · Advanced Science · doi:10.1002/advs.202304853 · Cited by 90 · cited 1×
“Several groups have now started closed-loop speech BCI studies specifically to restore lost speech using chronically implanted electrocorticography (ECoG) 16–19 and intracortical multielectrode arrays 20 .”
BackgroundInterim Safety Profile From the Feasibility Study of the BrainGate Neural Interface System
Rubin, D. B., Ajiboye, A. B., Barefoot, L., Bowker, M., Cash, S. S., Chen, D., Donoghue, J. P., Eskandar, E. N., Friehs, G., Grant, C., et al. (2023) · Neurology · doi:10.1212/wnl.0000000000201707 · Cited by 81 · cited 1×
“…that we recorded from ventral precentral gyrus, which is upstream of the neuronal injury incurred in many conditions, and that recent ECoG speech neuroprostheses were demonstrated in two individuals with brainstem stroke 16–18 , we predict that this approach will also work in other conditions 35 .”
BackgroundSelf-citationKatz, R. T., Haig, A. J., Clark, B. B., & Paola, R. D. (1992) · PubMed · Cited by 79
Long-term survival, prognosis, and life-care planning for 29 patients with chronic locked-in syndrome.
Katz, R. T., Haig, A., Clark, B., et al. (1992) · Archives of Physical Medicine and Rehabilitation · doi:10.5555/uri:pii:000399939290025r · Cited by 77 · cited 1×
“As communication fails, people report increased rates of isolation, depression, and decreased quality of life 2,3 ; losing communication often determines if a person will pursue or withdraw life-sustaining care in advanced ALS.”
BackgroundThomson, D. R., Besner, D., & Smilek, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00360 · Cited by 76
Sponheim, C., Papadourakis, V., Collinger, J. L., Downey, J., Weiss, J., Pentousi, L., Elliott, K., & Hatsopoulos, N. G. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac3eaf · Cited by 66
Young, D., Willett, F., Memberg, W. D., Murphy, B., Walter, B., Sweet, J., Miller, J., Hochberg, L. R., Kirsch, R. F., & Ajiboye, A. B. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aa9ee8 · Cited by 61
Self-citationRoussel, P., Godais, G. L., Bocquelet, F., Palma, M., Hongjie, J., Zhang, S., Giraud, A.-L., Mégevand, P., Miller, K., Gehrig, J., et al. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abb25e · Cited by 49
Direct speech reconstruction from sensorimotor brain activity with optimized deep learning models
Berezutskaya, J., Freudenburg, Z. V., Vansteensel, M. J., Aarnoutse, E. J., Ramsey, N. F., & van Gerven, M. A. J. (2023) · Journal of Neural Engineering · doi:10.1088/1741-2552/ace8be · Cited by 42 · cited 1×
“14 , but see 15 ).”
On the Limit of English Conversational Speech Recognition
Tüske, Z., Saon, G., & Kingsbury, B. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-211 · Cited by 41
BRAND: a platform for closed-loop experiments with deep network models
Ali, Y. H., Bodkin, K., Rigotti-Thompson, M., Patel, K., Card, N. S., Bhaduri, B., Nason-Tomaszewski, S. R., Mifsud, D. M., Hou, X., Nicolas, C., et al. (2024) · Journal of Neural Engineering · doi:10.1088/1741-2552/ad3b3a · Cited by 32 · cited 1×
“A signal processing system (NeuroPort System, Blackrock Neurotech) was used to acquire signals from the 256 implanted electrodes and transmit them to a computer running custom software 22 (Section S1.”
MethodSelf-citationDemonstration of a portable intracortical brain-computer interface
Weiss, J. M., Gaunt, R. A., Franklin, R., Boninger, M. L., & Collinger, J. L. (2019) · Brain-Computer Interfaces · doi:10.1080/2326263x.2019.1709260 · Cited by 32
Speech-related dorsal motor cortex activity does not interfere with iBCI cursor control
Stavisky, S. D., Willett, F. R., Avansino, D. T., Hochberg, L. R., Shenoy, K. V., & Henderson, J. M. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab5b72 · Cited by 25
Self-citationHosman, T., Pun, T. K., Kapitonava, A., Simeral, J. D., & Hochberg, L. R. (2023) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner52421.2023.10123740 · Cited by 23 · cited 1×
“Here, adopting the recent recognition that recent days' neural data can be used to calibrate an effective neural decoder for a new day 31,32 , we demonstrated that a speech decoder could similarly be used to provide >95% accuracy at the start of each session.”
MethodSelf-citationSynthesizing Speech by Decoding Intracortical Neural Activity from Dorsal Motor Cortex
Wairagkar, M., Hochberg, L. R., Brandman, D. M., & Stavisky, S. D. (2023) · 2023 11th International IEEE/EMBS Conference on Neural Engineering (NER) · doi:10.1109/ner52421.2023.10123880 · Cited by 22 · cited 1×
“While the demonstrated brain-to-text capabilities can provide widely useful communication, they do not capture the full expressive richness of voice; the more difficult challenge of closed-loop brain-to-voice synthesis remains an active area of speech BCI research 18,34,40 .”
BackgroundSelf-citationFan, C., Hahn, N., Kamdar, F., Avansino, D., Wilson, G. H., Hochberg, L., Shenoy, K. V., Henderson, J. M., & Willett, F. R. (2023) · arXiv · doi:10.48550/arxiv.2311.03611 · Cited by 19 · cited 4×
“Decoding performance continued to improve in subsequent research sessions as we collected more training data, optimized algorithm hyperparameters, added online decoder fine-tuning 23 , and expanded the training dataset to include personal use data.”
BackgroundMethodInfluentialSelf-citationFan, C., Hahn, N. V., Kamdar, F. B., Avansino, D. T., Wilson, G. H., Hochberg, L. R., Shenoy, K. V., Henderson, J. M., & Willett, F. R. (2023) · PubMed · Cited by 17
Self-citation(1969) · Anaesthesia · doi:10.1111/j.1365-2044.1969.tb02863.x · Cited by 1
Research works printed but not linked: 10
- 2023, Interim Safety Profile From the Feasibility Study of the BrainGate Neural Interface System, Neurology, 100, e1177
- 2002, Timing of speech deterioration in people with amyotrophic lateral sclerosis, J Med Speech-Lang Pathol, 10, 231
- Glasser MF, Coalson TS, Robinson EC, A multi-modal parcellation of human cerebral cortex. Nature 2016; 536(7615): 171– 8. 27437579 PMC4990127 · DOI · Paper page
- 10.1056/nejmoa2027540 · DOI · Paper page
- Metzger SL, Liu JR, Moses DA, Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis. Nat Commun 2022; 13(1): 6510. 36347863 PMC9643551 · DOI · Paper page
- Metzger SL, Littlejohn KT, Silva AB, A high-performance neuroprosthesis for speech decoding and avatar control. Nature 2023; 620(7976): 1037– 46. 37612505 PMC10826467 · DOI · Paper page
- Silva AB, Liu JR, Zhao L, Levy DF, Scott TL, Chang EF. A Neurosurgical Functional Dissection of the Middle Precentral Gyrus during Speech Production. J Neurosci 2022; 42(45): 8416– 26. 36351829 PMC9665919 · DOI · Paper page
- Ali YH, Bodkin K, Rigotti-Thompson M, BRAND: A platform for closed-loop experiments with deep network models [Internet]. 2023 [ cited 2023 Dec 11];2023.08.08.552473. Available from: 10.1101/2023.08.08.552473v1 PMC11021878 38579696 · doi:10.1101/2023.08.08.552473v1
- Godfrey JJ, Holliman EC, McDaniel J. SWITCHBOARD: telephone speech corpus for research and development [Internet]. In: [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing. 1992 [ cited 2023 Dec 11]. p. 517– 20 vol. 1. Available from: https://ieeexplore.ieee.org/document/225858 · DOI · Paper page
- Sussillo D, Stavisky SD, Kao JC, Ryu SI, Shenoy KV. Making brain–machine interfaces robust to future neural variability. Nat Commun 2016; 7(1): 13749. 27958268 PMC5159828 · DOI · Paper page
Cited by
Recent applications of EEG-based brain-computer-interface in the medical field
Liu, X., Wang, W.-L., Liu, M., Chen, M., Pereira, T., Doda, D. Y., Ke, Y., Wang, S., Dong, W., Tong, X., et al. (2025) · Military Medical Research · doi:10.1186/s40779-025-00598-z · Cited by 171
“Card et al. [172] recently reported a study involving a 45-year-old patient with ALS.”
Silva, A. B., Littlejohn, K. T., Liu, J. R., Moses, D. A., & Chang, E. F. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-024-00819-9 · Cited by 103
“In this approach, MEA activity is decoded into subword units and converted into sentences using language models; this language model output is used as a ‘pseudo-label’ to update the RNN decoding model 9,16,105 .”
InfluentialA streaming brain-to-voice neuroprosthesis to restore naturalistic communication
Littlejohn, K. T., Cho, C. J., Liu, J. R., Silva, A. B., Yu, B., Anderson, V. R., Kurtz-Miott, C. M., Brosler, S., Kashyap, A., Hallinan, I. P., et al. (2025) · Nature Neuroscience · doi:10.1038/s41593-025-01905-6 · Cited by 133
“The neural data generated from these attempts were used to train the decoding model 21,22,24–27 , which may prove challenging to adapt to someone who has difficulty with or has lost this ability.”
InfluentialInner speech in motor cortex and implications for speech neuroprostheses
Kunz, E. M., Abramovich Krasa, B., Kamdar, F., Avansino, D. T., Hahn, N., Yoon, S., Singh, A., Nason-Tomaszewski, S. R., Card, N. S., Jude, J. J., et al. (2025) · Cell · doi:10.1016/j.cell.2025.06.015 · Cited by 55
“All LLM-gener-ated code was verified by researchers.”
InfluentialSelf-citationAn instantaneous voice synthesis neuroprosthesisPreprint
Wairagkar, M., Card, N., Singer-Clark, T. S., et al. (2024) · bioRxiv · doi:10.1101/2024.08.14.607690 · Cited by 48
“We also demonstrated that this brain-to-voice speech neuroprosthesis could be paired with our previously-reported high accuracy brain-to-text decoder 1 , which essentially acted as closed-captioning ( Video 3 ).”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Card, N. S., Wairagkar, M., Iacobacci, C., Hou, X., Singer-Clark, T., Willett, F. R., Kunz, E. M., Fan, C., Vahdati Nia, M., Deo, D. R., Srinivasan, A., Choi, E. Y., Glasser, M. F., Hochberg, L. R., Henderson, J. M., Shahlaie, K., Brandman, D. M., & Stavisky, S. D.
- Affiliations
- Howard Hughes Medical Institute (US)
- ORCID
- Venue
- New England Journal of Medicine
- Publisher
- openRxiv
- Published
- 2023-12-26
- Volume
- 391
- Issue
- 7
- Pages
- 609-618
- ISSN
- 0028-4793 (ISSN-L)
- PMC
- PMC11030484
- OpenAlex
- W4401560881
- Semantic Scholar
- 54934763d368ce6497b5ed690a5a9cc8ca88b2e3
- Cited by
- 244
- References
- 39
- Citations by year
- 2026: 105 · 2025: 109 · 2024: 29 · 2023: 1
- FWCI
- 59.04
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- New England Journal of Medicine (publishedVersion)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- PubMed Central (submittedVersion, cc-by)
- Digital Commons@Becker (Washington University School of Medicine) (submittedVersion)
- Funders
- U.S. Department of Defense
- Howard Hughes Medical Institute
- U.S. Department of Veterans Affairs (I01 RX002295, A2295R)
- Burroughs Wellcome Fund
- Simons Foundation (872146SPI)
- A.P. Giannini Foundation
- Massachusetts General Hospital
- Giannini Foundation of Agricultural Economics
- Searle Scholars Program
- National Institutes of Health (5u01dc017844-06)
- Rehabilitation Research and Development Service
- University of California, Davis
- Wu Tsai Neurosciences Institute, Stanford University
- National Institute of Mental Health (R01MH060974)
- NIH Office of the Director
- National Institute on Deafness and Other Communication Disorders (U01DC17844, U01DC019430, DP2DC021055, U01- DC017844)
- Congressionally Directed Medical Research Programs (AL220043)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neural Engineering
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- brain-computer interface, amyotrophic lateral sclerosis, tetraparesis, intracortical microelectrode arrays, text-to-speech, conversational communication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.