Brain-Computer Interface: Applications to Speech Decoding and Synthesis to Augment Communication
Open accessAbstract
Damage or degeneration of motor pathways necessary for speech and other movements, as in brainstem strokes or amyotrophic lateral sclerosis (ALS), can interfere with efficient communication without affecting brain structures responsible for language or cognition. In the worst-case scenario, this can result in the locked in syndrome (LIS), a condition in which individuals cannot initiate communication and can only express themselves by answering yes/no questions with eye blinks or other rudimentary movements. Existing augmentative and alternative communication (AAC) devices that rely on eye tracking can improve the quality of life for people with this condition, but brain-computer interfaces (BCIs) are also increasingly being investigated as AAC devices, particularly when eye tracking is too slow or unreliable. Moreover, with recent and ongoing advances in machine learning and neural recording technologies, BCIs may offer the only means to go beyond cursor control and text generation on a computer, to allow real-time synthesis of speech, which would arguably offer the most efficient and expressive channel for communication. The potential for BCI speech synthesis has only recently been realized because of seminal studies of the neuroanatomical and neurophysiological underpinnings of speech production using intracranial electrocorticographic (ECoG) recordings in patients undergoing epilepsy surgery. These studies have shown that cortical areas responsible for vocalization and articulation are distributed over a large area of ventral sensorimotor cortex, and that it is possible to decode speech and reconstruct its acoustics from ECoG if these areas are recorded with sufficiently dense and comprehensive electrode arrays. In this article, we review these advances, including the latest neural decoding strategies that range from deep learning models to the direct concatenation of speech units. We also discuss state-of-the-art vocoders that are integral in constructing natural-sounding audio waveforms for speech BCIs. Finally, this review outlines some of the challenges ahead in directly synthesizing speech for patients with LIS.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Brain-computer interfaces can enable communication for people with paralysis by transforming cortical activity associated with attempted speech into text on a computer screen.
“Brain-computer interfaces can enable communication for people with paralysis by transforming cortical activity associated with attempted speech into text on a computer screen.”
From An Accurate and Rapidly Calibrating Speech Neuroprosthesis (2024) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · see citing works
Brain–computer interfaces for communication and control
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00057-3 · Cited by 7,927 · see citing works
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,882 · see citing works
Classification of primary progressive aphasia and its variants
Gorno-Tempini, M. L., Hillis, A. E., Weintraub, S., Kertesz, A., Mendez, M., Cappa, S. F., Ogar, J. M., Rohrer, J. D., Black, S., Boeve, B. F., et al. (2011) · Neurology · doi:10.1212/wnl.0b013e31821103e6 · Cited by 5,141 · see citing works
Language: An Introduction to the Study of Speech
Bloomfield, L., & Sapir, E. (1922) · The Classical World · doi:10.2307/4388302 · Cited by 3,204 · see citing works
Signal estimation from modified short-time Fourier transform
Griffin, D. W., & Lim, J. S. (1983) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1983.1172092 · Cited by 2,253 · see citing works
Qualitative Assessment of Verbal Fluency Performance in Frontotemporal Dementia
van den Berg, E., Jiskoot, L. C., Grosveld, M. J. H., Swieten, J. C. V., & Papma, J. M. (2017) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000477538 · Cited by 2,061 · see citing works
Unit selection in a concatenative speech synthesis system using a large speech database
Hunt, A., & Black, A. (1996) · 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings · doi:10.1109/icassp.1996.541110 · Cited by 1,508 · see citing works
An Integrated Brain-Machine Interface Platform With Thousands of Channels
Musk, E., & Neuralink (2019) · Journal of Medical Internet Research · doi:10.2196/16194 · Cited by 1,206 · see citing works
“Waveglow: a flow-based generative network for speech synthesis,”
Prenger, R., Valle, R., & Catanzaro, B. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683143 · Cited by 1,004 · see citing works
High-performance brain-to-text communication via handwriting
Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M., & Shenoy, K. V. (2021) · Nature · doi:10.1038/s41586-021-03506-2 · Cited by 969 · cited 1× · see citing works
“Significant advances have been made to assist these individuals through the use of other types of BCIs, including those using P300 [3], motor imagery [4], handwriting [5], and steady-state visually evoked potential [6].”
BackgroundSpeech synthesis from neural decoding of spoken sentences
Anumanchipalli, G. K., Chartier, J., & Chang, E. F. (2019) · Nature · doi:10.1038/s41586-019-1119-1 · Cited by 883 · see citing works
Ray, S., Crone, N. E., Niebur, E., Franaszczuk, P. J., & Hsiao, S. S. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2848-08.2008 · Cited by 722 · see citing works
Self-citationNeuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria
Moses, D. A., Metzger, S. L., Liu, J. R., Anumanchipalli, G. K., Makin, J. G., Sun, P. F., Chartier, J., Dougherty, M. E., Liu, P. M., Abrams, G. M., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2027540 · Cited by 582 · see citing works
Fully Implanted Brain–Computer Interface in a Locked-In Patient with ALS
Vansteensel, M. J., Pels, E. G. M., Bleichner, M. G., Branco, M. P., Denison, T., Freudenburg, Z. V., Gosselaar, P., Leinders, S., Ottens, T. H., Van Den Boom, M. A., et al. (2016) · New England Journal of Medicine · doi:10.1056/nejmoa1608085 · Cited by 578 · see citing works
Pandarinath, C., Nuyujukian, P., Blabe, C. H., Sorice, B. L., Saab, J., Willett, F. R., Hochberg, L. R., Shenoy, K. V., & Henderson, J. M. (2017) · eLife · doi:10.7554/elife.18554 · Cited by 566 · cited 1× · see citing works
“1 3 minute in conversational speech is more than 7 times that of eye tracking and 10 times of handwriting [7, 8].”
BackgroundAutomatic speech recognition and speech variability: A review
Benzeghiba, M., De Mori, R., Deroo, O., Dupont, S., Erbes, T., Jouvet, D., Fissore, L., Laface, P., Mertins, A., Ris, C., et al. (2007) · Speech Communication · doi:10.1016/j.specom.2007.02.006 · Cited by 529 · see citing works
High-frequency gamma oscillations and human brain mapping with electrocorticography
Crone, N. E., Sinai, A., & Korzeniewska, A. (2006) · Progress in Brain Research · doi:10.1016/s0079-6123(06)59019-3 · Cited by 526 · see citing works
Self-citationVarieties of the locked-in syndrome
Bauer, G., Gerstenbrand, F., & Rumpl, E. (1979) · Journal of Neurology · doi:10.1007/bf00313105 · Cited by 504 · see citing works
Smith, E., & Delargy, M. (2005) · BMJ · doi:10.1136/bmj.330.7488.406 · Cited by 437 · see citing works
LPCNET: Improving Neural Speech Synthesis through Linear Prediction
Valin, J.-M., & Skoglund, J. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682804 · Cited by 427 · see citing works
SampleRNN: An Unconditional End-to-End Neural Audio Generation ModelPreprint
Mehri, S., Kumar, K., Gulrajani, I., Kumar, R., Jain, S., Sotelo, J., Courville, A. C., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.07837 · Cited by 414 · see citing works
Benabid, A. L., Costecalde, T., Eliseyev, A., Charvet, G., Verney, A., Karakas, S., Foerster, M., Lambert, A., Morinière, B., Abroug, N., et al. (2019) · The Lancet Neurology · doi:10.1016/s1474-4422(19)30321-7 · Cited by 404 · see citing works
Long-term asynchronous decoding of arm motion using electrocorticographic signals in monkey
Chao (2010) · Frontiers in Neuroengineering · doi:10.3389/fneng.2010.00003 · Cited by 337 · see citing works
Research on speech motor control and its disorders
Kent, R. D. (2000) · Journal of Communication Disorders · doi:10.1016/s0021-9924(00)00023-x · Cited by 333 · see citing works
Machine translation of cortical activity to text with an encoder–decoder framework
Makin, J. G., Moses, D. A., & Chang, E. F. (2020) · Nature Neuroscience · doi:10.1038/s41593-020-0608-8 · Cited by 321 · see citing works
A Wireless Brain-Machine Interface for Real-Time Speech Synthesis
Guenther, F. H., Brumberg, J. S., Wright, E. J., Nieto-Castanon, A., Tourville, J. A., Panko, M., Law, R., Siebert, S. A., Bartels, J. L., Andreasen, D. S., et al. (2009) · PLoS ONE · doi:10.1371/journal.pone.0008218 · Cited by 306 · see citing works
Kübler, A., Furdea, A., Halder, S., Hammer, E. M., Nijboer, F., & Kotchoubey, B. (2009) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2008.04122.x · Cited by 304 · see citing works
Brain-to-text: decoding spoken phrases from phone representations in the brain
Herff, C., Heger, D., de Pesters, A., Telaar, D., Brunner, P., Schalk, G., & Schultz, T. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00217 · Cited by 297 · see citing works
Automatically clustering similar units for unit selection in speech synthesis
Black, A. W., & Taylor, P. (1997) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1997-219 · Cited by 286 · see citing works
Brain–computer interfaces for speech communication
Brumberg, J. S., Nieto-Castañón, A., Kennedy, P. R., & Guenther, F. H. (2010) · Speech Communication · doi:10.1016/j.specom.2010.01.001 · Cited by 267 · see citing works
Mental imagery of speech and movement implicates the dynamics of internal forward models
Tian, X. (2010) · Frontiers in Psychology · doi:10.3389/fpsyg.2010.00166 · Cited by 259 · see citing works
Pei, X., Barbour, D. L., Leuthardt, E. C., & Schalk, G. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/8/4/046028 · Cited by 237 · see citing works
Decoding spoken words using local field potentials recorded from the cortical surface
Kellis, S., Miller, K., Thomson, K., Brown, R., House, P., & Greger, B. (2010) · Journal of Neural Engineering · doi:10.1088/1741-2560/7/5/056007 · Cited by 235 · see citing works
Speech synthesis from ECoG using densely connected 3D convolutional neural networks
Angrick, M., Herff, C., Mugler, E., Tate, M. C., Slutzky, M. W., Krusienski, D. J., & Schultz, T. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab0c59 · Cited by 225 · see citing works
Real-time decoding of question-and-answer speech dialogue using human cortical activity
Moses, D. A., Leonard, M. K., Makin, J. G., & Chang, E. F. (2019) · Nature Communications · doi:10.1038/s41467-019-10994-4 · Cited by 224 · see citing works
Towards reconstructing intelligible speech from the human auditory cortex
Akbari, H., Khalighinejad, B., Herrero, J. L., Mehta, A. D., & Mesgarani, N. (2019) · Scientific Reports · doi:10.1038/s41598-018-37359-z · Cited by 221 · see citing works
Cortical gamma responses: Searching high and low
Crone, N. E., Korzeniewska, A., & Franaszczuk, P. J. (2011) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2010.10.013 · Cited by 201 · see citing works
Self-citationDecoding spectrotemporal features of overt and covert speech from the human cortex
Martin, S., Brunner, P., Holdgraf, C., Heinze, H.-J., Crone, N. E., Rieger, J., Schalk, G., Knight, R. T., & Pasley, B. N. (2014) · Frontiers in Neuroengineering · doi:10.3389/fneng.2014.00014 · Cited by 200 · see citing works
Self-citationEffects of postlingual deafness on speech production: Implications for the role of auditory feedback
Waldstein, R. S. (1990) · The Journal of the Acoustical Society of America · doi:10.1121/1.400107 · Cited by 200 · see citing works
Perkell, J. S., Guenther, F. H., Lane, H. L., Matthies, M. L., Perrier, P., Vick, J. C., Wilhelms-Tricarico, R. F., & Zandipour, M. (2000) · Journal of Phonetics · doi:10.1006/jpho.2000.0116 · Cited by 192 · see citing works
Mugler, E. M., Patton, J. L., Flint, R. D., Wright, Z. A., Schuele, S. U., Rosenow, J., Shih, J. J., Krusienski, D. J., & Slutzky, M. W. (2014) · Journal of Neural Engineering · doi:10.1088/1741-2560/11/3/035015 · Cited by 192 · see citing works
Survey: The locked-in syndrome: a syndrome looking for a therapy
León‐Carrión, J., van Eeckhout, P., Domínguez-Morales, M. D. R., & Pérez-Santamaría, F. J. (2002) · Brain Injury · doi:10.1080/02699050110119781 · Cited by 171 · see citing works
Word pair classification during imagined speech using direct brain recordings
Martin, S., Brunner, P., Iturrate, I., Millán, J. D. R., Schalk, G., Knight, R. T., & Pasley, B. N. (2016) · Scientific Reports · doi:10.1038/srep25803 · Cited by 171 · see citing works
Optimal spacing of surface electrode arrays for brain–machine interface applications
Slutzky, M. W., Jordan, L. R., Krieg, T. D., Chen, M., Mogul, D. J., & Miller, L. E. (2010) · Journal of Neural Engineering · doi:10.1088/1741-2560/7/2/026004 · Cited by 161 · see citing works
Perraudin, N., Balázs, P., & Søndergaard, P. L. (2013) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2013.6701851 · Cited by 152 · see citing works
Effect of delayed auditory feedback on normal speakers at two speech rates
Stuart, A., Kalinowski, J. S., Rastatter, M. P., & Lynch, K. (2002) · The Journal of the Acoustical Society of America · doi:10.1121/1.1466868 · Cited by 145 · see citing works
Ray, S., Hsiao, S. S., Crone, N. E., Franaszczuk, P. J., & Niebur, E. (2008) · Journal of Neuroscience · doi:10.1523/jneurosci.1588-08.2008 · Cited by 141 · see citing works
Self-citationA review of electrodes for the electrical brain signal recording
Im, C., & Seo, J.-M. (2016) · Biomedical Engineering Letters · doi:10.1007/s13534-016-0235-1 · Cited by 141 · see citing works
Stuttering amelioration at various auditory feedback delays and speech rates
Kalinowski, J. S., Stuart, A., Sark, S. A., et al. (1996) · International Journal of Language & Communication Disorders · doi:10.3109/13682829609033157 · Cited by 135 · see citing works
Altered auditory feedback and the treatment of stuttering: A review
Lincoln, M., Packman, A., & Onslow, M. (2006) · Journal of Fluency Disorders · doi:10.1016/j.jfludis.2006.04.001 · Cited by 132 · see citing works
An independent SSVEP-based brain–computer interface in locked-in syndrome
Lesenfants, D., Habbal, D., Lugo, Z., LeBeau, M., Horki, P., Amico, E., Pokorny, C., Gómez, F. A., Soddu, A., Muller-Putz, G. R., et al. (2014) · Journal of Neural Engineering · doi:10.1088/1741-2560/11/3/035002 · Cited by 127 · see citing works
Plug-and-play control of a brain–computer interface through neural map stabilization
Silversmith, D. B., Abiri, R., Hardy, N. F., Natraj, N., Tu-Chan, A., Chang, E. F., & Ganguly, K. (2021) · Nature Biotechnology · doi:10.1038/s41587-020-0662-5 · Cited by 126 · see citing works
Angrick, M., Ottenhoff, M. C., Diener, L., Ivucic, D., Ivucic, G., Goulis, S., Saal, J., Colon, A. J., Wagner, L., Krusienski, D. J., et al. (2021) · Communications Biology · doi:10.1038/s42003-021-02578-0 · Cited by 125 · see citing works
Decoding spoken phonemes from sensorimotor cortex with high-density ECoG grids
Ramsey, N. F., Salari, E., Aarnoutse, E. J., Vansteensel, M. J., Bleichner, M. G., & Freudenburg, Z. V. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.10.011 · Cited by 124 · see citing works
Herff, C., Diener, L., Angrick, M., Mugler, E., Tate, M. C., Goldrick, M. A., Krusienski, D. J., Slutzky, M. W., & Schultz, T. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.01267 · Cited by 118 · see citing works
Decoding Inner Speech Using Electrocorticography: Progress and Challenges Toward a Speech Prosthesis
Martin, S., Iturrate, I., Millán, J. D. R., Knight, R. T., & Pasley, B. N. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00422 · Cited by 115 · see citing works
Herff, C., Krusienski, D. J., & Kubben, P. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00123 · Cited by 115 · see citing works
The Potential for a Speech Brain–Computer Interface Using Chronic Electrocorticography
Rabbani, Q., Milsap, G., & Crone, N. E. (2019) · Neurotherapeutics · doi:10.1007/s13311-018-00692-2 · Cited by 106 · see citing works
Self-citationNeural ensemble dynamics in dorsal motor cortex during speech in people with paralysis
Stavisky, S. D., Willett, F. R., Wilson, G. H., Murphy, B. A., Rezaii, P., Avansino, D. T., Memberg, W. D., Miller, J. P., Kirsch, R. F., Hochberg, L. R., et al. (2019) · eLife · doi:10.7554/elife.46015 · Cited by 105 · see citing works
Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus
Wilson, G. H., Stavisky, S. D., Willett, F. R., Avansino, D. T., Kelemen, J. N., Hochberg, L. R., Henderson, J. M., Druckmann, S., & Shenoy, K. V. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abbfef · Cited by 103 · see citing works
Chen, Y., Shimotake, A., Matsumoto, R., Kunieda, T., Kikuchi, T., Miyamoto, S., Fukuyama, H., Takahashi, R., Ikeda, A., & Ralph, M. A. L. (2016) · Cortex · doi:10.1016/j.cortex.2016.02.015 · Cited by 96 · see citing works
Mugler, E. M., Tate, M. C., Livescu, K., Templer, J. W., Goldrick, M. A., & Slutzky, M. W. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.1206-18.2018 · Cited by 96 · see citing works
Pels, E., Aarnoutse, E. J., Leinders, S., Freudenburg, Z. V., Branco, M. P., van der Vijgh, B. H., Snijders, T. J., Denison, T., Vansteensel, M. J., & Ramsey, N. F. (2019) · Clinical Neurophysiology · doi:10.1016/j.clinph.2019.07.020 · Cited by 95 · see citing works
MacKay, D. G. (1968) · The Journal of the Acoustical Society of America · doi:10.1121/1.1910900 · Cited by 94 · see citing works
Degenhart, A. D., Eles, J. R., Dum, R. P., Mischel, J., Smalianchuk, I., Endler, B., Ashmore, R. C., Tyler‐Kabara, E. C., Hatsopoulos, N. G., Wang, W., et al. (2016) · Journal of Neural Engineering · doi:10.1088/1741-2560/13/4/046019 · Cited by 92 · see citing works
Moses, D. A., Mesgarani, N., Leonard, M. K., & Chang, E. F. (2016) · Journal of Neural Engineering · doi:10.1088/1741-2560/13/5/056004 · Cited by 92 · see citing works
Blakely, T. M., Miller, K. J., Rao, R. P. N., Holmes, M., & Ojemann, J. G. (2008) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2008.4650328 · Cited by 89 · see citing works
Electrocorticographic representations of segmental features in continuous speech
Lotte, F., Brumberg, J. S., Brunner, P., Gunduz, A., Ritaccio, A. L., Guan, C., & Schalk, G. (2015) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2015.00097 · Cited by 86 · see citing works
Role of the auditory system in speech production
Guenther, F. H., & Hickok, G. S. (2015) · Handbook of clinical neurology · doi:10.1016/b978-0-444-62630-1.00009-3 · Cited by 84 · see citing works
Caldwell, D. J., Ojemann, J. G., & Rao, R. P. N. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00804 · Cited by 81 · see citing works
Real-Time Control of an Articulatory-Based Speech Synthesizer for Brain Computer Interfaces
Bocquelet, F., Hueber, T., Laurent Girin, B., Savariaux, C., & Yvert, B. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005119 · Cited by 77 · see citing works
Brain2Char: a deep architecture for decoding text from brain recordings
Sun, P., Anumanchipalli, G. K., & Chang, E. F. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abc742 · Cited by 77 · see citing works
Review of subject: The locked-in syndrome: a syndrome looking for a therapy
León‐Carrión, J., van Eeckhout, P., & Domínguez-Morales, M. D. R. (2002) · Brain Injury · doi:10.1080/02699050110119466 · Cited by 74 · see citing works
Wang, X., Lorenzo-Trueba, J., Takaki, S., Juvela, L., & Yamagishi, J. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461452 · Cited by 71 · see citing works
Structured neuronal encoding and decoding of human speech features
Tankus, A., Fried, I., & Shoham, S. (2012) · Nature Communications · doi:10.1038/ncomms1995 · Cited by 69 · see citing works
A study of speech deterioration in post-lingually deafened adults
Cowie, R., Douglas‐Cowie, E., & Kerr, A. G. (1982) · The Journal of Laryngology & Otology · doi:10.1017/s002221510009229x · Cited by 64 · see citing works
Digital Voicing of Silent Speech
Gaddy, D., & Klein, D. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.445 · Cited by 60 · see citing works
On the non-uniqueness of the inverse MEG problem
Dassios, G., FOKAS, A. S., & Kariotou, F. (2005) · Inverse Problems · doi:10.1088/0266-5611/21/2/l01 · Cited by 56 · see citing works
Control of breathing in man; insights from the ‘locked-in’ syndrome
Heywood, P. G., Murphy, K., Corfield, D. R., Morrell, M. J., Howard, R., & Guz, A. (1996) · Respiration Physiology · doi:10.1016/0034-5687(96)00060-6 · Cited by 53 · see citing works
Neural decoding of single vowels during covert articulation using electrocorticography
Ikeda, S., Shibata, T., Nakano, N., Okada, R., Tsuyuguchi, N., Ikeda, K., & Kato, A. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00125 · Cited by 52 · see citing works
Effect of Altered Auditory Feedback on People Who Stutter During Scripted Telephone Conversations
Zimmerman, S., Kalinowski, J. S., Stuart, A., et al. (1997) · Journal of Speech Language and Hearing Research · doi:10.1044/jslhr.4005.1130 · Cited by 51 · see citing works
Antipova, E. A., Purdy, S. C., Blakeley, M., & Williams, S. (2008) · Journal of Fluency Disorders · doi:10.1016/j.jfludis.2008.09.002 · Cited by 51 · see citing works
Impact of vocal effort variability on automatic speech recognition
Zelinka, P., Sigmund, M., & Schimmel, J. (2012) · Speech Communication · doi:10.1016/j.specom.2012.01.002 · Cited by 51 · see citing works
Bouchard, K. E., & Chang, E. F. (2014) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/embc.2014.6945185 · Cited by 47 · see citing works
Airaksinen, M., Juvela, L., Bollepalli, B., Yamagishi, J., & Alku, P. (2018) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2018.2835720 · Cited by 46 · see citing works
Rupp, K., Roos, M. J., Milsap, G., Caceres, C. A., Ratto, C. R., Chevillet, M. A., Crone, N. E., & Wolmetz, M. E. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.12.074 · Cited by 44 · see citing works
Self-citationTowards reconstructing intelligible speech from the human auditory cortexPreprint
Akbari, H., Khalighinejad, B., Herrero, J. L., Mehta, A. D., & Mesgarani, N. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/350124 · Cited by 44 · see citing works
Chronic ambulatory electrocorticography from human speech cortex
Rao, V. R., Leonard, M. K., Kleen, J. K., Lucas, B. A., Mirro, E. A., & Chang, E. F. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.008 · Cited by 42 · see citing works
Real-time classification of auditory sentences using evoked cortical activity in humans
Moses, D. A., Leonard, M. K., & Chang, E. F. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aaab6f · Cited by 41 · see citing works
Using Brain Computer Interface for Synthesized Speech Communication for the Physically Disabled
Soman, S., & Murthy, B. K. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.02.023 · Cited by 38 · see citing works
Time course of speech changes in response to unanticipated short-term changes in hearing state
Perkell, J. S., Lane, H. L., Denny, M., et al. (2007) · The Journal of the Acoustical Society of America · doi:10.1121/1.2642349 · Cited by 35 · see citing works
Toward a Speech Neuroprosthesis
Chang, E. F., & Anumanchipalli, G. K. (2020) · JAMA · doi:10.1001/jama.2019.19813 · Cited by 35 · see citing works
Persistence of distal motor control in the locked in syndrome. Review of 11 patients
Richard, I., Péréon, Y., Guiheneu, P., Nogues, B., Perrouin-Verbe, B., & Mathé, J.-F. (1995) · Spinal Cord · doi:10.1038/sc.1995.135 · Cited by 33 · see citing works
A trainable excitation model for HMM-based speech synthesis
Maia, R., Toda, T., Zen, H., et al. (2007) · Interspeech (USB) · doi:10.21437/interspeech.2007-530 · Cited by 30 · see citing works
Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework
Köhler, J., Ottenhoff, M. C., Goulis, S., Angrick, M., Colon, A., Wagner, L., Tousseyn, S., Kubben, P., & Herff, C. (2022) · Neurons Behavior Data analysis and Theory · doi:10.51628/001c.57524 · Cited by 29 · see citing works
Speech Synthesis from ECoG using Densely Connected 3D Convolutional Neural NetworksPreprint
Angrick, M., Herff, C., Mugler, E. M., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/478644 · Cited by 28 · see citing works
León‐Carrión, J., von Wild, K. R. H., & Zitnay, G. A. (2006) · Taylor & Francis eBooks · doi:10.4324/9780203965238 · Cited by 26 · see citing works
The effects of delayed auditory and visual feedback on speech production
Chesters, J., Baghai-Ravary, L., & Möttönen, R. (2015) · The Journal of the Acoustical Society of America · doi:10.1121/1.4906266 · Cited by 26 · see citing works
Classification of mouth movements using 7 T fMRI
Bleichner, M. G., Jansma, J. M., Salari, E., Freudenburg, Z. V., Raemaekers, M. A. H., & Ramsey, N. F. (2015) · Journal of Neural Engineering · doi:10.1088/1741-2560/12/6/066026 · Cited by 23 · see citing works
Fitzgerald's Clinical Neuroanatomy and Neuroscience
Mtui, E. P., Gruener, G., & Dockery., P. (2015) · Cited by 18 · see citing works
Keyword Spotting Using Human Electrocorticographic Recordings
Milsap, G., Collard, M., Coogan, C. G., Rabbani, Q., Wang, Y., & Crone, N. E. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00060 · Cited by 18 · see citing works
Self-citationIntelligible speech synthesis from neural decoding of spoken sentencesPreprint
Anumanchipalli, G. K., Chartier, J., & Chang, E. F. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/481267 · Cited by 14 · see citing works
Mugler, E. M., Goldrick, M., Rosenow, J. M., Tate, M. C., & Slutzky, M. W. (2015) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7319597 · Cited by 10 · see citing works
Imagined speech can be decoded from low- and cross-frequency features in perceptual spacePreprint
Proix, T., Saa, J. F. D., Christen, A., Martin, S., Pasley, B. N., Knight, R. T., Tian, X., Poeppel, D., Doyle, W., Devinsky, O., et al. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.01.26.428315 · Cited by 6 · see citing works
Speech Spectrogram Estimation from Intracranial Brain Activity Using a Quantization Approach
Angrick, M., Herff, C., Johnson, G. D., Shih, J. J., Krusienski, D. J., & Schultz, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2946 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- Chang EF, Anumanchipalli GK. Toward a speech neuroprosthe- sis. JAMA. 2020;323:413-4.
- Kastner S, editor. Elife. 2017;6:e18554.
- Leon-Carrion J, von Wild KRH, Zitnay GA. Brain injury treat- ment: theories and practices. Taylor & Francis; 2006.
- Chomsky N. Syntactic structures [Internet]. Syntactic Struct. De Gruyter Mouton; 2009 [cited 2021 Oct 21]. Available from: https:// www. degru yter. com/ docum ent/ doi/ 10. 1515/ 97831 10218 329/ html. · DOI
- Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks. Adv Neural Inf Process Syst [Internet].
- Oord A van den, Dieleman S, Zen H, et al. Wavenet: a genera- tive model for raw audio. ArXiv Prepr ArXiv160903499. 2016.
- Kalchbrenner N, Elsen E, Simonyan K, et al. Efficient neural audio synthesis. Proc 35th Int Conf Mach Learn [Internet]. PMLR; 2018 [cited 2021 Dec 16]. p. 2410-2419. Available from: https:// proce edings. mlr. press/ v80/ kalch brenn er18a. html.
Cited by
Sorted by citations
An Accurate and Rapidly Calibrating Speech Neuroprosthesis
Card, N. S., Wairagkar, M., Iacobacci, C., Hou, X., Singer-Clark, T., Willett, F. R., Kunz, E. M., Fan, C., Vahdati Nia, M., Deo, D. R., et al. (2024) · New England Journal of Medicine · doi:10.1056/nejmoa2314132 · Cited by 238 · see citing works
Antiageing strategy for neurodegenerative diseases: from mechanisms to clinical advances
Jiang, Q., Liu, J., Huang, S., Wang, X.-Y., Chen, X., Liu, G.-H., Ye, K., Song, W., Masters, C. L., Wang, J., et al. (2025) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-025-02145-7 · Cited by 165 · see citing works
Representation of internal speech by single neurons in human supramarginal gyrus
Wandelt, S. K., Bjånes, D. A., Pejsa, K., Lee, B., Liu, C., & Andersen, R. A. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01867-y · Cited by 74 · see citing works
An accurate and rapidly calibrating speech neuroprosthesis
Card, N. S., Wairagkar, M., Iacobacci, C., Hou, X., Singer-Clark, T., Willett, F. R., Kunz, E. M., Fan, C., Vahdati Nia, M., Deo, D. R., et al. (2023) · doi:10.1101/2023.12.26.23300110 · Cited by 244 · see citing works
“Brain-computer interfaces (BCIs) are a promising assistive technology to meet patients’ fundamental need for fast and effortless communication by bypassing the damaged parts of the nervous system and directly decoding their intended speech from neural measurements (reviewed in 5 ).”
BackgroundOnline internal speech decoding from single neurons in a human participant
Wandelt, S. K., Bjånes, D. A., Pejsa, K., Lee, B., Liu, C., & Andersen, R. A. (2022) · doi:10.1101/2022.11.02.22281775 · Cited by 17 · see citing works
The ethical significance of user-control in AI-driven speech-BCIs: a narrative review
van Stuijvenberg, O. C., Samlal, D. P. S., Vansteensel, M. J., Broekman, M. L. D., & Jongsma, K. R. (2024) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2024.1420334 · Cited by 13 · see citing works
- Authors
- Luo, S., Rabbani, Q., & Crone, N. E.
- Affiliations
- Johns Hopkins University (US)
- Corresponding author
- Luo, Shiyu
- Venue
- Neurotherapeutics
- Publisher
- Elsevier BV
- Published
- 2022-01
- Volume
- 19
- Issue
- 1
- Pages
- 263-273
- ISSN
- 1878-7479 (ISSN-L)
- PMC
- PMC9130409
- OpenAlex
- W4210261408
- Cited by
- 106
- References
- 107
- Citations by year
- 2026: 21 · 2025: 39 · 2024: 29 · 2023: 14 · 2022: 3
- FWCI
- 10.95
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Neurotherapeutics (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Neurological Disorders and Stroke (UH3NS114439)
- National Institute on Deafness and Other Communication Disorders (U01DC016686)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Health Sciences › Medicine › Physiology › Voice and Speech Disorders
- Keywords
- brain-computer interface, speech decoding, locked-in syndrome, augmentative and alternative communication, intracranial electrocorticography, vocoder, amyotrophic lateral sclerosis, brainstem infarction, neural decoding, deep learning
- MeSH terms
- Electrocorticography, Communication, Humans, Quality of Life, Speech, Brain-Computer Interfaces
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
