Synthesizing Speech by Decoding Intracortical Neural Activity from Dorsal Motor Cortex
The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Losing the ability to speak due to brain injury or neurodegenerative diseases such as ALS can be debilitating. Brain-computer interfaces could potentially provide affected individuals a fast and intuitive way to communicate by decoding speech-related neural activity into a computer-synthesized voice. Current intracortical BCIs for communication using handwriting or point-and-click typing are substantially slower than natural speech and do not capture the full expressive range of speech. Recent studies have identified speech features from ECoG and sEEG recordings; however, intelligible speech synthesis has not yet been demonstrated. Our previous work has shown speech-related patterns in intracortical recordings from dorsal (arm/hand) motor cortex that enabled discrete word/phoneme classification. This motivates exploring an intracortical approach for continuous voice synthesis. Here, we present a neural decoding framework to synthesize speech by directly translating neural activity recorded from human motor cortex using intracortical multielectrode arrays into a low-dimensional speech feature space from which voice is synthesized.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
High-performance brain-to-text communication via handwriting
Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M., & Shenoy, K. V. (2021) · Nature · doi:10.1038/s41586-021-03506-2 · Cited by 1,024 · cited 2×
“Current intracortical BCIs have shown great potential for high accuracy communication by decoding neural activity related to attempted handwriting [9] or hand movements in point and click typing [10] in people with tetraplegia.”
BackgroundSelf-citationSpeech synthesis from neural decoding of spoken sentences
Anumanchipalli, G. K., Chartier, J., & Chang, E. F. (2019) · Nature · doi:10.1038/s41586-019-1119-1 · Cited by 883 · cited 3×
“Studies have found that articulatory trajectories, pitch and intonations are encoded in ECoG [1] allowing offline speech synthesis [2], [3].”
BackgroundResultNeuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria
Moses, D. A., Metzger, S. L., Liu, J. R., Anumanchipalli, G. K., Makin, J. G., Sun, P. F., Chartier, J., Dougherty, M. E., Liu, P. M., Abrams, G. M., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2027540 · Cited by 582
Pandarinath, C., Nuyujukian, P., Blabe, C. H., Sorice, B. L., Saab, J., Willett, F. R., Hochberg, L. R., Shenoy, K. V., & Henderson, J. M. (2017) · eLife · doi:10.7554/elife.18554 · Cited by 566 · cited 1×
“Current intracortical BCIs have shown great potential for high accuracy communication by decoding neural activity related to attempted handwriting [9] or hand movements in point and click typing [10] in people with tetraplegia.”
BackgroundSelf-citationSpeech synthesis from ECoG using densely connected 3D convolutional neural networks
Angrick, M., Herff, C., Mugler, E., Tate, M. C., Slutzky, M. W., Krusienski, D. J., & Schultz, T. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab0c59 · Cited by 225 · cited 3×
“Studies have found that articulatory trajectories, pitch and intonations are encoded in ECoG [1] allowing offline speech synthesis [2], [3].”
BackgroundResultMetzger, S. L., Liu, J. R., Moses, D. A., Dougherty, M. E., Seaton, M. P., Littlejohn, K. T., Chartier, J., Anumanchipalli, G. K., Tu-Chan, A., Ganguly, K., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33611-3 · Cited by 153
Angrick, M., Ottenhoff, M. C., Diener, L., Ivucic, D., Ivucic, G., Goulis, S., Saal, J., Colon, A. J., Wagner, L., Krusienski, D. J., et al. (2021) · Communications Biology · doi:10.1038/s42003-021-02578-0 · Cited by 125
Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis
Stavisky, S. D., Willett, F. R., Wilson, G. H., Murphy, B. A., Rezaii, P., Avansino, D. T., Memberg, W. D., Miller, J. P., Kirsch, R. F., Hochberg, L. R., et al. (2019) · eLife · doi:10.7554/elife.46015 · Cited by 105 · cited 3×
“Despite recording neural activity from an area of the motor cortex which has relatively low modulation during speech [12], we could reconstruct the coarse structure of speech in offline analysis.”
BackgroundResultSelf-citationDecoding spoken English from intracortical electrode arrays in dorsal precentral gyrus
Wilson, G. H., Stavisky, S. D., Willett, F. R., Avansino, D. T., Kelemen, J. N., Hochberg, L. R., Henderson, J. M., Druckmann, S., & Shenoy, K. V. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abbfef · Cited by 103
Self-citation
Cited by
An accurate and rapidly calibrating speech neuroprosthesis
Card, N. S., Wairagkar, M., Iacobacci, C., Hou, X., Singer-Clark, T., Willett, F. R., Kunz, E. M., Fan, C., Vahdati Nia, M., Deo, D. R., et al. (2023) · doi:10.1101/2023.12.26.23300110 · Cited by 244
“While the demonstrated brain-to-text capabilities can provide widely useful communication, they do not capture the full expressive richness of voice; the more difficult challenge of closed-loop brain-to-voice synthesis remains an active area of speech BCI research 18,34,40 .”
BackgroundSelf-citationAn Accurate and Rapidly Calibrating Speech Neuroprosthesis
Card, N. S., Wairagkar, M., Iacobacci, C., Hou, X., Singer-Clark, T., Willett, F. R., Kunz, E. M., Fan, C., Vahdati Nia, M., Deo, D. R., et al. (2024) · New England Journal of Medicine · doi:10.1056/nejmoa2314132 · Cited by 268
Self-citationSilva, A. B., Littlejohn, K. T., Liu, J. R., Moses, D. A., & Chang, E. F. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-024-00819-9 · Cited by 103
“Initial work to synthesize speech from neural activity in able speakers has leveraged a few approaches 110–112 , 125 .”
Direct speech reconstruction from sensorimotor brain activity with optimized deep learning models
Berezutskaya, J., Freudenburg, Z. V., Vansteensel, M. J., Aarnoutse, E. J., Ramsey, N. F., & van Gerven, M. A. J. (2023) · Journal of Neural Engineering · doi:10.1088/1741-2552/ace8be · Cited by 42
Direct Speech Reconstruction from Sensorimotor Brain Activity with Optimized Deep Learning Models
Berezutskaya, J., Freudenburg, Z. V., Vansteensel, M. J., Aarnoutse, E. J., Ramsey, N. F., & van Gerven, M. A. J. (2022) · doi:10.1101/2022.08.02.502503 · Cited by 42
Metadata
- Authors
- Wairagkar, M., Hochberg, L. R., Brandman, D. M., & Stavisky, S. D.
- Venue
- 2023 11th International IEEE/EMBS Conference on Neural Engineering (NER)
- Publisher
- IEEE
- Published
- 2023-04-24
- Pages
- 1-4
- ISSN
- 1948-3554 (ISSN-L)
- OpenAlex
- W4377089439
- Cited by
- 22
- References
- 11
- FWCI
- 11.89
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Synthesizing Speech by Decoding Intracortical Neural Activity from Dorsal Motor Cortex. https://app.cochat.ai/papers/10.1109/ner52421.2023.10123880
- The paper
- Wairagkar, M., Hochberg, L. R., Brandman, D. M., & Stavisky, S. D. (2023). Synthesizing Speech by Decoding Intracortical Neural Activity from Dorsal Motor Cortex. 2023 11th International IEEE/EMBS Conference on Neural Engineering (NER). https://doi.org/10.1109/ner52421.2023.10123880
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.