The DOI matches its Crossref record; 73 of the 76 references it cites were found in a registry or an index.
Speech synthesis from ECoG using densely connected 3D convolutional neural networks
Open accessPublished version of a preprint
Abstract
Objective . Direct synthesis of speech from neural signals could provide a fast and natural way of communication to people with neurological diseases. Invasively-measured brain activity (electrocorticography; ECoG) supplies the necessary temporal and spatial resolution to decode fast and complex processes such as speech production. A number of impressive advances in speech decoding using neural signals have been achieved in recent years, but the complex dynamics are still not fully understood. However, it is unlikely that simple linear models can capture the relation between neural activity and continuous spoken speech. Approach . Here we show that deep neural networks can be used to map ECoG from speech production areas onto an intermediate representation of speech (logMel spectrogram). The proposed method uses a densely connected convolutional neural network topology which is well-suited to work with the small amount of data available from each participant. Main results . In a study with six participants, we achieved correlations up to r = 0.69 between the reconstructed and original logMel spectrograms. We transfered our prediction back into an audible waveform by applying a Wavenet vocoder. The vocoder was conditioned on logMel features that harnessed a much larger, pre-existing data corpus to provide the most natural acoustic output. Significance . To the best of our knowledge, this is the first time that high-quality speech has been reconstructed from neural recordings during speech production using deep neural networks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous work in speakers with intact speech has demonstrated that intelligible speech can be synthesized from neural activity during vocalized or mimed speech10,11, but this has not been shown with someone who is paralysed.”
From A high-performance neuroprosthesis for speech decoding and avatar control (2023) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Another approach, pursued by Angrick et al.111 and Anumanchipalli et al.110, involved a two-stage decoding process. In the first stage, a deep-learning model is trained to regress neural activity (recorded from SEEG111 or ECoG110) to a time series of acoustic features, such as the mel-spectrogram (Fig. 3b). In the second stage, a speech synthesizer — a vocoder — is used to convert the acoustic rep”
From The speech neuroprosthesis (2024) · cites this paper as [111]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Brain–computer interfaces for communication and control
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00057-3 · Cited by 7,927
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Deep learning with convolutional neural networks for EEG decoding and visualization
Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., Hutter, F., Burgard, W., & Ball, T. (2017) · Human Brain Mapping · doi:10.1002/hbm.23730 · Cited by 3,751
Neuronal ensemble control of prosthetic devices by a human with tetraplegia
Hochberg, L. R., Serruya, M. D., Friehs, G. M., Mukand, J. A., Saleh, M., Caplan, A. H., Branner, A., Chen, D., Penn, R. D., & Donoghue, J. P. (2006) · Nature · doi:10.1038/nature04970 · Cited by 3,295
BCI2000: A General-Purpose Brain-Computer Interface (BCI) System
Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., & Wolpaw, J. R. (2004) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2004.827072 · Cited by 3,129
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791
Cited by
Sorted by citations
Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria
Moses, D. A., Metzger, S. L., Liu, J. R., Anumanchipalli, G. K., Makin, J. G., Sun, P. F., Chartier, J., Dougherty, M. E., Liu, P. M., Abrams, G. M., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2027540 · Cited by 582
A high-performance neuroprosthesis for speech decoding and avatar control
Metzger, S. L., Littlejohn, K. T., Silva, A. B., Moses, D. A., Seaton, M. P., Wang, R., Dougherty, M. E., Liu, J. R., Wu, P., Berger, M. A., et al. (2023) · Nature · doi:10.1038/s41586-023-06443-4 · Cited by 522
Decoding speech perception from non-invasive brain recordings
Défossez, A., Caucheteux, C., Rapin, J., Kabeli, O., & King, J.-R. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00714-5 · Cited by 322
He, B., Yuan, H., Meng, J., & Gao, S. (2020) · Springer eBooks · doi:10.1007/978-3-030-43395-6_4 · Cited by 150
Silent Speech Interfaces for Speech Restoration: A Review
González, J. A., Gomez-Alanis, A., Martín-Doñas, J. M., Pérez-Córdoba, J. L., & Gomez, A. M. (2020) · IEEE Access · doi:10.1109/access.2020.3026579 · Cited by 147
Kim, T., Shin, Y., Kang, K., Kim, K., Kim, G., Byeon, Y., Kim, H., Gao, Y., Lee, J. R., Son, G., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33457-9 · Cited by 150
Angrick, M., Ottenhoff, M. C., Diener, L., Ivucic, D., Ivucic, G., Goulis, S., Saal, J., Colon, A. J., Wagner, L., Krusienski, D. J., et al. (2021) · Communications Biology · doi:10.1038/s42003-021-02578-0 · Cited by 125
Self-citationDecoding Imagined and Spoken Phrases From Non-invasive Neural (MEG) Signals
Dash, D., Ferrari, P., & Wang, J. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00290 · Cited by 134
Light neural network with fewer parameters based on CNN for fault diagnosis of rotating machinery
Jin, T., Yan, C., Chen, C., Yang, Z., Tian, H., & Wang, S. (2021) · Measurement · doi:10.1016/j.measurement.2021.109639 · Cited by 123
Herff, C., Krusienski, D. J., & Kubben, P. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00123 · Cited by 123
Self-citationHerff, C., Diener, L., Angrick, M., Mugler, E., Tate, M. C., Goldrick, M. A., Krusienski, D. J., Slutzky, M. W., & Schultz, T. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.01267 · Cited by 118
Self-citationBrain-Computer Interface: Applications to Speech Decoding and Synthesis to Augment Communication
Luo, S., Rabbani, Q., & Crone, N. E. (2022) · Neurotherapeutics · doi:10.1007/s13311-022-01190-2 · Cited by 106
Decoding Movement From Electrocorticographic Activity: A Review
Volkova, K., Lebedev, M., Kaplan, A., & Ossadtchi, A. E. (2019) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2019.00074 · Cited by 118
A State-of-the-Art Review of EEG-Based Imagined Speech Decoding
Lopez-Bernal, D., Balderas, D., Ponce, P., & Molina, A. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.867281 · Cited by 107
Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis
Stavisky, S. D., Willett, F. R., Wilson, G. H., Murphy, B. A., Rezaii, P., Avansino, D. T., Memberg, W. D., Miller, J. P., Kirsch, R. F., Hochberg, L. R., et al. (2019) · eLife · doi:10.7554/elife.46015 · Cited by 105
Silva, A. B., Littlejohn, K. T., Liu, J. R., Moses, D. A., & Chang, E. F. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-024-00819-9 · Cited by 103
“This decoding model can be trained to transform neural activity into text, audible speech or orofacial movements. a , Recording of neural activity can be achieved using different neural interfaces such as electrocorticography (ECoG), microelectrode array (MEA) and stereoelectroencephalography (SEEG).”
Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus
Wilson, G. H., Stavisky, S. D., Willett, F. R., Avansino, D. T., Kelemen, J. N., Hochberg, L. R., Henderson, J. M., Druckmann, S., & Shenoy, K. V. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abbfef · Cited by 103
A neural speech decoding framework leveraging deep learning and speech synthesis
Chen, X., Wang, R., Khalilian-Gourtani, A., Yu, L., Dugan, P. C., Friedman, D. J., Doyle, W., Devinsky, O., Wang, Y., & Flinker, A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00824-8 · Cited by 115
Human motor decoding from neural signals: a review
Tam, W.-K., Wu, T., Zhao, Q., Keefer, E. W., & Yang, Z. (2019) · BMC Biomedical Engineering · doi:10.1186/s42490-019-0022-z · Cited by 106
An instantaneous voice-synthesis neuroprosthesis
Wairagkar, M., Card, N. S., Singer-Clark, T., Hou, X., Iacobacci, C., Miller, L. M., Hochberg, L. R., Brandman, D. M., & Stavisky, S. D. (2025) · Nature · doi:10.1038/s41586-025-09127-3 · Cited by 120
Angrick, M., Luo, S., Rabbani, Q., Candrea, D. N., Shah, S., Milsap, G., Anderson, W. S., Gordon, C. R., Rosenblatt, K. R., Clawson, L., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-60277-2 · Cited by 103
Self-citationLuo, S., Angrick, M., Coogan, C., Candrea, D. N., Wyse‐Sookoo, K., Shah, S., Rabbani, Q., Milsap, G. W., Weiss, A. R., Anderson, W. S., et al. (2023) · Advanced Science · doi:10.1002/advs.202304853 · Cited by 90
Self-citationTang, C., Xu, M., Yi, W., Zhang, Z., Occhipinti, E., Dong, C., Ravenscroft, D., Jung, S., Lee, S., Gao, S., et al. (2024) · npj Flexible Electronics · doi:10.1038/s41528-024-00315-1 · Cited by 102
A streaming brain-to-voice neuroprosthesis to restore naturalistic communication
Littlejohn, K. T., Cho, C. J., Liu, J. R., Silva, A. B., Yu, B., Anderson, V. R., Kurtz-Miott, C. M., Brosler, S., Kashyap, A., Hallinan, I. P., et al. (2025) · Nature Neuroscience · doi:10.1038/s41593-025-01905-6 · Cited by 104
Representation of internal speech by single neurons in human supramarginal gyrus
Wandelt, S. K., Bjånes, D. A., Pejsa, K., Lee, B., Liu, C., & Andersen, R. A. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01867-y · Cited by 74
Brain2Char: a deep architecture for decoding text from brain recordings
Sun, P., Anumanchipalli, G. K., & Chang, E. F. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abc742 · Cited by 77
High-resolution neural recordings improve the accuracy of speech decoding
Duraivel, S., Rahimpour, S., Chiang, C.-H., Trumpis, M., Wang, C., Barth, K., Harward, S. C., Lad, S. P., Friedman, A. H., Southwell, D. G., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-42555-1 · Cited by 75
Deep learning approaches for neural decoding across architectures and recording modalities
Livezey, J. A., & Glaser, J. I. (2020) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa355 · Cited by 94
Brain control of bimanual movement enabled by recurrent neural networks
Deo, D. R., Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M., & Shenoy, K. V. (2024) · Scientific Reports · doi:10.1038/s41598-024-51617-3 · Cited by 77
A Bimodal Deep Learning Architecture for EEG-fNIRS Decoding of Overt and Imagined Speech
Cooney, C., Folli, R., & Coyle, D. (2021) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3132861 · Cited by 60
Towards Voice Reconstruction from EEG during Imagined Speech
Lee, Y.-E., Lee, S., Kim, S., & Lee, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25745 · Cited by 63
Aphasia recovery by language training using a brain–computer interface: a proof-of-concept study
Musso, M., Hübner, D., Schwarzkopf, S., Bernodusson, M. L., LeVan, P., Weiller, C., & Tangermann, M. (2022) · Brain Communications · doi:10.1093/braincomms/fcac008 · Cited by 56
Dataset of Speech Production in intracranial Electroencephalography
Verwoert, M., Ottenhoff, M. C., Goulis, S., Colon, A., Wagner, L., Tousseyn, S., van Dijk, J. P., Kubben, P., & Herff, C. (2022) · Scientific Data · doi:10.1038/s41597-022-01542-9 · Cited by 56
Self-citationMiao, M., Hu, W., Yin, H., & Zhang, K. (2020) · Computational and Mathematical Methods in Medicine · doi:10.1155/2020/1981728 · Cited by 50
Roussel, P., Godais, G. L., Bocquelet, F., Palma, M., Jiang, H., Zhang, S., Giraud, A., Mégevand, P., Miller, K. J., Gehrig, J., et al. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abb25e · Cited by 56
Soroush, P. Z., Herff, C., Ries, S. K., Shih, J. J., Schultz, T., & Krusienski, D. J. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.119913 · Cited by 48
Self-citationJeong, J.-H., Cho, J.-H., Lee, B.-H., & Lee, S. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3211694 · Cited by 46
Decoding and synthesizing tonal language speech from brain activity
Liu, Y., Zhao, Z., Xu, M., Yu, H., Zhu, Y., Zhang, J., Bu, L., Zhang, X., Lu, J., Li, Y., et al. (2023) · Science Advances · doi:10.1126/sciadv.adh0478 · Cited by 50
RaviPrakash, H., Korostenskaja, M., Castillo, E. M., Lee, K. H., Salinas, C. M., Baumgartner, J. E., Anwar, S. M., Spampinato, C., & Bağcı, U. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00409 · Cited by 46
Direct speech reconstruction from sensorimotor brain activity with optimized deep learning models
Berezutskaya, J., Freudenburg, Z. V., Vansteensel, M. J., Aarnoutse, E. J., Ramsey, N. F., & van Gerven, M. A. J. (2023) · Journal of Neural Engineering · doi:10.1088/1741-2552/ace8be · Cited by 42
Silva, A. B., Liu, J. R., Metzger, S. L., Bhaya-Grossman, I., Dougherty, M. E., Seaton, M. P., Littlejohn, K. T., Tu-Chan, A., Ganguly, K., Moses, D. A., et al. (2024) · Nature Biomedical Engineering · doi:10.1038/s41551-024-01207-5 · Cited by 54
Data-driven models in human neuroscience and neuroengineering
Brunton, B. W., & Beyeler, M. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.06.008 · Cited by 42
Heelan, C., Lee, J.-H., O’Shea, R. T., Lynch, L., Brandman, D. M., Truccolo, W., & Nurmikko, A. V. (2019) · Communications Biology · doi:10.1038/s42003-019-0707-9 · Cited by 40
LipLearner: Customizable Silent Speech Interactions on Mobile Devices
Su, Z., Fang, S., & Rekimoto, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581465 · Cited by 43
MEG Sensor Selection for Neural Speech Decoding
Dash, D., Wisler, A. A., Ferrari, P., Davenport, E. M., Maldjian, J. A., & Wang, J. (2020) · IEEE Access · doi:10.1109/access.2020.3028831 · Cited by 40
Harnessing the Power of Artificial Intelligence in Otolaryngology and the Communication Sciences
Wilson, B. S., Tucci, D. L., Moses, D. A., Chang, E. F., Young, N. M., Zeng, F., Lesica, N. A., Bur, A. M., Kavookjian, H. L., Mussatto, C. C., et al. (2022) · Journal of the Association for Research in Otolaryngology · doi:10.1007/s10162-022-00846-2 · Cited by 37
An end-to-end 3D convolutional neural network for decoding attentive mental state
Zhang, Y., Cai, H. J., Nie, L., Xu, P., Zhao, S., & Guan, C. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.019 · Cited by 35
Audio representations for deep learning in sound synthesis: A review
Natsiou, A., & O’Leary, S. (2021) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa53542.2021.9686838 · Cited by 35
The Layer 7 Cortical Interface: A Scalable and Minimally Invasive Brain–Computer Interface Platform
Hettick, M., Ho, E., Poole, A. J., Monge, M., Papageorgiou, D., Takahashi, K., LaMarca, M., Trietsch, D., Reed, K., Murphy, M., et al. (2022) · doi:10.1101/2022.01.02.474656 · Cited by 31
Prefrontal High Gamma in ECoG Tags Periodicity of Musical Rhythms in Perception and Imagination
Herff, S. A., Herff, C., Milne, A. J., Johnson, G. D., Shih, J. J., & Krusienski, D. J. (2020) · eNeuro · doi:10.1523/eneuro.0413-19.2020 · Cited by 31
Self-citationRestoring Speech Using Brain–Computer Interfaces
Stavisky, S. D. (2025) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-110122-012818 · Cited by 42
Neural control of lexical tone production in human laryngeal motor cortex
Lu, J., Li, Y., Zhao, Z., Liu, Y., Zhu, Y., Mao, Y., Wu, J., & Chang, E. F. (2023) · Nature Communications · doi:10.1038/s41467-023-42175-9 · Cited by 26
Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework
Köhler, J., Ottenhoff, M. C., Goulis, S., Angrick, M., Colon, A., Wagner, L., Tousseyn, S., Kubben, P., & Herff, C. (2022) · Neurons Behavior Data analysis and Theory · doi:10.51628/001c.57524 · Cited by 37
Self-citationBrain implants that let you speak your mind
Pandarinath, C., & Ali, Y. H. (2019) · Nature · doi:10.1038/d41586-019-01181-y · Cited by 27
Tian, H., Li, X., Wei, Y., Ji, S., Yang, Q., Gou, G., Wang, X., Wu, F., Jian, J., Guo, H., et al. (2022) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2022.101075 · Cited by 28
NeuroVAD: Real-Time Voice Activity Detection from Non-Invasive Neuromagnetic Signals
Dash, D., Ferrari, P., Dutta, S., & Wang, J. (2020) · Sensors · doi:10.3390/s20082248 · Cited by 27
Classification of Articulator Movements and Movement Direction from Sensorimotor Cortex Activity
Salari, E., Freudenburg, Z. V., Branco, M. P., Aarnoutse, E. J., Vansteensel, M. J., & Ramsey, N. F. (2019) · Scientific Reports · doi:10.1038/s41598-019-50834-5 · Cited by 26
Speech-related dorsal motor cortex activity does not interfere with iBCI cursor control
Stavisky, S. D., Willett, F. R., Avansino, D. T., Hochberg, L. R., Shenoy, K. V., & Henderson, J. M. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab5b72 · Cited by 25
Hettick, M., Ho, E., Poole, A. J., Monge, M., Papageorgiou, D., Takahashi, K., LaMarca, M., Trietsch, D., Reed, K., Murphy, M. E., et al. (2025) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01501-w · Cited by 43
Decoding Intracranial EEG With Machine Learning: A Systematic Review
Mirchi, N., Warsi, N. M., Zhang, F., Wong, S. M., Suresh, H., Mithani, K., Erdman, L., & Ibrahim, G. M. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.913777 · Cited by 26
Distributed feedforward and feedback cortical processing supports human speech production
Wang, R., Chen, X., Khalilian-Gourtani, A., Yu, L., Dugan, P. C., Friedman, D. J., Doyle, W., Devinsky, O., Wang, Y., & Flinker, A. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2300255120 · Cited by 25
Wu, X., Wellington, S., Fu, Z., & Zhang, D. (2024) · Journal of Neural Engineering · doi:10.1088/1741-2552/ad593a · Cited by 24
Petrosyan, A., Voskoboinikov, A., Sukhinin, D., Makarova, A. V., Skalnaya, A. A., Arkhipova, N., Sinkin, M. V., & Ossadtchi, A. E. (2022) · Journal of Neural Engineering · doi:10.1088/1741-2552/aca1e1 · Cited by 26
Synthesizing Speech by Decoding Intracortical Neural Activity from Dorsal Motor Cortex
Wairagkar, M., Hochberg, L. R., Brandman, D. M., & Stavisky, S. D. (2023) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner52421.2023.10123880 · Cited by 25
Angrick, M., Luo, S., Rabbani, Q., Candrea, D. N., Shah, S., Milsap, G. W., Anderson, W. S., Gordon, C. R., Rosenblatt, K. R., Clawson, L., et al. (2023) · doi:10.1101/2023.06.30.23291352 · Cited by 95
Self-citationTowards decoding individual words from non-invasive brain recordings
d’Ascoli, S., Bel, C., Rapin, J., Banville, H. J., Benchetrit, Y., Pallier, C., & King, J.-R. (2025) · Nature Communications · doi:10.1038/s41467-025-65499-0 · Cited by 44
ReHEarSSE: Recognizing Hidden-in-the-Ear Silently Spelled Expressions
Dong, X., Chen, Y., Nishiyama, Y., Sezaki, K., Wang, Y., Christofferson, K., & Mariakakis, A. T. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642095 · Cited by 21
Wu, X., Metcalfe, B., He, S., Tan, H., & Zhang, D. (2024) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2024.3421551 · Cited by 24
Anti-interference self-powered acoustic fabric for complex acoustic environments
Zhao, J., Yao, Y., Lei, W., Li, Z., Liu, A., Ye, M., Wu, J., Guo, S., & Guo, W. (2023) · Nano Energy · doi:10.1016/j.nanoen.2023.108534 · Cited by 19
A brain-to-text framework for decoding natural tonal sentences
Zhang, D., Wang, Z., Qian, Y., Zhao, Z., Liu, Y., Hao, X., Li, W., Lu, S., Zhu, H., Chen, L., et al. (2024) · Cell Reports · doi:10.1016/j.celrep.2024.114924 · Cited by 25
Electroencephalography, Magnetoencephalography, and Cognitive Reserve: A Systematic Review
Balart-Sánchez, S. A., Bittencourt-Villalpando, M., van der Naalt, J., & Maurits, N. M. (2020) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acaa132 · Cited by 18
Online recognition of handwritten characters from scalp-recorded brain activities during handwriting
Pei, L., & Ouyang, G. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac01a0 · Cited by 19
Sequence-to-Sequence Voice Reconstruction for Silent Speech in a Tonal Language
Li, H., Lin, H., Wang, Y., Wang, H., Zhang, M., Gao, H., Ai, Q., Luo, Z., & Li, G. (2022) · Brain Sciences · doi:10.3390/brainsci12070818 · Cited by 18
Concerns in the Blurred Divisions Between Medical and Consumer Neurotechnology
Paek, A. Y., Brantley, J. A., Evans, B. J., & Contreras-Vidal, J. L. (2020) · IEEE Systems Journal · doi:10.1109/jsyst.2020.3032609 · Cited by 22
Online internal speech decoding from single neurons in a human participant
Wandelt, S. K., Bjånes, D. A., Pejsa, K., Lee, B., Liu, C., & Andersen, R. A. (2022) · doi:10.1101/2022.11.02.22281775 · Cited by 17
Neural Speech Decoding for Amyotrophic Lateral Sclerosis
Dash, D., Ferrari, P., Hernandez, A., Heitzman, D., Austin, S. G., & Wang, J. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3071 · Cited by 19
Cooney, C., Folli, R., & Coyle, D. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104783 · Cited by 16
Synthesizing Speech from ECoG with a Combination of Transformer-Based Encoder and Neural Vocoder
Shigemi, K., Komeiji, S., Mitsuhashi, T., Iimura, Y., Suzuki, H., Sugano, H., Shinoda, K., Yatabe, K., & Tanaka, T. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097004 · Cited by 18
Array-wide uniform PEDOT:PSS electroplating from potentiostatic deposition
Shin, Y., Ryu, J., Bai, T., Qiang, Y., Qi, Y., Li, G., Huang, Y., Seo, K. J., & Fang, H. (2024) · Biosensors and Bioelectronics · doi:10.1016/j.bios.2024.116418 · Cited by 17
Brain Computer Interfaces for Assisted Communication in Paralysis and Quality of Life
Chaudhary, U., Chander, B. S., Ohry, A., Jaramillo-Gonzalez, A., Lulé, D. E., & Birbaumer, N. (2021) · International Journal of Neural Systems · doi:10.1142/s0129065721300035 · Cited by 19
Towards a Speaker Independent Speech-BCI Using Speaker Adaptation
Dash, D., Wisler, A. A., Ferrari, P., & Wang, J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-3109 · Cited by 16
Towards Closed-Loop Speech Synthesis from Stereotactic EEG: A Unit Selection Approach
Angrick, M., Ottenhoff, M. C., Diener, L., Ivucic, D., Ivucic, G., Goulis, S., Colon, A., Wagner, L., Krusienski, D. J., Kubben, P., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747300 · Cited by 15
Self-citationWilliams, S. C., Horsfall, H. L., Funnell, J. P., HANRAHAN, J. G., Schaefer, A. T., Muirhead, W. R., & Marcus, H. J. (2022) · World Neurosurgery · doi:10.1016/j.wneu.2022.05.062 · Cited by 16
Emerging Opportunities for Advancing Cognitive Neuroscience
Hasson, U., & Nusbaum, H. C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.007 · Cited by 15
Human enhancement through the lens of experimental and speculative neurotechnologies
Teunisse, W., Youssef, S., & Schmidt, M. R. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.179 · Cited by 13
Angrick, M., Ottenhoff, M. C., Goulis, S., Colon, A., Wagner, L., Krusienski, D. J., Kubben, P., Schultz, T., & Herff, C. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9629711 · Cited by 12
Self-citationAdvancing Speech Synthesis using EEG
Krishna, G., Tran, C., Carnahan, M., & Tewfik, A. H. (2021) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner49283.2021.9441306 · Cited by 15
Meng, K., Grayden, D. B., Cook, M. J., Vogrin, S. J., & Goodarzy, F. (2021) · International Winter Conference on Brain-Computer Interface (BCI) · doi:10.1109/bci51272.2021.9385355 · Cited by 18
Transformer-based neural speech decoding from surface and depth electrode signals
Chen, J., Chen, X., Wang, R., Le, C., Khalilian-Gourtani, A., Jensen, E., Dugan, P. C., Doyle, W., Devinsky, O., Friedman, D. J., et al. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/adab21 · Cited by 25
Decoding Speech Evoked Jaw Motion from Non-invasive Neuromagnetic Oscillations
Dash, D., Ferrari, P., & Wang, J. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207448 · Cited by 12
Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory
Rusnac, A.-L., & Grigore, O. (2022) · Applied Sciences · doi:10.3390/app122211873 · Cited by 12
Meng, K., Goodarzy, F., Kim, E., Park, Y. J., Kim, J. S., Cook, M. J., Chung, C. K., & Grayden, D. B. (2023) · Journal of Neural Engineering · doi:10.1088/1741-2552/ace7f6 · Cited by 13
Huggins, J. E., Guger, C., Aarnoutse, E. J., Allison, B. Z., Anderson, C. W., Bedrick, S. D., Besio, W. G., Chavarriaga, R., Collinger, J. L., Do, A. H., et al. (2019) · Brain-Computer Interfaces · doi:10.1080/2326263x.2019.1697163 · Cited by 15
Self-citationDeep CNN Sparse Coding for Real Time Inhaler Sounds Classification
Ntalianis, V., Fakotakis, N., Nousias, S., Lalos, A. S., Birbas, M., Zacharaki, E. I., & Μουστάκας, Κ. (2020) · Sensors · doi:10.3390/s20082363 · Cited by 12
Role of Brainwaves in Neural Speech Decoding
Dash, D., Ferrari, P., & Wang, J. (2020) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco47968.2020.9287714 · Cited by 13
Improving Silent Speech BCI Training Procedures Through Transfer from Overt to Silent Speech
Rekrut, M., Selim, A. M., & Krüger, A. (2022) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53654.2022.9945447 · Cited by 11
Neural correlates of unstructured motor behaviors
Gabriel, P., Chen, K., Alasfour, A., Pailla, T., Doyle, W., Devinsky, O., Friedman, D. J., Dugan, P. C., Melloni, L., Thesen, T., et al. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab355c · Cited by 10
Speech Activity Detection from Stereotactic EEG
Soroush, P. Z., Angrick, M., Shih, J. J., Schultz, T., & Krusienski, D. J. (2021) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc52423.2021.9659058 · Cited by 11
Self-citationImagined, Intended, and Spoken Speech Envelope Synthesis from Neuromagnetic Signals
Dash, D., Ferrari, P., Berstis, K., & Wang, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87802-3_13 · Cited by 9
Magnetometers vs Gradiometers for Neural Speech Decoding
Dash, D., Ferrari, P., Abbas Babajani-feremi, A., Borna, A., Schwindt, P. D. D., & Wang, J. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9630489 · Cited by 12
Metadata
- Authors
- Angrick, M., Herff, C., Mugler, E., Tate, M. C., Slutzky, M. W., Krusienski, D. J., & Schultz, T.
- Affiliations
- University of Bremen (DE)
- Northwestern University (US)
- Virginia Commonwealth University (US)
- Corresponding author
- Angrick, Miguel
- Herff, Christian
- ORCID
- Venue
- Journal of Neural Engineering
- Publisher
- IOP Publishing
- Published
- 2019-04-16
- Volume
- 16
- Issue
- 3
- Pages
- 036019
- ISSN
- 1741-2552 (ISSN-L)
- 1741-2560
- PMC
- PMC6822609
- OpenAlex
- W2952609044
- Cited by
- 225
- References
- 76
- Citations by year
- 2026: 18 · 2025: 36 · 2024: 35 · 2023: 28 · 2022: 34 · 2021: 30 · 2020: 29 · 2019: 15
- FWCI
- 12.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Neural Engineering (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- National Science Foundation (1608140)
- National Institutes of Health (F32DC015708)
- National Center for Advancing Translational Sciences (UL1TR000150)
- Bundesministerium für Bildung und Forschung (01GQ1602)
- Doris Duke Charitable Foundation (2011039)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- speech synthesis, electrocorticography, neural speech decoding, 3D convolutional neural networks, speech production, brain-computer interface, augmentative and alternative communication
- MeSH terms
- Electrocorticography, Cerebral Cortex, Communication Aids for Disabled, Communication Devices for People with Disabilities, Humans, Photic Stimulation, Speech, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
