Speech synthesis from ECoG using densely connected 3D convolutional neural networks
Open accessPublished version of a preprint
Abstract
Objective . Direct synthesis of speech from neural signals could provide a fast and natural way of communication to people with neurological diseases. Invasively-measured brain activity (electrocorticography; ECoG) supplies the necessary temporal and spatial resolution to decode fast and complex processes such as speech production. A number of impressive advances in speech decoding using neural signals have been achieved in recent years, but the complex dynamics are still not fully understood. However, it is unlikely that simple linear models can capture the relation between neural activity and continuous spoken speech. Approach . Here we show that deep neural networks can be used to map ECoG from speech production areas onto an intermediate representation of speech (logMel spectrogram). The proposed method uses a densely connected convolutional neural network topology which is well-suited to work with the small amount of data available from each participant. Main results . In a study with six participants, we achieved correlations up to r = 0.69 between the reconstructed and original logMel spectrograms. We transfered our prediction back into an audible waveform by applying a Wavenet vocoder. The vocoder was conditioned on logMel features that harnessed a much larger, pre-existing data corpus to provide the most natural acoustic output. Significance . To the best of our knowledge, this is the first time that high-quality speech has been reconstructed from neural recordings during speech production using deep neural networks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior work has synthesized intelligible speech from neural activity during vocalized or mimed speech in speakers with intact speech, but not in a paralysed person.
“Previous work in speakers with intact speech has demonstrated that intelligible speech can be synthesized from neural activity during vocalized or mimed speech10,11, but this has not been shown with someone who is paralysed.”
From A high-performance neuroprosthesis for speech decoding and avatar control (2023) · cites this paper as [10]
The Discussion (page unavailable) says the study reconstructed high-quality speech from neural recordings during overt speech in motor-intact participants and that speech representations in paralysed patients need further research. The Results (page unavailable) say intelligibility was evaluated with STOI, but the available text does not establish that the synthesized speech was intelligible. This paper studied vocalized speech, not mimed speech. (LLM, conf=0.78)
Experimental — accuracy not yet measuredAngrick et al. and Anumanchipalli et al. used a two-stage decoding process in which a deep-learning model regresses neural activity to acoustic features and a vocoder converts them to audible speech.
“Another approach, pursued by Angrick et al.111 and Anumanchipalli et al.110, involved a two-stage decoding process. In the first stage, a deep-learning model is trained to regress neural activity (recorded from SEEG111 or ECoG110) to a time series of acoustic features, such as the mel-spectrogram (Fig. 3b). In the second stage, a speech synthesizer — a vocoder — is used to convert the acoustic rep”
From The speech neuroprosthesis (2024) · cites this paper as [111]
The Decoding Approach section describes the claimed two-stage process: a DenseNet maps neural activity to mel-scaled spectrograms, and a WaveNet vocoder produces audio. However, the citing sentence assigns SEEG to Angrick et al.; the Experiment and Data Recording section says they recorded ECoG using a cortical electrode grid. Page numbers are unavailable in our copy. (LLM, conf=0.90)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
Brain–computer interfaces for communication and control
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00057-3 · Cited by 7,927 · see citing works
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · see citing works
Deep learning with convolutional neural networks for EEG decoding and visualization
Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., Hutter, F., Burgard, W., & Ball, T. (2017) · Human Brain Mapping · doi:10.1002/hbm.23730 · Cited by 3,751 · see citing works
Neuronal ensemble control of prosthetic devices by a human with tetraplegia
Hochberg, L. R., Serruya, M. D., Friehs, G. M., Mukand, J. A., Saleh, M., Caplan, A. H., Branner, A., Chen, D., Penn, R. D., & Donoghue, J. P. (2006) · Nature · doi:10.1038/nature04970 · Cited by 3,295 · see citing works
BCI2000: A General-Purpose Brain-Computer Interface (BCI) System
Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., & Wolpaw, J. R. (2004) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2004.827072 · Cited by 3,022 · see citing works
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791 · see citing works
Rare and common variants: twenty arguments
Gibson, G. (2012) · Nature Reviews Genetics · doi:10.1038/nrg3118 · Cited by 1,278 · see citing works
A short-time objective intelligibility measure for time-frequency weighted noisy speech
Taal, C., Hendriks, R. C., Heusdens, R. R., & Jensen, J. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5495701 · Cited by 1,111 · see citing works
A Scale for the Measurement of the Psychological Magnitude Pitch
Stevens, S. S., Volkmann, J., & Newman, E. B. (1937) · The Journal of the Acoustical Society of America · doi:10.1121/1.1915893 · Cited by 1,107 · cited 2× · see citing works
“The compression is based on the usage of the mel-scale which condenses frequency bands in accordance to the human perception [53].”
BackgroundMethodPhonetic Feature Encoding in Human Superior Temporal Gyrus
Mesgarani, N., Cheung, C., Johnson, K., & Chang, E. F. (2014) · Science · doi:10.1126/science.1245994 · Cited by 993 · see citing works
Ajiboye, A. B., Willett, F. R., Young, D. R., Memberg, W. D., Murphy, B. A., Miller, J. P., Walter, B. L., Sweet, J. A., Hoyen, H. A., Keith, M. W., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)30601-3 · Cited by 862 · see citing works
Computational neuroanatomy of speech production
Hickok, G. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3158 · Cited by 837 · see citing works
Statistical parametric speech synthesis using deep neural networks
Zen, H., Senior, A., & Schuster, M. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639215 · Cited by 818 · see citing works
Ray, S., Crone, N. E., Niebur, E., Franaszczuk, P. J., & Hsiao, S. S. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.2848-08.2008 · Cited by 722 · see citing works
Inferring single-trial neural population dynamics using sequential auto-encoders
Pandarinath, C., O’Shea, D. J., Collins, J., Józefowicz, R., Stavisky, S. D., Kao, J. C., Trautmann, E. M., Kaufman, M. T., Ryu, S. I., Hochberg, L. R., et al. (2018) · Nature Methods · doi:10.1038/s41592-018-0109-9 · Cited by 672 · see citing works
Reconstructing Speech from Human Auditory Cortex
Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., Knight, R. T., & Chang, E. F. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001251 · Cited by 604 · see citing works
Salimans, T., Karpathy, A., Chen, X., & Kingma, D. P. (2017) · arXiv · doi:10.48550/arxiv.1701.05517 · Cited by 584 · see citing works
Pandarinath, C., Nuyujukian, P., Blabe, C. H., Sorice, B. L., Saab, J., Willett, F. R., Hochberg, L. R., Shenoy, K. V., & Henderson, J. M. (2017) · eLife · doi:10.7554/elife.18554 · Cited by 566 · cited 1× · see citing works
“While our study uses overtly-produced speech in motor-intact participants, there are recent advances in decoding motor function [67, 68, 69, 70] in paralyzed patients that might be extended to the decoding of speech from motor areas in the future.”
BackgroundInterpretable deep neural networks for single-trial EEG classification
Sturm, I., Lapuschkin, S., Samek, W., & Müller, K. (2016) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2016.10.008 · Cited by 380 · see citing works
Turner, G. S., Tjaden, K., & Weismer, G. G. (1995) · Journal of Speech Language and Hearing Research · doi:10.1044/jshr.3805.1001 · Cited by 328 · see citing works
A Wireless Brain-Machine Interface for Real-Time Speech Synthesis
Guenther, F. H., Brumberg, J. S., Wright, E. J., Nieto-Castanon, A., Tourville, J. A., Panko, M., Law, R., Siebert, S. A., Bartels, J. L., Andreasen, D. S., et al. (2009) · PLoS ONE · doi:10.1371/journal.pone.0008218 · Cited by 306 · see citing works
Brain-to-text: decoding spoken phrases from phone representations in the brain
Herff, C., Heger, D., de Pesters, A., Telaar, D., Brunner, P., Schalk, G., & Schultz, T. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00217 · Cited by 297 · see citing works
Self-citationSpeaker-Dependent WaveNet Vocoder
Tamamori, A., Hayashi, T., Kobayashi, K., Takeda, K., & Toda, T. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-314 · Cited by 276 · see citing works
Parallel WaveNet: Fast High-Fidelity Speech SynthesisPreprint
van den Oord, A., Li, Y., Babuschkin, I., Simonyan, K., Vinyals, O., Kavukcuoglu, K., van den Driessche, G., Lockhart, E., Cobo, L. C., Stimberg, F., et al. (2017) · arXiv (Cornell University) · Cited by 262 · see citing works
Ito, K. A. (2024) · TIB Data Manager · doi:10.57702/yamk7ojd · Cited by 259 · see citing works
The equilibrium function of the otolith organs of the thornback ray (Raja clavata)
Lowenstein, O. E., & Roberts, T. D. M. (1949) · The Journal of Physiology · doi:10.1113/jphysiol.1949.sp004448 · Cited by 243 · see citing works
Decoding spoken words using local field potentials recorded from the cortical surface
Kellis, S., Miller, K., Thomson, K., Brown, R., House, P., & Greger, B. (2010) · Journal of Neural Engineering · doi:10.1088/1741-2560/7/5/056007 · Cited by 235 · see citing works
Biosignal-Based Spoken Communication: A Survey
Schultz, T., Wand, M., Hueber, T., Krusienski, D. J., Herff, C., & Brumberg, J. S. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2752365 · Cited by 201 · see citing works
Self-citationDecoding spectrotemporal features of overt and covert speech from the human cortex
Martin, S., Brunner, P., Holdgraf, C., Heinze, H.-J., Crone, N. E., Rieger, J., Schalk, G., Knight, R. T., & Pasley, B. N. (2014) · Frontiers in Neuroengineering · doi:10.3389/fneng.2014.00014 · Cited by 200 · see citing works
Mugler, E. M., Patton, J. L., Flint, R. D., Wright, Z. A., Schuele, S. U., Rosenow, J., Shih, J. J., Krusienski, D. J., & Slutzky, M. W. (2014) · Journal of Neural Engineering · doi:10.1088/1741-2560/11/3/035015 · Cited by 192 · see citing works
Self-citationA recurrent neural network for closed-loop intracortical brain–machine interface decoders
Sussillo, D., Nuyujukian, P., Fan, J. M., Kao, J. C., Stavisky, S. D., Ryu, S. I., & Shenoy, K. V. (2012) · Journal of Neural Engineering · doi:10.1088/1741-2560/9/2/026027 · Cited by 183 · see citing works
The Control of Vocal Pitch in Human Laryngeal Motor Cortex
Dichter, B. K., Breshears, J. D., Leonard, M. K., & Chang, E. F. (2018) · Cell · doi:10.1016/j.cell.2018.05.016 · Cited by 183 · see citing works
Word pair classification during imagined speech using direct brain recordings
Martin, S., Brunner, P., Iturrate, I., Millán, J. D. R., Schalk, G., Knight, R. T., & Pasley, B. N. (2016) · Scientific Reports · doi:10.1038/srep25803 · Cited by 171 · see citing works
Intonational speech prosody encoding in the human auditory cortex
Tang, C., Hamilton, L. S., & Chang, E. F. (2017) · Science · doi:10.1126/science.aam8577 · Cited by 154 · see citing works
Effect of delayed auditory feedback on normal speakers at two speech rates
Stuart, A., Kalinowski, J. S., Rastatter, M. P., & Lynch, K. (2002) · The Journal of the Acoustical Society of America · doi:10.1121/1.1466868 · Cited by 145 · see citing works
Language and motor speech skills in children with cerebral palsy
Pirilä, S., van der Meere, J., Pentikainen, T., Ruusu-Niemi, P., Korpela, R., Kilpinen, J., & Nieminen, P. (2006) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2006.06.002 · Cited by 145 · see citing works
The Tracking of Speech Envelope in the Human Cortex
Kubanek, J., Brunner, P., Gunduz, A., Poeppel, D., & Schalk, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0053398 · Cited by 130 · see citing works
Deep learning for epileptic intracranial EEG data
Antoniades, A., Spyrou, L., Took, C. C., & Sanei, S. (2016) · 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/mlsp.2016.7738824 · Cited by 129 · see citing works
Reconstructing the spectrotemporal modulations of real-life sounds from fMRI response patterns
Santoro, R., Moerel, M., Martino, F. D., Valente, G., Uǧurbil, K., Yacoub, E. S., & Formisano, E. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1617622114 · Cited by 129 · see citing works
Decoding spoken phonemes from sensorimotor cortex with high-density ECoG grids
Ramsey, N. F., Salari, E., Aarnoutse, E. J., Vansteensel, M. J., Bleichner, M. G., & Freudenburg, Z. V. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.10.011 · Cited by 124 · see citing works
EEG Classification of Covert Speech Using Regularized Neural Networks
Sereshkeh, A. R., Trott, R. E., Bricout, A., & Chau, T. T. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2758164 · Cited by 123 · see citing works
Seeliger, K., Fritsche, M., Güçlü, U., Schoenmakers, S., Schoffelen, J., Bosch, S., & van Gerven, M. A. J. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.07.018 · Cited by 118 · see citing works
Automatic Speech Recognition from Neural Signals: A Focused Review
Herff, C., & Schultz, T. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00429 · Cited by 117 · see citing works
Self-citationModeling the Dynamics of Human Brain Activity with Recurrent Neural Networks
Güçlü, U., & van Gerven, M. A. J. (2017) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2017.00007 · Cited by 116 · see citing works
The Potential for a Speech Brain–Computer Interface Using Chronic Electrocorticography
Rabbani, Q., Milsap, G., & Crone, N. E. (2019) · Neurotherapeutics · doi:10.1007/s13311-018-00692-2 · Cited by 106 · see citing works
Impairment of Speech Intelligibility in Men with Amyotrophic Lateral Sclerosis
Kent, R. D., Kent, J. F., Weismer, G. G., Sufit, R. L., Rosenbek, J. C., Martin, R. E., & Brooks, B. R. (1990) · Journal of Speech and Hearing Disorders · doi:10.1044/jshd.5504.721 · Cited by 98 · see citing works
Mugler, E. M., Tate, M. C., Livescu, K., Templer, J. W., Goldrick, M. A., & Slutzky, M. W. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.1206-18.2018 · Cited by 96 · see citing works
Self-citationMoses, D. A., Mesgarani, N., Leonard, M. K., & Chang, E. F. (2016) · Journal of Neural Engineering · doi:10.1088/1741-2560/13/5/056004 · Cited by 92 · see citing works
Electrocorticographic representations of segmental features in continuous speech
Lotte, F., Brumberg, J. S., Brunner, P., Gunduz, A., Ritaccio, A. L., Guan, C., & Schalk, G. (2015) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2015.00097 · Cited by 86 · see citing works
15th Int. Conf on Machine Learning
Cristianini, N., Shawe-Taylor, J. S., & Sybacek, P. (1998) · Cited by 76 · see citing works
Brumberg, J. S., Krusienski, D. J., Chakrabarti, S., Gunduz, A., Brunner, P., Ritaccio, A. L., & Schalk, G. (2016) · PLoS ONE · doi:10.1371/journal.pone.0166872 · Cited by 71 · see citing works
Self-citationRiès, S., Dhillon, R. K., Clarke, A., King‐Stephens, D., Laxer, K. D., Weber, P. B., Kuperman, R. A., Auguste, K. I., Brunner, P., Schalk, G., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1620669114 · Cited by 70 · see citing works
Key considerations in designing a speech brain-computer interface
Bocquelet, F., Hueber, T., Laurent Girin, B., Chabardès, S., & Yvert, B. (2016) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2017.07.002 · Cited by 70 · see citing works
Investigating deep learning for fNIRS based BCI
Hennrich, J., Herff, C., Heger, D., & Schultz, T. (2015) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7318984 · Cited by 67 · see citing works
Self-citationProgress in speech decoding from the electrocorticogram
Chakrabarti, S., Sandberg, H. M., Brumberg, J. S., & Krusienski, D. J. (2015) · Biomedical Engineering Letters · doi:10.1007/s13534-015-0175-1 · Cited by 65 · see citing works
Self-citationMotor cortical activity changes during neuroprosthetic-controlled object interaction
Downey, J. E., Brane, L., Gaunt, R. A., Tyler-Kabara, E. C., Boninger, M. L., & Collinger, J. L. (2017) · Scientific Reports · doi:10.1038/s41598-017-17222-3 · Cited by 63 · see citing works
Psychoacoustic Speech Tests: A Modified Rhyme Test
House, A. S., Williams, C. E., Hecker, M. H. L., & Kryter, K. D. (1963) · The Journal of the Acoustical Society of America · doi:10.1121/1.2142744 · Cited by 62 · see citing works
Livezey, J. A., Bouchard, K. E., & Chang, E. F. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007091 · Cited by 57 · see citing works
Effects of Laryngeal Cancer on Voice and Swallowing
Starmer, H. M., Tippett, D. C., & Webster, K. T. (2008) · Otolaryngologic Clinics of North America · doi:10.1016/j.otc.2008.01.018 · Cited by 55 · see citing works
Experiments on Parallel Training of Deep Neural Network using Model AveragingPreprint
Su, H., & Chen, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.01239 · Cited by 53 · see citing works
A Deep Learning Architecture to Detect Events in EEG Signals During Sleep
Chambon, S., Thorey, V., Arnal, P. J., Mignot, E. J.-M., & Gramfort, A. (2018) · Machine learning for signal processing ... · doi:10.1109/mlsp.2018.8517067 · Cited by 51 · see citing works
The use of intracranial recordings to decode human language: Challenges and opportunities
Martin, S., Millán, J. D. R., Knight, R. T., & Pasley, B. N. (2016) · Brain and Language · doi:10.1016/j.bandl.2016.06.003 · Cited by 48 · see citing works
ON the Use of Wavenet as a Statistical Vocoder
Adiga, N., Tsiaras, V., & Stylianou, Y. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462393 · Cited by 47 · see citing works
Physiological properties of brain-machine interface input signals
Slutzky, M. W., & Flint, R. D. (2017) · Journal of Neurophysiology · doi:10.1152/jn.00070.2017 · Cited by 45 · see citing works
Self-citationTowards direct speech synthesis from ECoG: A pilot study
Herff, C., Johnson, G. D., Diener, L., Shih, J. J., Krusienski, D. J., & Schultz, T. (2016) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2016.7591004 · Cited by 41 · see citing works
Self-citationIljina, O., Derix, J., Schirrmeister, R. T., Schulze‐Bonhage, A., Auer, P. L., Aertsen, A., & Ball, T. (2017) · Brain-Computer Interfaces · doi:10.1080/2326263x.2017.1330611 · Cited by 37 · see citing works
Effect of Delayed Auditory Feedback, Speech Rate, and Sex on Speech Production
Stuart, A., & Kalinowski, J. S. (2015) · Perceptual and Motor Skills · doi:10.2466/23.25.pms.120v17x2 · Cited by 15 · see citing works
Modeling brain responses to perceived speech with LSTM networks
Berezutskaya, J., Freudenburg, Z. V., Ramsey, N. F., Güçlü, U., & van Gerven, M. A. J. (2017) · Radboud Repository (Radboud University) · Cited by 5 · see citing works
interpretation of convolutional neural networks for speech regression from electrocorticography.
Angrick, M., Herff, C., Johnson, G. D., Shih, J. J., Krusienski, D. J., & Schultz, T. (2018) · The European Symposium on Artificial Neural Networks · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
3 of the 76 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [66]Kalchbrenner N, Elsen E, Simonyan K, Noury S, Casagrande N, Lockhart E, Stimberg F, Oord A. v. d., Dieleman S, and Kavukcuoglu K, “ Efficient neural audio synthesis,” arXiv preprint arXiv: 1802 08435, 2018.
Cited by
Sorted by citations
Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria
Moses, D. A., Metzger, S. L., Liu, J. R., Anumanchipalli, G. K., Makin, J. G., Sun, P. F., Chartier, J., Dougherty, M. E., Liu, P. M., Abrams, G. M., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2027540 · Cited by 582 · see citing works
A high-performance neuroprosthesis for speech decoding and avatar control
Metzger, S. L., Littlejohn, K. T., Silva, A. B., Moses, D. A., Seaton, M. P., Wang, R., Dougherty, M. E., Liu, J. R., Wu, P., Berger, M. A., et al. (2023) · Nature · doi:10.1038/s41586-023-06443-4 · Cited by 522 · see citing works
Decoding speech perception from non-invasive brain recordings
Défossez, A., Caucheteux, C., Rapin, J., Kabeli, O., & King, J.-R. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00714-5 · Cited by 242 · see citing works
He, B., Yuan, H., Meng, J., & Gao, S. (2020) · Springer eBooks · doi:10.1007/978-3-030-43395-6_4 · Cited by 148 · see citing works
Silent Speech Interfaces for Speech Restoration: A Review
González, J. A., Gomez-Alanis, A., Martín-Doñas, J. M., Pérez-Córdoba, J. L., & Gomez, A. M. (2020) · IEEE Access · doi:10.1109/access.2020.3026579 · Cited by 141 · see citing works
Kim, T., Shin, Y., Kang, K., Kim, K., Kim, G., Byeon, Y., Kim, H., Gao, Y., Lee, J. R., Son, G., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-33457-9 · Cited by 142 · see citing works
Angrick, M., Ottenhoff, M. C., Diener, L., Ivucic, D., Ivucic, G., Goulis, S., Saal, J., Colon, A. J., Wagner, L., Krusienski, D. J., et al. (2021) · Communications Biology · doi:10.1038/s42003-021-02578-0 · Cited by 125 · see citing works
Self-citationDecoding Imagined and Spoken Phrases From Non-invasive Neural (MEG) Signals
Dash, D., Ferrari, P., & Wang, J. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00290 · Cited by 121 · see citing works
Light neural network with fewer parameters based on CNN for fault diagnosis of rotating machinery
Jin, T., Yan, C., Chen, C., Yang, Z., Tian, H., & Wang, S. (2021) · Measurement · doi:10.1016/j.measurement.2021.109639 · Cited by 124 · see citing works
Herff, C., Krusienski, D. J., & Kubben, P. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00123 · Cited by 115 · see citing works
Self-citationHerff, C., Diener, L., Angrick, M., Mugler, E., Tate, M. C., Goldrick, M. A., Krusienski, D. J., Slutzky, M. W., & Schultz, T. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.01267 · Cited by 118 · see citing works
Self-citationBrain-Computer Interface: Applications to Speech Decoding and Synthesis to Augment Communication
Luo, S., Rabbani, Q., & Crone, N. E. (2022) · Neurotherapeutics · doi:10.1007/s13311-022-01190-2 · Cited by 106 · see citing works
Decoding Movement From Electrocorticographic Activity: A Review
Volkova, K., Lebedev, M., Kaplan, A., & Ossadtchi, A. E. (2019) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2019.00074 · Cited by 108 · see citing works
A State-of-the-Art Review of EEG-Based Imagined Speech Decoding
Lopez-Bernal, D., Balderas, D., Ponce, P., & Molina, A. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.867281 · Cited by 107 · see citing works
Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis
Stavisky, S. D., Willett, F. R., Wilson, G. H., Murphy, B. A., Rezaii, P., Avansino, D. T., Memberg, W. D., Miller, J. P., Kirsch, R. F., Hochberg, L. R., et al. (2019) · eLife · doi:10.7554/elife.46015 · Cited by 105 · see citing works
Silva, A. B., Littlejohn, K. T., Liu, J. R., Moses, D. A., & Chang, E. F. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-024-00819-9 · Cited by 103 · see citing works
“This decoding model can be trained to transform neural activity into text, audible speech or orofacial movements. a , Recording of neural activity can be achieved using different neural interfaces such as electrocorticography (ECoG), microelectrode array (MEA) and stereoelectroencephalography (SEEG).”
Decoding spoken English from intracortical electrode arrays in dorsal precentral gyrus
Wilson, G. H., Stavisky, S. D., Willett, F. R., Avansino, D. T., Kelemen, J. N., Hochberg, L. R., Henderson, J. M., Druckmann, S., & Shenoy, K. V. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abbfef · Cited by 103 · see citing works
A neural speech decoding framework leveraging deep learning and speech synthesis
Chen, X., Wang, R., Khalilian-Gourtani, A., Yu, L., Dugan, P. C., Friedman, D. J., Doyle, W., Devinsky, O., Wang, Y., & Flinker, A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00824-8 · Cited by 102 · see citing works
Human motor decoding from neural signals: a review
Tam, W.-K., Wu, T., Zhao, Q., Keefer, E. W., & Yang, Z. (2019) · BMC Biomedical Engineering · doi:10.1186/s42490-019-0022-z · Cited by 100 · see citing works
An instantaneous voice-synthesis neuroprosthesis
Wairagkar, M., Card, N. S., Singer-Clark, T. S., Hou, X., Iacobacci, C., Miller, L. M., Hochberg, L. R., Brandman, D. M., & Stavisky, S. D. (2025) · Nature · doi:10.1038/s41586-025-09127-3 · Cited by 98 · see citing works
Angrick, M., Luo, S., Rabbani, Q., Candrea, D. N., Shah, S., Milsap, G., Anderson, W. S., Gordon, C. R., Rosenblatt, K. R., Clawson, L., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-60277-2 · Cited by 96 · see citing works
Self-citationLuo, S., Angrick, M., Coogan, C., Candrea, D. N., Wyse‐Sookoo, K., Shah, S., Rabbani, Q., Milsap, G. W., Weiss, A. R., Anderson, W. S., et al. (2023) · Advanced Science · doi:10.1002/advs.202304853 · Cited by 90 · see citing works
Self-citationTang, C., Xu, M., Yi, W., Zhang, Z., Occhipinti, E., Dong, C., Ravenscroft, D., Jung, S., Lee, S., Gao, S., et al. (2024) · npj Flexible Electronics · doi:10.1038/s41528-024-00315-1 · Cited by 89 · see citing works
A streaming brain-to-voice neuroprosthesis to restore naturalistic communication
Littlejohn, K. T., Cho, C. J., Liu, J. R., Silva, A. B., Yu, B., Anderson, V. R., Kurtz-Miott, C. M., Brosler, S., Kashyap, A., Hallinan, I. P., et al. (2025) · Nature Neuroscience · doi:10.1038/s41593-025-01905-6 · Cited by 93 · see citing works
Representation of internal speech by single neurons in human supramarginal gyrus
Wandelt, S. K., Bjånes, D. A., Pejsa, K., Lee, B., Liu, C., & Andersen, R. A. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01867-y · Cited by 74 · see citing works
Brain2Char: a deep architecture for decoding text from brain recordings
Sun, P., Anumanchipalli, G. K., & Chang, E. F. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abc742 · Cited by 77 · see citing works
High-resolution neural recordings improve the accuracy of speech decoding
Duraivel, S., Rahimpour, S., Chiang, C.-H., Trumpis, M., Wang, C., Barth, K., Harward, S. C., Lad, S. P., Friedman, A. H., Southwell, D. G., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-42555-1 · Cited by 75 · see citing works
Deep learning approaches for neural decoding across architectures and recording modalities
Livezey, J. A., & Glaser, J. I. (2020) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa355 · Cited by 84 · see citing works
Brain control of bimanual movement enabled by recurrent neural networks
Deo, D. R., Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M., & Shenoy, K. V. (2024) · Scientific Reports · doi:10.1038/s41598-024-51617-3 · Cited by 73 · see citing works
A Bimodal Deep Learning Architecture for EEG-fNIRS Decoding of Overt and Imagined Speech
Cooney, C., Folli, R., & Coyle, D. (2021) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3132861 · Cited by 56 · see citing works
Towards Voice Reconstruction from EEG during Imagined Speech
Lee, Y.-E., Lee, S., Kim, S., & Lee, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25745 · Cited by 56 · see citing works
Aphasia recovery by language training using a brain–computer interface: a proof-of-concept study
Musso, M., Hübner, D., Schwarzkopf, S., Bernodusson, M. L., LeVan, P., Weiller, C., & Tangermann, M. (2022) · Brain Communications · doi:10.1093/braincomms/fcac008 · Cited by 56 · see citing works
Dataset of Speech Production in intracranial Electroencephalography
Verwoert, M., Ottenhoff, M. C., Goulis, S., Colon, A., Wagner, L., Tousseyn, S., van Dijk, J. P., Kubben, P., & Herff, C. (2022) · Scientific Data · doi:10.1038/s41597-022-01542-9 · Cited by 51 · see citing works
Self-citationSoroush, P. Z., Herff, C., Ries, S. K., Shih, J. J., Schultz, T., & Krusienski, D. J. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.119913 · Cited by 48 · see citing works
Self-citationMiao, M., Hu, W., Yin, H., & Zhang, K. (2020) · Computational and Mathematical Methods in Medicine · doi:10.1155/2020/1981728 · Cited by 50 · see citing works
Roussel, P., Godais, G. L., Bocquelet, F., Palma, M., Jiang, H., Zhang, S., Giraud, A., Mégevand, P., Miller, K. J., Gehrig, J., et al. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abb25e · Cited by 49 · see citing works
Jeong, J.-H., Cho, J.-H., Lee, B.-H., & Lee, S. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3211694 · Cited by 45 · see citing works
Decoding and synthesizing tonal language speech from brain activity
Liu, Y., Zhao, Z., Xu, M., Yu, H., Zhu, Y., Zhang, J., Bu, L., Zhang, X., Lu, J., Li, Y., et al. (2023) · Science Advances · doi:10.1126/sciadv.adh0478 · Cited by 46 · see citing works
RaviPrakash, H., Korostenskaja, M., Castillo, E. M., Lee, K. H., Salinas, C. M., Baumgartner, J. E., Anwar, S. M., Spampinato, C., & Bağcı, U. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00409 · Cited by 46 · see citing works
Direct speech reconstruction from sensorimotor brain activity with optimized deep learning models
Berezutskaya, J., Freudenburg, Z. V., Vansteensel, M. J., Aarnoutse, E. J., Ramsey, N. F., & van Gerven, M. A. J. (2023) · Journal of Neural Engineering · doi:10.1088/1741-2552/ace8be · Cited by 42 · see citing works
Silva, A. B., Liu, J. R., Metzger, S. L., Bhaya-Grossman, I., Dougherty, M. E., Seaton, M. P., Littlejohn, K. T., Tu-Chan, A., Ganguly, K., Moses, D. A., et al. (2024) · Nature Biomedical Engineering · doi:10.1038/s41551-024-01207-5 · Cited by 42 · see citing works
Data-driven models in human neuroscience and neuroengineering
Brunton, B. W., & Beyeler, M. (2019) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2019.06.008 · Cited by 42 · see citing works
Heelan, C., Lee, J.-H., O’Shea, R. T., Lynch, L., Brandman, D. M., Truccolo, W., & Nurmikko, A. V. (2019) · Communications Biology · doi:10.1038/s42003-019-0707-9 · Cited by 40 · see citing works
LipLearner: Customizable Silent Speech Interactions on Mobile Devices
Su, Z., Fang, S., & Rekimoto, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581465 · Cited by 40 · see citing works
MEG Sensor Selection for Neural Speech Decoding
Dash, D., Wisler, A. A., Ferrari, P., Davenport, E. M., Maldjian, J. A., & Wang, J. (2020) · IEEE Access · doi:10.1109/access.2020.3028831 · Cited by 38 · see citing works
Harnessing the Power of Artificial Intelligence in Otolaryngology and the Communication Sciences
Wilson, B. S., Tucci, D. L., Moses, D. A., Chang, E. F., Young, N. M., Zeng, F., Lesica, N. A., Bur, A. M., Kavookjian, H. L., Mussatto, C. C., et al. (2022) · Journal of the Association for Research in Otolaryngology · doi:10.1007/s10162-022-00846-2 · Cited by 36 · see citing works
An end-to-end 3D convolutional neural network for decoding attentive mental state
Zhang, Y., Cai, H. J., Nie, L., Xu, P., Zhao, S., & Guan, C. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.019 · Cited by 35 · see citing works
Audio representations for deep learning in sound synthesis: A review
Natsiou, A., & O’Leary, S. (2021) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa53542.2021.9686838 · Cited by 35 · see citing works
The Layer 7 Cortical Interface: A Scalable and Minimally Invasive Brain–Computer Interface Platform
Hettick, M., Ho, E., Poole, A. J., Monge, M., Papageorgiou, D., Takahashi, K., LaMarca, M., Trietsch, D., Reed, K., Murphy, M., et al. (2022) · doi:10.1101/2022.01.02.474656 · Cited by 31 · see citing works
Prefrontal High Gamma in ECoG Tags Periodicity of Musical Rhythms in Perception and Imagination
Herff, S. A., Herff, C., Milne, A. J., Johnson, G. D., Shih, J. J., & Krusienski, D. J. (2020) · eNeuro · doi:10.1523/eneuro.0413-19.2020 · Cited by 31 · see citing works
Self-citationRestoring Speech Using Brain–Computer Interfaces
Stavisky, S. D. (2025) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-110122-012818 · Cited by 33 · see citing works
Neural control of lexical tone production in human laryngeal motor cortex
Lu, J., Li, Y., Zhao, Z., Liu, Y., Zhu, Y., Mao, Y., Wu, J., & Chang, E. F. (2023) · Nature Communications · doi:10.1038/s41467-023-42175-9 · Cited by 26 · see citing works
Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework
Köhler, J., Ottenhoff, M. C., Goulis, S., Angrick, M., Colon, A., Wagner, L., Tousseyn, S., Kubben, P., & Herff, C. (2022) · Neurons Behavior Data analysis and Theory · doi:10.51628/001c.57524 · Cited by 29 · see citing works
Self-citationBrain implants that let you speak your mind
Pandarinath, C., & Ali, Y. H. (2019) · Nature · doi:10.1038/d41586-019-01181-y · Cited by 27 · see citing works
Tian, H., Li, X., Wei, Y., Ji, S., Yang, Q., Gou, G., Wang, X., Wu, F., Jian, J., Guo, H., et al. (2022) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2022.101075 · Cited by 27 · see citing works
NeuroVAD: Real-Time Voice Activity Detection from Non-Invasive Neuromagnetic Signals
Dash, D., Ferrari, P., Dutta, S., & Wang, J. (2020) · Sensors · doi:10.3390/s20082248 · Cited by 27 · see citing works
Classification of Articulator Movements and Movement Direction from Sensorimotor Cortex Activity
Salari, E., Freudenburg, Z. V., Branco, M. P., Aarnoutse, E. J., Vansteensel, M. J., & Ramsey, N. F. (2019) · Scientific Reports · doi:10.1038/s41598-019-50834-5 · Cited by 26 · see citing works
Hettick, M., Ho, E., Poole, A. J., Monge, M., Papageorgiou, D., Takahashi, K., LaMarca, M., Trietsch, D., Reed, K., Murphy, M. E., et al. (2025) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01501-w · Cited by 33 · see citing works
Speech-related dorsal motor cortex activity does not interfere with iBCI cursor control
Stavisky, S. D., Willett, F. R., Avansino, D. T., Hochberg, L. R., Shenoy, K. V., & Henderson, J. M. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab5b72 · Cited by 25 · see citing works
Decoding Intracranial EEG With Machine Learning: A Systematic Review
Mirchi, N., Warsi, N. M., Zhang, F., Wong, S. M., Suresh, H., Mithani, K., Erdman, L., & Ibrahim, G. M. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.913777 · Cited by 25 · see citing works
Distributed feedforward and feedback cortical processing supports human speech production
Wang, R., Chen, X., Khalilian-Gourtani, A., Yu, L., Dugan, P. C., Friedman, D. J., Doyle, W., Devinsky, O., Wang, Y., & Flinker, A. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2300255120 · Cited by 23 · see citing works
Wu, X., Wellington, S., Fu, Z., & Zhang, D. (2024) · Journal of Neural Engineering · doi:10.1088/1741-2552/ad593a · Cited by 22 · see citing works
Petrosyan, A., Voskoboinikov, A., Sukhinin, D., Makarova, A. V., Skalnaya, A. A., Arkhipova, N., Sinkin, M. V., & Ossadtchi, A. E. (2022) · Journal of Neural Engineering · doi:10.1088/1741-2552/aca1e1 · Cited by 23 · see citing works
Synthesizing Speech by Decoding Intracortical Neural Activity from Dorsal Motor Cortex
Wairagkar, M., Hochberg, L. R., Brandman, D. M., & Stavisky, S. D. (2023) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner52421.2023.10123880 · Cited by 22 · see citing works
Angrick, M., Luo, S., Rabbani, Q., Candrea, D. N., Shah, S., Milsap, G. W., Anderson, W. S., Gordon, C. R., Rosenblatt, K. R., Clawson, L., et al. (2023) · doi:10.1101/2023.06.30.23291352 · Cited by 95 · see citing works
Self-citationTowards decoding individual words from non-invasive brain recordings
d’Ascoli, S., Bel, C., Rapin, J., Banville, H. J., Benchetrit, Y., Pallier, C., & King, J.-R. (2025) · Nature Communications · doi:10.1038/s41467-025-65499-0 · Cited by 20 · see citing works
ReHEarSSE: Recognizing Hidden-in-the-Ear Silently Spelled Expressions
Dong, X., Chen, Y., Nishiyama, Y., Sezaki, K., Wang, Y., Christofferson, K., & Mariakakis, A. T. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642095 · Cited by 21 · see citing works
Wu, X., Metcalfe, B., He, S., Tan, H., & Zhang, D. (2024) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2024.3421551 · Cited by 21 · see citing works
Anti-interference self-powered acoustic fabric for complex acoustic environments
Zhao, J., Yao, Y., Lei, W., Li, Z., Liu, A., Ye, M., Wu, J., Guo, S., & Guo, W. (2023) · Nano Energy · doi:10.1016/j.nanoen.2023.108534 · Cited by 19 · see citing works
A brain-to-text framework for decoding natural tonal sentences
Zhang, D., Wang, Z., Qian, Y., Zhao, Z., Liu, Y., Hao, X., Li, W., Lu, S., Zhu, H., Chen, L., et al. (2024) · Cell Reports · doi:10.1016/j.celrep.2024.114924 · Cited by 19 · see citing works
Electroencephalography, Magnetoencephalography, and Cognitive Reserve: A Systematic Review
Balart-Sánchez, S. A., Bittencourt-Villalpando, M., van der Naalt, J., & Maurits, N. M. (2020) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acaa132 · Cited by 18 · see citing works
Online recognition of handwritten characters from scalp-recorded brain activities during handwriting
Pei, L., & Ouyang, G. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac01a0 · Cited by 18 · see citing works
Sequence-to-Sequence Voice Reconstruction for Silent Speech in a Tonal Language
Li, H., Lin, H., Wang, Y., Wang, H., Zhang, M., Gao, H., Ai, Q., Luo, Z., & Li, G. (2022) · Brain Sciences · doi:10.3390/brainsci12070818 · Cited by 18 · see citing works
Concerns in the Blurred Divisions Between Medical and Consumer Neurotechnology
Paek, A. Y., Brantley, J. A., Evans, B. J., & Contreras-Vidal, J. L. (2020) · IEEE Systems Journal · doi:10.1109/jsyst.2020.3032609 · Cited by 18 · see citing works
Online internal speech decoding from single neurons in a human participant
Wandelt, S. K., Bjånes, D. A., Pejsa, K., Lee, B., Liu, C., & Andersen, R. A. (2022) · doi:10.1101/2022.11.02.22281775 · Cited by 17 · see citing works
Neural Speech Decoding for Amyotrophic Lateral Sclerosis
Dash, D., Ferrari, P., Hernandez, A., Heitzman, D., Austin, S. G., & Wang, J. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3071 · Cited by 17 · see citing works
Cooney, C., Folli, R., & Coyle, D. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104783 · Cited by 17 · see citing works
Synthesizing Speech from ECoG with a Combination of Transformer-Based Encoder and Neural Vocoder
Shigemi, K., Komeiji, S., Mitsuhashi, T., Iimura, Y., Suzuki, H., Sugano, H., Shinoda, K., Yatabe, K., & Tanaka, T. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097004 · Cited by 15 · see citing works
Array-wide uniform PEDOT:PSS electroplating from potentiostatic deposition
Shin, Y., Ryu, J., Bai, T., Qiang, Y., Qi, Y., Li, G., Huang, Y., Seo, K. J., & Fang, H. (2024) · Biosensors and Bioelectronics · doi:10.1016/j.bios.2024.116418 · Cited by 17 · see citing works
Brain Computer Interfaces for Assisted Communication in Paralysis and Quality of Life
Chaudhary, U., Chander, B. S., Ohry, A., Jaramillo-Gonzalez, A., Lulé, D. E., & Birbaumer, N. (2021) · International Journal of Neural Systems · doi:10.1142/s0129065721300035 · Cited by 17 · see citing works
Towards a Speaker Independent Speech-BCI Using Speaker Adaptation
Dash, D., Wisler, A. A., Ferrari, P., & Wang, J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-3109 · Cited by 16 · see citing works
Towards Closed-Loop Speech Synthesis from Stereotactic EEG: A Unit Selection Approach
Angrick, M., Ottenhoff, M. C., Diener, L., Ivucic, D., Ivucic, G., Goulis, S., Colon, A., Wagner, L., Krusienski, D. J., Kubben, P., et al. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747300 · Cited by 16 · see citing works
Self-citationWilliams, S. C., Horsfall, H. L., Funnell, J. P., HANRAHAN, J. G., Schaefer, A. T., Muirhead, W. R., & Marcus, H. J. (2022) · World Neurosurgery · doi:10.1016/j.wneu.2022.05.062 · Cited by 16 · see citing works
Emerging Opportunities for Advancing Cognitive Neuroscience
Hasson, U., & Nusbaum, H. C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.007 · Cited by 15 · see citing works
Human enhancement through the lens of experimental and speculative neurotechnologies
Teunisse, W., Youssef, S., & Schmidt, M. R. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.179 · Cited by 13 · see citing works
Angrick, M., Ottenhoff, M. C., Goulis, S., Colon, A., Wagner, L., Krusienski, D. J., Kubben, P., Schultz, T., & Herff, C. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9629711 · Cited by 14 · see citing works
Self-citationAdvancing Speech Synthesis using EEG
Krishna, G., Tran, C., Carnahan, M., & Tewfik, A. H. (2021) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner49283.2021.9441306 · Cited by 14 · see citing works
Meng, K., Grayden, D. B., Cook, M. J., Vogrin, S. J., & Goodarzy, F. (2021) · International Winter Conference on Brain-Computer Interface (BCI) · doi:10.1109/bci51272.2021.9385355 · Cited by 14 · see citing works
Transformer-based neural speech decoding from surface and depth electrode signals
Chen, J., Chen, X., Wang, R., Le, C., Khalilian-Gourtani, A., Jensen, E., Dugan, P. C., Doyle, W., Devinsky, O., Friedman, D. J., et al. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/adab21 · Cited by 13 · see citing works
Decoding Speech Evoked Jaw Motion from Non-invasive Neuromagnetic Oscillations
Dash, D., Ferrari, P., & Wang, J. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207448 · Cited by 12 · see citing works
Imaginary Speech Recognition Using a Convolutional Network with Long-Short Memory
Rusnac, A.-L., & Grigore, O. (2022) · Applied Sciences · doi:10.3390/app122211873 · Cited by 12 · see citing works
Meng, K., Goodarzy, F., Kim, E., Park, Y. J., Kim, J. S., Cook, M. J., Chung, C. K., & Grayden, D. B. (2023) · Journal of Neural Engineering · doi:10.1088/1741-2552/ace7f6 · Cited by 12 · see citing works
Huggins, J. E., Guger, C., Aarnoutse, E. J., Allison, B. Z., Anderson, C. W., Bedrick, S. D., Besio, W. G., Chavarriaga, R., Collinger, J. L., Do, A. H., et al. (2019) · Brain-Computer Interfaces · doi:10.1080/2326263x.2019.1697163 · Cited by 14 · see citing works
Self-citationDeep CNN Sparse Coding for Real Time Inhaler Sounds Classification
Ntalianis, V., Fakotakis, N., Nousias, S., Lalos, A. S., Birbas, M., Zacharaki, E. I., & Μουστάκας, Κ. (2020) · Sensors · doi:10.3390/s20082363 · Cited by 12 · see citing works
Role of Brainwaves in Neural Speech Decoding
Dash, D., Ferrari, P., & Wang, J. (2020) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco47968.2020.9287714 · Cited by 12 · see citing works
Improving Silent Speech BCI Training Procedures Through Transfer from Overt to Silent Speech
Rekrut, M., Selim, A. M., & Krüger, A. (2022) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53654.2022.9945447 · Cited by 11 · see citing works
Neural correlates of unstructured motor behaviors
Gabriel, P., Chen, K., Alasfour, A., Pailla, T., Doyle, W., Devinsky, O., Friedman, D. J., Dugan, P. C., Melloni, L., Thesen, T., et al. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab355c · Cited by 10 · see citing works
Speech Activity Detection from Stereotactic EEG
Soroush, P. Z., Angrick, M., Shih, J. J., Schultz, T., & Krusienski, D. J. (2021) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc52423.2021.9659058 · Cited by 11 · see citing works
Self-citationImagined, Intended, and Spoken Speech Envelope Synthesis from Neuromagnetic Signals
Dash, D., Ferrari, P., Berstis, K., & Wang, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87802-3_13 · Cited by 10 · see citing works
Magnetometers vs Gradiometers for Neural Speech Decoding
Dash, D., Ferrari, P., Abbas Babajani-feremi, A., Borna, A., Schwindt, P. D. D., & Wang, J. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9630489 · Cited by 10 · see citing works
- Authors
- Angrick, M., Herff, C., Mugler, E., Tate, M. C., Slutzky, M. W., Krusienski, D. J., & Schultz, T.
- Affiliations
- University of Bremen (DE)
- Northwestern University (US)
- Virginia Commonwealth University (US)
- Corresponding author
- Angrick, Miguel
- Herff, Christian
- ORCID
- Venue
- Journal of Neural Engineering
- Publisher
- IOP Publishing
- Published
- 2019-04-16
- Volume
- 16
- Issue
- 3
- Pages
- 036019
- ISSN
- 1741-2552 (ISSN-L)
- 1741-2560
- PMC
- PMC6822609
- OpenAlex
- W2952609044
- Cited by
- 225
- References
- 76
- Citations by year
- 2026: 18 · 2025: 36 · 2024: 35 · 2023: 28 · 2022: 34 · 2021: 30 · 2020: 29 · 2019: 15
- FWCI
- 12.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Neural Engineering (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- National Science Foundation (1608140)
- National Institutes of Health (F32DC015708)
- National Center for Advancing Translational Sciences (UL1TR000150)
- Bundesministerium für Bildung und Forschung (01GQ1602)
- Doris Duke Charitable Foundation (2011039)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- speech synthesis, electrocorticography, neural speech decoding, 3D convolutional neural networks, speech production, brain-computer interface, augmentative and alternative communication
- MeSH terms
- Electrocorticography, Cerebral Cortex, Communication Aids for Disabled, Communication Devices for People with Disabilities, Humans, Photic Stimulation, Speech, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
