Reconstructing Speech from Human Auditory Cortex
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Abstract
How the human auditory system extracts perceptually relevant acoustic features of speech is unknown. To address this question, we used intracranial recordings from nonprimary auditory cortex in the human superior temporal gyrus to determine what acoustic information in speech sounds can be reconstructed from population neural activity. We found that slow and intermediate temporal fluctuations, such as those corresponding to syllable rate, were accurately reconstructed using a linear model based on the auditory spectrogram. However, reconstruction of fast temporal fluctuations, such as syllable onsets and offsets, required a nonlinear sound representation based on temporal modulation energy. Reconstruction accuracy was highest within the range of spectro-temporal fluctuations that have been found to be critical for speech intelligibility. The decoded speech representations allowed readout and identification of individual words directly from brain activity during single trial sound presentations. These findings reveal neural encoding mechanisms of speech acoustic parameters in higher order human auditory cortex.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





![Figure 6 Schematic of nonlinear modulation model. (A) The input spectrogram (top left) is transformed by a linear modulation filter bank (right) followed by a nonlinear magnitude operation (not shown). This nonlinear operation extracts the modulation energy of the incoming spectrogram and generates phase invariance to local fluctuations in the spectrogram envelope. The input representation is the two-dimensional spectrogram S(f,t) across frequency f and time t. The output (bottom left) is the four-dimensional modulation energy representation M(s,r,f,t) across spectral modulation scale s, temporal modulation rate r, frequency f, and time t. In the full modulation representation [18], negative rates by convention correspond to upward frequency sweeps, while positive rates correspond to downward frequency sweeps. Accuracy for positive and negative rates was averaged unless otherwise shown. See Materials and Methods. (B) Schematic of linear (spectrogram envelope) and nonlinear (modulation energy) temporal coding. Left: acoustic waveform (black curve) and spectrogram of a temporally modulated tone. The linear spectrogram model (top) assumes that neural responses are a linear function of the spectrogram envelope (plotted for the tone center frequency channel, top right). In this case, the instantaneous output may be high or low and does not directly indicate the modulation rate of the envelope. The nonlinear modulation model (bottom) assumes that neural responses are a linear function of modulation energy. This is an amplitude-based coding scheme (plotted for the peak modulation channel, bottom right). The nonlinear modulation model explicitly estimates the modulation rate by taking on a constant value for a constant rate [32].](https://figures.cochat.ai/paper-figures/d8911f10b353486dba673852c364a006/reconstructing-speech-from-human-auditory-cortex-figure-6.jpg)
![Figure 7 Example of nonlinear modulation coding and reconstruction. (A) Top: the spectrogram of an isolated word (“waldo”) presented aurally to one participant. Blue curve plots the spectrogram envelope, summed over all frequencies. Left panels: induced high gamma responses (black curves, trial averaged) at four different STG sites. Temporal modulation energy of the stimulus (dashed red curves) is overlaid (computed from 2, 4, 8, and 16 Hz modulation filters and normalized to maximum value). Dashed black lines indicate baseline response level. Right panels: nonlinear modulation rate tuning curves for each site (estimated from nonlinear STRFs). Shaded regions and error bars indicate SEM. (B) Original spectrogram (top), modulation-based reconstruction (middle), and spectrogram-based reconstruction (bottom), linearly decoded from a fixed set of STG electrodes. The modulation reconstruction is projected into the spectrogram domain using an iterative projection algorithm and an overcomplete set of modulation filters [18]. The displayed spectrogram is averaged over 100 random initializations of the algorithm.](https://figures.cochat.ai/paper-figures/2d2eaf31d85c437bb041f38dd7527a14/reconstructing-speech-from-human-auditory-cortex-figure-7.jpg)

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Neural activity in the STG encodes acoustic features — for example, recurring speech sound patterns in the mel-spectrogram — during speech perception53,54 and may be used to reconstruct perceived speech55,56.”
From The speech neuroprosthesis (2024) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680 · cited 1×
“The modulation model therefore represents an auditory analog to the classical energy model of complex cells in the visual system [32–36], which are invariant to the spatial phase of visual stimuli.”
MethodFundamentals of speech recognition
Rabiner, L. R., & Juang, B.-H. F. (1993) · Cited by 7,680
The cortical organization of speech processing
Hickok, G., & Poeppel, D. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2113 · Cited by 5,495 · cited 2×
“For speech comprehension, it is hypothesized that intermediate and higher-order auditory areas extract or construct information-rich features of speech, while discarding nonessential low-level acoustic information [4,5,9, 10,43].”
BackgroundSpatiotemporal energy models for the perception of motion
Adelson, E. H., & Bergen, J. R. (1985) · Journal of the Optical Society of America A · doi:10.1364/josaa.2.000284 · Cited by 3,442 · cited 2×
“Such amplitude-based coding schemes are broadly referred to as ‘‘energy models’’ [32,33].”
BackgroundDISCONNEXION SYNDROMES IN ANIMALS AND MAN
GESCHWIND, N. (1965) · Brain · doi:10.1093/brain/88.2.237 · Cited by 3,403 · cited 5×
“To recover the acoustic waveform from the spectrogram, we used an iterative projection algorithm to estimate the phase [8].”
BackgroundMethodInfluentialTIMIT Acoustic-Phonetic Continuous Speech Corpus
Garofolo, J. S., Lamel, L. F., Fisher, W. M., Pallett, D. S., Dahlgren, N. L., Zue, V. W., & Fiscus, J. G. (2020) · Americanae (AECID Library) · doi:10.35111/17gk-bn40 · Cited by 2,508
Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing
Rauschecker, J. P., & Scott, S. K. (2009) · Nature Neuroscience · doi:10.1038/nn.2331 · Cited by 1,766 · cited 5×
“Beyond primary auditory areas, further processing in intermediate and higher-order auditory cortex likely results in additional stimulus transformations [5].”
BackgroundResultInfluentialLogothetis, N. K., & Wandell, B. A. (2004) · Annual Review of Physiology · doi:10.1146/annurev.physiol.66.082602.092845 · Cited by 1,568 · cited 1×
“This is consistent with robust speech-induced high gamma responses reported in previous intracranial studies [24–29] and with observed correlations between high gamma power and local spike rate [30].”
ResultIdentifying natural images from human brain activity
Kay, K. N., Naselaris, T., Prenger, R. J., & Gallant, J. L. (2008) · Nature · doi:10.1038/nature06713 · Cited by 1,286 · cited 2×
“An important application of this approach has also been demonstrated in the study of visual object recognition in the primate visual system [36,49,50].”
BackgroundMethod
Cited by
Items 301–400 of 694
Shirzhiyan, Z., Shamsi, E., Jafarpisheh, A. S., & Jafari, A. H. (2019) · Speech Communication · doi:10.1016/j.specom.2019.09.003 · Cited by 15
Levin, M., Balberg, M., & Zaltz, Y. (2022) · Applied Sciences · doi:10.3390/app122312063 · Cited by 15
Fine-grained Artificial Neurons in Audio-transformers for Disentangling Neural Auditory Encoding
Zhou, M., Liu, X., Liu, D., Wu, Z., Liu, Z., Zhao, L., Zhu, D., Guo, L., Han, J., Liu, T., et al. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.503 · Cited by 15
Cushing, D., Gazzaley, A. H., & Morsella, E. (2017) · Consciousness and Cognition · doi:10.1016/j.concog.2017.07.003 · Cited by 15
Sellers, K. K., Chung, J. E., Zhou, J., Triplett, M., Dawes, H. E., Haque, R., & Chang, E. F. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac1984 · Cited by 14
“used to link neural signals to individual and synergistic finger movements [44] and decode control of individual prosthetic fingers [45]; 4 mm pitch arrays were used to map speech processing in human nonprimary auditory cortex [46].”
MethodSelf-citationFujiwara, Y., & Ushiba, J. (2022) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2022.882290 · Cited by 14
Pei, C., Qiu, Y., Li, F., Huang, X., Si, Y., Li, Y., Zhang, X., Chen, C., Liu, Q., Cao, Z., et al. (2022) · Cerebral Cortex · doi:10.1093/cercor/bhac376 · Cited by 14
Cho, H., Fonken, Y. M., Adamek, M., Jimenez, R., Lin, J. J., Schalk, G., Knight, R. T., & Brunner, P. (2023) · Cerebral Cortex · doi:10.1093/cercor/bhad155 · Cited by 14
“Based on previous decoding and encoding approaches, HFA is the most prominent signal containing information on stimulusspecific predictions (Chang et al. 2010; Flinker et al. 2011; Pasley et al. 2012; Martin et al. 2014; Holdgraf et al. 2016).”
BackgroundSelf-citationMice can learn phonetic categories
Saunders, J. L., & Wehr, M. (2019) · The Journal of the Acoustical Society of America · doi:10.1121/1.5091776 · Cited by 14
A neural-based vocoder implementation for evaluating cochlear implant coding strategies
Boghdady, N. E., Kegel, A., Lai, W. K., & Dillier, N. (2016) · Hearing Research · doi:10.1016/j.heares.2016.01.005 · Cited by 14
Shahin, A. J., Shen, S., & Kerlin, J. R. (2017) · Language Cognition and Neuroscience · doi:10.1080/23273798.2017.1283428 · Cited by 14
“…2004; Beauchamp et al., 2010; Heinrich, Carlyon, Davis, & Johnsrude, 2008, 2011; Hickok, 2009; Hickok & Poeppel, 2000; Miller & D’Esposito, 2005; Pasley et al., 2012; Patterson, Uppenkamp, Johnsrude, & Griffiths, 2002; Rauschecker & Scott, 2009; Rauschecker, Tian, & Hauser, 1995; Tian, Reser,…”
The shuffle estimator for explainable variance in fMRI experiments
Benjamini, Y., & Yu, B. (2013) · The Annals of Applied Statistics · doi:10.1214/13-aoas681 · Cited by 14
“The prediction accuracy of their fitted models on a separate validation image set was surprisingly high given the richness of the input class, inspiring many studies of rich stimuli class encoding [Pasley et al. (2012), Pereira, Detre and Botvinick (2011)].”
MethodA computational modelling framework for assessing information transmission with cochlear implants
Leclère, T., Johannesen, P. T., Wijetillake, A., Segovia-Martínez, M., & Lopez‐Poveda, E. A. (2023) · Hearing Research · doi:10.1016/j.heares.2023.108744 · Cited by 13
“…it must be decided whether the envelope for each channel should be reconstructed separately or not, and how a single information metric can be derived from those multiple reconstructions (successful attempts have already been re-ported by Mesgarani et al. 2009 , or Pasley et al. 2012 ).”
BackgroundMethodInfluentialBrain rhythms in mental time travel
Polyn, S. M., & Sederberg, P. B. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.06.084 · Cited by 13
“Similarly, Pasley et al. (2012) , recording from higher order auditory cortex, were able to reconstruct speech stimuli heard by the participant based on gamma-band activity.”
BackgroundInfluentialCoarse behavioral context decoding
Alasfour, A., Gabriel, P., Jiang, X., Shamie, I., Melloni, L., Thesen, T., Dugan, P. C., Friedman, D. J., Doyle, W., Devinsky, O., et al. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aaee9c · Cited by 13
“These cortical regions of interest have been shown to be engaged in motor [19] and language [20] related behaviors, respectively.”
BackgroundBinder, J. R. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-407794-2.00037-7 · Cited by 13
Open-vocabulary Auditory Neural Decoding Using fMRI-prompted LLMPreprint
Chen, X., Du, C., Liu, C., Wang, Y., & He, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.07840 · Cited by 13
fMRI2GES: Co-Speech Gesture Reconstruction From fMRI Signal With Dual Brain Decoding Alignment
Zhu, C., Shao, J., Lin, J., Wang, Y., Wang, J., Tang, J., & Li, K. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3558125 · Cited by 12
“At the same time, the researchers try to analyze the relationship between brain activity and di ff erent sound characteristics to decode the speech content heard by the subject [9], [36], [37], [38], [39].”
Wired for Words: The Neural Architecture of Language
Hickok, G. S. (2025) · The MIT Press eBooks · doi:10.7551/mitpress/9000.001.0001 · Cited by 12
Trébuchon, A., Alario, F. ‐., & Liégeois‐Chauvel, C. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.702773 · Cited by 12
“All these patterns reveal the representation of stimulus features which can be used to predict responses to novel stimuli or reconstruct the presented stimuli from pattern of cortical activity (Pasley et al., 2012).”
BackgroundSelective attention decoding in bimodal cochlear implant users
Dolhopiatenko, H., & Nogueira, W. (2023) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.1057605 · Cited by 12
“In order to investigate how speech features are decoded from the neural representation, one can apply the backward model (Mesgarani et al., 2009; Ding and Simon, 2012; Pasley et al., 2012; Mirkovic et al., 2015; OSullivan et al., 2015; Crosse et al., 2016).”
MethodPreparatory delta phase response is correlated with naturalistic speech comprehension performance
Li, J., Hong, B., Nolte, G., Engel, A. K., & Zhang, D. (2021) · Cognitive Neurodynamics · doi:10.1007/s11571-021-09711-z · Cited by 12
Ufer, C., & Blank, H. (2023) · Language Cognition and Neuroscience · doi:10.1080/23273798.2023.2166679 · Cited by 12
“…regions correspond to the following studies: 1: Hsieh et al., 2012; 2: Joanisse & DeSouza, 2014; 3: Okada et al., 2010; 4: Evans & Davis, 2015; 5: Pasley et al., 2012; 6: McGettigan et al., 2012; 7: Chang et al., 2010; 8: Obleser et al., 2010; 9: Arsenault & Buchs-baum, 2015; 10: Correia et al.,…”
Zhang, Q., Hu, X., Luo, H., Li, J., Zhang, X., & Zhang, B. (2016) · European Journal of Neuroscience · doi:10.1111/ejn.13164 · Cited by 12
“, 2012), among which the results obtained using surface electrode arrays in human brains (Chang et al., 2010; Pasley et al., 2012; Mesgarani et al., 2014) are”
BackgroundFukushima, M., Saunders, R. C., Fujii, N., Averbeck, B. B., & Mishkin, M. (2014) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2014.6945188 · Cited by 12
Farrokhi, B., & Erfanian, A. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aab290 · Cited by 12
Involuntary mental rotation and visuospatial imagery from external control
Cushing, D., Gazzaley, A. H., & Morsella, E. (2019) · Consciousness and Cognition · doi:10.1016/j.concog.2019.102809 · Cited by 12
Involuntary attentional shifts as a function of set and processing fluency
Gardner, K., Walker, E. B., Li, Y., Gazzaley, A. H., & Morsella, E. (2020) · Acta Psychologica · doi:10.1016/j.actpsy.2020.103009 · Cited by 12
“”(7) In addition, some neuroimaging data (which did not stem from RITs) corroborate that subjects do not confabulate about their reported mental events (Mason et al., 2007; McVay & Kane, 2010; Mitchell et al., 2007; Pasley et al., 2012; Wyland, Kelley, Macrae, Gordon, & Heatherton, 2003).”
BackgroundFarrokhi, B., & Erfanian, A. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab848b · Cited by 12
Entrainment in Human Auditory Cortex: Mechanism and Functions
Poeppel, D., & Teng, X. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-805408-6.00018-x · Cited by 12
Cortical Representation of Species-Specific Vocalizations in Guinea Pig
Šuta, D., Popelář, J., Burianova, J. S., & Syka, J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065432 · Cited by 12
“Recent work by Pasley et al [14] showed that some attributes of human speech can be reconstructed from population neural activity in the auditory cortex using a linear model based on the PLOS ONE | www.plosone.org 1 June 2013 | Volume 8 | Issue 6 | e65432 auditory spectrogram, but some elements required a nonlinear so…”
MethodIntracranial Electrophysiology in Language Research
Flinker, A., Piai, V., & Knight, R. T. (2018) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198786825.013.43 · Cited by 12
“, were the first to use high gamma activity in STRF modeling providing evidence for a robust acoustic representation in STG which consistently reconstructs neural responses based on the acoustic input (Pasley et al., 2012).”
BackgroundSelf-citationLiu, C., Du, C., Chen, X., & He, H. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2026.9010004 · Cited by 11
Theological and Ethical Aspects of Mind Transfer in Transhumanism
Osiński, G. (2021) · Scientia et Fides · doi:10.12775/setf.2021.005 · Cited by 11
Recognizing Tonal and Nontonal Mandarin Sentences for EEG-Based Brain–Computer Interface
Yang, S.-R., Jung, T., Lin, C., Huang, K., Wei, C.-S., Chiueh, H., Hsin, Y.-L., Liou, G.-T., & Wang, L. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3137251 · Cited by 11
“…[37]; generate neural encoding mode of perceived phonemes [32], words [38], and sentences [22]; reconstruct acoustic properties of perceived [39]; generate natural-sounding synthetic speech from brain activity [24]; and immediately identify volunteers’ spoken responses to a set of standard…”
BackgroundHa, J., Baek, S.-C., Lim, Y., & Chung, J. H. (2023) · Scientific Reports · doi:10.1038/s41598-023-49990-6 · Cited by 11
Unattended processing of hierarchical pitch variations in spoken sentences
Li, X., & Chen, Y. (2018) · Brain and Language · doi:10.1016/j.bandl.2018.05.004 · Cited by 11
“This attention-modulation effect on speech processing has been observed at both the high-level language brain areas and the low-level acoustic brain areas (Ding & Simon, 2012a, b; Kerlin, Shahin, & Miller, 2010; Mesgarani & Chang, 2012).”
Venezia, J. H., Leek, M. R., & Lindeman, M. P. (2020) · Journal of Speech Language and Hearing Research · doi:10.1044/2020_jslhr-19-00324 · Cited by 11
Etard, O., Messaoud, R. B., Gaugain, G., & Reichenbach, T. (2022) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_01811 · Cited by 11
“to track slow (<8 Hz) amplitude fluctuations in speech (Ding & Simon, 2012b; Pasley et al., 2012; Lalor & Foxe, 2010; Nourski et al., 2009).”
BackgroundInfluentialBhat, J., Pitt, M. A., & Shahin, A. J. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00173 · Cited by 11
“, and D’Esposito, M. (2005). Perceptual fusion and stimulus coinci-”
BackgroundResultInfluentialReconstructing Speech Stimuli From Human Auditory Cortex Activity Using a WaveNet Approach
Wang, R., Wang, Y., & Flinker, A. (2018) · IEEE Signal Processing in Medicine and Biology Symposium · doi:10.1109/spmb.2018.8615605 · Cited by 11
Self-citationA Bi-LSTM Based Network with Attention Mechanism for EEG Visual Classification
Mo, L., Wang, Y., Zhou, W., Shen, X., & Kong, W. (2021) · IEEE International Conference on Unmanned Systems (ICUS) · doi:10.1109/icus52573.2021.9641318 · Cited by 10
“Many researchers dedicate to exploring and analysing an EEG signal based communication mode between human brain and the outside world [3], [4].”
BackgroundEEG and fMRI coupling and decoupling based on joint independent component analysis (jICA)
Heugel, N., Beardsley, S. A., & Liebenthal, E. (2022) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2022.109477 · Cited by 10
Stuart, M., Lesaja, S., Shih, J. J., Schultz, T., Manic, M., & Krusienski, D. J. (2022) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2022.3207624 · Cited by 10
“ECoG signals have been shown to successfully control the movement of an upper-limb neuroprosthetic [3] or typing interface [4], as well as decoding speech processes [5].”
BackgroundRuffini, G., Lopez-Sola, E., Palma, R., Sanchez-Todo, R., Vohryzek, J., Castaldo, F., & Friston, K. J. (2026) · Neural Computation · doi:10.1162/neco.a.1537 · Cited by 10
The effects of attention and visual input on the representation of natural speech in EEG
O’Sullivan, J., Crosse, M. J., Power, A. J., & Lalor, E. C. (2013) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6610122 · Cited by 10
Keshavarzi, M., Choisdealbha, Á. N., Attaheri, A., Rocha, S., Brusini, P., Gibbon, S., Boutris, P., Mead, N., Olawole‐Scott, H., Ahmed, H., et al. (2023) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2023.110036 · Cited by 10
“…even silent reading (Martin et al., 2014). u A range of inverse mapping techniques have been employed to find the best approximation of the acoustic stimulus from the population of evoked neural activity (Crosse et al., 2015; Mesgarani et al., 2009; O'Sullivan et al., 2015; Pasley et al., 2012).”
MethodAn alternative approach for auditory attention tracking using single-trial EEG
Ekin, B., Atlas, L., Mirbagheri, M., & Lee, A. K. C. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7471771 · Cited by 10
“neural responses in the auditory cortex [18, 19], the EEG data were digitally filtered offline with a band-pass filter between 2 and 8 Hz, and the speech envelopes were low-pass filtered below 8 Hz, similar to preprocessing of [9].”
MethodPerceptual Encoding of Natural Speech Sounds Revealed by the N1 Event-Related Potential Response
Pereira, O., Gao, Y. A., & Toscano, J. C. (2018) · Auditory Perception & Cognition · doi:10.1080/25742442.2018.1545106 · Cited by 10
Decoding and encoding of visual patterns using magnetoencephalographic data represented in manifolds
Kuo, P., Chen, Y., Chen, L., & Hsieh, J. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.07.046 · Cited by 10
“Pasley et al. (2012) applied space-time Gabor filters to video sequences in an encoding model.”
Mind Wars: Brain Science and the Military
Moreno, J. D. (2013) · Bioethics News · doi:10.1007/bf03351549 · Cited by 10
Deschepper, R., Six, S., Vandeweghe, N., Couck, M., Gidron, Y., Depoorter, A., & Bilsen, J. J. R. (2017) · Methodological Innovations · doi:10.1177/2059799117703119 · Cited by 10
“By measuring neural activity, they were able to decode individual words (Kaku, 2014; Martin et al., 2014; Pasley et al., 2012).”
BackgroundDas, N., Bertrand, A., & Francart, T. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/312827 · Cited by 10
Auditory features modelling reveals sound envelope representation in striate cortexPreprint
Martinelli, A., Handjaras, G., Betta, M., Leo, A., Cecchetti, L., Pietrini, P., Ricciardi, E., & Bottari, D. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.04.15.043174 · Cited by 10
A systematic review of EEG-based Imagined Speech decoding
Jin, Z., Li, D., & Huang, S. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113563 · Cited by 9
Neural auditory processing of parameterized speech envelopes
Gransier, R., & Wouters, J. M. (2021) · Hearing Research · doi:10.1016/j.heares.2021.108374 · Cited by 9
EEG-based auditory attention decoding with audiovisual speech for hearing-impaired listeners
Wang, B., Xu, X., Niu, Y., Wu, C., Wu, X. H., & Chen, J. (2023) · Cerebral Cortex · doi:10.1093/cercor/bhad325 · Cited by 9
Temporal alignment of electrocorticographic recordings for upper limb movement
Talakoub, O., Popović, M. R., Navarro, J., Hamani, C., Fonoff, E. T., & Wong, W. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00431 · Cited by 9
“1109/RBME.2011.2172408 Sergio, L. E., and Kalaska, J. F. (1998). Changes in the temporal pattern of primary”
BackgroundInfluentialDecoding Lip Movements During Continuous Speech using Electrocorticography
Lesaja, S., Herff, C., Johnson, G. D., Shih, J. J., Schultz, T., & Krusienski, D. J. (2019) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2019.8716914 · Cited by 9
“Recent studies have shown that it is possible to decode certain aspects of continuously-spoken speech from ECoG signals including phonemes [5], words [6] and sentences [7] or reconstruct accoustic properties of perceived [8] and produced speech [9], [10].”
BackgroundMohammadi, Y., Graversen, C., Østergaard, J., Andersen, O. K., & Reichenbach, T. (2023) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_02016 · Cited by 9
“When listening to speech, cortical activity tracks the lowfrequency amplitude modulation (envelope) of the speech signal (Ding, Melloni, Zhang, Tian, & Poeppel, 2015; Ding & Simon, 2013; Giraud & Poeppel, 2012; Pasley et al., 2012).”
BackgroundOvert speech decoding from cortical activity: a comparison of different linear methods
Godais, G. L., Roussel, P., Bocquelet, F., Aubert, M., Kahane, P., Chabardès, S., & Yvert, B. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1124065 · Cited by 9
“Spectrograms of perceived words and sentences from a limited set have been reconstructed from the auditory cortex by linear regression (Pasley et al., 2012), and spectrograms of produced speech have been decoded from electrocorticography (ECoG) recordings of temporal areas (Herff et al., 2016).”
BackgroundMethodInterconnected growing self-organizing maps for auditory and semantic acquisition modeling
Cao, M., Li, A., Fang, Q., Kaufmann, E., & Kröger, B. J. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00236 · Cited by 9
“To overcome this problem, in our approach, the speech signals are converted into neural representations of auditory states ( or spectrograms) according to the method outlined in Kannampuzha et al. (2011) and Pasley et al. (2012).”
MethodA Graphical Model for Online Auditory Scene Modulation Using EEG Evidence for Attention
Haghighi, M., Moghadamfalahi, M., Akçakaya, M., Shinn‐Cunningham, B. G., & Deniz Erdogmus, B. (2017) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2017.2712419 · Cited by 9
“These observations have lead to efforts to use recorded cortical measurements to reconstruct an attended speech stimulus [21].”
BackgroundLauteslager, T., O’Sullivan, J., Reilly, R. B., & Lalor, E. C. (2014) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2014.6943841 · Cited by 9
Towards a Speech BCI Using ECoG
Leuthardt, E. C., Cunningham, J. P., & Barbour, D. L. (2013) · Springer briefs in electrical and computer engineering · doi:10.1007/978-3-642-36083-1_10 · Cited by 9
Towards Using EEG to Improve ASR Accuracy
Chen, Y.-N., Chang, K.-M. K., & Mostow, J. (2012) · North American Chapter of the Association for Computational Linguistics · Cited by 9
Wang, S., Deng, Y., Bao, Z., Zhan, X., & Duan, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.26301 · Cited by 10
EEG-based decoding of auditory attention to conversations with turn-taking speakers
de Ryck, I. V., Heintz, N., Rotaru, I., Geirnaert, S., Bertrand, A., & Francart, T. (2026) · Hearing Research · doi:10.1016/j.heares.2026.109539 · Cited by 8
Direct Cortical Neurophysiology of Speech Perception
Leonard, M. K., & Chang, E. F. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-407794-2.00039-0 · Cited by 8
Self-citationHurmalainen, A., Gemmeke, J. F., & Virtanen, T. I. (2013) · Lirias · Cited by 8
The reflexive imagery task: An experimental paradigm for neuroimaging
Cho, H., Dou, W., Reyes, Z., Geisler, M. W., & Morsella, E. (2018) · AIMS neuroscience · doi:10.3934/neuroscience.2018.2.97 · Cited by 8
EEG Correlates of Involuntary Cognitions in the Reflexive Imagery Task
Dou, W., Allen, A. K., Cho, H., Bhangal, S., Cook, A. J., Morsella, E., & Geisler, M. W. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00482 · Cited by 8
Spatiotemporal dynamics of human high gamma discriminate naturalistic behavioral states
Alasfour, A., Gabriel, P., Jiang, X., Shamie, I., Melloni, L., Thesen, T., Dugan, P. C., Friedman, D. J., Doyle, W., Devinsky, O., et al. (2022) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010401 · Cited by 8
“These cortical regions of interest have been shown to be engaged in motor [27] and language [28] related behaviors, respectively.”
BackgroundGonering, B., & Corina, D. P. (2023) · Frontiers in Language Sciences · doi:10.3389/flang.2023.1176233 · Cited by 8
Woolley, S. M. N. (2013) · Springer eBooks · doi:10.1007/978-1-4614-8400-4_4 · Cited by 8
Encoding of natural sounds by variance of the cortical local field potential
Ding, N., Simon, J. Z., Shamma, S., & David, S. V. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00652.2015 · Cited by 8
Self-citationThe phonetics-phonology relationship in the neurobiology of language
Grimaldi, M. (2018) · De Gruyter eBooks · doi:10.1515/9781501506734-003 · Cited by 8
Neural Coding of Noisy and Reverberant Speech in Human Auditory CortexPreprint
Puvvada, K. C., Villafañe-Delgado, M., Brodbeck, C. J., & Simon, J. Z. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/229153 · Cited by 8
“Here, using Magnetoencephalography (MEG) recordings of human subjects listening to continuous speech, and linear system methods of neural response prediction (encoding) and stimulus reconstruction (decoding) (Di Liberto et al. 2015; Ding and Simon 2012b; Pasley et al. 2012), we investigated the effect of noise and rev…”
BackgroundMethodLinguistics and human brain: a perspective of computational neuroscience
Zhang, F., Chai, B., Wu, Y., Siok, W. T., & Wang, N. (2026) · Cognitive Neurodynamics · doi:10.1007/s11571-026-10496-2 · Cited by 7
“Decoding studies further demonstrated that phonetic, lexical, and semantic information can be recovered from cortical activity during speech processing, especially in intracranial recordings (Pasley et al. 2012; Moses et al. 2016; Khanna et al. 2024).”
Individual Word Classification During Imagined Speech Using Intracranial Recordings
Martin, S., Iturrate, I., Brunner, P., Millán, J. D. R., Schalk, G., Knight, R. T., & Pasley, B. N. (2019) · Springer briefs in electrical and computer engineering · doi:10.1007/978-3-030-05668-1_8 · Cited by 7
Self-citationSpeech Perception with Noise Vocoding and Background Noise: An EEG and Behavioral Study
Dong, Y. X., & Gai, Y. (2021) · Journal of the Association for Research in Otolaryngology · doi:10.1007/s10162-021-00787-2 · Cited by 7
“In particular, the frequency range of 2─8 Hz in cortical responses is found to be linearly correlated to speech envelope (Pasley et al. 2012; Zion Golumbic et al. 2013; O’Sullivan et al. 2015).”
BackgroundInfluentialLinguistic representation of vowels in speech imagery EEG
Nitta, T., Horikawa, J., Iribe, Y., Taguchi, R., Katsurada, K., Shinohara, S., & Kawai, G. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1163578 · Cited by 7
“In the field of neural decoding for direct communication in brain-computer interfaces (BCIs), research is progressing for detecting spoken signals from multi-channel electrocorticograms (ECoGs) at the brain cortex (Knight and Heinze, 2008; Pasley et al., 2012; Bouchard et al., 2013; Flinker et al., 2015; Herff and Sch…”
BackgroundA Framework for Speech Activity Detection Using Adaptive Auditory Receptive Fields
Carlin, M. A., & Elhilali, M. (2015) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2015.2481179 · Cited by 7
“speech [89] and the nature of how top-down adaptive feedback”
Fast and robust estimation of spectro-temporal receptive fields using stochastic approximations
Meyer, A. F., Diepenbrock, J.-P., Ohl, F. W., & Anemüller, J. (2015) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2015.02.009 · Cited by 7
The effect of topic familiarity and volatility of auditory scene on selective auditory attention
Park, J. J., Baek, S.-C., Suh, M., Choi, J., Kim, S. J., & Lim, Y. (2023) · Hearing Research · doi:10.1016/j.heares.2023.108770 · Cited by 7
“( Ding and Simon, 2012a , 2012b ; Mesgarani and Chang, 2012 ; Zion Golumbic et al., 2013 ).”
Le sujet neurocognitif et l’éducation : un paradoxe ?
Caussidier, C. (2014) · Éducation et socialisation · doi:10.4000/edso.979 · Cited by 7
Consciousness and the Invention of Morel
Perogamvros, L. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00061 · Cited by 7
“…that the dreaming experience is multimodal, an ideal dream decoder should be based not only on sensory decoding models (Nishimoto et al., 2011; Pasley et al., 2012) (see section “Decoding Perceptual Reality from Brain Activity”), but also on semantic (Schwartz, 2004; Mitchell et al., 2008;…”
BackgroundBiologically inspired methods in speech recognition and synthesis: closing the loop
Bekolay, T. (2016) · Figshare · doi:10.6084/m9.figshare.2064072 · Cited by 7
BP-GPT: Auditory Neural Decoding Using fMRI-prompted LLM
Chen, X., Du, C., Liu, C., Wang, Y., & He, H. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890142 · Cited by 6
Meta-Learning an In-Context Transformer Model of Human Higher Visual Cortex
Yu, M.-L., Nan, M., Adeli, H., Prince, J. S., Pyles, J. A., Wehbe, L., Henderson, M. M., Tarr, M. J., & Luo, A. F. (2025) · neural information processing systems · doi:10.52202/085713-5606 · Cited by 6
Schneider, F., & Blank, H. (2026) · PLoS Biology · doi:10.1371/journal.pbio.3003588 · Cited by 6
TF-NSSE: A time–frequency domain neuro-steered speaker extractor
Qiu, Z., Gu, J., Yao, D., Li, J., & Yan, Y. (2023) · Applied Acoustics · doi:10.1016/j.apacoust.2023.109519 · Cited by 6
Inhibition does not affect the timing code for vocalizations in the mouse auditory midbrain
Dimitrov, A. G., Cummins, G. I., Mayko, Z. M., & Portfors, C. V. (2014) · Frontiers in Physiology · doi:10.3389/fphys.2014.00140 · Cited by 6
“…in the population activity of neurons in the human auditory cortex can be used to accurately identify segments of speech (Mesgarani and Chang, 2012; Pasley et al., 2012). www.frontiersin.org April 2014 | Volume 5 | Article 140 | 15 Given the much higher information capacity and energy efficiency…”
MethodO’Sullivan, C., Weible, A. P., & Wehr, M. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.01394 · Cited by 6
“Early in the auditory pathway, speech-evoked neural activity appears to encode the acoustic structure of speech sounds, whereas in higher order auditory areas, it appears to encode progressively more categorical aspects of speech (Chang et al., 2010; Flinker et al., 2011; Tsunada et al., 2011, 2016; Pasley et al., 201…”
BackgroundTemporal Coherence Principle in Scene Analysis
Shamma, S., & Elhilali, M. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-809324-5.24252-1 · Cited by 6
Self-citationAttentional Inhibition Ability Predicts Neural Representation During Challenging Auditory Streaming
Belo, J., Clerc, M., & Schön, D. (2025) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-024-01260-2 · Cited by 6
“…in the presence of concurrent streams in general and in a dichotic context (i.e., different sound in each ear) in particular, the cortical tracking of the attended source is enhanced compared with the unattended source (O’Sullivan et al., 2015; Pasley et al., 2012; Zion Golumbic et al., 2013).”
Kernel Convolution Model for Decoding Sounds from Time-Varying Neural Responses
Faisal, A., Nora, A., Seol, J., Renvall, H., & Salmelin, R. H. (2015) · International Workshop on Pattern Recognition in NeuroImaging · doi:10.1109/prni.2015.10 · Cited by 6
“to investigate how the primary auditory cortex neurons encode spectro-temporal features in invasive recordings of ferrets [2] and humans [3], to study the robustness and the extent to which perceptual aspects are coded in the cortical representation [4], and to characterizing stimulus-response function of auditory neu…”
BackgroundBioelectric Measurements: Magnetoencephalography
Mäkelä, J. P. (2014) · Elsevier eBooks · doi:10.1016/b978-0-444-53632-7.00508-6 · Cited by 6
Wireless Neural Interface Design
Yeager, D. J. (2014) · eScholarship (California Digital Library) · Cited by 6
Analyzing the perception of natural music with EEG and ECoG
Sturm, I. (2016) · DepositOnce · doi:10.14279/depositonce-4980 · Cited by 6
“ECoG offers unprecedented opportunities in challenging analysis conditions, such as those created by naturalistic listening scenarios (Brunet et al., 2014, Derix et al., 2014, Kubanek et al., 2013, Majima et al., 2014, Pasley et al., 2012, Potes et al., 2012).”
BackgroundMethod
Metadata
- Authors
- Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., Knight, R. T., & Chang, E. F.
- Affiliations
- University of California, Berkeley (US)
- University of Maryland, College Park (US)
- University of California, San Francisco (US)
- Johns Hopkins University (US)
- Corresponding author
- Pasley, Brian N.
- ORCID
- Venue
- PLoS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-01-31
- Volume
- 10
- Issue
- 1
- Pages
- e1001251
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC3269422
- OpenAlex
- W2005066437
- Cited by
- 604
- References
- 61
- Citations by year
- 2026: 21 · 2025: 30 · 2024: 28 · 2023: 35 · 2022: 36 · 2021: 32 · 2020: 51 · 2019: 66 · 2018: 42 · 2017: 53 · 2016: 43 · 2015: 51 · 2014: 42 · 2013: 50 · 2012: 24
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- National Institutes of Health (2r01ns021135-33, 5r00ns065120-04)
- U.S. Public Health Service (P04813)
- National Institute on Deafness and Other Communication Disorders (K99DC010439, K99DC012804, R01DC012379)
- National Institute of Neurological Disorders and Stroke (NS-40596, NS-21135, K99NS065120, R56NS021135, R37NS021135, R01NS040596, R00-NS065120, R01 NS-021135)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Keywords
- speech perception, auditory cortex, superior temporal gyrus, neural decoding, speech reconstruction, acoustic features, syllable rate, word recognition, nonlinear decoding, speech intelligibility
- MeSH terms
- Algorithms, Auditory Cortex, Brain Mapping, Computer Simulation, Electrodes, Implanted, Electroencephalography, Female, Humans, Male, Models, Biological, Speech Acoustics, Linear Models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.