The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
The Hierarchical Cortical Organization of Human Speech Processing
Open accessAbstract
Speech comprehension requires that the brain extract semantic meaning from the spectral features represented at the cochlea. To investigate this process, we performed an fMRI experiment in which five men and two women passively listened to several hours of natural narrative speech. We then used voxelwise modeling to predict BOLD responses based on three different feature spaces that represent the spectral, articulatory, and semantic properties of speech. The amount of variance explained by each feature space was then assessed using a separate validation dataset. Because some responses might be explained equally well by more than one feature space, we used a variance partitioning analysis to determine the fraction of the variance that was uniquely explained by each feature space. Consistent with previous studies, we found that speech comprehension involves hierarchical representations starting in primary auditory areas and moving laterally on the temporal lobe: spectral features are found in the core of A1, mixtures of spectral and articulatory in STG, mixtures of articulatory and semantic in STS, and semantic in STS and beyond. Our data also show that both hemispheres are equally and actively involved in speech perception and interpretation. Further, responses as early in the auditory hierarchy as in STS are more correlated with semantic than spectral representations. These results illustrate the importance of using natural speech in neurolinguistic research. Our methodology also provides an efficient way to simultaneously test multiple specific hypotheses about the representations of speech without using block designs and segmented or synthetic speech. SIGNIFICANCE STATEMENT To investigate the processing steps performed by the human brain to transform natural speech sound into meaningful language, we used models based on a hierarchical set of speech features to predict BOLD responses of individual voxels recorded in an fMRI experiment while subjects listened to natural speech. Both cerebral hemispheres were actively involved in speech processing in large and equal amounts. Also, the transformation from spectral features to semantic elements occurs early in the cortical speech-processing stream. Our experimental and analytical approaches are important alternatives and complements to standard approaches that use segmented speech and block designs, which report more laterality in speech processing and associated semantic processing to higher levels of cortex than reported here.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Welch, P. (1967) · IEEE Transactions on Audio and Electroacoustics · doi:10.1109/tau.1967.1161901 · Cited by 12,277
Cortical Surface-Based Analysis
Dale, A. M., Fischl, B., & Sereno, M. I. (1999) · NeuroImage · doi:10.1006/nimg.1998.0395 · Cited by 10,910
A global optimisation method for robust affine registration of brain images
Jenkinson, M., & Smith, S. (2001) · Medical Image Analysis · doi:10.1016/s1361-8415(01)00036-6 · Cited by 6,872
Speaking: from intention to articulation
(1989) · Choice Reviews Online · doi:10.5860/choice.27-1947 · Cited by 5,003
Binder, J. R., Desai, R. H., Graves, W. W., & Conant, L. L. (2009) · Cerebral Cortex · doi:10.1093/cercor/bhp055 · Cited by 4,060
Speaking. From Intention to Articulation
Caplan, D. (1990) · Trends in Neurosciences · doi:10.1016/0166-2236(90)90151-y · Cited by 4,008
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866
Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing
Rauschecker, J. P., & Scott, S. K. (2009) · Nature Neuroscience · doi:10.1038/nn.2331 · Cited by 1,766
Natural speech reveals the semantic maps that tile human cerebral cortex
Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016) · Nature · doi:10.1038/nature17637 · Cited by 1,504 · cited 14×
“We then used a voxelwise modeling framework (Nishimoto and Gallant, 2011; Huth et al., 2012, 2016) to build predictive models based on three distinct feature spaces: spectral features, articulatory features, and semantic features.”
BackgroundMethodResultInfluentialSelf-citationThe anatomy of language: a review of 100 fMRI studies published in 2009
Price, C. J. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2010.05444.x · Cited by 1,387
Human temporal lobe activation by speech and nonspeech sounds
Binder, J. R. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.5.512 · Cited by 1,345 · cited 1×
“In a typical experiment, responses are measured to stimuli that differ along a single dimension of interest (Binder et al., 2000; Leaver and Rauschecker, 2010); for review, see DeWitt and Rauschecker, 2012), and then subtraction is used to find brain regions that respond significantly more to one…”
MethodPredicting Human Brain Activity Associated with the Meanings of Nouns
Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K.-M., Malave, V. L., Mason, R. A., & Just, M. A. (2008) · Science · doi:10.1126/science.1152876 · Cited by 1,180
Selective cortical representation of attended speaker in multi-talker speech perception
Mesgarani, N., & Chang, E. F. (2012) · Nature · doi:10.1038/nature11020 · Cited by 1,007
Phonetic Feature Encoding in Human Superior Temporal Gyrus
Mesgarani, N., Cheung, C., Johnson, K., & Chang, E. F. (2014) · Science · doi:10.1126/science.1245994 · Cited by 993
Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies
Nishimoto, S., Vu, A. T., Naselaris, T., Benjamini, Y., Yu, B., & Gallant, J. L. (2011) · Current Biology · doi:10.1016/j.cub.2011.08.031 · Cited by 933
Self-citationProcessing of Complex Sounds in the Macaque Nonprimary Auditory Cortex
Rauschecker, J. P., Tian, B., & Hauser, M. (1995) · Science · doi:10.1126/science.7701330 · Cited by 929
Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party”
Zion Golumbic, E. M., Ding, N., Bickel, S., Lakatos, P., Schevon, C. A., McKhann, G. M., Goodman, R. R., Emerson, R., Mehta, A. D., Simon, J. Z., et al. (2013) · Neuron · doi:10.1016/j.neuron.2012.12.037 · Cited by 909
Huth, A. G., Nishimoto, S., Vu, A. T., & Gallant, J. L. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.014 · Cited by 896
Self-citationListening to speech activates motor areas involved in speech production
Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2004) · Nature Neuroscience · doi:10.1038/nn1263 · Cited by 885 · cited 1×
“In the speech-related sPMv (Wilson et al., 2004), we found that 43% of voxels in the left hemisphere and 41% of voxels in the right hemisphere were story responsive.”
BackgroundTopographic mapping of a hierarchy of temporal receptive windows using a narrated story
Lerner, Y., Honey, C. J., Silbert, L. J., & Hasson, U. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3684-10.2011 · Cited by 834
Functional Specialization in Rhesus Monkey Auditory Cortex
Tian, B., Reser, D., Durham, A., Kustov, A., & Rauschecker, J. P. (2001) · Science · doi:10.1126/science.1058911 · Cited by 728
The neuroanatomical and functional organization of speech perception
Scott, S. K., & Johnsrude, I. S. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(02)00037-1 · Cited by 705
Functional organization of human sensorimotor cortex for speech articulation
Bouchard, K. E., Mesgarani, N., Johnson, K., & Chang, E. F. (2013) · Nature · doi:10.1038/nature11911 · Cited by 686 · cited 1×
“…appear in intermediate auditory processing areas in the temporal lobe (DeWitt and Rauschecker, 2012; Mesgarani et al., 2014), while representations based on vocal gestures are thought to appear in premotor areas of the frontal lobe and in Broca’s areas (Bouchard et al., 2013; Flinker et al., 2015).”
BackgroundCerebral lateralization of language in normal left-handed people studied by functional MRI
Pujol, J., Deus, J., Losilla, J. M., & Capdevila, A. (1999) · Neurology · doi:10.1212/wnl.52.5.1038 · Cited by 686
Visser, M., Jefferies, E., & Lambon Ralph, M. A. (2010) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21309 · Cited by 647
The Neural Mechanisms of Speech Comprehension: fMRI studies of Semantic Ambiguity
Rodd, J. M., Davis, M. H., & Johnsrude, I. S. (2005) · Cerebral Cortex · doi:10.1093/cercor/bhi009 · Cited by 632
Hierarchical and asymmetric temporal sensitivity in human auditory cortices
Boemio, A., Fromm, S., Braun, A., & Poeppel, D. (2005) · Nature Neuroscience · doi:10.1038/nn1409 · Cited by 582
Low-Frequency Cortical Entrainment to Speech Reflects Phoneme-Level Processing
Di Liberto, G. M., O’Sullivan, J. A., & Lalor, E. C. (2015) · Current Biology · doi:10.1016/j.cub.2015.08.030 · Cited by 578
Phase-Locked Responses to Speech in Human Auditory Cortex are Enhanced During Comprehension
Peelle, J. E., Gross, J., & Davis, M. H. (2013) · Cerebral Cortex · doi:10.1093/cercor/bhs118 · Cited by 568
Modulation spectra of natural sounds and ethological theories of auditory processing
Singh, N. C., & Theunissen, F. E. (2003) · The Journal of the Acoustical Society of America · doi:10.1121/1.1624067 · Cited by 521
Self-citationRedefining the role of Broca’s area in speech
Flinker, A., Korzeniewska, A., Shestyuk, A. Y., Franaszczuk, P. J., Dronkers, N. F., Knight, R. T., & Crone, N. E. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1414491112 · Cited by 487
Phoneme and word recognition in the auditory ventral stream
DeWitt, I., & Rauschecker, J. P. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1113427109 · Cited by 465
Distinct Cortical Pathways for Music and Speech Revealed by Hypothesis-Free Voxel Decomposition
Norman-Haignere, S., Kanwisher, N. G., & McDermott, J. H. (2015) · Neuron · doi:10.1016/j.neuron.2015.11.035 · Cited by 465
The Modulation Transfer Function for Speech Intelligibility
Elliott, T. M., & Theunissen, F. E. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000302 · Cited by 461
Self-citationDepireux, D. A., Simon, J. Z., Klein, D. J., & Shamma, S. A. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.85.3.1220 · Cited by 428
A computational model of filtering, detection, and compression in the cochlea
Lyon, R. (2005) · ICASSP '82. IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1982.1171644 · Cited by 411
Leaver, A. M., & Rauschecker, J. P. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.0296-10.2010 · Cited by 376
Attention during natural vision warps semantic representation across the human brain
Çukur, T., Nishimoto, S., Huth, A. G., & Gallant, J. L. (2013) · Nature Neuroscience · doi:10.1038/nn.3381 · Cited by 351
Self-citationSpectrotemporal Receptive Fields in the Lemniscal Auditory Thalamus and Cortex
Miller, L. M., Escabí, M. A., Read, H. L., & Schreiner, C. E. (2002) · Journal of Neurophysiology · doi:10.1152/jn.00395.2001 · Cited by 350
MP RAGE: a three-dimensional, T1-weighted, gradient-echo sequence--initial experience in the brain.
Brant-Zawadzki, M., Gillan, G. D., & Nitz, W. R. (1992) · Radiology · doi:10.1148/radiology.182.3.1535892 · Cited by 342
Spectro-temporal modulation transfer functions and speech intelligibility
Chi, T., Gao, Y., Guyton, M. C., Ru, P., & Shamma, S. (1999) · The Journal of the Acoustical Society of America · doi:10.1121/1.428100 · Cited by 336
Event-Related fMRI of the Auditory Cortex
Belin, P., Zatorre, R. J., Hoge, R., Evans, A. C., & Pike, B. (1999) · NeuroImage · doi:10.1006/nimg.1999.0480 · Cited by 293
Towards a New Neurobiology of Language
Poeppel, D., Emmorey, K., Hickok, G., & Pylkkänen, L. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.3244-12.2012 · Cited by 289
Right-Hemisphere Auditory Cortex Is Dominant for Coding Syllable Patterns in Speech
Abrams, D. A., Nicol, T., Zecker, S., & Kraus, N. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.0187-08.2008 · Cited by 282
Wehbe, L., Murphy, B., Talukdar, P., Fyshe, A., Ramdas, A., & Mitchell, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0112575 · Cited by 249
Neurobiological roots of language in primate audition: common computational properties
Bornkessel-Schlesewsky, I., Schlesewsky, M., Small, S. L., & Rauschecker, J. P. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.12.008 · Cited by 246
Localization of sublexical speech perception components
Turkeltaub, P. E., & Branch Coslett, H. (2010) · Brain and Language · doi:10.1016/j.bandl.2010.03.008 · Cited by 245
The cortical analysis of speech-specific temporal structure revealed by responses to sound quilts
Overath, T., McDermott, J. H., Zarate, J. M., & Poeppel, D. (2015) · Nature Neuroscience · doi:10.1038/nn.4021 · Cited by 241
Santoro, R., Moerel, M., De Martino, F., Goebel, R., Ugurbil, K., Yacoub, E., & Formisano, E. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003412 · Cited by 240
‘What’, ‘where’ and ‘how’ in auditory cortex
Belin, P., & Zatorre, R. J. (2000) · Nature Neuroscience · doi:10.1038/79890 · Cited by 235
Feature Analysis of Natural Sounds in the Songbird Auditory Forebrain
Sen, K., Theunissen, F. E., & Doupe, A. J. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.86.3.1445 · Cited by 225
Self-citationDiscrimination of speech from nonspeech based on multiscale spectro-temporal Modulations
Mesgarani, N., Slaney, M., & Shamma, S. A. (2006) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tsa.2005.858055 · Cited by 224
A Neurosemantic Theory of Concrete Noun Representation Based on the Underlying Brain Codes
Just, M. A., Cherkassky, V. L., Aryal, S., & Mitchell, T. M. (2010) · PLoS ONE · doi:10.1371/journal.pone.0008622 · Cited by 207
Suga, N., O'Neill, W. E., & Manabe, T. (1978) · Science · doi:10.1126/science.644320 · Cited by 205
Degree of language lateralization determines susceptibility to unilateral brain lesions
Knecht, S., Flöel, A., Dräger, B., Breitenstein, C., Sommer, J., Henningsen, H., Ringelstein, E. B., & Pascual-Leone, A. (2002) · Nature Neuroscience · doi:10.1038/nn868 · Cited by 196
Rethinking the neurological basis of language
Stowe, L. A., Haverkort, M., & Zwarts, F. (2005) · Lingua · doi:10.1016/j.lingua.2004.01.013 · Cited by 172
Fritz, J. B., Elhilali, M., David, S. V., & Shamma, S. A. (2007) · Hearing Research · doi:10.1016/j.heares.2007.01.009 · Cited by 167
Nishimoto, S., & Gallant, J. L. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.6801-10.2011 · Cited by 161
Self-citationHullett, P. W., Hamilton, L. S., Mesgarani, N., Schreiner, C. E., & Chang, E. F. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.1779-15.2016 · Cited by 160
Predicting neuronal responses during natural vision
David, S. V., & Gallant, J. L. (2005) · Network Computation in Neural Systems · doi:10.1080/09548980500464030 · Cited by 147
Self-citationLeft Posterior Temporal Regions are Sensitive to Auditory Categorization
Desai, R. H., Liebenthal, E., Waldron, E. J., & Binder, J. R. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20081 · Cited by 127
Auditory-Motor Processing of Speech Sounds
Möttönen, R., Dutton, R., & Watkins, K. E. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhs110 · Cited by 126
Lescroart, M. D., Stansbury, D. E., & Gallant, J. L. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00135 · Cited by 124
Self-citationHuman auditory cortex is sensitive to the perceived clarity of speech
Wild, C. J., Davis, M. H., & Johnsrude, I. S. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.035 · Cited by 116
Hsu, A., Borst, A., & Theunissen, F. E. (2004) · Network Computation in Neural Systems · doi:10.1088/0954-898x/15/2/002 · Cited by 116
Self-citationRapid tuning shifts in human auditory cortex enhance speech intelligibility
Holdgraf, C., de Heer, W. A., Pasley, B. N., Rieger, J. W., Crone, N. E., Lin, J. J., Knight, R. T., & Theunissen, F. E. (2016) · Nature Communications · doi:10.1038/ncomms13654 · Cited by 107
Self-citationFunctional Groups in the Avian Auditory System
Woolley, S. M. N., Gill, P. R., Fremouw, T., & Theunissen, F. E. (2009) · Journal of Neuroscience · doi:10.1523/jneurosci.2042-08.2009 · Cited by 97
Self-citationEvidence Optimization Techniques for Estimating Stimulus-Response Functions
Sahani, M., & Linden, J. F. (2002) · UCL Discovery (University College London) · Cited by 95
Noise-invariant Neurons in the Avian Auditory Cortex: Hearing the Song in Noise
Moore, R. C., Lee, T., & Theunissen, F. E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002942 · Cited by 76
Self-citationFedorenko, E. G., Nieto-Castañón, A., & Kanwisher, N. (2011) · Brain and Language · doi:10.1016/j.bandl.2011.01.001 · Cited by 73
Slaney, M. (1997) · Cited by 47
Common and distinct neural substrates for the perception of speech rhythm and intonation
Zhang, L., Shu, H., Zhou, F., Wang, X., & Li, P. (2010) · Human Brain Mapping · doi:10.1002/hbm.20922 · Cited by 47
Cited by
Kell, A. J. E., Yamins, D. L. K., Shook, E. N., Norman-Haignere, S. V., & McDermott, J. H. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.044 · Cited by 544
“Speech-selective responses emerge only in non-primary areas (de Heer et al., 2017; Evans and Davis, 2015; Mesgarani et al., 2014; Okada et al., 2010; Overath et al., 2015) and cannot be accounted for by modulation features standardly used to model primary areas (Norman-Haignere et al., 2015;…”
Broderick, M. P., Anderson, A. J., Di Liberto, G. M., Crosse, M. J., & Lalor, E. C. (2018) · Current Biology · doi:10.1016/j.cub.2018.01.080 · Cited by 421
The Encoding of Speech Sounds in the Superior Temporal Gyrus
Yi, H. G., Leonard, M. K., & Chang, E. F. (2019) · Neuron · doi:10.1016/j.neuron.2019.04.023 · Cited by 417
Semantic reconstruction of continuous language from non-invasive brain recordings
Tang, J., LeBel, A., Jain, S., & Huth, A. G. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01304-9 · Cited by 442
“This is likely a consequence of our decoding framework—the speech network is known to be consistently engaged in language processing, but it tends to represent lower-level articulatory and auditory features 6 , whereas our decoder operates on higher-level semantic features of entire word sequences.”
BackgroundSelf-citationKeep it real: rethinking the primacy of experimental control in cognitive neuroscience
Nastase, S. A., Goldstein, A., & Hasson, U. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117254 · Cited by 344
Metadata
- Authors
- de Heer, W. A., Huth, A. G., Griffiths, T. L., Gallant, J. L., & Theunissen, F. E.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2017-06-06
- Volume
- 37
- Issue
- 27
- Pages
- 6539-6557
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2622627557
- Cited by
- 277
- References
- 74
- FWCI
- 11.88
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.