Reconstructing the spectrotemporal modulations of real-life sounds from fMRI response patterns
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Significance: The sounds we encounter in everyday life (e.g. speech, voices, animal cries, wind, rain) are complex and various. How the human brain analyses their acoustics remains largely unknown. This research shows that mathematical modelling in combination with high spatial resolution functional magnetic resonance imaging enables reverse engineering of the human brain computations underlying real-life listening. Importantly, the research reveals that even general auditory processing mechanisms in the human brain are optimized for fine-grained analysis of the most behaviorally relevant sounds (i.e., speech, voices). Most likely, this observation reflects the evolutionary feat that, for humans, discriminating between speech sounds is more crucial than distinguishing, for example, between barking dogs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C. (1995) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910330508 · Cited by 3,202
Human temporal lobe activation by speech and nonspeech sounds
Binder, J. R. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.5.512 · Cited by 1,345 · cited 1×
“Beyond the acoustic analysis, fMRI response patterns in the superior temporal cortex relate to higher levels of sound processing, as required, for example, for the perceptual and cognitive processing of speech (24, 25), voice (26, 48), or music (49, 50).”
BackgroundPoeppel, D. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00107-3 · Cited by 1,315
Goebel, R., Esposito, F., & Formisano, E. (2006) · Human Brain Mapping · doi:10.1002/hbm.20249 · Cited by 1,053 · cited 1×
“We performed cortex-based alignment (52) of all 10 subjects included in this study.”
MethodSelf-citationReconstructing Visual Experiences from Brain Activity Evoked by Natural Movies
Nishimoto, S., Vu, A. T., Naselaris, T., Benjamini, Y., Yu, B., & Gallant, J. L. (2011) · Current Biology · doi:10.1016/j.cub.2011.08.031 · Cited by 933
Reconstructing Speech from Human Auditory Cortex
Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., Knight, R. T., & Chang, E. F. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001251 · Cited by 604 · cited 3×
“Although consistent with recent reports of speech spectrogram reconstruction from ECoG recordings (12), this result is surprising, given the low temporal resolution of fMRI and the coarse temporal sampling [repetition time (TR) = 2.”
BackgroundResultWho “Is Saying What”? Brain-based decoding of human voice and speech
Formisano, E., De Martino, F., Bonte, M., & Goebel, R. (2008) · Science · doi:10.1126/science.1164318 · Cited by 599 · cited 1×
“Beyond the acoustic analysis, fMRI response patterns in the superior temporal cortex relate to higher levels of sound processing, as required, for example, for the perceptual and cognitive processing of speech (24, 25), voice (26, 48), or music (49, 50).”
BackgroundSelf-citationMiyawaki, Y., Uchida, H., Yamashita, O., Sato, M.-A., Morito, Y., Tanabe, H. C., Sadato, N., & Kamitani, Y. (2008) · Neuron · doi:10.1016/j.neuron.2008.11.004 · Cited by 543 · cited 1×
“Unlike the more common classification-based decoding, which only allows discriminating between a small set of stimulus categories, this approach embeds a representational model of the stimuli in terms of elementary features, thereby enabling the identification of individual arbitrary stimuli from brain response patter…”
Recurrent turnover of senescent cells during regeneration of a complex structure
Yun, M. H., Davaapil, H., & Brockes, J. P. (2015) · eLife · doi:10.7554/elife.05505 · Cited by 385
Representation of the temporal envelope of sounds in the human brain
Giraud, A.-L., Lorenzi, C., Ashburner, J., Wable, J., Johnsrude, I., Frackowiak, R., & Kleinschmidt, A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.3.1588 · Cited by 353
Cited by
Speech synthesis from ECoG using densely connected 3D convolutional neural networks
Angrick, M., Herff, C., Mugler, E., Tate, M. C., Slutzky, M. W., Krusienski, D. J., & Schultz, T. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab0c59 · Cited by 225
“In [24], Santoro et al reconstructed the spectro-temporal modulations of real-life sounds from fMRI response patterns.”
BackgroundTowards reconstructing intelligible speech from the human auditory cortex
Akbari, H., Khalighinejad, B., Herrero, J. L., Mehta, A. D., & Mesgarani, N. (2019) · Scientific Reports · doi:10.1038/s41598-018-37359-z · Cited by 221
Herff, C., Diener, L., Angrick, M., Mugler, E., Tate, M. C., Goldrick, M. A., Krusienski, D. J., Slutzky, M. W., & Schultz, T. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.01267 · Cited by 118
Hierarchy of speech-driven spectrotemporal receptive fields in human auditory cortex
Venezia, J. H., Thurman, S. M., Richards, V. M., & Hickok, G. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.11.049 · Cited by 47
Salient experiences are represented by unique transcriptional signatures in the mouse brain
Mukherjee, D., Ignatowska-Jankowska, B. M., Itskovits, E., Gonzales, B. J., Turm, H., Izakson, L., Haritan, D., Bleistein, N., Cohen, C., Amit, I., et al. (2018) · eLife · doi:10.7554/elife.31220 · Cited by 46
Metadata
- Authors
- Santoro, R., Moerel, M., De Martino, F., Valente, G., Ugurbil, K., Yacoub, E., & Formisano, E.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2017-04-18
- Volume
- 114
- Issue
- 18
- Pages
- 4799-4804
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2607268939
- Cited by
- 129
- References
- 10
- FWCI
- 7.73
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reconstructing the spectrotemporal modulations of real-life sounds from fMRI response patterns. https://app.cochat.ai/papers/10.1073/pnas.1617622114
- The paper
- Santoro, R., Moerel, M., De Martino, F., Valente, G., Ugurbil, K., Yacoub, E., & Formisano, E. (2017). Reconstructing the spectrotemporal modulations of real-life sounds from fMRI response patterns. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1617622114
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.