Neural Encoding of Auditory Features during Music Perception and Imagery
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Despite many behavioral and neuroimaging investigations, it remains unclear how the human cortex represents spectrotemporal sound features during auditory imagery, and how this representation compares to auditory perception. To assess this, we recorded electrocorticographic signals from an epileptic patient with proficient music ability in 2 conditions. First, the participant played 2 piano pieces on an electronic piano with the sound volume of the digital keyboard on. Second, the participant replayed the same piano pieces, but without auditory feedback, and the participant was asked to imagine hearing the music in his mind. In both conditions, the sound output of the keyboard was recorded, thus allowing precise time-locking between the neural activity and the spectrotemporal content of the music imagery. This novel task design provided a unique opportunity to apply receptive field modeling techniques to quantitatively study neural encoding during auditory mental imagery. In both conditions, we built encoding models to predict high gamma neural activity (70-150 Hz) from the spectrogram representation of the recorded sound. We found robust spectrotemporal receptive fields during auditory imagery with substantial, but not complete overlap in frequency tuning and cortical location compared to receptive fields measured during auditory perception.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kosslyn, S. M., Ganis, G., & Thompson, W. L. (2001) · Nature Reviews Neuroscience · doi:10.1038/35090055 · Cited by 1,894 · cited 3×
“This is in accordance with previous research that showed partial overlap across conditions (Griffiths 1999; Kosslyn et al. 2001; Yoo et al. 2001; Halpern et al. 2004; Bunzeck et al. 2005).”
BackgroundResultNatural speech reveals the semantic maps that tile human cerebral cortex
Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016) · Nature · doi:10.1038/nature17637 · Cited by 1,504
Phonetic Feature Encoding in Human Superior Temporal Gyrus
Mesgarani, N., Cheung, C., Johnson, K., & Chang, E. F. (2014) · Science · doi:10.1126/science.1245994 · Cited by 993 · cited 2×
“For instance, encoding models have revealed the neural tuning properties of various speech features, such as acoustic, phonetic, and semantic representations (Pasley et al. 2012; Tankus et al. 2012; Mesgarani et al. 2014; Lotte et al. 2015; Huth et al. 2016).”
BackgroundSelf-citationAuditory-motor interaction revealed by fMRI: speech, music, and working memory in area Spt
Hickok, G., Buchsbaum, B., Humphries, C., & Muftuler, T. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903322307393 · Cited by 700 · cited 1×
“2009), the frontal cortex, the sylvian parietal temporal area (Hickok et al. 2003), ventrolateral and dorsolateral cortices (Meyer et al.”
BackgroundCortical activity during motor execution, motor imagery, and imagery-based online feedback
Miller, K. J., Schalk, G., Fetz, E. E., den Nijs, M., Ojemann, J. G., & Rao, R. P. N. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0913697107 · Cited by 622 · cited 1×
“Similar patterns of partial functional and anatomical overlap have been found in the visual system (Kosslyn and Thompson 2000, 2003) and in the motor system (Roth et al. 1996; Miller et al. 2010).”
BackgroundReconstructing Speech from Human Auditory Cortex
Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., Knight, R. T., & Chang, E. F. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001251 · Cited by 604 · cited 6×
“Auditory perception has been extensively studied in animal models and humans using spectrotemporal receptive field (STRFs) analysis (Aertsen et al. 1981; Clopton and Backoff 1991; Theunissen et al. 2000; Chi et al. 2005; Pasley et al. 2012), which identifies the time-frequency stimulus features encoded by a neuron or…”
BackgroundMethodInfluentialSelf-citationWhen is early visual cortex activated during visual mental imagery?
Kosslyn, S. M., & Thompson, W. L. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.5.723 · Cited by 515
Neural decoding of visual imagery during sleep
Horikawa, T., Tamaki, M., Miyawaki, Y., & Kamitani, Y. (2013) · Science · doi:10.1126/science.1234330 · Cited by 498
Induced electrocorticographic gamma activity during auditory perception
Crone, N. E., Boatman, D., Gordon, B., & Hao, L. (2001) · Clinical Neurophysiology · doi:10.1016/s1388-2457(00)00545-9 · Cited by 494
Hearing in the Mind's Ear: A PET Investigation of Musical Imagery and Perception
Zatorre, R. J., Halpern, A. R., Perry, D. W., Meyer, E., & Evans, A. C. (1996) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1996.8.1.29 · Cited by 439 · cited 3×
“Studies have shown the importance of both hemispheres for auditory perception and imagination (Zatorre and Halpern 1993; Zatorre et al. 1996; Griffiths 1999; Rauschecker 2001; Halpern et al. 2004; Kraemer et al. 2005).”
BackgroundMental Concerts: Musical Imagery and Auditory Cortex
Zatorre, R. J., & Halpern, A. R. (2005) · Neuron · doi:10.1016/j.neuron.2005.06.013 · Cited by 420
Auditory imagery: Empirical findings
Hubbard, T. L. (2010) · Psychological Bulletin · doi:10.1037/a0018436 · Cited by 310
Sound of silence activates auditory cortex
Kraemer, D. J. M., Macrae, C. N., Green, A. E., & Kelley, W. M. (2005) · Nature · doi:10.1038/434158a · Cited by 278
Decoding spectrotemporal features of overt and covert speech from the human cortex
Martin, S., Brunner, P., Holdgraf, C., Heinze, H.-J., Crone, N. E., Rieger, J., Schalk, G., Knight, R. T., & Pasley, B. N. (2014) · Frontiers in Neuroengineering · doi:10.3389/fneng.2014.00014 · Cited by 200 · cited 2×
“Finally, to further assess the reconstruction accuracy, we evaluated the ability to identify isolated piano notes from the test set auditory spectrogram reconstructions—using similar approach as in (Pasley et al. 2012; Martin et al. 2014).”
BackgroundMethodSelf-citationPerceptual restoration of masked speech in human cortex
Leonard, M. K., Baud, M. O., Sjerps, M. J., & Chang, E. F. (2016) · Nature Communications · doi:10.1038/ncomms13619 · Cited by 170
Self-citationMind's Ear in a Musician: Where and When in the Brain
Schürmann, M., Raij, T., Fujiki, N., & Hari, R. (2002) · NeuroImage · doi:10.1006/nimg.2002.1098 · Cited by 67
Cited by
Towards reconstructing intelligible speech from the human auditory cortex
Akbari, H., Khalighinejad, B., Herrero, J. L., Mehta, A. D., & Mesgarani, N. (2019) · Scientific Reports · doi:10.1038/s41598-018-37359-z · Cited by 221
Decoding Inner Speech Using Electrocorticography: Progress and Challenges Toward a Speech Prosthesis
Martin, S., Iturrate, I., Millán, J. D. R., Knight, R. T., & Pasley, B. N. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00422 · Cited by 115
“This article is adapted from the following doctorate thesis: Understanding and decoding imagined speech using intracranial recordings in the human brain (Martin, 2017).”
BackgroundMethodSelf-citationCommon and distinct neural representations of imagined and perceived speech
Lu, L., Han, M., Zou, G., Zheng, L., & Gao, J.-H. (2023) · Cerebral Cortex · doi:10.1093/cercor/bhac519 · Cited by 11
Metadata
- Authors
- Martin, S., Mikutta, C., Leonard, M. K., Hungate, D., Koelsch, S., Shamma, S., Chang, E. F., Millán, J. D. R., Knight, R. T., & Pasley, B. N.
- Venue
- Cerebral Cortex
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-10-27
- Volume
- 28
- Issue
- 12
- Pages
- 4222-4233
- ISSN
- 1047-3211 (ISSN-L)
- 1460-2199
- OpenAlex
- W2767197763
- Cited by
- 61
- References
- 16
- FWCI
- 2.43
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.