The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Multiresolution spectrotemporal analysis of complex sounds
Abstract
A computational model of auditory analysis is described that is inspired by psychoacoustical and neurophysiological findings in early and central stages of the auditory system. The model provides a unified multiresolution representation of the spectral and temporal features likely critical in the perception of sound. Simplified, more specifically tailored versions of this model have already been validated by successful application in the assessment of speech intelligibility [Elhilali et al., Speech Commun. 41(2-3), 331–348 (2003); Chi et al., J. Acoust. Soc. Am. 106, 2719–2732 (1999)] and in explaining the perception of monaural phase sensitivity [R. Carlyon and S. Shamma, J. Acoust. Soc. Am. 114, 333–348 (2003)]. Here we provide a more complete mathematical formulation of the model, illustrating how complex signals are transformed through various stages of the model, and relating it to comparable existing models of auditory processing. Furthermore, we outline several reconstruction algorithms to resynthesize the sound from the model output so as to evaluate the fidelity of the representation and contribution of different features and cues to the sound percept.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural Networks: A Comprehensive Foundation
Haykin, S. (1998) · Cited by 30,070
Discrete-Time Signal Processing
Oppenheim, A. V., & Schafer, R. W. (1989) · Cited by 12,572
Numerical Methods for Unconstrained Optimization and Nonlinear Equations
Dennis, J. E., & Schnabel, R. B. (1996) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611971200 · Cited by 7,615
Phase retrieval algorithms: a comparison
Fienup, J. R. (1982) · Applied Optics · doi:10.1364/ao.21.002758 · Cited by 5,596
A practical algorithm for the determination of phase from image and diffraction plane pictures
Gerchberg, R. W. (1972) · Optik · Cited by 4,519
Perception of the speech code.
Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967) · Psychological Review · doi:10.1037/h0020279 · Cited by 3,722
Speech Recognition with Primarily Temporal Cues
Shannon, R. V., Zeng, F.-G., Kamath, V., Wygonski, J., & Ekelid, M. (1995) · Science · doi:10.1126/science.270.5234.303 · Cited by 3,024
Hermansky, H., & Morgan, N. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.326616 · Cited by 1,808
Synaptic depression and cortical gain control
Abbott, L. F., Varela, J. A., Sen, K., & Nelson, S. B. (1997) · Science · doi:10.1126/science.275.5297.221 · Cited by 1,690
Jones, J. P., & Palmer, L. A. (1987) · Journal of Neurophysiology · doi:10.1152/jn.1987.58.6.1233 · Cited by 1,658
Cited by
Cortical oscillations and speech processing: emerging computational principles and operations
Giraud, A.-L., & Poeppel, D. (2012) · Nature Neuroscience · doi:10.1038/nn.3063 · Cited by 1,917
Anden, J., & Mallat, S. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2326991 · Cited by 666
Reconstructing Speech from Human Auditory Cortex
Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., Knight, R. T., & Chang, E. F. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001251 · Cited by 604
“We investigated an alternative, nonlinear model based on modulation (described in detail in [18]).”
BackgroundMethodInfluentialSelf-citationPhase recovery, MaxCut and complex semidefinite programming
Waldspurger, I., d’Aspremont, A., & Mallat, S. (2015) · Mathematical Programming · doi:10.1007/s10107-013-0738-9 · Cited by 573
A complementary systems account of word learning: neural and behavioural evidence
Davis, M. H., & Gaskell, M. G. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0111 · Cited by 572
Kell, A. J. E., Yamins, D. L. K., Shook, E. N., Norman-Haignere, S. V., & McDermott, J. H. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.044 · Cited by 544
Phase Retrieval Using Alternating Minimization
Netrapalli, P., Jain, P., & Sanghavi, S. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2448516 · Cited by 535
Temporal modulations in speech and music
Ding, N., Patel, A. D., Chen, L., Butler, H., Luo, C., & Poeppel, D. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.011 · Cited by 502
Distinct Cortical Pathways for Music and Speech Revealed by Hypothesis-Free Voxel Decomposition
Norman-Haignere, S., Kanwisher, N. G., & McDermott, J. H. (2015) · Neuron · doi:10.1016/j.neuron.2015.11.035 · Cited by 465
Cortical entrainment to continuous speech: functional roles and interpretations
Ding, N., & Simon, J. Z. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00311 · Cited by 443
Automatic speech emotion recognition using modulation spectral features
Wu, S., Falk, T. H., & Chan, W.-Y. (2011) · Speech Communication · doi:10.1016/j.specom.2010.08.013 · Cited by 389
Distinct sensitivity to spectrotemporal modulation supports brain asymmetry for speech and melody
Albouy, P., Benjamin, L., Morillon, B., & Zatorre, R. J. (2020) · Science · doi:10.1126/science.aaz3468 · Cited by 298
Human Screams Occupy a Privileged Niche in the Communication Soundscape
Arnal, L. H., Flinker, A., Kleinschmidt, A., Giraud, A.-L., & Poeppel, D. (2015) · Current Biology · doi:10.1016/j.cub.2015.06.043 · Cited by 296
Towards reconstructing intelligible speech from the human auditory cortex
Akbari, H., Khalighinejad, B., Herrero, J. L., Mehta, A. D., & Mesgarani, N. (2019) · Scientific Reports · doi:10.1038/s41598-018-37359-z · Cited by 221
Decoding spectrotemporal features of overt and covert speech from the human cortex
Martin, S., Brunner, P., Holdgraf, C., Heinze, H.-J., Crone, N. E., Rieger, J., Schalk, G., Knight, R. T., & Pasley, B. N. (2014) · Frontiers in Neuroengineering · doi:10.3389/fneng.2014.00014 · Cited by 200
“This representation was generated by an affine wavelet transform of the sound pressure waveform using a 128 channel-auditory filter bank mimicking the frequency analysis of the auditory periphery (Chi et al., 2005).”
MethodVisual Imagery and Perception Share Neural Representations in the Alpha Frequency Band
Xie, S., Kaiser, D., & Cichy, R. M. (2020) · Current Biology · doi:10.1016/j.cub.2020.04.074 · Cited by 139
Decoding Inner Speech Using Electrocorticography: Progress and Challenges Toward a Speech Prosthesis
Martin, S., Iturrate, I., Millán, J. D. R., Knight, R. T., & Pasley, B. N. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00422 · Cited by 115
Neural representation of spectral and temporal information in speech
Young, E. D. (2008) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2007.2151 · Cited by 104
Spectro-temporal analysis of speech using 2-d Gabor filters
Ezzat, T., Bouvrie, J., & Poggio, T. (2007) · Interspeech 2007 · doi:10.21437/interspeech.2007-236 · Cited by 66
Model metamers reveal divergent invariances between biological and artificial neural networks
Feather, J., Leclerc, G., Mądry, A., & McDermott, J. H. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01442-0 · Cited by 59
“Analogous results with two classical sensory system models (HMAX 3,8 and a spectrotemporal modulation filterbank 42 ), which further illustrate the general applicability of the method, are shown in Extended Data Figs.”
ResultHierarchy of speech-driven spectrotemporal receptive fields in human auditory cortex
Venezia, J. H., Thurman, S. M., Richards, V. M., & Hickok, G. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.11.049 · Cited by 47
Stress and emotion recognition using log-Gabor filter analysis of speech spectrograms
He, L., Lech, M., Maddage, N., & Allen, N. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349454 · Cited by 29
Autoencoding sensory substitution
Tóth, V., & Parkkonen, L. (2019) · Unpublished · doi:10.13140/rg.2.2.10576.87048 · Cited by 7
Metadata
- Authors
- Chi, T., Ru, P., & Shamma, S. A.
- Venue
- The Journal of the Acoustical Society of America
- Publisher
- Acoustical Society of America (ASA)
- Published
- 2005-08-01
- Volume
- 118
- Issue
- 2
- Pages
- 887-906
- ISSN
- 0001-4966 (ISSN-L)
- 1520-8524
- OpenAlex
- W2054139811
- Cited by
- 673
- References
- 70
- FWCI
- 6.07
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.