The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.
The Natural Statistics of Audiovisual Speech
Open accessAbstract
Humans, like other animals, are exposed to a continuous stream of signals, which are dynamic, multimodal, extended, and time varying in nature. This complex input space must be transduced and sampled by our sensory systems and transmitted to the brain where it can guide the selection of appropriate actions. To simplify this process, it's been suggested that the brain exploits statistical regularities in the stimulus space. Tests of this idea have largely been confined to unimodal signals and natural scenes. One important class of multisensory signals for which a quantitative input space characterization is unavailable is human speech. We do not understand what signals our brain has to actively piece together from an audiovisual speech stream to arrive at a percept versus what is already embedded in the signal structure of the stream itself. In essence, we do not have a clear understanding of the natural statistics of audiovisual speech. In the present study, we identified the following major statistical features of audiovisual speech. First, we observed robust correlations and close temporal correspondence between the area of the mouth opening and the acoustic envelope. Second, we found the strongest correlation between the area of the mouth opening and vocal tract resonances. Third, we observed that both area of the mouth opening and the voice envelope are temporally modulated in the 2-7 Hz frequency range. Finally, we show that the timing of mouth movements relative to the onset of the voice is consistently between 100 and 300 ms. We interpret these data in the context of recent neural theories of speech which suggest that speech communication is a reciprocally coupled, multisensory event, whereby the outputs of the signaler are matched to the neural processes of the receiver.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neuronal Oscillations in Cortical Networks
Buzsáki, G., & Draguhn, A. (2004) · Science · doi:10.1126/science.1099745 · Cited by 6,576
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Speech Recognition with Primarily Temporal Cues
Shannon, R. V., Zeng, F.-G., Kamath, V., Wygonski, J., & Ekelid, M. (1995) · Science · doi:10.1126/science.270.5234.303 · Cited by 3,024
High Gamma Power Is Phase-Locked to Theta Oscillations in Human Neocortex
Canolty, R. T., Edwards, E., Dalal, S. S., Soltani, M., Nagarajan, S. S., Kirsch, H. E., Berger, M. S., Barbaro, N. M., & Knight, R. T. (2006) · Science · doi:10.1126/science.1128115 · Cited by 2,650
TIMIT Acoustic-Phonetic Continuous Speech Corpus
Garofolo, J. S., Lamel, L. F., Fisher, W. M., Pallett, D. S., Dahlgren, N. L., Zue, V. W., & Fiscus, J. G. (2020) · Americanae (AECID Library) · doi:10.35111/17gk-bn40 · Cited by 2,508
Natural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460 · cited 1×
“scenes [8] and auditory soundscapes [9].”
Is neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463
Self-citationPoeppel, D. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00107-3 · Cited by 1,315
(1991) · Choice Reviews Online · doi:10.5860/choice.29-0605 · Cited by 1,284
Lakatos, P., Shah, A. S., Knuth, K. H., Ulbert, I., Karmos, G., & Schroeder, C. E. (2005) · Journal of Neurophysiology · doi:10.1152/jn.00263.2005 · Cited by 1,237
Cited by
An integrated theory of language production and comprehension
Pickering, M. J., & Garrod, S. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12001495 · Cited by 1,569
Brain-to-brain coupling: a mechanism for creating and sharing a social world
Hasson, U., Ghazanfar, A. A., Galantucci, B., Garrod, S., & Keysers, C. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.12.007 · Cited by 1,140
Self-citationA temporal sampling framework for developmental dyslexia
Goswami, U. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.10.001 · Cited by 912
Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party”
Zion Golumbic, E. M., Ding, N., Bickel, S., Lakatos, P., Schevon, C. A., McKhann, G. M., Goodman, R. R., Emerson, R., Mehta, A. D., Simon, J. Z., et al. (2013) · Neuron · doi:10.1016/j.neuron.2012.12.037 · Cited by 909
Chronux: A platform for analyzing neural signals
Bokil, H., Andrews, P., Kulkarni, J. E., Mehta, S., & Mitra, P. P. (2010) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2010.06.020 · Cited by 865
Speech rhythms and multiplexed oscillatory sensory coding in the human brain
Gross, J., Hoogenboom, N., Thut, G., Schyns, P., Panzeri, S., Belin, P., & Garrod, S. (2013) · PLoS Biology · doi:10.1371/journal.pbio.1001752 · Cited by 643
Neural Oscillations Carry Speech Rhythm through to Comprehension
Peelle, J. E., & Davis, M. H. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00320 · Cited by 571
Phase-Locked Responses to Speech in Human Auditory Cortex are Enhanced During Comprehension
Peelle, J. E., Gross, J., & Davis, M. H. (2013) · Cerebral Cortex · doi:10.1093/cercor/bhs118 · Cited by 568
Infants deploy selective attention to the mouth of a talking face when learning speech
Lewkowicz, D. J., & Hansen-Tift, A. M. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1114783109 · Cited by 526
Temporal modulations in speech and music
Ding, N., Patel, A. D., Chen, L., Butler, H., Luo, C., & Poeppel, D. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.011 · Cited by 502
Speech rhythms and their neural foundations
Poeppel, D., & Assaneo, M. F. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0304-4 · Cited by 476
Multimodal Language Processing in Human Communication
Holler, J., & Levinson, S. C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.05.006 · Cited by 441
Huss, M., Verney, J. P., Fosker, T., Mead, N., & Goswami, U. (2011) · Cortex · doi:10.1016/j.cortex.2010.07.010 · Cited by 371
Ince, R. A. A., Giordano, B. L., Kayser, C., Rousselet, G. A., Gross, J., & Schyns, P. G. (2017) · Human Brain Mapping · doi:10.1002/hbm.23471 · Cited by 366
Transitions in neural oscillations reflect prediction errors generated in audiovisual speech
Arnal, L. H., Wyart, V., & Giraud, A.-L. (2011) · Nature Neuroscience · doi:10.1038/nn.2810 · Cited by 353
Keitel, A., Gross, J., & Kayser, C. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2004473 · Cited by 312
The Evolution of Rhythm Processing
Kotz, S. A., Ravignani, A., & Fitch, W. T. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.08.002 · Cited by 304
The emergence of multisensory systems through perceptual narrowing
Lewkowicz, D. J., & Ghazanfar, A. A. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.08.004 · Cited by 297
Self-citationFontolan, L., Morillon, B., Liegeois-Chauvel, C., & Giraud, A.-L. (2014) · Nature Communications · doi:10.1038/ncomms5694 · Cited by 276
Multisensory Integration: Flexible Use of General Operations
van Atteveldt, N., Murray, M. M., Thut, G., & Schroeder, C. E. (2014) · Neuron · doi:10.1016/j.neuron.2014.02.044 · Cited by 272
Lip Movements Generation at a Glance
Chen, L., Li, Z., Maddox, R. K., Duan, Z., & Xu, C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_32 · Cited by 258
Visual Input Enhances Selective Speech Envelope Tracking in Auditory Cortex at a “Cocktail Party”
Zion Golumbic, E., Cogan, G. B., Schroeder, C. E., & Poeppel, D. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3675-12.2013 · Cited by 258
Coupled oscillator dynamics of vocal turn-taking in monkeys
Takahashi, D. Y., Narayanan, D. Z., & Ghazanfar, A. A. (2013) · Current Biology · doi:10.1016/j.cub.2013.09.005 · Cited by 257
Self-citationDual Neural Routing of Visual Facilitation in Speech Processing
Arnal, L. H., Morillon, B., Kell, C. A., & Giraud, A.-L. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3194-09.2009 · Cited by 241
Assaneo, M. F., & Poeppel, D. (2018) · Science Advances · doi:10.1126/sciadv.aao3842 · Cited by 164
Costantini, M., Robinson, J., Migliorati, D., Donno, B., Ferri, F., & Northoff, G. (2016) · Cognition · doi:10.1016/j.cognition.2016.08.010 · Cited by 107
Predictive Coding in Sensory Cortex
Kok, P., & de Lange, F. P. (2015) · An Introduction to Model-Based Cognitive Neuroscience · doi:10.1007/978-1-4939-2236-9_11 · Cited by 98
Temporally resolved neural dynamics underlying handwriting
Pei, L., Longcamp, M., Leung, F. K.-S., & Ouyang, G. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118578 · Cited by 23
Metadata
- Authors
- Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A., & Ghazanfar, A. A.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2009-07-17
- Volume
- 5
- Issue
- 7
- Pages
- e1000436
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2099940393
- Cited by
- 630
- References
- 78
- FWCI
- 20.09
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.