The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
The use of visible speech cues for improving auditory detection of spoken sentences
Abstract
Classic accounts of the benefits of speechreading to speech recognition treat auditory and visual channels as independent sources of information that are integrated fairly early in the speech perception process. The primary question addressed in this study was whether visible movements of the speech articulators could be used to improve the detection of speech in noise, thus demonstrating an influence of speechreading on the ability to detect, rather than recognize, speech. In the first experiment, ten normal-hearing subjects detected the presence of three known spoken sentences in noise under three conditions: auditory-only (A), auditory plus speechreading with a visually matched sentence (AVM) and auditory plus speechreading with a visually unmatched sentence (AVUM). When the speechread sentence matched the target sentence, average detection thresholds improved by about 1.6 dB relative to the auditory condition. However, the amount of threshold reduction varied significantly for the three target sentences (from 0.8 to 2.2 dB). There was no difference in detection thresholds between the AVUM condition and the A condition. In a second experiment, the effects of visually matched orthographic stimuli on detection thresholds was examined for the same three target sentences in six subjects who participated in the earlier experiment. When the orthographic stimuli were presented just prior to each trial, average detection thresholds improved by about 0.5 dB relative to the A condition. However, unlike the AVM condition, the detection improvement due to orthography was not dependent on the target sentence. Analyses of correlations between area of mouth opening and acoustic envelopes derived from selected spectral regions of each sentence (corresponding to the wide-band speech, and first, second, and third formant regions) suggested that AVM threshold reduction may be determined by the degree of auditory-visual temporal coherence, especially between the area of lip opening and the envelope derived from mid- to high-frequency acoustic energy. Taken together, the data (for these sentences at least) suggest that visual cues derived from the dynamic movements of the fact during speech production interact with time-aligned auditory cues to enhance sensitivity in auditory detection. The amount of visual influence depends in part on the degree of correlation between acoustic envelopes and visible movement of the articulators.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Transformed Up-Down Methods in Psychoacoustics
Levitt, H. (1971) · The Journal of the Acoustical Society of America · doi:10.1121/1.1912375 · Cited by 5,477
Visual Contribution to Speech Intelligibility in Noise
Sumby, W. H., & Pollack, I. (1954) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907309 · Cited by 2,583
An Analysis of Perceptual Confusions Among Some English Consonants
Miller, G. A., & Nicely, P. E. (1955) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907526 · Cited by 1,869
Activation of Auditory Cortex During Silent Lipreading
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., McGuire, P. K., Woodruff, P. W. R., Iversen, S. D., & David, A. S. (1997) · Science · doi:10.1126/science.276.5312.593 · Cited by 1,025
The intelligibility of speech as a function of the context of the test materials.
Miller, G. A., Heise, G. A., & Lichten, W. (1951) · Journal of Experimental Psychology · doi:10.1037/h0062491 · Cited by 769
Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors
Meredith, M., Nemitz, J., & Stein, B. (1987) · The Journal of Neuroscience · doi:10.1523/jneurosci.07-10-03215.1987 · Cited by 754
Quantitative association of vocal-tract and facial behavior
Yehia, H., Rubin, P., & Vatikiotis-Bateson, E. (1998) · Speech Communication · doi:10.1016/s0167-6393(98)00048-x · Cited by 420
Detection in noise by spectro-temporal pattern analysis
Hall, J. W., Haggard, M. P., & Fernandes, M. A. (1984) · The Journal of the Acoustical Society of America · doi:10.1121/1.391005 · Cited by 369
Seeing speech: visual information from lip movements modifies activity in the human auditory cortex
Sams, M., Aulanko, R., Hämäläinen, M., Hari, R., Lounasmaa, O. V., Lu, S.-T., & Simola, J. (1991) · Neuroscience Letters · doi:10.1016/0304-3940(91)90914-f · Cited by 367
Grant, K. W., Walden, B. E., & Seitz, P. F. (1998) · The Journal of the Acoustical Society of America · doi:10.1121/1.422788 · Cited by 342
Self-citation
Cited by
Acoustic Communication in Noise
Brumm, H., & Slabbekoorn, H. (2005) · Advances in the Study of Behavior · doi:10.1016/s0065-3454(05)35004-2 · Cited by 973
Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party”
Zion Golumbic, E. M., Ding, N., Bickel, S., Lakatos, P., Schevon, C. A., McKhann, G. M., Goodman, R. R., Emerson, R., Mehta, A. D., Simon, J. Z., et al. (2013) · Neuron · doi:10.1016/j.neuron.2012.12.037 · Cited by 909
Ross, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C., & Foxe, J. J. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl024 · Cited by 650
The Natural Statistics of Audiovisual Speech
Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A., & Ghazanfar, A. A. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000436 · Cited by 630
Auditory processing in autism spectrum disorder: A review
O’Connor, K. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.11.008 · Cited by 466
Krahmer, E., & Swerts, M. (2007) · Journal of Memory and Language · doi:10.1016/j.jml.2007.06.005 · Cited by 386
Reading Speech from Still and Moving Faces: The Neural Substrates of Visible Speech
Calvert, G. A., & Campbell, R. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321107828 · Cited by 332
Listening to talking faces: motor cortical activation during speech perception
Skipper, J. I., Nusbaum, H. C., & Small, S. L. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.11.006 · Cited by 324
Bimodal speech: early suppressive visual effects in human auditory cortex
Besle, J., Fort, A., Delpuech, C., & Giard, M. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03670.x · Cited by 323
Perceptual Fusion and Stimulus Coincidence in the Cross-Modal Integration of Speech
Miller, L. M., & D'Esposito, M. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0896-05.2005 · Cited by 322
Concept representation reflects multimodal abstraction: A framework for embodied semantics
Fernandino, L., Binder, J. R., Desai, R. H., Pendl, S. L., Humphries, C. J., Gross, W. L., Conant, L. L., & Seidenberg, M. S. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhv020 · Cited by 305
A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion
Nath, A. R., & Beauchamp, M. S. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.07.024 · Cited by 280
Seeing to hear better: evidence for early audio-visual interactions in speech identification
SCHWARTZ, J. (2004) · Cognition · doi:10.1016/s0010-0277(04)00054-x · Cited by 279
Seeing to hear better: evidence for early audio-visual interactions in speech identification
Schwartz, J.-L., Berthommier, F., & Savariaux, C. (2004) · Cognition · doi:10.1016/j.cognition.2004.01.006 · Cited by 275
Visual Input Enhances Selective Speech Envelope Tracking in Auditory Cortex at a “Cocktail Party”
Zion Golumbic, E., Cogan, G. B., Schroeder, C. E., & Poeppel, D. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3675-12.2013 · Cited by 258
Speech perception as a multimodal phenomenon
Rosenblum, L. D. (2008) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2008.00615.x · Cited by 193
“Importantly, this research shows that the amount of improvement depends on the degree to which the visible extent of mouth opening is correlated with the changing auditory amplitude of the speech (Grant & Seitz, 2000).”
Tian, X., & Poeppel, D. (2013) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00381 · Cited by 172
Stanney, K., Samman, S., Reeves, L., Hale, K., Buff, W., Bowers, C., Goldiez, B., Nicholson, D., & Lackey, S. (2004) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1702_7 · Cited by 65
Metadata
- Authors
- Grant, K. W., & Seitz, P.-F.
- Venue
- The Journal of the Acoustical Society of America
- Publisher
- Acoustical Society of America (ASA)
- Published
- 2000-09-01
- Volume
- 108
- Issue
- 3
- Pages
- 1197-1208
- ISSN
- 0001-4966 (ISSN-L)
- 1520-8524
- OpenAlex
- W2010929291
- Cited by
- 521
- References
- 26
- FWCI
- 6.86
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.