The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
The use of visible speech cues for improving auditory detection of spoken sentences
Abstract
Classic accounts of the benefits of speechreading to speech recognition treat auditory and visual channels as independent sources of information that are integrated fairly early in the speech perception process. The primary question addressed in this study was whether visible movements of the speech articulators could be used to improve the detection of speech in noise, thus demonstrating an influence of speechreading on the ability to detect, rather than recognize, speech. In the first experiment, ten normal-hearing subjects detected the presence of three known spoken sentences in noise under three conditions: auditory-only (A), auditory plus speechreading with a visually matched sentence (AVM) and auditory plus speechreading with a visually unmatched sentence (AVUM). When the speechread sentence matched the target sentence, average detection thresholds improved by about 1.6 dB relative to the auditory condition. However, the amount of threshold reduction varied significantly for the three target sentences (from 0.8 to 2.2 dB). There was no difference in detection thresholds between the AVUM condition and the A condition. In a second experiment, the effects of visually matched orthographic stimuli on detection thresholds was examined for the same three target sentences in six subjects who participated in the earlier experiment. When the orthographic stimuli were presented just prior to each trial, average detection thresholds improved by about 0.5 dB relative to the A condition. However, unlike the AVM condition, the detection improvement due to orthography was not dependent on the target sentence. Analyses of correlations between area of mouth opening and acoustic envelopes derived from selected spectral regions of each sentence (corresponding to the wide-band speech, and first, second, and third formant regions) suggested that AVM threshold reduction may be determined by the degree of auditory-visual temporal coherence, especially between the area of lip opening and the envelope derived from mid- to high-frequency acoustic energy. Taken together, the data (for these sentences at least) suggest that visual cues derived from the dynamic movements of the fact during speech production interact with time-aligned auditory cues to enhance sensitivity in auditory detection. The amount of visual influence depends in part on the degree of correlation between acoustic envelopes and visible movement of the articulators.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Transformed Up-Down Methods in Psychoacoustics
Levitt, H. (1971) · The Journal of the Acoustical Society of America · doi:10.1121/1.1912375 · Cited by 5,477
Visual Contribution to Speech Intelligibility in Noise
Sumby, W. H., & Pollack, I. (1954) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907309 · Cited by 2,583
An Analysis of Perceptual Confusions Among Some English Consonants
Miller, G. A., & Nicely, P. E. (1955) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907526 · Cited by 1,869
Activation of Auditory Cortex During Silent Lipreading
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., McGuire, P. K., Woodruff, P. W. R., Iversen, S. D., & David, A. S. (1997) · Science · doi:10.1126/science.276.5312.593 · Cited by 1,025
The intelligibility of speech as a function of the context of the test materials.
Miller, G. A., Heise, G. A., & Lichten, W. (1951) · Journal of Experimental Psychology · doi:10.1037/h0062491 · Cited by 769
Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors
Meredith, M., Nemitz, J., & Stein, B. (1987) · The Journal of Neuroscience · doi:10.1523/jneurosci.07-10-03215.1987 · Cited by 754
Quantitative association of vocal-tract and facial behavior
Yehia, H., Rubin, P., & Vatikiotis-Bateson, E. (1998) · Speech Communication · doi:10.1016/s0167-6393(98)00048-x · Cited by 420
Detection in noise by spectro-temporal pattern analysis
Hall, J. W., Haggard, M. P., & Fernandes, M. A. (1984) · The Journal of the Acoustical Society of America · doi:10.1121/1.391005 · Cited by 369
Seeing speech: visual information from lip movements modifies activity in the human auditory cortex
Sams, M., Aulanko, R., Hämäläinen, M., Hari, R., Lounasmaa, O. V., Lu, S.-T., & Simola, J. (1991) · Neuroscience Letters · doi:10.1016/0304-3940(91)90914-f · Cited by 367
Grant, K. W., Walden, B. E., & Seitz, P. F. (1998) · The Journal of the Acoustical Society of America · doi:10.1121/1.422788 · Cited by 342
Self-citationDevelopment of a Quantitative Description of Vowel Articulation
Stevens, K. N., & House, A. S. (1955) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907943 · Cited by 315
Intermodal timing relations and audio-visual speech recognition by normal-hearing adults
McGrath, M., & Summerfield, Q. (1985) · The Journal of the Acoustical Society of America · doi:10.1121/1.392336 · Cited by 201
Measures of auditory–visual integration in nonsense syllables and sentences
Grant, K. W., & Seitz, P. F. (1998) · The Journal of the Acoustical Society of America · doi:10.1121/1.423751 · Cited by 193
Self-citationCrossmodal Integration in the Identification of Consonant Segments
Braida, L. D. (1991) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640749108400991 · Cited by 171
An Acoustical Theory of Vowel Production and Some of its Implications
Stevens, K. N., & House, A. S. (1961) · Journal of Speech and Hearing Research · doi:10.1044/jshr.0404.303 · Cited by 114
Evaluating the articulation index for auditory–visual input
Grant, K. W., & Braida, L. D. (1991) · The Journal of the Acoustical Society of America · doi:10.1121/1.400733 · Cited by 113
Self-citationSpence, C., Ranson, J., & Driver, J. (2000) · Perception & Psychophysics · doi:10.3758/bf03205560 · Cited by 112
Interval of Time Uncertainty in Auditory Detection
Egan, J. P., Greenberg, G. Z., & Schulman, A. I. (1961) · The Journal of the Acoustical Society of America · doi:10.1121/1.1908795 · Cited by 92
The visuotopic component of the multisensory map in the deep laminae of the cat superior colliculus
Meredith, M. A., & Stein, B. (1990) · Journal of Neuroscience · doi:10.1523/jneurosci.10-11-03727.1990 · Cited by 80
Can speech perception be influenced by simultaneous presentation of print?
Frost, R., Repp, B. H., & Katz, L. D. (1988) · Journal of Memory and Language · doi:10.1016/0749-596x(88)90018-6 · Cited by 76
Single Band Amplitude Envelope Cues as an Aid to Speechreading
Grant, K. W., Braida, L. D., & Renn, R. J. (1991) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640749108400990 · Cited by 37
Self-citationDetectability of auditory signals presented without defined observation intervals
Watson, C. S., & Nichols, T. L. (1976) · The Journal of the Acoustical Society of America · doi:10.1121/1.380915 · Cited by 36
Auditory and Audiovisual Feature Transmission in Hearing-Impaired Adults
Walden, B. E., Prosek, R. A., & Worthington, D. W. (1975) · Journal of Speech and Hearing Research · doi:10.1044/jshr.1802.272 · Cited by 35
Lexical Mediation between Sight and Sound in Speechreading
Repp, B. H., Frost, R., & Zsiga, E. C. (1992) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640749208401313 · Cited by 22
Frost, R. (1991) · Cognition · doi:10.1016/0010-0277(91)90053-7 · Cited by 16
Masking protection in the perception of auditory objects
Gordon, P. C. (2000) · Speech Communication · doi:10.1016/s0167-6393(99)00053-9 · Cited by 14
Cited by
Acoustic Communication in Noise
Brumm, H., & Slabbekoorn, H. (2005) · Advances in the Study of Behavior · doi:10.1016/s0065-3454(05)35004-2 · Cited by 973
Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party”
Zion Golumbic, E. M., Ding, N., Bickel, S., Lakatos, P., Schevon, C. A., McKhann, G. M., Goodman, R. R., Emerson, R., Mehta, A. D., Simon, J. Z., et al. (2013) · Neuron · doi:10.1016/j.neuron.2012.12.037 · Cited by 909
Ross, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C., & Foxe, J. J. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl024 · Cited by 650
The Natural Statistics of Audiovisual Speech
Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A., & Ghazanfar, A. A. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000436 · Cited by 630
Auditory processing in autism spectrum disorder: A review
O’Connor, K. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2011.11.008 · Cited by 466
Metadata
- Authors
- Grant, K. W., & Seitz, P.-F.
- Venue
- The Journal of the Acoustical Society of America
- Publisher
- Acoustical Society of America (ASA)
- Published
- 2000-09-01
- Volume
- 108
- Issue
- 3
- Pages
- 1197-1208
- ISSN
- 0001-4966 (ISSN-L)
- 1520-8524
- OpenAlex
- W2010929291
- Cited by
- 521
- References
- 26
- FWCI
- 6.86
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.