Visual Contribution to Speech Intelligibility in Noise
The DOI matches its Crossref record.
Abstract
Oral speech intelligibility tests were conducted with, and without, supplementary visual observation of the speaker's facial and lip movements. The difference between these two conditions was examined as a function of the speech-to-noise ratio and of the size of the vocabulary under test. The visual contribution to oral speech intelligibility (relative to its possible contribution) is, to a first approximation, independent of the speech-to-noise ratio under test. However, since there is a much greater opportunity for the visual contribution at low speech-to-noise ratios, its absolute contribution can be exploited most profitably under these conditions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous studies show that non-verbal facial gestures often account for a substantial portion of the perceived feeling and attitude of a speaker18,19 and that face-to-face communication enhances social connectivity20 and intelligibility21.”
From A high-performance neuroprosthesis for speech decoding and avatar control (2023) · cites this paper as [21]
In the citing paper
“Non-verbal facial expressions that accompany virtual or face-to-face communication provide numerous benefits compared with audio-only communication, including improved conveyance of emotion and attitude130,131, that provides increased clarity132.”
From The speech neuroprosthesis (2024) · cites this paper as [132]
In the citing paper
“Estimates suggest that the information gain in this situation is the equivalent of boosting the audible signal by up to 15 dB (Sumby and Pollack, 1954)”
From Behavioral, Perceptual, and Neural Alterations in Sensory and Multisensory Function in Autism Spectrum Disorder (2015) · cites this paper as Sumby and Pollack, 1954
In the citing paper
“Seeing the face ofa speaker can significantly embellish a degraded or noisy auditory speech signal so that it functionally raises the signal-tonoise ratio by as much as 22 dB”
From The McGurk effect in infants (1997) · cites this paper as Sumby & Pollack, 1954
Citations
Cited by
Items 901–1,000 of 1,005
Auditory-visual scenes for hearing research
van de Par, S., Ewert, S. D., Hládek, Ľ., Kirsch, C., Schütze, J., Llorca-Bofí, J., Grimm, G., Hendrikse, M. M. E., Kollmeier, B., & Seeber, B. U. (2022) · Acta Acustica · doi:10.1051/aacus/2022032 · Cited by 23
Postnatal Experiences Influence How the Brain Integrates Information from Different Senses
Stein, B. E., Perrault, T. J., Stanford, T. R., & Rowland, B. A. (2009) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.021.2009 · Cited by 29
Buchan, J. N., & Munhall, K. G. (2011) · Perception · doi:10.1068/p6939 · Cited by 32
Vision of tongue movements bias auditory speech perception
D’Ausilio, A., Bartoli, E., Maffongelli, L., Berry, J., & Fadiga, L. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.08.018 · Cited by 24
“For instance, vision of the articulators enhances accurate auditory perception in noise (Sumby & Pollack, 1954).”
BackgroundHigh-intensity sound increases the size of visually perceived objects
Takeshima, Y., & Gyoba, J. (2012) · Attention Perception & Psychophysics · doi:10.3758/s13414-012-0403-z · Cited by 30
Electrocorticography Reveals Enhanced Visual Cortex Responses to Visual Speech
Schepers, I. M., Yoshor, D., & Beauchamp, M. S. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhu127 · Cited by 22
Minar, N. J., & Lewkowicz, D. J. (2017) · Developmental Science · doi:10.1111/desc.12604 · Cited by 27
Face viewing behavior predicts multisensory gain during speech perception
Rennig, J., Wegner‐Clemens, K., & Beauchamp, M. S. (2019) · Psychonomic Bulletin & Review · doi:10.3758/s13423-019-01665-y · Cited by 28
Strand, J. F., Brown, V. A., & Barbour, D. L. (2020) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01713-y · Cited by 25
Magnotti, J. F., Dzeda, K. B., Wegner‐Clemens, K., Rennig, J., & Beauchamp, M. S. (2020) · Cortex · doi:10.1016/j.cortex.2020.10.002 · Cited by 29
Effects of audio–visual integration on the detection of masked speech and non-speech sounds
Eramudugolla, R., Henderson, R., & Mattingley, J. B. (2010) · Brain and Cognition · doi:10.1016/j.bandc.2010.09.005 · Cited by 29
“Recognition of speech in noise is similarly improved by presenting matched, synchronous visual stimuli relative to presenting the auditory stimuli alone (Ross, Saint-Amour, Leavitt, Javitt, & Foxe, 2007; Sumby & Pollack, 1954). ll rights reserved. mudugolla).”
BackgroundLeybaert, J., Macchi, L., Huyse, A., Champoux, F., Bayard, C., Colin, C., & Berthommier, F. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00422 · Cited by 35
“In face-to-face communication, speech perception is a multimodal process involving both auditory and visual modalities (Sumby and Pollack, 1954; Grant and Seitz, 2000).”
BackgroundEffect of Dual Sensory Loss on Auditory Localization: Implications for Intervention
Simon, H. J., & Levitt, H. (2007) · Trends in Amplification · doi:10.1177/1084713807308209 · Cited by 27
Recognition of Visual Speech Elements Using Hidden Markov Models
Foo, S. W., & Dong, L. (2002) · Lecture notes in computer science · doi:10.1007/3-540-36228-2_75 · Cited by 23
“And even earlier, Sumby and Pollack proved that visual clues could lead to better perception of speech especially under noisy environment[ 2 ].”
BackgroundGeorge, . (2010) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2010.00193 · Cited by 29
“Seeing a speaker for example considerably improves identification of acoustic speech in noisy conditions (Sumby and Pollack, 1954) but it can also, under good listening conditions, lead to audiovisual illusions such as the famous “McGurk effect,” in which seeing someone producing one phoneme while hearing another one…”
A comparison of models for fusion of the auditory and visual sensors in speech perception
Robert-Ribes, J., Schwartz, J., & Escudier, P. (1995) · Artificial Intelligence Review · doi:10.1007/bf00849043 · Cited by 23
“A number of investigators have studied this effect of noise on speech intelligibility according to whether the message is heard only, or heard with the speaker's face in view ( Sumby and Pollack 1954, Binnie et al. 1974, Erber 1969, 1975, Summerfield 1979; Benoit, Mohamadi, and Kandel, in press).”
BackgroundInformative speaking: A summary and bibliography of related research
Petrie, C. R. (1963) · Speech Monographs · doi:10.1080/03637756309375362 · Cited by 23
3. Audio-visual Integration During Bilingual Language Processing
Marian, V. (2009) · Multilingual Matters eBooks · doi:10.21832/9781847691262-005 · Cited by 34
Intra- versus intermodal integration in young and older adults.
Spehar, B. P., Tye-Murray, N., & Sommers, M. (2008) · Journal of the Acoustical Society of America · doi:10.1121/1.2890748 · Cited by 23
Speech intelligibility of virtual humans
Devesse, A., Dudek, A., Wieringen, A. V., & Wouters, J. M. (2018) · International Journal of Audiology · doi:10.1080/14992027.2018.1511922 · Cited by 24
“This visual speech benefit is in agreement with research demonstrating that visual cues improve speech understanding in noisy conditions (Sumby and Pollack 1954; MacLeod and Summerfield 1987, 1990; Grant, Walden and Seitz 1998) and provides support for the applicability of our virtual humans in…”
BackgroundResultMultisensory perception reflects individual differences in processing temporal correlations
Nidiffer, A. R., Diederich, A., Ramachandran, R., & Wallace, M. T. (2018) · Scientific Reports · doi:10.1038/s41598-018-32673-y · Cited by 28
Hobeika, L., Taffou, M., Carpentier, T., Warusfel, O., & Viaud‐Delmon, I. (2019) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2019.108534 · Cited by 23
The relationship between multisensory associative learning and multisensory integration
Lauzon, S., Abraham, A. E., Curcin, K., Butler, B. E., & Stevenson, R. A. (2022) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2022.108336 · Cited by 29
Ahmed, F., Nidiffer, A. R., O’Sullivan, A. E., Zuk, N. J., & Lalor, E. C. (2023) · NeuroImage · doi:10.1016/j.neuroimage.2023.120143 · Cited by 28
Seismic Signaling is Crucial for Female Mate Choice in a Multimodal Signaling Wolf Spider
Wilgers, D. J., & Hebets, E. A. (2012) · Ethology · doi:10.1111/j.1439-0310.2012.02023.x · Cited by 22
Alm, M., & Behne, D. M. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01014 · Cited by 25
Suprasegmental Lexical Stress Cues in Visual Speech can Guide Spoken-Word Recognition
Jesse, A., & McQueen, J. M. (2013) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2013.834371 · Cited by 22
“Seeing a speaker typically improves speech perception (Miller & Nicely, 1955; Reisberg, McLean, & Goldfield, 1987; Sumby & Pollack, 1954), as visual speech contains information that is redundant and complementary to that in auditory speech (Jesse & Massaro, 2010; Walden, Prosek, & Worthington,…”
BackgroundDevelopment of an audiovisual speech perception app for children with autism spectrum disorders
IRWIN, J. R., Preston, J. L., Brancazio, L., D’angelo, M., & Turcios, J. (2014) · Clinical Linguistics & Phonetics · doi:10.3109/02699206.2014.966395 · Cited by 29
“Moreover, visual speech information plays an especially helpful role in the perception of auditory speech in noise (Sumby & Pollack, 1954), a condition that has been reported to be difficult for children with ASD (Alcántara, Weisblatt, Moore, & Bolton, 2004).”
Spatial Frequency Requirements and Gaze Strategy in Visual-Only and Audiovisual Speech Perception
Wilson, A., Alsius, A., Paré, M., & Munhall, K. G. (2016) · Journal of Speech Language and Hearing Research · doi:10.1044/2016_jslhr-s-15-0092 · Cited by 22
Chikara, R. K., & Ko, L. (2019) · Brain Sciences · doi:10.3390/brainsci9090216 · Cited by 22
How Visual Cues for when to Listen Aid Selective Auditory Attention
Varghese, L. A., Ozmeral, E. J., Best, V., & Shinn‐Cunningham, B. G. (2012) · Journal of the Association for Research in Otolaryngology · doi:10.1007/s10162-012-0314-7 · Cited by 22
A Case of Impaired Auditory and Visual Speech Prosody Perception after Right Hemisphere Damage
Nicholson, K. G., Baum, S. R., Cuddy, L. L., & Munhall, K. G. (2002) · Neurocase · doi:10.1076/neur.8.3.314.16195 · Cited by 23
“Furthermore, being able to see the talker’s face improves the intelligibility of speech when the auditory information is impoverished, due to either a hearing impairment (e.g. Bernstein et al., 2000) or a noisy environment (e.g. Sumby and Pollack, 1954; Macleod and Summerfield, 1987).”
BackgroundDoes face familiarity influence speechreadability?
Lander, K., & Davies, R. (2008) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470210801908476 · Cited by 22
“Under noisy conditions, viewing the talking face supplements the auditory signal, increasing perceptual accuracy (Sumby & Pollack, 1954 ).”
BackgroundA cross-modal account for synchronic and diachronic patterns of /f/ and /θ/ in English
McGuire, G. L., & Babel, M. (2012) · Laboratory Phonology Journal of the Association for Laboratory Phonology · doi:10.1515/lp-2012-0014 · Cited by 26
The Role of Auditory and Visual Speech in Word Learning at 18 Months and in Adulthood
Mélanie Havy, A., Foroud, A., Fais, L., & Werker, J. F. (2017) · Child Development · doi:10.1111/cdev.12715 · Cited by 22
“In adults, empirical studies have revealed that visible speech highlights distinctions that are not acoustically salient and can override auditory confusion (Sumby & Pollack, 1954).”
BackgroundDoes audiovisual speech perception use information about facial configuration?
Hietanen, J. K., Manninen, P., Sams, M. E., & Surakka, V. (2001) · The European Journal of Cognitive Psychology · doi:10.1080/09541440126006 · Cited by 22
“Speechreading is important even for people with normal hearing, especially when the auditory information is degraded or embedded in noise (e.g., Dodd, 1977; Erber, 1969; Neely, 1956; O’Neill, 1954; Sanders & Goodrich, 1971; Sumby & Pollack, 1954).”
BackgroundThe perception of speaking rate using visual information from a talker’s face
Green, K. P. (1987) · Perception & Psychophysics · doi:10.3758/bf03207990 · Cited by 22
Burnham, D. K., Reynolds, J., Vatikiotis‐Bateson, E., Yehia, H. C., Ciocca, V., Morris, R. H., Hill, H. C. H., Vignali, G., Bollwerk, S., Tam, H., et al. (2006) · Research Online (University of Wollongong) · Cited by 22
Multiple cameras for audio-visual speech recognition in an automotive environment
Navarathna, R., Dean, D., Sridharan, S., & Lucey, P. (2012) · Computer Speech & Language · doi:10.1016/j.csl.2012.07.005 · Cited by 21
“All of this interest led to the first work on audio-visual speech processing, published by Sumby and Pollack (1954). n this work, Sumby and Pollack examined the contribution of visual factors to oral speech intelligibility as a function f the speech-to-noise (SNR) ratio and the size of the…”
BackgroundThézé, R., Gadiri, M. A., Albert, L., Provost, A., Giraud, A., & Mégevand, P. (2020) · Scientific Reports · doi:10.1038/s41598-020-72375-y · Cited by 21
The role of iconic gestures and mouth movements in face-to-face communication
Krason, A., Fenton, R., Varley, R. A., & Vigliocco, G. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-021-02009-5 · Cited by 29
“Mouth movements are particularly useful in adverse listening conditions (Ma et al., 2009; Reisberg et al., 1987; Ross et al., 2007; Schwartz et al., 2004; Sumby & Pollack, 1954).”
BackgroundSemantic Cues Modulate Children’s and Adults’ Processing of Audio-Visual Face Mask Speech
Schwarz, J., Li, K., Sim, J. H., Zhang, Y., Worster, E., Post, B., Gibson, J. L., & McDougall, K. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.879156 · Cited by 27
“, by noise (Sumby and Pollack, 1954; Lansing and McConkie, 2003), leading adults to shift their attention from the eyes to the mouth (Vatikiotis-Bateson et al.”
BackgroundPhonetic Recalibration in Audiovisual Speech
Vroomen, J., & Baart, M. (2011) · Frontiers in neuroscience · doi:10.1201/b11092-24 · Cited by 27
Audiovisual speech perception and word recognition
Massaro, D. W., & Jesse, A. (2012) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198568971.013.0002 · Cited by 23
Leybaert, J., & Colin, C. (2007) · Enfance · doi:10.3917/enf.593.0245 · Cited by 28
“…à la lecture labiale, améliore la compréhen- sion de la parole des sujets entendants, surtout lorsque le signal de parole est masqué par du bruit (Sumby & Pollack, 1954), des sujets avec déficience auditive (Erber, 1972) et des sujets sourds munis d’un IC (Tyler, Parkinson, Woodworth, Lowder,…”
BackgroundThe role of sound intensity and stop-consonant voicing on McGurk fusions and combinations
Colin, C., Radeau, M., Deltenre, P., Démolin, D., & Soquet, A. (2002) · The European Journal of Cognitive Psychology · doi:10.1080/09541440143000203 · Cited by 21
Directional hearing aid benefit in listeners with severe hearing loss
Ricketts, T. A., & Hornsby, B. W. Y. (2006) · International Journal of Audiology · doi:10.1080/14992020500258602 · Cited by 21
“When speech recognition performance is very poor, listeners are primarily relying on visual information (Sumby & Pollack, 1954).”
BackgroundHertrich, I., Dietrich, S., & Ackermann, H. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00530 · Cited by 26
“Depending on environmental conditions, lip reading can significantly enhance speech perception (Sumby and Pollack, 1954; Ma et al., 2009).”
BackgroundCrossmodal Correspondence Between Auditory Pitch and Visual Elevation Affects Temporal Ventriloquism
Orchard-Mills, E., der Burg, E. V., & Alais, D. (2015) · Perception · doi:10.1177/0301006615622320 · Cited by 22
“However, semantic information can also influence multisensory binding (McGurk & MacDonald, 1976; Sumby & Pollack, 1954).”
BackgroundTurbo Automatic Speech Recognition
Receveur, S., Weiss, R., & Fingscheidt, T. (2016) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2016.2520364 · Cited by 21
“Here, further information sources such as additional acoustic channels [21], [22], modalities [23]–[28], or models [29], [30] are exploited by the speech recognizer.”
BackgroundStacey, P. C., Kitterick, P. T., Morris, S., & Sumner, C. J. (2016) · Hearing Research · doi:10.1016/j.heares.2016.04.002 · Cited by 21
“These findings therefore imply that the larger body of data on audio-visual integration in conditions of normal, undegraded speech (e.g. Tye-Murray et al., 2010; Sumby and Pollack, 1954), and studies of audio-visual integration in hearing impaired listeners (e.”
BackgroundResultFletcher, M. D., Song, H., & Perry, S. W. (2020) · Scientific Reports · doi:10.1038/s41598-020-69697-2 · Cited by 21
Objective Assessment of Speech Intelligibility in Crowded Public Spaces
Brungart, D. S., Barrett, M. E., Cohen, J. I., Fodor, C., Yancey, C. M., & Gordon-Salant, S. M. (2020) · Ear and Hearing · doi:10.1097/aud.0000000000000943 · Cited by 21
DARE: Deceiving Audio–Visual speech Recognition model
Mishra, S., Gupta, A. K., & Gupta, P. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107503 · Cited by 21
Chu, Y.-H., Lin, F., Chou, Y.-J., Tsai, K. W., Kuo, W., & Jääskeläinen, I. P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0080265 · Cited by 21
“Further, as a phenomenon demonstrating that visual information has access to the auditory system at relatively early sound processing stages, presentation of certain combinations of incongruent phonetic sounds and articulatory gestures can result in illusory third-category phonetic percepts, for instance, visual /ga/…”
BackgroundAudio-Visual Perception of Sinewave Speech in an Adult Cochlear Implant User: A Case Study
Goh, W. D., Pisoni, D. B., Kirk, K. I., & Remez, R. E. (2001) · Ear and Hearing · doi:10.1097/00003446-200110000-00005 · Cited by 27
“S’s performance deals with assessing the contribution of the visual information relative to the possible information available in the absence of visual stimulation (Sumby & Pollack, 1954).”
BackgroundAudiovisual perception of natural speech is impaired in adult dyslexics: An ERP study
Rüsseler, J., Gerth, I., Heldmann, M., & Münte, T. F. (2014) · Neuroscience · doi:10.1016/j.neuroscience.2014.12.023 · Cited by 24
“This manipulation was motivated by the fact that visual information has been shown to be recruited for speech perception in particular when listening conditions are adverse (i.e., in noisy environments) (Sumby and Pollack, 1954; Schwartz et al., 2004; Ross et al., 2007).”
A hybrid ANN/HMM audio-visual speech recognition system.
Heckmann, M., Berthommier, F., & Kroschel, K. (2001) · AVSP · Cited by 26
LANGUAGE EFFECTS ON THE DEGREE OF VISUAL INFLUENCE IN AUDIOVISUAL SPEECH PERCEPTION
Chen, Y., & Hazan, V. L. (2007) · Cited by 21
Chemosensory Integration and the Perception of Flavor
Prescott, J., & Stevenson, R. J. (2015) · Wiley eBooks · doi:10.1002/9781118971758.ch45 · Cited by 25
Activation in the angular gyrus and in the pSTS is modulated by face primes during voice recognition
Hölig, C., Föcker, J., Best, A., Röder, B., & Büchel, C. (2017) · Human Brain Mapping · doi:10.1002/hbm.23540 · Cited by 23
“For instance, it has been demonstrated that seeing a talker’s facial movements can facilitate auditory speech perception [Sumby and Pollack, 1954] and that incongruent visual lip movements can create illusionary auditory”
BackgroundChildren With SLI Can Exhibit Reduced Attention to a Talker's Mouth
Pons, F., Sanz‐Torrent, M., Ferinu, L., Birulés, J., & Andreu, L. (2018) · Language Learning · doi:10.1111/lang.12276 · Cited by 20
“…comprehension compared to perceiving the auditory information only (Chandrasekaran, Trubanova, Stillittano, Caplier, & Ghazanfar, 2009; King & Calvert, 2001; Reisberg, McLean, & Goldfield, 1987; Rosenblum, 2008; Sumby & Pollack, 1954; Summerfield, 1979; Yehia, Rubin, & Vatikiotis-Bateson, 1998).”
Development and evaluation of video recordings for the OLSA matrix sentence test
Llorach, G., Kirschner, F., Grimm, G., Zokoll, M. A., Wagener, K. C., & Hohmann, V. (2021) · International Journal of Audiology · doi:10.1080/14992027.2021.1930205 · Cited by 20
Worster, E., MacSweeney, M., Pimperton, H., Kyle, F. E., Harris, M., Beedie, I., Ralph‐Lewis, A., & Hulme, C. (2020) · Journal of Speech Language and Hearing Research · doi:10.1044/2020_jslhr-20-00159 · Cited by 26
“When visual speech is congruent with auditory speech, it can enhance speech perception in hearing adults and children (Knowland et al., 2016; Lusk & Mitchel, 2016; Sumby & Pollack, 1954), while incongruent visual information can disrupt it.”
BackgroundFeasibility of Virtual Reality Audiological Testing: Prospective Study
Seol, H. Y., Kang, S., Lim, J., Hong, S. H., & Moon, I. J. (2021) · JMIR Serious Games · doi:10.2196/26976 · Cited by 27
“All of these findings are consistent with previous literature [44-49].”
ResultIntelligibility and recall of sentences spoken by adult and child talkers wearing face masks
Truong, T. L., & Weber, A. (2021) · The Journal of the Acoustical Society of America · doi:10.1121/10.0006098 · Cited by 23
“Visual cues, such as lip and jaw movements, can provide crucial linguistic information about speech sounds (e.g., Campbell, 2008; Summerfield, 1992).”
BackgroundChawarska, K., Lewkowicz, D. J., Feiner, H., Macari, S. L., & Vernetti, A. (2022) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13595 · Cited by 22
“In adults, attention to such cues increases speech comprehension (Grant & Seitz, 2000; MacLeod & Summerfield, 1987; Shahin & Miller, 2009; Sumby & Pollack, 1954; Summerfield, 1979) and speeds up the neural processing of speech (van Wassenhove, Grant, & Poeppel, 2005).”
BackgroundLip movements enhance speech representations and effective connectivity in auditory dorsal stream
Zhang, L., & Du, Y. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119311 · Cited by 19
“In face-to-face communication, we routinely extract visual speech cues from the speaker’s articulatory movements, which substantially benefits speech comprehension especially in challenging listening conditions (Ross et al., 2007; Sumby and Pollack, 1954) and in hearing impaired senior populations (Puschmann et al.”
BackgroundImproved speech intelligibility in the presence of congruent vibrotactile speech input
Schulte, A. J., Marozeau, J., Ruhe, A., Büchner, A., Kral, A., & Innes-Brown, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-48893-w · Cited by 22
Schall, S., & von Kriegstein, K. (2014) · PLoS ONE · doi:10.1371/journal.pone.0086325 · Cited by 22
“Seeing the moving mouth, lips and tongue of a speaker improves speech comprehension substantially [1–3].”
BackgroundDoes visual speech information affect word segmentation?
Sell, A. J., & Kaschak, M. P. (2009) · Memory & Cognition · doi:10.3758/mc.37.6.889 · Cited by 23
Read My Lips: Brain Dynamics Associated with Audiovisual Integration and Deviance Detection
Tse, C., Gratton, G., Garnsey, S. M., Novak, M. A., & Fabiani, M. (2015) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00812 · Cited by 20
“…and face movements involved in producing speech can improve its intelligibility, especially in noisy environments (Gilbert, Lansing, & Garnsey, 2012; Winkler, Horvath, Weisz, & Trejo, 2009; Ross, Saint-Amour, Leavitt, Javitt, & Foxe, 2007; Sams, Möttönen, & Sihvonen, 2005; Sumby & Pollack, 1954).”
Winkler, I., Horváth, J., Weisz, J., & Trejo, L. J. (2009) · Biological Psychology · doi:10.1016/j.biopsycho.2009.08.011 · Cited by 23
“…experiments agree that seeing the speaker’s face improves the intelligibility of speech in noisy environments (e.g., Erber, 1975; Helfer and Freyman, 2005; Sams et al., 2005; Sommers et al., 2005; Sumby and Pollack, 1954) and can disambiguate ambiguous speech utterances (Bertelson et al., 2003).”
Reduced audiovisual integration in synesthesia – evidence from bimodal speech perception
Sinke, C., Neufeld, J., Zedler, M., Emrich, H. M., Bleich, S., Münte, T. F., & Szycik, G. R. (2012) · Journal of Neuropsychology · doi:10.1111/jnp.12006 · Cited by 23
Nonnative Audiovisual Speech Perception in Noise: Dissociable Effects of the Speaker and Listener
Xie, Z., Yi, H., & Chandrasekaran, B. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114439 · Cited by 20
Greater BOLD Variability in Older Compared with Younger Adults during Audiovisual Speech Perception
Baum, S. H., & Beauchamp, M. S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0111121 · Cited by 23
Barakova, E. I., & Chonnaparamutt, W. (2009) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2009.933626 · Cited by 21
“Under normal circumstances, multisensory stimulation leads to enhanced perceptions of, and facilitated responses to, objects in the environment (e.g., [5]–[ 7 ]).”
BackgroundAhveninen, J., Jääskeläinen, I. P., Belliveau, J. W., Hämäläinen, M., Lin, F., & Raij, T. (2012) · PLoS ONE · doi:10.1371/journal.pone.0038511 · Cited by 21
Speaker identity supports phonetic category learning.
Mani, N., & Schneider, S. L. (2012) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/a0030402 · Cited by 23
“…phonemic contrast /p/ – /k/ is facilitated by simultaneous presentation of the visually discriminable mouth movements required to make these sounds (Sumby and Pollack, 1954; Liberman, et al., 1967; McGurk and MacDonald, 1976; Kuhl and Meltzoff, 1982; 1984; Hollich et al., 2005, Hazan et al.,…”
Zekveld, A., Kramer, S., Kessens, J., Vlaming, M., & Houtgast, T. (2008) · Ear and Hearing · doi:10.1097/aud.0b013e31818005bd · Cited by 24
“0196/0202/08/2906-0838/0 • Ear & Hearing • Copyright © 2008 by Lippincott Williams & Wilkins • Printed in the U.S.A. 838 (lipreading) increases the comprehension of speech in difficult listening situations (e.g., Sumby & Pollack, 1954).”
BackgroundThe Auditory-Visual Speech Benefit on Working Memory in Older Adults with Hearing Impairment
Frtusova, J. B., & Phillips, N. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00490 · Cited by 24
“…(AV) speech, in which both auditory and visual speech cues (i.e., lip, tongue, and face movements) are available, enhances speech recognition (e.g., Sumby and Pollack, 1954; Klucharev et al., 2003; Bernstein and Grant, 2009; Ma et al., 2009; Tanaka et al., 2009; Fraser et al., 2010; Winneke and…”
BackgroundJuan, C., Cappe, C., Alric, B., Roby, B., Gilardeau, S., Barone, P., & Girard, P. (2017) · PLoS ONE · doi:10.1371/journal.pone.0172480 · Cited by 26
Audiovisual Speech Integration
Beauchamp, M. S. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-407794-2.00042-0 · Cited by 27
Breaking down the cocktail party: Attentional modulation of cerebral audiovisual speech processing
Wikman, P. A., Sahari, E., Salmela, V. R., Leminen, A., Leminen, M., Laine, M., & Alho, K. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117365 · Cited by 25
Novel Two-Stage Audiovisual Speech Filtering in Noisy Environments
Abel, A. K., & Hussain, A. (2013) · Cognitive Computation · doi:10.1007/s12559-013-9231-2 · Cited by 22
“The relationship between audio and visual aspects of speech perception has been established since pioneering work by Sumby and Pollack in 1954 [15], which demonstrated that lip reading improves the intelligibility of speech in noise when audiovisual perception is compared with equivalent audio-only perception.”
Audio-Visual Speech Cue Combination
Arnold, D. H., Tear, M. J., Schindel, R., & Roseboom, W. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010217 · Cited by 22
“It is well established that audio-visual interactions can shape the apparent content of speech [3] and improve the sensitivity of speech recognition [9].”
BackgroundHearing impairment and audiovisual speech integration ability: a case study report
Altieri, N. A., & Hudock, D. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00678 · Cited by 20
“While a listener’s hearing ability certainly influences language performance, decades of research has revealed that cues obtained from the visual modality affect speech recognition capabilities (e.g., Sumby and Pollack, 1954; McGurk and MacDonald, 1976; Massaro, 2004).”
BackgroundInfluentialAge-related sensitive periods influence visual language discrimination in adults
Weikum, W., Vouloumanos, A., Navarra, J., Soto‐Faraco, S., Sebastián‐Gallés, N., & Werker, J. F. (2013) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2013.00086 · Cited by 23
Speech discrimination skills in deaf infants before and after cochlear implantation
Horn, D. L., Houston, D. M., & Miyamoto, R. T. (2007) · Hearing Balance and Communication · doi:10.1080/16513860701659438 · Cited by 20
“The use of natural audiovisual recordings of a speaker producing the non-words in an infant directed manner probably made the task easier for infants to accomplish than an auditory-only task because they could use visual information (28).”
BackgroundJantvik, T., Gustafsson, L., & Papliński, A. P. (2011) · Neural Computation · doi:10.1162/neco_a_00149 · Cited by 23
“It has long been known that visual speech significantly increases auditory speech comprehension under noisy auditory conditions (Sumby & Pollack, 1954 ), and one study has shown that audiovisual speech yields comprehension improvement even when auditory conditions are good (Remez, 2005 ).”
BackgroundNirme, J., Haake, M., Åhlander, V. L., Brännström, K. J., & Sahlén, B. S. (2018) · Logopedics Phoniatrics Vocology · doi:10.1080/14015439.2018.1455894 · Cited by 24
“Adults’ speech recognition in noise has been found to improve when a speaker’s face is visible [7].”
BackgroundHardison, D. M. (2005) · Speech Communication · doi:10.1016/j.specom.2005.02.002 · Cited by 23
“Listeners are often more aware of the contribution of visual information when compensating for noise (e.g., Sumby and Pollack, 1954; Summerfield, 1979) or impaired hearing ability (e.g., Walden et al., 1977).”
BackgroundPark, W. J., & Tadin, D. (2018) · Wiley eBooks · doi:10.1002/9781119170174.epcn210 · Cited by 21
Perceptual Organization of Speech
Remez, R. E. (2021) · Wiley eBooks · doi:10.1002/9781119184096.ch1 · Cited by 21
Exploring the boundaries of gesture-speech integration during language comprehension
Kelly, S. D. (2017) · Gesture Studies · doi:10.1075/gs.7.12kel · Cited by 30
Ricketts, T. A., Picou, E. M., Shehorn, J., & Dittberner, A. B. (2019) · Journal of Speech Language and Hearing Research · doi:10.1044/2019_jslhr-h-19-0013 · Cited by 22
“…cues have been shown to improve performance over a wide range of signal-to-noise ratios (SNRs; Erber, 1975; Grant & Seitz, 1998; O’Neill, 1954; Sumby & Pollack, 1954), particularly in difficult listening situations, such as when usable audibility is limited (Bernstein & Grant, 2009; Helfer &…”
BackgroundVSEGAN: Visual Speech Enhancement Generative Adversarial Network
Xu, X., Wang, Y., Xu, D., Peng, Y., Zhang, C., Jia, J., & Chen, B. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747187 · Cited by 84
Newman, R. S., Kirby, L. A., Von Holzen, K., & Redcay, E. (2021) · Journal of Neurodevelopmental Disorders · doi:10.1186/s11689-020-09348-9 · Cited by 17
Borowiak, K., Schelinski, S., & von Kriegstein, K. (2018) · NeuroImage: Clinical · doi:10.1016/j.nicl.2018.09.019 · Cited by 17
Auditory Deficits in Audiovisual Speech Perception in Adult Asperger’s Syndrome: fMRI Study
Tietze, F.-A., Hundertmark, L., Roy, M., Zerr, M., Sinke, C., Wiswede, D., Walter, M., Münte, T. F., & Szycik, G. R. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02286 · Cited by 16
“Under noisy conditions AV information (in comparison to auditory only information) results in considerable improvements in intelligibility (Sumby and Pollack, 1954; Schwartz et al., 2004; Ross et al., 2007a).”
Background
Metadata
- Authors
- Sumby, W. H., & Pollack, I.
- Affiliations
- Human Factors Operations Research Laboratories, Washington 25, D. C.
- Venue
- The Journal of the Acoustical Society of America
- Publisher
- Acoustical Society of America (ASA)
- Published
- 1954-03-01
- Volume
- 26
- Issue
- 2
- Pages
- 212-215
- ISSN
- 0001-4966 (ISSN-L)
- 1520-8524
- OpenAlex
- W2014621385
- Cited by
- 2,583
- References
- 0
- Citations by year
- 2026: 58 · 2025: 80 · 2024: 82 · 2023: 101 · 2022: 105 · 2021: 136 · 2020: 100 · 2019: 123 · 2018: 102 · 2017: 129 · 2016: 105 · 2015: 107 · 2014: 110 · 2013: 122 · 2012: 96
- FWCI
- 1.81
- Type
- journal article
- Language
- en
- Update notices
- correction (1954-07-01)
- Open access
- closed
- Locations
- The Journal of the Acoustical Society of America (publishedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
- Health Sciences › Health Professions › Speech and Hearing › Hearing Loss and Rehabilitation
- Keywords
- speech intelligibility in noise, visual speech perception, lip reading, speech-to-noise ratio, vocabulary size, oral speech, speech intelligibility test
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.