Visual Contribution to Speech Intelligibility in Noise
The DOI matches its Crossref record.
Abstract
Oral speech intelligibility tests were conducted with, and without, supplementary visual observation of the speaker's facial and lip movements. The difference between these two conditions was examined as a function of the speech-to-noise ratio and of the size of the vocabulary under test. The visual contribution to oral speech intelligibility (relative to its possible contribution) is, to a first approximation, independent of the speech-to-noise ratio under test. However, since there is a much greater opportunity for the visual contribution at low speech-to-noise ratios, its absolute contribution can be exploited most profitably under these conditions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous studies show that non-verbal facial gestures often account for a substantial portion of the perceived feeling and attitude of a speaker18,19 and that face-to-face communication enhances social connectivity20 and intelligibility21.”
From A high-performance neuroprosthesis for speech decoding and avatar control (2023) · cites this paper as [21]
In the citing paper
“Non-verbal facial expressions that accompany virtual or face-to-face communication provide numerous benefits compared with audio-only communication, including improved conveyance of emotion and attitude130,131, that provides increased clarity132.”
From The speech neuroprosthesis (2024) · cites this paper as [132]
In the citing paper
“Estimates suggest that the information gain in this situation is the equivalent of boosting the audible signal by up to 15 dB (Sumby and Pollack, 1954)”
From Behavioral, Perceptual, and Neural Alterations in Sensory and Multisensory Function in Autism Spectrum Disorder (2015) · cites this paper as Sumby and Pollack, 1954
In the citing paper
“Seeing the face ofa speaker can significantly embellish a degraded or noisy auditory speech signal so that it functionally raises the signal-tonoise ratio by as much as 22 dB”
From The McGurk effect in infants (1997) · cites this paper as Sumby & Pollack, 1954
Citations
Cited by
Multisensory integration: current issues from the perspective of the single neuron
Stein, B. E., & Stanford, T. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2331 · Cited by 1,598
Brain-to-brain coupling: a mechanism for creating and sharing a social world
Hasson, U., Ghazanfar, A. A., Galantucci, B., Garrod, S., & Keysers, C. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.12.007 · Cited by 1,140
Cortical oscillations and sensory predictions
Arnal, L. H., & Giraud, A.-L. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.05.003 · Cited by 1,114
Activation of Auditory Cortex During Silent Lipreading
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., McGuire, P. K., Woodruff, P. W. R., Iversen, S. D., & David, A. S. (1997) · Science · doi:10.1126/science.276.5312.593 · Cited by 1,025
Crossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies
Calvert, G. A. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1110 · Cited by 1,012
“For example, speech comprehension is improved when the speaker can be seen as well as heard (Sumby and Pollack, 1954; Reisberg, 1987), but impaired if the lip and mouth movements are semantically incongruent (Dodd, 1977).”
BackgroundAcoustic Communication in Noise
Brumm, H., & Slabbekoorn, H. (2005) · Advances in the Study of Behavior · doi:10.1016/s0065-3454(05)35004-2 · Cited by 973
Neuronal oscillations and multisensory interaction in primary auditory cortex
Lakatos, P., Chen, C.-M., O'Connell, M. N., Mills, A., & Schroeder, C. E. (2007) · Neuron · doi:10.1016/j.neuron.2006.12.011 · Cited by 961
“The best known example of this effect at a perceptual level is the demonstration over 50 years ago by Sumby and Pollack (1954) that viewing a speaker’s lip movements amplifies the subjective loudness of spoken words.”
BackgroundComplex signal function: developing a framework of testable hypotheses
Hebets, E. A., & Papaj, D. R. (2005) · Behavioral Ecology and Sociobiology · doi:10.1007/s00265-004-0865-7 · Cited by 958
Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party”
Zion Golumbic, E. M., Ding, N., Bickel, S., Lakatos, P., Schevon, C. A., McKhann, G. M., Goodman, R. R., Emerson, R., Mehta, A. D., Simon, J. Z., et al. (2013) · Neuron · doi:10.1016/j.neuron.2012.12.037 · Cited by 909
“It is known that viewing a talking face contributes to speech processing, particularly under noisy auditory conditions (Bishop and Miller, 2009; Sumby and Pollack, 1954).”
BackgroundCritical periods in speech perception: New directions
Werker, J. F., & Hensch, T. K. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015104 · Cited by 774
“For example, it is easier to understand degraded speech if visual information from talking faces is also provided (Sumby & Pollack 1954).”
BackgroundRecent advances in the automatic recognition of audiovisual speech
Pomianos, G., Neti, C., Gravier, G., Garg, A., & Senior, A. W. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817150 · Cited by 752
“The visual modality benefit to speech intelligibility in noise has been quantified as far back as in 1954 [6].”
BackgroundReceiver psychology and the evolution of multicomponent signals
Rowe, C. (1999) · Animal Behaviour · doi:10.1006/anbe.1999.1242 · Cited by 704
Musicians have enhanced subcortical auditory and audiovisual processing of speech and music
Musacchia, G., Sams, M., Skoe, E., & Kraus, N. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0701498104 · Cited by 678
The motor theory of speech perception reviewed
Galantucci, B., Fowler, C. A., & Turvey, M. T. (2006) · Psychonomic Bulletin & Review · doi:10.3758/bf03193857 · Cited by 672
“Another example is the finding that listeners perceive speech in noise more accurately when they can see the speaker than when they cannot ( Sumby & Pollack, 1954 ).”
BackgroundRoss, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C., & Foxe, J. J. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl024 · Cited by 650
“In the past, it has been suggested that the gain provided by AV stimulation continues to increase as the information available from the auditory modality decreases (Sumby and Pollack 1954; Erber 1969).”
BackgroundMethodResultInfluentialTemporal window of integration in auditory-visual speech perception
van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.01.001 · Cited by 621
“For nstance, a congruent facial display (articulating the audio peech) provides critical cues in noisy environments (Helfer, 997; MacLeod & Summerfield, 1990; Sumby & Pollack, 1954) nd benefits hearing-impaired listeners (Grant, Walden, & Seitz, 998).”
BackgroundIssues in the Classification of Multimodal Communication Signals
Partan, S. R., & Marler, P. (2005) · The American Naturalist · doi:10.1086/431246 · Cited by 596
Integration of Letters and Speech Sounds in the Human Brain
van Atteveldt, N., Formisano, E., Goebel, R., & Blomert, L. (2004) · Neuron · doi:10.1016/j.neuron.2004.06.025 · Cited by 544
“Sumby and Pollack (1954) showed sis of individual cortical anatomy revealed that letters that lip reading can improve speech perception, espeand speech sounds are integrated in heteromodal cially in situations when the auditory input is degraded.”
BackgroundA high-performance neuroprosthesis for speech decoding and avatar control
Metzger, S. L., Littlejohn, K. T., Silva, A. B., Moses, D. A., Seaton, M. P., Wang, R., Dougherty, M. E., Liu, J. R., Wu, P., Berger, M. A., et al. (2023) · Nature · doi:10.1038/s41586-023-06443-4 · Cited by 522
Extraction of visual features for lipreading
Matthews, I., Cootes, T. F., Bangham, J. A., Cox, S., & Harvey, R. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.982900 · Cited by 537
“Two of these are top-down approaches that fit a model of the inner and outer lip contours and derive lipreading features from a principal component analysis of shape or shape and appearance, respectively.”
BackgroundInfants deploy selective attention to the mouth of a talking face when learning speech
Lewkowicz, D. J., & Hansen-Tift, A. M. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1114783109 · Cited by 526
The use of visible speech cues for improving auditory detection of spoken sentences
Grant, K. W., & Seitz, P.-F. (2000) · The Journal of the Acoustical Society of America · doi:10.1121/1.1288668 · Cited by 521
Infants' perception of expressive behaviors: Differentiation of multimodal information.
Walker‐Andrews, A. S. (1997) · Psychological Bulletin · doi:10.1037//0033-2909.121.3.437 · Cited by 514
“Sumby and Pollack (1954) reported that to watch a speaker's lip is tantamount to a gain of about 20 dB SPL in intensity.”
BackgroundVisual Modulation of Neurons in Auditory Cortex
Kayser, C., Petkov, C. I., & Logothetis, N. K. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm187 · Cited by 509
“A classical example is the well-known cocktail party where visual information helps to segregate the voice of a speaker from background noise (Sumby and Polack 1954).”
BackgroundNeuronal oscillations and visual amplification of speech
Schroeder, C. E., Lakatos, P., Kajikawa, Y., Partan, S., & Puce, A. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.01.002 · Cited by 498
Oviatt, S. (2007) · Human Factors and Ergonomics · doi:10.1201/9781410615862.ch21 · Cited by 485
Multisensory temporal integration in autism spectrum disorders
Stevenson, R. A., Siemann, J. K., Schneider, B. C., Eberly, H. E., Woynaroski, T. G., Camarata, S. M., & Wallace, M. T. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.3615-13.2014 · Cited by 482
“Multisensory binding and integration convey a host of behavioral gains, including substantial benefits in speech comprehension (Sumby and Pollack, 1954).”
BackgroundPerception of intersensory synchrony: A tutorial review
Vroomen, J., & Keetels, M. (2010) · Attention, Perception, & Psychophysics · doi:10.3758/app.72.4.871 · Cited by 464
“The multisensory nature of the world is highly advantageous, because it increases perceptual reliability and saliency, and, as a result, it can enhance learning, discrimination, or the speed of a reaction to the stimulus (e.g., Sumby & Pollack, 1954; Summerfield, 1987).”
Enhanced multisensory integration in older adults
Laurienti, P. J., Burdette, J. H., Maldjian, J. A., & Wallace, M. T. (2006) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2005.05.024 · Cited by 451
“Relating these findings to a realworld situation, the use of video telephones could provide redundant visual and auditory cues resulting in amplification of the perceived volume of the speaker [61] and increased listener comprehension [15] [16] [23].”
BackgroundWayne, R. V., & Johnsrude, I. S. (2015) · Ageing Research Reviews · doi:10.1016/j.arr.2015.06.002 · Cited by 449
Audio-visual speech modeling for continuous speech recognition
Dupont, S., & Luettin, J. (2000) · IEEE Transactions on Multimedia · doi:10.1109/6046.865479 · Cited by 598
Quantifying the contribution of vision to speech perception in noise
Macleod, A., & Summerfield, Q. (1987) · British Journal of Audiology · doi:10.3109/03005368709077786 · Cited by 426
Broderick, M. P., Anderson, A. J., Di Liberto, G. M., Crosse, M. J., & Lalor, E. C. (2018) · Current Biology · doi:10.1016/j.cub.2018.01.080 · Cited by 421
Murray, M. M., Molholm, S., Michel, C. M., Heslenfeld, D. J., Ritter, W., Javitt, D. C., Schroeder, C. E., & Foxe, J. J. (2005) · Cerebral Cortex · doi:10.1093/cercor/bhh197 · Cited by 413
Enhancement of Perceived Visual Intensity by Auditory Stimuli: A Psychophysical Analysis
Stein, B. E., London, N., Wilkinson, L. K., & Price, D. D. (1996) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1996.8.6.497 · Cited by 404
Predictive top-down integration of prior knowledge during speech perception
Sohoglu, E., Peelle, J. E., Carlyon, R. P., & Davis, M. H. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.5069-11.2012 · Cited by 388
“Speech perception is an ideal context in which to study integration effects because in everyday listening we constantly exploit prior information—such as a speaker’s lip movements or semantic context—to interpret incoming speech signals (Sumby, 1954; Miller and Isard, 1963).”
BackgroundVisual Language Discrimination in Infancy
Weikum, W. M., Vouloumanos, A., Navarra, J., Soto-Faraco, S., Sebastián-Gallés, N., & Werker, J. F. (2007) · Science · doi:10.1126/science.1137686 · Cited by 385
Multisensory Visual-Auditory Object Recognition in Humans: a High-density Electrical Mapping Study
Molholm, S. (2004) · Cerebral Cortex · doi:10.1093/cercor/bhh007 · Cited by 384
“McGurk and MacDonald, 1976), or enhance speech perception in a noisy environment ( Sumby and Pollack, 1954; Campbell and Dodd, 1980; MacLeod and Summerfield, 1990; Thompson, 1995).”
BackgroundSensory integration and the perceptual experience of persons with autism
Iarocci, G., & McDonald, J. (2006) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-005-0044-3 · Cited by 376
“These enhancements were hypothesized to underlie the perceptual gains resulting from concurrent audiovisual stimulation (e.g., Sumby & Pollack, 1954).”
BackgroundBaum, S. H., Stevenson, R. A., & Wallace, M. T. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.007 · Cited by 370
Audiovisual Integration of Speech Falters under High Attention Demands
Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005) · Current Biology · doi:10.1016/j.cub.2005.03.046 · Cited by 380
“The visual-only condition produced hardly any correct responses, confirming that information available from silent speech actions alone is usually insufficient to enable word identification under open-response set conditions [11, 12].”
BackgroundThe Leading Sense: Supramodal Control of Neurophysiological Context by Attention
Lakatos, P., O'Connell, M. N., Barczak, A., Mills, A., Javitt, D. C., & Schroeder, C. E. (2009) · Neuron · doi:10.1016/j.neuron.2009.10.014 · Cited by 381
“It has been shown that visual inputs support audiovisual speech processing in noisy environmental conditions (Sumby and Pollack, 1954) and that attention to the visual modality plays an important role in this process (Senkowski et al., 2008b).”
Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli
Laurienti, P. J., Burdette, J. H., Wallace, M. T., Yen, Y.-F., Field, A. S., & Stein, B. E. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902317361930 · Cited by 369
“…enhance or depress stimulus detection and feature recognition, depending on the nature of the individual stimuli and how they are paired (Driver & Spence, 1998; Stein & Meredith, 1993; Stein, London, Wilkinson, & Price, 1996; Welch & Warren, 1986; McGurk & MacDonald, 1976; Sumby & Pollack, 1954).”
BackgroundSkipper, J. I., van Wassenhove, V., Nusbaum, H. C., & Small, S. L. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhl147 · Cited by 369
“Adding visible facial movements to speech enhances speech recognition comparable with removing up to 20 dB of noise from the auditory signal (Sumby and Pollack 1954).”
BackgroundTemporal constraints on the McGurk effect
Munhall, K. G., Gribble, P., Sacco, L., & Ward, M. (1996) · Perception & Psychophysics · doi:10.3758/bf03206811 · Cited by 358
Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A. (1997) · Perception & Psychophysics · doi:10.3758/bf03211902 · Cited by 367
Visual Prosody and Speech Intelligibility: Head Movement Improves Auditory Speech Perception
Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., & Vatikiotis-Bateson, E. (2004) · Psychological Science · doi:10.1111/j.0963-7214.2004.01502010.x · Cited by 482
“It is well known that the intelligibility of degraded auditory speech is enhanced when listeners view a talker’s lip movements ( Sumby & Pollack, 1954 ).”
BackgroundFunctional imaging of human crossmodal identification and object recognition
Amedi, A., von Kriegstein, K., van Atteveldt, N. M., Beauchamp, M. S., & Naumer, M. J. (2005) · Experimental Brain Research · doi:10.1007/s00221-005-2396-5 · Cited by 353
“2003), and this is consistent with behavioral findings (Sumby and Pollack 1954).”
BackgroundResultSound alters activity in human V1 in association with illusory visual perception
Watkins, S., Shams, L., Tanaka, S., Haynes, J.-D., & Rees, G. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.016 · Cited by 343
“…integration in primate striate cortex”, J. Neurosci., vol. 22, pp.5749 - 5759, 2002 聴覚連合野における聴覚と体性感覚の感覚統合プロセス [7] J.J. Foxe, G.R. Wylie, A. Martinez, C.E. Schroeder, D.C. Javitt, D. Guilfoyle, W. Ritter and M.M. Murray, ”Auditory-somatosensory multisensory processing in auditory association…”
Grant, K. W., Walden, B. E., & Seitz, P. F. (1998) · The Journal of the Acoustical Society of America · doi:10.1121/1.422788 · Cited by 342
Cognitive aging and auditory information processing
Pichora-Fuller, M. K. (2003) · International Journal of Audiology · doi:10.3109/14992020309074641 · Cited by 339
Seeing, Doing, and Knowing: A Philosophical Theory of Sense Perception
Matthen, M. (2005) · doi:10.1093/0199268509.001.0001 · Cited by 333
Reading Speech from Still and Moving Faces: The Neural Substrates of Visible Speech
Calvert, G. A., & Campbell, R. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321107828 · Cited by 332
“…a topic of interest to clinical researchers interested in the implications of hearing loss, deafness ( Jeffers & Barley, 1971), or hearing in noise (Sumby & Pollack, 1954), but is increasingly seen to have implications for understanding the mechanisms of speech and language processing more…”
BackgroundResponse amplification in sensory-specific cortices during crossmodal binding
Calvert, G. A., Brammer, M. J., Bullmore, E. T., Campbell, R., Iversen, S. D., & David, A. S. (1999) · NeuroReport · doi:10.1097/00001756-199908200-00033 · Cited by 331
Multisensory Processing in Review: from Physiology to Behaviour
Alais, D., Newell, F., & Mamassian, P. (2010) · Seeing and Perceiving · doi:10.1163/187847510x488603 · Cited by 324
Listening to talking faces: motor cortical activation during speech perception
Skipper, J. I., Nusbaum, H. C., & Small, S. L. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2004.11.006 · Cited by 324
Bimodal speech: early suppressive visual effects in human auditory cortex
Besle, J., Fort, A., Delpuech, C., & Giard, M. (2004) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2004.03670.x · Cited by 323
Perceptual Fusion and Stimulus Coincidence in the Cross-Modal Integration of Speech
Miller, L. M., & D'Esposito, M. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0896-05.2005 · Cited by 322
A data-based method for assessing and reducing human error to improve operational performance
Williams, J. C. (2003) · Conference Record for 1988 IEEE Fourth Conference on Human Factors and Power Plants, · doi:10.1109/hfpp.1988.27540 · Cited by 320
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation
Michelsanti, D., Tan, Z.-H., Zhang, S.-X., Xu, Y., Yu, M., Yu, D., & Jensen, J. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3066303 · Cited by 316
“Although simpler to conduct, this kind of setting has the disadvantage of completely ignoring the visual information, which has an impact on speech perception [174], [236].”
BackgroundWallace, M. T., & Stevenson, R. A. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.08.005 · Cited by 315
“…in a multisensory (i.e., audiovisual) context within a noisy environment (Bishop & Miller, 2009; Erber, 1975; Girin, Schwartz, & Feng, 2001; Grant & Walden, 1996; Grant, Walden, & Seitz, 1998; MacLeod & Summerfield, 1987; Robert-Ribes et al., 1998; Stevenson & James, 2009; Sumby & Pollack, 1954).”
BackgroundThe Intermodal Representation of Speech in Infants
Kuhl, P. K., & Meltzoff, A. N. (1984) · Infant Behavior and Development · doi:10.1016/s0163-6383(84)80050-8 · Cited by 310
State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications
Zollhöfer, M., Thies, J., Garrido, P., Bradley, D., Beeler, T., Pérez, P., Stamminger, M., Nießner, M., & Theobalt, C. (2018) · Computer Graphics Forum · doi:10.1111/cgf.13382 · Cited by 307
Cross-modal interactions in the perception of musical performance
VINES, B., KRUMHANSL, C., WANDERLEY, M., & LEVITIN, D. (2006) · Cognition · doi:10.1016/j.cognition.2005.09.003 · Cited by 306
“A number of experiments have shown that the intelligibility of speech is significantly increased when the speaker’s face can be seen as well as heard (MacLeod & Summerfield, 1987; Massaro, 1987, 1998; Repp, Frost, & Zsiga, 1992; Sumby & Pollack, 1954).”
BackgroundAttention and the multiple stages of multisensory integration: A review of audiovisual studies
Koelewijn, T., Bronkhorst, A., & Theeuwes, J. (2010) · Acta Psychologica · doi:10.1016/j.actpsy.2010.03.010 · Cited by 306
“This attribution of voices to congruent sources is generally bene fi cial and results in improved perception under noisy circumstances (Sumby & Pollack, 1954).”
Auditory–visual integration of talker gender in vowel perception
Johnson, K., Strand, E. A., & D'Imperio, M. (1999) · Journal of Phonetics · doi:10.1006/jpho.1999.0100 · Cited by 305
“For example, Sumby & Pollack (1954) found that the speech reception 1&&Linguistic theory is concerned primarily with an ideal speaker-listener, in a completely homogeneous speech-community, who knows its language perfectly and is una!ected by such grammatically irrelevant conditions as memory…”
BackgroundEvidence that cochlear-implanted deaf patients are better multisensory integrators
Rouger, J., Lagleyre, S., Fraysse, B., Deneve, S., Deguine, O., & Barone, P. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0609419104 · Cited by 301
Do early sensory cortices integrate cross-modal information?
Kayser, C., & Logothetis, N. K. (2007) · Brain Structure and Function · doi:10.1007/s00429-007-0154-0 · Cited by 293
“And concerning the above example of the cocktail party, a classic study found that the visual input corresponds to a hearing improvement that equivalents about 15–20 dB of sound intensity (Sumby and Polack 1954).”
BackgroundSeeing to hear better: evidence for early audio-visual interactions in speech identification
SCHWARTZ, J. (2004) · Cognition · doi:10.1016/s0010-0277(04)00054-x · Cited by 279
The processing of audio-visual speech: empirical and neural bases
Campbell, R. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2155 · Cited by 282
“Fifty years ago, it was established that seeing the talker could give an improvement in the comprehension of auditory speech in noise equivalent to that produced by an increase of up to 15 dB in signalto-noise ratio (Sumby & Pollack 1954).”
BackgroundPolysensory Interactions along Lateral Temporal Regions Evoked by Audiovisual Speech
Wright, T. M. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.10.1034 · Cited by 277
Seeing to hear better: evidence for early audio-visual interactions in speech identification
Schwartz, J.-L., Berthommier, F., & Savariaux, C. (2004) · Cognition · doi:10.1016/j.cognition.2004.01.006 · Cited by 275
The Bilingual Language Interaction Network for Comprehension of Speech
SHOOK, A., & MARIAN, V. (2013) · Bilingualism: Language and Cognition · doi:10.1017/s1366728912000466 · Cited by 275
“Visual input at the phonological level can also help perception – a classic study by Sumby and Pollack (1954) showed that corroborating visual information improves word recognition in noise.”
Neural mechanisms for synthesizing sensory information and producing adaptive behaviors
Stein, B. E. (1998) · Experimental Brain Research · doi:10.1007/s002210050553 · Cited by 264
“That is why being able to see a speaker's face makes it far easier to understand the spoken message, especially in a noisy room (Sumby and Pollack 1954).”
BackgroundFoxe, J. J., Molholm, S., Del Bene, V. A., Frey, H.-P., Russo, N. N., Blanco, D., Saint-Amour, D., & Ross, L. A. (2015) · Cerebral Cortex · doi:10.1093/cercor/bht213 · Cited by 263
Hickok, G. (2012) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2012.06.004 · Cited by 260
“Visual speech has a significant effect on speech recognition (Dodd, 1977; Sumby & Pollack, 1954), so much so that mismatched visual speech information can alter auditory speech percepts under some conditions (McGurk & MacDonald, 1976).”
Visual Input Enhances Selective Speech Envelope Tracking in Auditory Cortex at a “Cocktail Party”
Zion Golumbic, E., Cogan, G. B., Schroeder, C. E., & Poeppel, D. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3675-12.2013 · Cited by 258
Aging and Speech-on-Speech Masking
Helfer, K. S., & Freyman, R. L. (2008) · Ear & Hearing · doi:10.1097/aud.0b013e31815d638b · Cited by 256
“The averaged group scores were used to derive relative group differences using the following formula (adapted from Sumby & Pollack, 1954): Relative difference (younger subject score older subject score)/100 older subject score The relative group differences are shown in Figure 3.”
MethodSpeech-associated gestures, Broca’s area, and the human mirror system
Skipper, J. I., Goldin-Meadow, S., Nusbaum, H. C., & Small, S. L. (2007) · Brain and Language · doi:10.1016/j.bandl.2007.02.008 · Cited by 252
“…motor areas have been claimed (Skipper et al., 2005, 2006, 2007) to contribute to the improvement of phonetic recognition when mouth movements are observed during speech perception (see Grant & Greenberg, 2001; Reisberg, McLean, & Goldfield, 1987; Risberg & Lubker, 1978; Sumby & Pollack , 1954).”
BackgroundAudio-visual integration in multimodal communication
Chen, T., & Rao, R. R. (1998) · Proceedings of the IEEE · doi:10.1109/5.664274 · Cited by 251
“A study by Sumby and Pollack [17] showed that when noisy environments are encountered, visual information can lead to significant improvement in recognition by humans.”
BackgroundNarrowing of intersensory speech perception in infancy
Pons, F., Lewkowicz, D. J., Soto-Faraco, S., & Sebastián-Gallés, N. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0904134106 · Cited by 249
“In general, audiovisual redundancy increases the overall salience of the information (26) and, as a result, enhances perception, learning, and discrimination (12, 27–29).”
BackgroundAuditory-visual speech perception examined by fMRI and PET
Sekiyama, K., Kanno, I., Miura, S., & Sugita, Y. (2003) · Neuroscience Research · doi:10.1016/s0168-0102(03)00214-1 · Cited by 249
“They facilitate speech perception when auditory speech is degraded (e.g. Sumby and Pollack, 1954; Rosen et al., 1981).”
BackgroundDual Neural Routing of Visual Facilitation in Speech Processing
Arnal, L. H., Morillon, B., Kell, C. A., & Giraud, A.-L. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3194-09.2009 · Cited by 241
Prediction and constraint in audiovisual speech perception
Peelle, J. E., & Sommers, M. S. (2015) · Cortex · doi:10.1016/j.cortex.2015.03.006 · Cited by 246
“Consequently, it is not surprising that speech perception is almost always better when listeners can both see and hear a talker compared with listening alone (Erber, 1975; Sumby & Pollack, 1954).”
Burnham, D., & Dodd, B. (2004) · Developmental Psychobiology · doi:10.1002/dev.20032 · Cited by 247
Sommers, M. S., Tye-Murray, N., & Spehar, B. (2005) · Ear and Hearing · doi:10.1097/00003446-200506000-00003 · Cited by 246
“Moreover, the benefits of combining auditory and visual speech information increase with the difficulty of auditory-only (A) perception (Sumby & Pollack, 1954).”
BackgroundInfluentialSpeech perception without hearing
Bernstein, L. E., Tucker, P. E., & Demorest, M. E. (2000) · Perception & Psychophysics · doi:10.3758/bf03205546 · Cited by 241
Green, K. P., Kuhl, P. K., Meltzoff, A. N., & Stevens, E. B. (1991) · Perception & Psychophysics · doi:10.3758/bf03207536 · Cited by 239
Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex
Kayser, C., Petkov, C. I., Augath, M., & Logothetis, N. K. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.4737-06.2007 · Cited by 240
“For example, our perception of speech profits mostly from simultaneous visual cues when the auditory signal is corrupted by background noise (Sumby and Polack, 1954).”
Multisensory Processes: A Balancing Act across the Lifespan
Murray, M. M., Lewkowicz, D. J., Amedi, A., & Wallace, M. T. (2016) · Trends in Neurosciences · doi:10.1016/j.tins.2016.05.003 · Cited by 240
Spatial and temporal factors during processing of audiovisual speech: a PET study
Macaluso, E., George, N., Dolan, R., Spence, C., & Driver, J. (2004) · NeuroImage · doi:10.1016/j.neuroimage.2003.09.049 · Cited by 237
“Visual cues (lip movements and other visual information from mouth and face) can aid in hearing what has been said and in perceiving where the speech signal came from, especially in noisy environments (e.g., Bertelson and de Gelder, in press; Driver and Spence, 1994; Sumby and Pollack, 1954).”
BackgroundEye movement of perceivers during audiovisualspeech perception
Vatikiotis-Bateson, E., Eigsti, I.-M., Yano, S., & Munhall, K. G. (1998) · Perception & Psychophysics · doi:10.3758/bf03211929 · Cited by 236
“Intelligibility is maintained better at higher levels of masking noise when both visual and acoustic cues are available (Sumby & Pollack, 1954).”
BackgroundOn the perceptual organization of speech.
Remez, R. E., Rubin, P. E., Berns, S. M., Pardo, J. S., & Lang, J. M. (1994) · Psychological Review · doi:10.1037/0033-295x.101.1.129 · Cited by 234
The Effects of Visibility on Interaction in a Dyad
Argyle, M., Lalljee, M., & Cook, M. (1968) · Human Relations · doi:10.1177/001872676802100101 · Cited by 235
The emergence of Cognitive Hearing Science
ARLINGER, S., LUNNER, T., LYXELL, B., & KATHLEEN PICHORA‐FULLER, M. (2009) · Scandinavian Journal of Psychology · doi:10.1111/j.1467-9450.2009.00753.x · Cited by 231
“It has been known since the post-WWII era that word recognition can remain highly accurate in adverse SNR conditions if visual speech cues are available (Sumby & Pollack, 1954).”
BackgroundBimodal bilinguals co-activate both languages during spoken comprehension
Shook, A., & Marian, V. (2012) · Cognition · doi:10.1016/j.cognition.2012.05.014 · Cited by 231
“Shahin & Miller, 2009; Sumby & Pollack, 1954; Summerfield , 1987).”
Chen, T. (2001) · IEEE Signal Processing Magazine · doi:10.1109/79.911195 · Cited by 230
The Experimental Analysis of Social Performance
Argyle, M., & Kendon, A. (1967) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60342-1 · Cited by 228
Semantics and the multisensory brain: How meaning modulates processes of audio-visual integration
Doehrmann, O., & Naumer, M. J. (2008) · Brain Research · doi:10.1016/j.brainres.2008.03.071 · Cited by 228
Pisoni, D. B. (2000) · Ear and Hearing · doi:10.1097/00003446-200002000-00010 · Cited by 222
Metadata
- Authors
- Sumby, W. H., & Pollack, I.
- Affiliations
- Human Factors Operations Research Laboratories, Washington 25, D. C.
- Venue
- The Journal of the Acoustical Society of America
- Publisher
- Acoustical Society of America (ASA)
- Published
- 1954-03-01
- Volume
- 26
- Issue
- 2
- Pages
- 212-215
- ISSN
- 0001-4966 (ISSN-L)
- 1520-8524
- OpenAlex
- W2014621385
- Cited by
- 2,583
- References
- 0
- Citations by year
- 2026: 58 · 2025: 80 · 2024: 82 · 2023: 101 · 2022: 105 · 2021: 136 · 2020: 100 · 2019: 123 · 2018: 102 · 2017: 129 · 2016: 105 · 2015: 107 · 2014: 110 · 2013: 122 · 2012: 96
- FWCI
- 1.81
- Type
- journal article
- Language
- en
- Update notices
- correction (1954-07-01)
- Open access
- closed
- Locations
- The Journal of the Acoustical Society of America (publishedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
- Health Sciences › Health Professions › Speech and Hearing › Hearing Loss and Rehabilitation
- Keywords
- speech intelligibility in noise, visual speech perception, lip reading, speech-to-noise ratio, vocabulary size, oral speech, speech intelligibility test
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.