Bimodal speech perception: an examination across languages

Hearing lips and seeing voices

MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183

Strong Inference: Certain systematic methods of scientific thinking may produce much more rapid progress than others.

Platt, J. R. (1964) · Science · doi:10.1126/science.146.3642.347 · Cited by 2,907

Software for a cascade/parallel formant synthesizer

Klatt, D. H. (1980) · The Journal of the Acoustical Society of America · doi:10.1121/1.383940 · Cited by 1,585

Patterns of sounds

Maddieson, I. (1984) · Medical Entomology and Zoology · Cited by 1,037

Continuous Shading of Curved Surfaces

Gouraud, H. (1971) · IEEE Transactions on Computers · doi:10.1109/t-c.1971.223313 · Cited by 700

Speech perception by ear and eye a paradigm for psychological inquiry

Massaro, D. W. (1987) · Cited by 585

Parameterized Models for Facial Animation

Parke (1982) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.1982.1674492 · Cited by 487

Use of Visual Information for Phonetic Perception

Summerfield, Q. A. (1979) · Phonetica · doi:10.1159/000259969 · Cited by 320

McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility

Sekiyama, K., & Tohkura, Y. (1991) · The Journal of the Acoustical Society of America · doi:10.1121/1.401660 · Cited by 286

A parametric model for human faces.

Parke, F. I. (1974) · Defense Technical Information Center (DTIC) · Cited by 272

The McGurk effect in infants

Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A. (1997) · Perception & Psychophysics · doi:10.3758/bf03211902 · Cited by 367

Visual Prosody and Speech Intelligibility: Head Movement Improves Auditory Speech Perception

Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., & Vatikiotis-Bateson, E. (2004) · Psychological Science · doi:10.1111/j.0963-7214.2004.01502010.x · Cited by 482

Learning and representation in speech and language

Kuhl, P. K. (1994) · Current Opinion in Neurobiology · doi:10.1016/0959-4388(94)90128-7 · Cited by 329

INFLUENCES ON INFANT SPEECH PROCESSING: Toward a New Synthesis

Werker, J. F., & Tees, R. C. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.509 · Cited by 312

Auditory-visual speech perception examined by fMRI and PET

Sekiyama, K., Kanno, I., Miura, S., & Sugita, Y. (2003) · Neuroscience Research · doi:10.1016/s0168-0102(03)00214-1 · Cited by 256

Auditory–visual speech integration by prelinguistic infants: Perception of an emergent consonant in the McGurk effect

Burnham, D. K., & Dodd, B. J. (2004) · Developmental Psychobiology · doi:10.1002/dev.20032 · Cited by 267

Eye movement of perceivers during audiovisualspeech perception

Vatikiotis‐Bateson, E., Eigsti, I., Yano, S., & Munhall, K. G. (1998) · Perception & Psychophysics · doi:10.3758/bf03211929 · Cited by 252

Lip-Reading Aids Word Recognition Most in Moderate Noise: A Bayesian Explanation Using High-Dimensional Feature Space

Ma, W. J., Zhou, X., Ross, L. A., Foxe, J. J., & Parra, L. C. (2009) · PLoS ONE · doi:10.1371/journal.pone.0004638 · Cited by 200

Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects

Sekiyama, K. (1997) · Perception & Psychophysics · doi:10.3758/bf03206849 · Cited by 177

Effects of Phonetic Context on Audio-Visual Intelligibility of French

Benoı̂t, C., Mohamadi, T., & Kandel, S. (1994) · Journal of Speech Language and Hearing Research · doi:10.1044/jshr.3705.1195 · Cited by 163

Impact of language on development of auditory‐visual speech perception

Sekiyama, K., & Burnham, D. K. (2008) · Developmental Science · doi:10.1111/j.1467-7687.2008.00677.x · Cited by 159

Listening to a second language through the ears of a first

Cutler, A. (2000) · Interpreting International Journal of Research and Practice in Interpreting · doi:10.1075/intp.5.1.02cut · Cited by 132

Hearing lips in a second language: visual articulatory information enables the perception of second language sounds

Navarra, J., & Soto‐Faraco, S. (2005) · Psychological Research · doi:10.1007/s00426-005-0031-5 · Cited by 149

An Exploration of Why Preschoolers Perform Differently Than Do Adults in Audiovisual Speech Perception Tasks

Desjardins, R. N., Rogers, J., & Werker, J. F. (1997) · Journal of Experimental Child Psychology · doi:10.1006/jecp.1997.2379 · Cited by 126

The Role of Audiovisual Speech and Orthographic Information in Nonnative Speech Production

Erdener, D., & Burnham, D. K. (2005) · Language Learning · doi:10.1111/j.0023-8333.2005.00303.x · Cited by 121

The use of visual cues in the perception of non-native consonant contrasts

Hazan, V. L., Sennema, A., Faulkner, A., Ortega-Llebaría, M., Iba, M., & Chung, H. (2006) · The Journal of the Acoustical Society of America · doi:10.1121/1.2166611 · Cited by 118

Enhanced audiovisual integration with aging in speech perception: a heightened McGurk effect in older adults

Sekiyama, K., Soshi, T., & Sakamoto, S. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00323 · Cited by 120

Visual, orthographic, phonological, and lexical influences in reading.

Massaro, D. W., & Cohen, M. M. (1994) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.20.6.1107 · Cited by 84

Seeing pitch: Visual information for lexical tones of Mandarin-Chinese

Chen, T. H., & Massaro, D. W. (2008) · The Journal of the Acoustical Society of America · doi:10.1121/1.2839004 · Cited by 81

Dispersed activation in the left temporal cortex for speech-reading in congenitally deaf people

MacSweeney, M., Campbell, R., Calvert, G. A., McGuire, P. K., David, A. S., John Suckling, A., Andrew, C., Woll, B., & Brammer, M. J. (2001) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2000.0393 · Cited by 75

Multisensory integration of speech signals: the relationship between space and time

Jones, J. A., & Jarick, M. (2006) · Experimental Brain Research · doi:10.1007/s00221-006-0634-0 · Cited by 71

Studies in sound symbolism

Abelin, Å. (1999) · Gothenburg University Publications Electronic Archive (Gothenburg University) · Cited by 69

Catalan speakers’ perception of word stress in unaccented contexts

Ortega-Llebaría, M., del Mar, M., & Prieto, P. (2010) · The Journal of the Acoustical Society of America · doi:10.1121/1.3268506 · Cited by 67

Seeing and hearing rotated faces: Influences of facial orientation on visual and audiovisual speech recognition.

Jordan, T. R., & Bevan, K. M. (1997) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.23.2.388 · Cited by 63

A reanalysis of McGurk data suggests that audiovisual fusion in speech perception is subject-dependent

Schwartz, J. (2010) · The Journal of the Acoustical Society of America · doi:10.1121/1.3293001 · Cited by 70

Audiovisual sentence recognition not predicted by susceptibility to the McGurk effect

Engen, K. J. V., Xie, Z., & Chandrasekaran, B. (2016) · Attention Perception & Psychophysics · doi:10.3758/s13414-016-1238-9 · Cited by 61

Cue Integration in Categorical Tasks: Insights from Audio-Visual Speech Perception

Bejjanki, V. R., Clayards, M. A., Knill, D. C., & Aslin, R. Ν. (2011) · PLoS ONE · doi:10.1371/journal.pone.0019812 · Cited by 61

Cross-modal discrepancies in coarticulation and the integration of speech information: The McGurk effect with mismatched vowels.

Green, K. P., & Gerdman, A. (1995) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.21.6.1409 · Cited by 56

Bimodal Speech Perception by Native and Nonnative Speakers of English: Factors Influencing the McGurk Effect

Hardison, D. M. (1996) · Language Learning · doi:10.1111/j.1467-1770.1996.tb00640.x · Cited by 59

Enhancing foreign language tutors – In search of the golden speaker

Probst, K., Yan, K., & Eskénazi, M. (2002) · Speech Communication · doi:10.1016/s0167-6393(01)00009-7 · Cited by 54

Speechreading: illusion or window into pattern recognition

Massaro, D. W. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01360-1 · Cited by 53

Multisensory speech perception without the left superior temporal sulcus

Baum, S. H., Martin, R. C., Hamilton, A. C., & Beauchamp, M. S. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.05.034 · Cited by 54

Perceiving speech from inverted faces

Massaro, D. W., & Cohen, M. M. (1996) · Perception & Psychophysics · doi:10.3758/bf03206832 · Cited by 50

The supramodal brain: implications for auditory perception

Rosenblum, L. D., Dias, J. W., & Dorsi, J. (2016) · Journal of Cognitive Psychology · doi:10.1080/20445911.2016.1181691 · Cited by 58

Effects of separating auditory and visual sources on audiovisual integration of speech

Jones, J. A., & Munhall, K. G. (1997) · Canadian acoustics · Cited by 49

Linguistic experience and audio-visual perception of non-native fricatives

Wang, Y., Behne, D. M., & Jiang, H. (2008) · The Journal of the Acoustical Society of America · doi:10.1121/1.2956483 · Cited by 48

Effects of intrastimulus modality change on audiovisual time-to-arrival judgments

Gordon, M. S., & Rosenblum, L. D. (2005) · Perception & Psychophysics · doi:10.3758/bf03193516 · Cited by 46

Influence of native language phonetic system on audio-visual speech perception

Wang, Y., Behne, D. M., & Jiang, H. (2009) · Journal of Phonetics · doi:10.1016/j.wocn.2009.04.002 · Cited by 45

Differences in auditory-visual speech perception between Japanese and Americans: McGurk effect as a function of incompatibility.

Sekiyama, K. (1994) · Journal of the Acoustical Society of Japan (E) · doi:10.1250/ast.15.143 · Cited by 45

Kinematics-based synthesis of realistic talking faces

Kuratate, T., Yehia, H. C., & Vatikiotis‐Bateson, E. (1998) · Cited by 39

Use of visual information in speech perception: Evidence for a visual rate effect both with and without a McGurk effect

Brancazio, L., & Miller, J. L. (2005) · Perception & Psychophysics · doi:10.3758/bf03193531 · Cited by 38

Audio-visual speech scene analysis: Characterization of the dynamics of unbinding and rebinding the McGurk effect

Nahorna, O., Berthommier, F., & Schwartz, J. (2015) · The Journal of the Acoustical Society of America · doi:10.1121/1.4904536 · Cited by 35

Bayes factor of model selection validates FLMP

Massaro, D. W., Cohen, M. M., Campbell, C. S., & Rodriguez, T. (2001) · Psychonomic Bulletin & Review · doi:10.3758/bf03196136 · Cited by 35

Cross-linguistic comparisons in the integration of visual and auditory speech

Massaro, D. W., Cohen, M. M., & Smeele, P. M. T. (1995) · Memory & Cognition · doi:10.3758/bf03210561 · Cited by 35

Mandarin speech perception by ear and eye follows a universal principle

Chen, T. H., & Massaro, D. W. (2004) · Perception & Psychophysics · doi:10.3758/bf03194976 · Cited by 34

Developmental factors and the non-native speaker effect in auditory-visual speech perception

Chen, Y., & Hazan, V. L. (2009) · The Journal of the Acoustical Society of America · doi:10.1121/1.3158823 · Cited by 31

Selective attention to the mouth of a talker in Japanese-learning infants and toddlers: Its relationship with vocabulary and compensation for noise

Sekiyama, K., Hisanaga, S., & Mugitani, R. (2021) · Cortex · doi:10.1016/j.cortex.2021.03.023 · Cited by 25

The McGurk phenomenon in Italian listeners.

Bovo, R., Ciorba, A., Prosser, S., & Martini, A. (2009) · PubMed Central · Cited by 25

Issues in the development of auditory-visual speech perception: adults, infants, and children

Sekiyama, K., & Burnham, D. K. (2004) · Interspeech (USB) · doi:10.21437/interspeech.2004-412 · Cited by 24

Audiovisual Speech Perception and the McGurk Effect

Rosenblum, L. D. (2019) · Oxford Research Encyclopedia of Linguistics · doi:10.1093/acrefore/9780199384655.013.420 · Cited by 24

A comparison of models for fusion of the auditory and visual sensors in speech perception

Robert-Ribes, J., Schwartz, J., & Escudier, P. (1995) · Artificial Intelligence Review · doi:10.1007/bf00849043 · Cited by 23

Measuring and Modeling Speech Production

Rubin, P. E., & Vatikiotis‐Bateson, E. (1998) · Springer eBooks · doi:10.1007/978-3-642-76220-8_8 · Cited by 22

Audiovisual speech perception and word recognition

Massaro, D. W., & Jesse, A. (2012) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198568971.013.0002 · Cited by 21

LANGUAGE EFFECTS ON THE DEGREE OF VISUAL INFLUENCE IN AUDIOVISUAL SPEECH PERCEPTION

Chen, Y., & Hazan, V. L. (2007) · Cited by 21

Multisensory integration processes underlying speech perception as revealed by the McGurk illusion

Marques, L. M., Lapenta, O. M., Costa, T. L., & Boggio, P. S. (2016) · Language Cognition and Neuroscience · doi:10.1080/23273798.2016.1190023 · Cited by 19

Multimodal Speech Perception: A Paradigm for Speech Science

Massaro, D. W. (2002) · Text, speech and language technology · doi:10.1007/978-94-017-2367-1_4 · Cited by 19

Multimodal Second-Language Communication: Research Findings and Pedagogical Implications

Hardison, D. M., & Pennington, M. C. (2020) · RELC Journal · doi:10.1177/0033688220966635 · Cited by 18

Phonology in Context

Pennington, M. C. (2007) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230625396 · Cited by 18

Segmental differences in the visual contribution to speech intelligibility

Nielsen, K. Y. (2004) · The Journal of the Acoustical Society of America · doi:10.1121/1.4784676 · Cited by 18

Auditory and visual information in speech perception: A developmental perspective

Taitelbaum-Swead, R., & Fostick, L. (2016) · Clinical Linguistics & Phonetics · doi:10.3109/02699206.2016.1151938 · Cited by 17

Auditory-visual speech perception development in Japanese and English speakers.

Sekiyama, K., Burnham, D. K., Tam, H., & Erdener, D. (2003) · AVSP · Cited by 17

The visual element in phonological perception and learning

Hardison, D. M. (2007) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230625396_6 · Cited by 16

A few factors which affect the degree of incorporating lip-read information into speech perception

Sekiyama, K., Tohkura, Y., & Umeda, M. (2002) · International Conference on Spoken Language Processing (ICSLP) · doi:10.1109/icslp.1996.607896 · Cited by 13

Speech imagery recalibrates speech-perception boundaries

Scott, M. (2016) · Attention Perception & Psychophysics · doi:10.3758/s13414-016-1087-6 · Cited by 12

Why the FLMP should not be applied to McGurk data ...or how to better compare models in the Bayesian framework.

Schwartz, J. (2003) · AVSP · Cited by 11

The early maximum likelihood estimation model of audiovisual integration in speech perception

Andersen, T. (2015) · The Journal of the Acoustical Society of America · doi:10.1121/1.4916691 · Cited by 10

Segmental differences in the visual contribution to speech inteligibility

Nielsen, K. Y. (2004) · Interspeech (USB) · doi:10.21437/interspeech.2004-435 · Cited by 9

Input modality effects in foreign accent

Markham, D., & Nagano-Madsen, Y. (2002) · International Conference on Spoken Language Processing (ICSLP) · doi:10.1109/icslp.1996.607894 · Cited by 8

The effect of auditory and visual degradation on audiovisual perception of native and non-native speakers

Hazan, V. L., & Li, E. (2008) · Interspeech (USB) · doi:10.21437/interspeech.2008-359 · Cited by 8

O efeito dos estimulos auditivo e visual na percepção dos marcadores prosodicos lexicais e graficos usados na escrita do portugues brasileiro

Pacheco, V. (2006) · doi:10.47749/t/unicamp.2006.391098 · Cited by 8

Processing of audiovisual stimuli in aphasic and non-brain-damaged listeners

de Kok, D., Jonkers, R., & Bastiaanse, R. (2011) · Aphasiology · doi:10.1080/02687038.2011.608840 · Cited by 7

The McGurk Effect: Auditory Visual Speech Perception’s Piltdown Man

Massaro, D. W. (2017) · International Conference on Auditory-Visual Speech Processing (AVSP) · doi:10.21437/avsp.2017-25 · Cited by 6

Overview of audiovisual speech processing

Vatikiotis‐Bateson, E., & Kuratate, T. (2012) · Nippon Onkyo Gakkaishi/Acoustical science and technology/Nihon Onkyo Gakkaishi · doi:10.1250/ast.33.135 · Cited by 6

Perceiving Articulatory Events

Rosenblum, L. D. (2004) · Elsevier eBooks · doi:10.1016/b978-012515851-0/50010-2 · Cited by 6

Primacy of Multimodal Speech Perception for the Brain and Science

Rosenblum, L. D., & Dorsi, J. (2021) · Wiley eBooks · doi:10.1002/9781119184096.ch2 · Cited by 6

Remote Simultaneous Interpreting from Home or Hub: Accuracy of Numbers from English into Mandarin Chinese

Cheung, A. K. F. (2022) · Corpora and intercultural studies · doi:10.1007/978-981-19-6680-4_6 · Cited by 5

Disentangling unisensory from fusion effects in the attentional modulation of McGurk effects: a Bayesian modeling study suggests that fusion is attention-dependent

Schwartz, J., Tiippana, K. M., & Andersen, T. (2010) · HAL · Cited by 5

Comparison of McGurk Effect across Three Consonant-Vowel Combinations in Kannada

Devaraju, D. S., Uppunda, A. K., & Maruthy, S. (2018) · Journal of Audiology & Otology · doi:10.7874/jao.2018.00234 · Cited by 4

English vowel perception by non-native speakers: impact of audio and visual training modalities

Reyes, Y. P., & Hazan, V. L. (2021) · ONOMAZEIN · doi:10.7764/onomazein.51.04 · Cited by 4

The McGurk Effect and the Primacy of Multisensory Perception

Dias, J. W., Cook, T. C., & Rosenblum, L. D. (2017) · doi:10.1093/acprof:oso/9780199794607.003.0115 · Cited by 4

Aero-tactile influence on speech perception of voicing continua.

Goldenberg, D., Tiede, M., & Whalen, D. H. (2015) · ICPhS · Cited by 3

The Mcgurk effect at various auditory signal-to-noise ratios in american and Finnish listeners.

Tiippana, K. M., Hayes, E. A., Möttönen, R., Kraus, N., & Sams, M. E. (2010) · AVSP · Cited by 3

The contribution of the visual modality to vowel perception in native and non-native speakers

Rankin, S. M., & Solé, M.-J. (2024) · Journal of Phonetics · doi:10.1016/j.wocn.2024.101375 · Cited by 2

The Development of Auditory-Visual Speech Perception across Languages and Age

Erdener, D., Sekiyama, K., & Burnham, D. K. (2010) · Cited by 2

Congruent aero-tactile stimuli bias perception of voicing continua

Goldenberg, D., Tiede, M., Bennett, R. T., & Whalen, D. H. (2022) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.879981 · Cited by 2

Effect of native language experience on audio-visual perception of English fricatives by Korean and Mandarin natives.

Wang, Y., Behne, D. M., Jiang, H., & Feehan, A. (2007) · Cited by 2

Multimodal Speech: Two or Three Senses are Better than One

Schwartz, J., Escudier, P., & Teissier, P. (2009) · doi:10.1002/9780470611180.ch11 · Cited by 2

Non-Linguistic Influences on Infants' Nonnative Phoneme Perception: Exaggerated Prosody and Visual Speech Information Improve Discrimination

Ostroff, W. L. (2000) · Cited by 2

A Comparison of Models for Fusion of the Auditory and Visual Sensors in Speech Perception

Robert-Ribes, J., Schwartz, J., & Escudier, P. (1995) · Springer eBooks · doi:10.1007/978-94-009-1639-5_7 · Cited by 2

Mirrors and toothaches: commonplace manipulations of non-auditory feedback availability change perceived speech intelligibility

Casserly, E. D., & Marino, F. (2024) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2024.1462922 · Cited by 2

Perceptual Information Fusion in Humans and Machines

Salah, A. A. (2007) · Cited by 1

視覚と聴覚による音声知覚―言語/文化による差とその発達―

薫, 積. (2011) · 認知科学 = Cognitive studies : bulletin of the Japanese Cognitive Science Society · doi:10.11225/jcss.18.387 · Cited by 1

Using an Avatar, RFID & Speech Recognition for Hands-Free Data Collection

Nemeth, K. J., Arbuckle, N. B., Snead, A. E., Bowers, D. M., Burneka, C., & Quill, L. L. (2008) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120805202406 · Cited by 1

Importance of Visual Support Through Lipreading in the Identification of Words in Spanish Language

Gómez‐Vicente, V., Esquiva, G., Lancho, C., Benzerdjeb, K., Jeréz, A. A., & Ausó, E. (2024) · Language and Speech · doi:10.1177/00238309241270741 · Cited by 2

An Auditory-Visual Conflict of Emotions - Evidence from

Karuppali, S., Bhat, J. S., Krůpa, P., & Shreya, S. (2012) · Advances in life science and technology · Cited by 1

Hearing Lips and Seeing Voices: Illusion and Serendipity in Auditory‐Visual Perception Research

Macdonald, J. B. (2006) · doi:10.1002/9780470777930.ch6 · Cited by 1

The McGurk effect in infants

Johnson, J. A., Rosenblum, L. D., & Schmuckler, M. A. (1995) · The Journal of the Acoustical Society of America · doi:10.1121/1.411551 · Cited by 1

Native and nonnative audio-visual perception of English fricatives in quiet and cafe-noise backgrounds

Wang, Y., Behne, D. M., Jiang, H., & Danyluck, C. M. (2006) · Interspeech (USB) · doi:10.21437/interspeech.2006-295

Audiovisual speech perception in 4-month-old infants

Desjardins, R. N. (2009) · cIRcle (University of British Columbia) · doi:10.14288/1.0088069

An Auditory-Visual Conflict of Emotions - Evidence from

Karuppali, S., Bhat, J. S., & Krůpa, P. (2012)

Cross-linguistic comparisons in the integration of visual and auditory speech

Massaro, D. W., Cohen, M. M., & Smeele, P. M. T. (1995) · Memory & Cognition · doi:10.3758/bf03210561 · Cited by 35

The Relationship between Unimodal and Bimodal Speech Perception of Mandarin Speakers: A Universal Principle

Chen, T. H., Massaro, D. W., & Massaro, D. W. (2007)

Mandarin speech perception by ear and eye follows a universal principle

Chen, T. H., & Massaro, D. W. (2004) · Perception & Psychophysics · doi:10.3758/bf03194976 · Cited by 34

La langue et les lèvres: Cross-language influences on bimodal speech perception.

Werker, J. F., Frost, P. E., & McGuirk, H. (1992) · Canadian Journal of Psychology/Revue Canadienne de Psychologie · doi:10.1037/h0084331 · Cited by 50

Bimodal Speech Perception by Native and Nonnative Speakers of English: Factors Influencing the McGurk Effect

Hardison, D. M. (1999) · Language Learning · doi:10.1111/0023-8333.49.s1.7 · Cited by 32