Evaluation and integration of visual and auditory information in speech perception.

Abstract

Three experiments were carried out to investigate the evaluation and integration of visual and auditory information in speech perception. In the first two experiments, subjects identified /ba/ or /da/ speech events consisting of high-quality synthetic syllables ranging from /ba/ to /da/ combined with a videotaped /ba/ or /da/ or neutral articulation. Although subjects were specifically instructed to report what they heard, visual articulation made a large contribution to identification. The tests of quantitative models provide evidence for the integration of continuous and independent, as opposed to discrete or nonindependent, sources of information. The reaction times for identification were primarily correlated with the perceived ambiguity of the speech event. In a third experiment, the speech events were identified with an unconstrained set of response alternatives. In addition to /ba/ and /da/ responses, the /bda/ and /tha/ responses were well described by a combination of continuous and independent features. This body of results provides strong evidence for a fuzzy logical model of perceptual recognition.

Cites (3)

1. Hearing lips and seeing voices
   MCGURK, HARRY, MACDONALD, JOHN · 1976 · Nature
2. The discovery of processing stages: Extensions of Donders' method
   Sternberg, Saul · 1969 · Acta Psychologica
3. Speech perception by the chinchilla: Identification functions for synthetic VOT stimuli
   Kuhl, Patricia K., Miller, James D. · 1978 · The Journal of the Acoustical Society of America

Show all 3 references

Cited by (12)

1. Speech listening specifically modulates the excitability of tongue muscles: a TMS study
   Fadiga, Luciano, Craighero, Laila, Buccino, Giovanni, Rizzolatti, Giacomo · 2002 · European Journal of Neuroscience
2. Shortlist B: A Bayesian model of continuous speech recognition.
   Norris, Dennis, McQueen, James M. · 2008 · Psychological Review
3. Musicians have enhanced subcortical auditory and audiovisual processing of speech and music
   Musacchia, Gabriella, Sams, Mikko, Skoe, Erika, Kraus, Nina · 2007 · Proceedings of the National Academy of Sciences
4. Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning
   Kasabov, N. · 2001 · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics)
5. Categorical perception
   Goldstone, Robert L., Hendrickson, Andrew T. · 2010 · WIREs Cognitive Science

See all 199