Reading Fluent Speech from Talking Faces: Typical Brain Networks and Individual Differences
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Listeners are able to extract important linguistic information by viewing the talker's face—a process known as “speechreading.” Previous studies of speechreading present small closed sets of simple words and their results indicate that visual speech processing engages a wide network of brain regions in the temporal, frontal, and parietal lobes that are likely to underlie multiple stages of the receptive language system. The present study further explored this network in a large group of subjects by presenting naturally spoken sentences which tap the richer complexities of visual speech processing. Four different baselines (blank screen, static face, nonlinguistic facial gurning, and auditory speech) enabled us to determine the hierarchy of neural processing involved in speechreading and to test the claim that visual input reliably accesses sound-based representations in the auditory cortex. In contrast to passively viewing a blank screen, the static-face condition evoked activation bilaterally across the border of the fusiform gyrus and cerebellum, and in the medial superior frontal gyrus and left precentral gyrus (p < .05, whole brain corrected). With the static face as baseline, the gurning face evoked bilateral activation in the motion-sensitive region of the occipital cortex, whereas visual speech additionally engaged the middle temporal gyrus, inferior and middle frontal gyri, and the inferior parietal lobe, particularly in the left hemisphere. These latter regions are implicated in lexical stages of spoken language processing. Although auditory speech generated extensive bilateral activation across both superior and middle temporal gyri, the group-averaged pattern of speechreading activation failed to include any auditory regions along the superior temporal gyrus, suggesting that fluent visual speech does not always involve sound-based coding of the visual input. An important finding from the individual subject analyses was that activation in the superior temporal gyrus did reach significance (p < .001, small-volume corrected) for a subset of the group. Moreover, the extent of the left-sided superior temporal gyrus activity was strongly correlated with speech-reading performance. Skilled speechreading was also associated with activations and deactivations in other brain regions, suggesting that individual differences reflect the efficiency of a circuit linking sensory, perceptual, memory, cognitive, and linguistic processes rather than the operation of a single component process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Multimodality image registration by maximization of mutual information
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., & Suetens, P. (1997) · IEEE Transactions on Medical Imaging · doi:10.1109/42.563664 · Cited by 4,547 · cited 1×
“The 64-slice EPI was co-registered to the mean of the realigned images using an algorithm based on mutual information (Maes et al., 1997).”
MethodVisual Contribution to Speech Intelligibility in Noise
Sumby, W. H., & Pollack, I. (1954) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907309 · Cited by 2,583 · cited 2×
“Thus, linguistic facial movements are particularly informative for speech perception when the auditory signal is degraded by noise (Sumby & Pollack, 1954), reverberation, or the distortions introduced by a sensorineural hearing impairment (Jeffers & Barley, 1971).”
BackgroundTowards a neural basis of auditory sentence processing
Friederici, A. D. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01839-8 · Cited by 2,231 · cited 6×
“944 Journal of Cognitive Neuroscience Volume 17, Number 6 temporal gyrus bilaterally, likely to play a role in semantic identification (Friederici, 2002).”
BackgroundInfluentialA Parametric Manipulation of Factors Affecting Task-induced Deactivation in Functional Neuroimaging
McKiernan, K. A., Kaufman, J. N., Kucera-Thompson, J., & Binder, J. R. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321593117 · Cited by 1,080 · cited 1×
“A region of the brain that has consistently been associated with task difficulty is the posterior cingulate cortex, with activity decreasing as the task places a greater load on stimulus discriminability and short-term memory (McKiernan et al., 2003).”
BackgroundCalvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954
Defining a left-lateralized response specific to intelligible speech using fMRI
Narain, C. (2003) · Cerebral Cortex · doi:10.1093/cercor/bhg083 · Cited by 246 · cited 2×
“Does the speechreading of syllable or word lists activate a similar neural network to that of sentences? Certainly, the extensive literature on auditory speech processing indicates that distributed regions in the left hemisphere contribute to semantic processing; with a left temporo-parietal region, that encompasses t…”
BackgroundModulation of activity in temporal cortex during generation of inner speech
Shergill, S. S., Brammer, M. J., Fukuda, R., Bullmore, E., Amaro, E., Murray, R. M., & McGuire, P. K. (2002) · Human Brain Mapping · doi:10.1002/hbm.10046 · Cited by 182
Cited by
Rashid, B., Damaraju, E., Pearlson, G. D., & Calhoun, V. D. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00897 · Cited by 405
Multisensory Processing in Review: from Physiology to Behaviour
Alais, D., Newell, F., & Mamassian, P. (2010) · Seeing and Perceiving · doi:10.1163/187847510x488603 · Cited by 324
Brain regions involved in human movement perception: A quantitative voxel‐based meta‐analysis
Grosbras, M., Beaton, S., & Eickhoff, S. B. (2012) · Human Brain Mapping · doi:10.1002/hbm.21222 · Cited by 284
The processing of audio-visual speech: empirical and neural bases
Campbell, R. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2155 · Cited by 282
A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion
Nath, A. R., & Beauchamp, M. S. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.07.024 · Cited by 280
Metadata
- Authors
- Hall, D. A., Fussell, C., & Summerfield, A. Q.
- Venue
- Journal of Cognitive Neuroscience
- Publisher
- MIT Press - Journals
- Published
- 2005-06-01
- Volume
- 17
- Issue
- 6
- Pages
- 939-953
- ISSN
- 0898-929X (ISSN-L)
- 1530-8898
- OpenAlex
- W2139961014
- Cited by
- 113
- References
- 8
- FWCI
- 2.18
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.