CoChat
Sign Up
Cited by 47Multisensory perception and integrationNeuroscience and Music PerceptionNeurobiology of Language and Bilingualism

Lipreading and Covert Speech Production Similarly Modulate Human Auditory-Cortex Responses to Pure Tones

Open access
Kauramäki, Jaakko, Jääskeläinen, Iiro P., Hari, Riitta, Möttönen, Riikka, Rauschecker, Josef P., Sams, Mikko·2010·The Journal of Neuroscience
DOI 10.1523/jneurosci.1950-09.2010PMC PMC2832801
Open this paper in CoChat — free accountOpen PDF

Abstract

Watching the lips of a speaker enhances speech perception. At the same time, the 100 ms response to speech sounds is suppressed in the observer's auditory cortex. Here, we used whole-scalp 306-channel magnetoencephalography (MEG) to study whether lipreading modulates human auditory processing already at the level of the most elementary sound features, i.e., pure tones. We further envisioned the temporal dynamics of the suppression to tell whether the effect is driven by top-down influences. Nineteen subjects were presented with 50 ms tones spanning six octaves (125–8000 Hz) (1) during “lipreading,” i.e., when they watched video clips of silent articulations of Finnish vowels /a/, /i/, /o/, and /y/, and reacted to vowels presented twice in a row; (2) during a visual control task; (3) during a still-face passive control condition; and (4) in a separate experiment with a subset of nine subjects, during covert production of the same vowels. Auditory-cortex 100 ms responses (N100m) were equally suppressed in the lipreading and covert-speech-production tasks compared with the visual control and baseline tasks; the effects involved all frequencies and were most prominent in the left hemisphere. Responses to tones presented at different times with respect to the onset of the visual articulation showed significantly increased N100m suppression immediately after the articulatory gesture. These findings suggest that the lipreading-related suppression in the auditory cortex is caused by top-down influences, possibly by an efference copy from the speech-production system, generated during both own speech and lipreading.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1523/jneurosci.1950-09.2010

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 61

  1. Besle J Fort A Delpuech C Giard MH Bimodal speech: early suppressive visual effects in human auditory cortex Eur J Neurosci 2004 20 2225 2234 15450102 PMC1885424
  2. Busse L Roberts KC Crist RE Weissman DH Woldorff MG The spread of attention across modalities and space in a multisensory object Proc Natl Acad Sci USA 2005 102 18751 18756 16339900 PMC1317940
  3. Calvert GA Bullmore ET Brammer MJ Campbell R Williams SCR McGuire PK Woodruff PWR Iversen SD David AS Activation of auditory cortex during silent lipreading Science 1997 276 593 596 9110978
  4. Cappe C Barone P Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey Eur J Neurosci 2005 22 2886 2902 16324124
  5. Cappe C Morel A Barone P Rouiller EM The thalamocortical projection systems in primate: An anatomical support for multisensory and sensorimotor interplay Cereb Cortex 2009a 19 2025 2037 19150924 PMC2722423
  6. Cappe C Rouiller E Barone P Multisensory anatomical pathways Hear Res 2009b 258 28 36 19410641
  7. Curio G Neuloh G Numminen J Jousmäki V Hari R Speaking modifies voice-evoked activity in the human auditory cortex Hum Brain Mapp 2000 9 183 191 10770228 PMC6871984
  8. Ghazanfar AA Maier JX Hoffman KL Logothetis NK Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex J Neurosci 2005 25 5004 5012 15901781 PMC6724848
  9. Hackett TA De La Mothe LA Ulbert I Karmos G Smiley J Schroeder CE Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane J Comp Neurol 2007 502 924 952 17444488
  10. Hari R Grandori F Hoke M Romani G The neuromagnetic method in the study of the human auditory cortex Auditory evoked magnetic fields and potentials. Advances in audiology 1990 Vol 6 Basel Karger 222 282
  11. Heinks-Maldonado TH Mathalon DH Gray M Ford JM Fine-tuning of auditory cortex during speech production Psychophysiology 2005 42 180 190 15787855
  12. Heinks-Maldonado TH Nagarajan SS Houde JF Magnetoencephalographic evidence for a precise forward model in speech production NeuroReport 2006 17 1375 1379 16932142 PMC4060597
  13. Hertrich I Mathiak K Lutzenberger W Menning H Ackermann H Sequential audiovisual interactions during speech perception: A whole-head MEG study Neuropsychologia 2007 45 1342 1354 17067640
  14. Hertrich I Mathiak K Lutzenberger W Ackermann H Time course of early audiovisual interactions during speech and nonspeech central auditory processing: A magnetoencephalography study J Cogn Neurosci 2009 21 259 274 18510440
  15. Houde JF Nagarajan SS Sekihara K Merzenich MM Modulation of the auditory cortex during speech: An MEG study J Cogn Neurosci 2002 14 1125 1138 12495520
  16. Hromádka T DeWeese MR Zador AM Sparse representation of sounds in the unanesthetized auditory cortex PLoS Biol 2008 6 e16 18232737 PMC2214813
  17. Jääskeläinen IP Ahveninen J Belliveau JW Raij T Sams M Short-term plasticity in auditory cognition Trends Neurosci 2007 30 653 661 17981345
  18. Jääskeläinen IP Kauramäki J Tujunen J Sams M Formant transition-specific adaptation by lipreading of left auditory cortex N1m NeuroReport 2008 19 93 97 18281900
  19. Jääskeläinen IP Ojanen V Ahveninen J Auranen T Levänen S Möttönen R Tarnanen I Sams M Adaptation of neuromagnetic N1 responses to phonetic stimuli by visual speech in humans NeuroReport 2004 15 2741 2744 15597045
  20. Kaas JH Hackett TA Subdivisions of auditory cortex and processing streams in primates Proc Natl Acad Sci USA 2000 97 11793 11799 11050211 PMC34351
  21. Kayser C Logothetis NK Directed interactions between auditory and superior temporal cortices and their role in sensory integration Front Integr Neurosci 2009 3 7 19503750 PMC2691153
  22. Klucharev V Möttönen R Sams M Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception Cogn Brain Res 2003 18 65 75 14659498
  23. Lakatos P Chen CM O'Connell MN Mills A Schroeder CE Neuronal oscillations and multisensory interaction in primary auditory cortex Neuron 2007 53 279 292 17224408 PMC3717319
  24. Lewis JW Van Essen DC Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey J Comp Neurol 2000 428 112 137 11058227
  25. Loveless N Hari R Hämäläinen M Tiihonen J Evoked responses of human auditory cortex may be enhanced by preceding stimuli Electroencephalogr Clin Neurophysiol 1989 74 217 227 2470578
  26. Lütkenhöner B Krumbholz K Seither-Preisler A Studies of tonotopy based on wave N100 of the auditory evoked field are problematic NeuroImage 2003 19 935 949 12880822
  27. MacSweeney M Amaro E Calvert GA Campbell R David AS McGuire P Williams SC Woll B Brammer MJ Silent speechreading in the absence of scanner noise: an event-related fMRI study NeuroReport 2000 11 1729 1733 10852233
  28. Martikainen MH Kaneko Ki Hari R Suppressed responses to self-triggered sounds in the human auditory cortex Cereb Cortex 2005 15 299 302 15238430
  29. Massaro DW Cohen MM Smeele PMT Perception of asynchronous and conflicting visual and auditory speech J Acoust Soc Am 1996 100 1777 1786 8817903
  30. Miki K Watanabe S Kakigi R Interaction between auditory and visual stimulus relating to the vowel sounds in the auditory cortex in humans: a magnetoencephalographic study Neurosci Lett 2004 357 199 202 15003284
  31. Musacchia G Sams M Nicol T Kraus N Seeing speech affects acoustic information processing in the human brainstem Exp Brain Res 2006 168 1 10 16217645 PMC2535928
  32. Nishitani N Hari R Viewing lip forms: Cortical dynamics Neuron 2002 36 1211 1220 12495633
  33. Nishitani N Schurmann M Amunts K Hari R Broca's region: From action to language Physiology 2005 20 60 69 15653841
  34. Numminen J Curio G Differential effects of overt, covert and replayed speech on vowel-evoked responses of the human auditory cortex Neurosci Lett 1999 272 29 32 10507535
  35. Otazu GH Tai LH Yang Y Zador AM Engaging in an auditory task suppresses responses in auditory cortex Nat Neurosci 2009 12 646 654 19363491 PMC4084972
  36. Pantev C Bertrand O Eulitz C Verkindt C Hampson S Schuierer G Elbert T Specific tonotopic organizations of different areas of the human auditory cortex revealed by simultaneous magnetic and electric recordings Electroencephalogr Clin Neurophysiol 1995 94 26 40 7530637
  37. Papanicolaou AC Raz N Loring DW Eisenberg HM Brain stem evoked response suppression during speech production Brain Lang 1986 27 50 55 3947943
  38. Paus T Perry DW Zatorre RJ Worsley KJ Evans AC Modulation of cerebral blood flow in the human auditory cortex during speech: role of motor-to-sensory discharges Eur J Neurosci 1996 8 2236 2246 8950088
  39. Pekkola J Ojanen V Autti T Jääskeläinen IP Möttönen R Tarkiainen A Sams M Primary auditory cortex activation by visual speech: an fMRI study at 3 T NeuroReport 2005 16 125 128 15671860
  40. Rauschecker JP Tian B Hauser M Processing of complex sounds in the macaque nonprimary auditory cortex Science 1995 268 111 114 7701330
  41. Rizzolatti G Arbib MA Language within our grasp Trends Neurosci 1998 21 188 194 9610880
  42. Rizzolatti G Craighero L The mirror-neuron system Annu Rev Neurosci 2004 27 169 192 15217330
  43. Rockland KS Ojima H Multisensory convergence in calcarine visual areas in macaque monkey Int J Psychophysiol 2003 50 19 26 14511833
  44. Sams M Hari R Rif J Knuutila J The human auditory sensory memory trace persists about 10 sec: Neuromagnetic evidence J Cogn Neurosci 1993 5 363 370 23972223
  45. Sams M Möttönen R Sihvonen T Seeing and hearing others and oneself talk Cogn Brain Res 2005 23 429 435 15820649
  46. Schroeder CE Foxe JJ The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex Cogn Brain Res 2002 14 187 198 12063142
  47. Schroeder CE Lakatos P Kajikawa Y Partan S Puce A Neuronal oscillations and visual amplification of speech Trends Cogn Sci 2008 12 106 113 18280772 PMC3987824
  48. Schroeder CE Smiley J Fu KG McGinnis T O'Connell MN Hackett TA Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing Int J Psychophysiol 2003 50 5 17 14511832
  49. Sherman SM Guillery RW The role of the thalamus in the flow of information to the cortex Phil Trans R Soc B 2002 357 1695 1708 12626004 PMC1693087
  50. Skipper JI van Wassenhove V Nusbaum HC Small SL Hearing lips and seeing voices: How cortical areas supporting speech production mediate audiovisual speech perception Cereb Cortex 2007 17 2387 2399 17218482 PMC2896890
  51. Steinstraeter O Teismann I Wollbrink A Suntrup S Stoeckigt K Dziewas R Pantev C Local sphere-based co-registration for SAM group analysis in subjects without individual MRI Exp Brain Res 2009 193 387 396 19011844
  52. Stekelenburg JJ Vroomen J An event-related potential investigation of the time-course of temporal ventriloquism NeuroReport 2005 16 641 644 15812324
  53. Stekelenburg JJ Vroomen J Neural correlates of multisensory integration of ecologically valid audiovisual events J Cogn Neurosci 2007 19 1964 1973 17892381
  54. Talavage TM Sereno MI Melcher JR Ledden PJ Rosen BR Dale AM Tonotopic organization in human auditory cortex revealed by progressions of frequency sensitivity J Neurophysiol 2004 91 1282 1296 14614108
  55. Valtonen J May P Mäkinen V Tiitinen H Visual short-term memory load affects sensory processing of irrelevant sounds in human auditory cortex Cogn Brain Res 2003 17 358 367 12880906
  56. Wang X Lu T Snider RK Liang L Sustained firing in auditory cortex evoked by preferred stimuli Nature 2005 435 341 346 15902257
  57. van Atteveldt NM Formisano E Blomert L Goebel R The effect of temporal asynchrony on the multisensory integration of letters and speech sounds Cereb Cortex 2007 17 962 974 16751298
  58. van Wassenhove V Grant KW Poeppel D Visual speech speeds up the neural processing of auditory speech Proc Natl Acad Sci USA 2005 102 1181 1186 15647358 PMC545853
  59. van Wassenhove V Grant KW Poeppel D Temporal window of integration in auditory-visual speech perception Neuropsychologia 2007 45 598 607 16530232
  60. Warren RM Riener KR Bashford JA Jr Brubaker BS Spectral redundancy: intelligibility of sentences heard through narrow spectral slits Percept Psychophys 1995 57 175 182 7885815
  61. Woods DL Alho K Algazi A Intermodal selective attention. I. Effects on event-related potentials to lateralized auditory and visual stimuli Electroencephalogr Clin Neurophysiol 1992 82 341 345 1374703

Cited by

Sorted by citations

Authors
Kauramäki, J., Jääskeläinen, I. P., Hari, R., Möttönen, R., Rauschecker, J. P., & Sams, M.
Affiliations
  • University of Oxford (GB)
  • Georgetown University (US)
ORCID
Venue
The Journal of Neuroscience
Publisher
Society for Neuroscience
Published
2010-01-27
Volume
30
Issue
4
Pages
1314-1321
ISSN
  • 0270-6474 (ISSN-L)
  • 1529-2401
DOI
10.1523/jneurosci.1950-09.2010
PMC
PMC2832801
OpenAlex
W2111691469
Cited by
47
References
4
Citations by year
2023: 2 · 2022: 2 · 2020: 1 · 2019: 2 · 2018: 1 · 2017: 4 · 2016: 3 · 2015: 8 · 2014: 7 · 2013: 6 · 2012: 4
FWCI
7.12
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Neuroscience (publishedVersion, cc-by-nc-sa)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • Aaltodoc (Aalto University) (acceptedVersion, other-oa)
  • Aaltodoc (Aalto University) (submittedVersion)
Funders
  • National Science Foundation (0519127)
  • Academy of Finland (213938)
  • Emil Aaltosen Säätiö
  • National Institutes of Health
  • National Eye Institute (R01EY018923)
  • National Institute on Deafness and Other Communication Disorders (R01DC003489, RC1DC010720)
  • National Institute of Neurological Disorders and Stroke (R01NS052494)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
Keywords
lip reading, auditory cortex, N100m, pure tones, top-down modulation, efference copy, speech perception, auditory suppression, temporal dynamics, left hemisphere, vowels
MeSH terms
Acoustic Stimulation, Adult, Auditory Cortex, Auditory Pathways, Auditory Perception, Brain Mapping, Evoked Potentials, Auditory, Female, Humans, Functional Laterality, Lipreading, Male, Nerve Net, Neural Inhibition, Neuropsychological Tests, Perceptual Masking, Photic Stimulation, Pitch Discrimination, Reaction Time, Speech Acoustics, Speech Perception, Visual Perception, Magnetoencephalography, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review