The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Integration of Visual and Auditory Information by Superior Temporal Sulcus Neurons Responsive to the Sight of Actions
Abstract
Processing of complex visual stimuli comprising facial movements, hand actions, and body movements is known to occur in the superior temporal sulcus (STS) of humans and nonhuman primates. The STS is also thought to play a role in the integration of multimodal sensory input. We investigated whether STS neurons coding the sight of actions also integrated the sound of those actions. For 23% of neurons responsive to the sight of an action, the sound of that action significantly modulated the visual response. The sound of the action increased or decreased the visually evoked response for an equal number of neurons. In the neurons whose visual response was increased by the addition of sound (but not those neurons whose responses were decreased), the audiovisual integration was dependent upon the sound of the action matching the sight of the action. These results suggest that neurons in the STS form multisensory representations of observed actions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Action recognition in the premotor cortex
Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996) · Brain · doi:10.1093/brain/119.2.593 · Cited by 4,968
Understanding motor events: a neurophysiological study
di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., & Rizzolatti, G. (1992) · Experimental Brain Research · doi:10.1007/bf00230027 · Cited by 3,539
Cortical mechanisms of human imitation
Iacoboni, M., Woods, R. P., Brass, M., Bekkering, H., Mazziotta, J. C., & Rizzolatti, G. (1999) · Science · doi:10.1126/science.286.5449.2526 · Cited by 2,683 · cited 1×
“Understanding actions could be based upon a matching of sensory input to the motor representation of the same action within premotor cortex (Bekkering & Wohlschlaeger, 2002; Prinz, 2002; Viviani, 2002; Iacoboni et al., 1999).”
BackgroundA cortical area selective for visual processing of the human body
Downing, P. E., Jiang, Y., Shuman, M., & Kanwisher, N. (2001) · Science · doi:10.1126/science.1063414 · Cited by 2,043
(1994) · Choice Reviews Online · doi:10.5860/choice.31-4365 · Cited by 1,882
Hearing Sounds, Understanding Actions: Action Representation in Mirror Neurons
Kohler, E., Keysers, C., Umiltà, M. A., Fogassi, L., Gallese, V., & Rizzolatti, G. (2002) · Science · doi:10.1126/science.1070311 · Cited by 1,801 · cited 2×
“Matching of motor and sensory representations in the premotor cortex extends to auditory signals as well as visual in- put (Kohler et al., 2002).”
BackgroundVisual properties of neurons in a polysensory area in superior temporal sulcus of the macaque.
Bruce, C., Desimone, R., & Gross, C. G. (1981) · Journal of Neurophysiology · doi:10.1152/jn.1981.46.2.369 · Cited by 1,227
Meredith, M. A., & Stein, B. E. (1986) · Journal of Neurophysiology · doi:10.1152/jn.1986.56.3.640 · Cited by 1,183
Temporal Cortex Activation in Humans Viewing Eye and Mouth Movements
Puce, A., Allison, T., Bentin, S., Gore, J. C., & McCarthy, G. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-06-02188.1998 · Cited by 1,077
Giard, M. H., & Peronnet, F. (1999) · Journal of Cognitive Neuroscience · doi:10.1162/089892999563544 · Cited by 1,072
Crossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies
Calvert, G. A. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1110 · Cited by 1,012
Calvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954
Cortical connections of visual area MT in the macaque
Ungerleider, L. G., & Desimone, R. (1986) · Journal of Comparative Neurology · doi:10.1002/cne.902480204 · Cited by 953
Understanding the recognition of facial identity and facial expression
Calder, A. J., & Young, A. W. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1724 · Cited by 948
Processing of Complex Sounds in the Macaque Nonprimary Auditory Cortex
Rauschecker, J. P., Tian, B., & Hauser, M. (1995) · Science · doi:10.1126/science.7701330 · Cited by 929
Brain Areas Involved in Perception of Biological Motion
Grossman, E., Donnelly, M., Price, R., Pickens, D., Morgan, V., Neighbor, G., & Blake, R. (2000) · Journal of Cognitive Neuroscience · doi:10.1162/089892900562417 · Cited by 915
Interactions Among Converging Sensory Inputs in the Superior Colliculus
Meredith, M. A., & Stein, B. E. (1983) · Science · doi:10.1126/science.6867718 · Cited by 841
Seltzer, B., & Pandya, D. N. (1978) · Brain Research · doi:10.1016/0006-8993(78)90584-x · Cited by 782
Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus
Beauchamp, M. S., Lee, K. E., Argall, B. D., & Martin, A. (2004) · Neuron · doi:10.1016/s0896-6273(04)00070-4 · Cited by 773
Global and fine information coded by single neurons in the temporal visual cortex
Sugase, Y., Yamane, S., Ueno, S., & Kawano, K. (1999) · Nature · doi:10.1038/23703 · Cited by 760
Empathy and the somatotopic auditory mirror system in humans
Gazzola, V., Aziz-Zadeh, L., & Keysers, C. (2006) · Current Biology · doi:10.1016/j.cub.2006.07.072 · Cited by 752
Ferrari, P. F., Gallese, V., Rizzolatti, G., & Fogassi, L. (2003) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2003.02601.x · Cited by 728
Electrophysiology and brain imaging of biological motion
Puce, A., & Perrett, D. (2003) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.2002.1221 · Cited by 693
Self-citationHasselmo, M. E., Rolls, E. T., & Baylis, G. C. (1989) · Behavioural Brain Research · doi:10.1016/s0166-4328(89)80054-3 · Cited by 657
Temporal dynamics of cortical representation for action
Nishitani, N., & Hari, R. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.2.913 · Cited by 625 · cited 3×
“Neural signals were recorded to hard disk for off-line filtering and analysis.”
BackgroundMethodFrameworks of analysis for the neural representation of animate objects and actions
Perrett, D. I., Harries, M. H., Bevan, R., Thomas, S., Benson, P. J., Mistlin, A. J., Chitty, A. J., Hietanen, J. K., & Ortega, J. E. (1989) · Journal of Experimental Biology · doi:10.1242/jeb.146.1.87 · Cited by 616
Self-citationMultiple visual areas in the caudal superior temporal sulcus of the macaque
Desimone, R., & Ungerleider, L. G. (1986) · Journal of Comparative Neurology · doi:10.1002/cne.902480203 · Cited by 590
The posterior thalamic region and its cortical projection in new world and old world monkeys
Burton, H., & Jones, E. G. (1976) · Journal of Comparative Neurology · doi:10.1002/cne.901680204 · Cited by 555 · cited 3×
“The STS in the monkey has substantial reciprocal connections with the superior colliculus by way of the pulvinar (Lui, Gregory, Blanks, & Giolli, 1995; Burton & Jones, 1976; Benevento & Fallon, 1975).”
BackgroundMethodVisual areas in the temporal cortex of the macaque
Desimone, R., & Gross, C. G. (1979) · Brain Research · doi:10.1016/0006-8993(79)90699-1 · Cited by 550
The ascending projections of the superior colliculus in the rhesus monkey (Macaca mulatta)
Benevento, L. A., & Fallon, J. H. (1975) · Journal of Comparative Neurology · doi:10.1002/cne.901600306 · Cited by 458
Functional subdivisions of the temporal lobe neocortex
Baylis, G., Rolls, E., & Leonard, C. (1987) · The Journal of Neuroscience · doi:10.1523/jneurosci.07-02-00330.1987 · Cited by 453
The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex
Schroeder, C. E., & Foxe, J. J. (2002) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00073-3 · Cited by 416
Representation and integration of multiple sensory inputs in primate superior colliculus
Wallace, M. T., Wilkinson, L. K., & Stein, B. E. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.76.2.1246 · Cited by 412
Perrett, D. I., Smith, P., Potter, D. D., Mistlin, A. J., Head, A., Milner, A. D., & Jeeves, M. A. (1984) · PubMed · Cited by 404
Self-citationSpatial factors determine the activity of multisensory neurons in cat superior colliculus
Alex Meredith, M., & Stein, B. E. (1986) · Brain Research · doi:10.1016/0006-8993(86)91648-3 · Cited by 388
Responses of Anterior Superior Temporal Polysensory (STPa) Neurons to “Biological Motion” Stimuli
Oram, M. W., & Perrett, D. I. (1994) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1994.6.2.99 · Cited by 383
Self-citationTowards a unifying neural theory of social cognition
Keysers, C., & Gazzola, V. (2006) · Progress in Brain Research · doi:10.1016/s0079-6123(06)56021-2 · Cited by 367
Further observations on parieto-temporal connections in the rhesus monkey
Seltzer, B., & Pandya, D. N. (1984) · Experimental Brain Research · doi:10.1007/bf00237280 · Cited by 357
Oram, M. W., & Perrett, D. I. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.76.1.109 · Cited by 345
Self-citationAuditory receptive fields in primate superior colliculus shift with changes in eye position
Jay, M. F., & Sparks, D. L. (1984) · Nature · doi:10.1038/309345a0 · Cited by 340
Perrett, D. I., Smith, P. A. J., Mistlin, A. J., Chitty, A. J., Head, A. S., Potter, D. D., Broennimann, R., Milner, A. D., & Jeeves, M. A. (1985) · Behavioural Brain Research · doi:10.1016/0166-4328(85)90089-0 · Cited by 332
Self-citationSeltzer, B., & Pandya, D. N. (1994) · Journal of Comparative Neurology · doi:10.1002/cne.903430308 · Cited by 327
Cortical connections of inferior temporal area TEO in macaque monkeys
Distler, C., Boussaoud, D., Desimone, R., & Ungerleider, L. G. (1993) · Journal of Comparative Neurology · doi:10.1002/cne.903340111 · Cited by 322
Spatial determinants of multisensory integration in cat superior colliculus neurons
Meredith, M. A., & Stein, B. E. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.75.5.1843 · Cited by 322
Time course of neural responses discriminating different views of the face and head
Oram, M. W., & Perrett, D. I. (1992) · Journal of Neurophysiology · doi:10.1152/jn.1992.68.1.70 · Cited by 315
Self-citationEnhancement of selective listening by illusory mislocation of speech sounds due to lip-reading
Driver, J. (1996) · Nature · doi:10.1038/381066a0 · Cited by 315
Hikosaka, K., Iwai, E., Saito, H., & Tanaka, K. (1988) · Journal of Neurophysiology · doi:10.1152/jn.1988.60.5.1615 · Cited by 296
Benevento, L. A., Fallon, J., Davis, B. J., & Rezak, M. (1977) · Experimental Neurology · doi:10.1016/0014-4886(77)90112-1 · Cited by 288
Integration of multiple sensory modalities in cat cortex
Wallace, M., Meredith, M. A., & Stein, B. (1992) · Experimental Brain Research · doi:10.1007/bf00227844 · Cited by 235
Neural correlates of cross-modal binding
Bushara, K. O., Hanakawa, T., Immisch, I., Toma, K., Kansaku, K., & Hallett, M. (2003) · Nature Neuroscience · doi:10.1038/nn993 · Cited by 226
Facial expressions linked to monkey calls
Ghazanfar, A. A., & Logothetis, N. K. (2003) · Nature · doi:10.1038/423937a · Cited by 216
Two Cortical Areas Mediate Multisensory Integration in Superior Colliculus Neurons
Jiang, W., Wallace, M. T., Jiang, H., Vaughan, J. W., & Stein, B. E. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.85.2.506 · Cited by 206
Fort, A. (2002) · Cerebral Cortex · doi:10.1093/cercor/12.10.1031 · Cited by 181
Chapter 3 Beyond the face: exploring rapid influences of context on face processing
de Gelder, B., Meeren, H. K. M., Righart, R., den Stock, J. V., van de Riet, W. A. C., & Tamietto, M. (2006) · Progress in brain research · doi:10.1016/s0079-6123(06)55003-4 · Cited by 180
Intrinsic connections and architectonics of the superior temporal sulcus in the rhesus monkey
Seltzer, B., & Pandya, D. N. (1989) · Journal of Comparative Neurology · doi:10.1002/cne.902900402 · Cited by 177
Voice processing in human and non-human primates
Belin, P. (2006) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2006.1933 · Cited by 166
The Role of Vision in the Perception of Speech
Dodd, B. J. (1977) · Perception · doi:10.1068/p060031 · Cited by 155
Mechanisms of Within- and Cross-Modality Suppression in the Superior Colliculus
Kadunce, D. C., Vaughan, J. W., Wallace, M. T., Benedek, G., & Stein, B. E. (1997) · Journal of Neurophysiology · doi:10.1152/jn.1997.78.6.2834 · Cited by 149
Meaningful interactions can enhance visual discrimination of human agents
Neri, P., Luu, J. Y., & Levi, D. M. (2006) · Nature Neuroscience · doi:10.1038/nn1759 · Cited by 144
Saleem, K. S., Suzuki, W., Tanaka, K., & Hashikawa, T. (2000) · Journal of Neuroscience · doi:10.1523/jneurosci.20-13-05083.2000 · Cited by 143
Two Corticotectal Areas Facilitate Multisensory Orientation Behavior
Jiang, W., Jiang, H., & Stein, B. E. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902760807230 · Cited by 142
Colombo, M., & Gross, C. G. (1994) · Behavioral Neuroscience · doi:10.1037/0735-7044.108.3.443 · Cited by 123
Sensory Modality Specificity of Neural Activity Related to Memory in Visual Cortex
Gibson, J. R., & Maunsell, J. H. R. (1997) · Journal of Neurophysiology · doi:10.1152/jn.1997.78.3.1263 · Cited by 121
Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects
Stevenson, R. A., Geoghegan, M. L., & James, T. W. (2006) · Experimental Brain Research · doi:10.1007/s00221-006-0770-6 · Cited by 119
Bruce, C. J., Desimone, R., & Gross, C. G. (1986) · Journal of Neurophysiology · doi:10.1152/jn.1986.55.5.1057 · Cited by 114
Vision influences tactile perception at body sites that cannot be viewed directly
Tipper, S. P., Phillips, N., Dancer, C., Lloyd, D. M., Howard, L. M., & McGlone, F. P. (2001) · Experimental Brain Research · doi:10.1007/s002210100743 · Cited by 114
Superior colliculus lesions preferentially disrupt multisensory orientation
Burnett, L., Stein, B. E., Chaponis, D. M., & Wallace, M. T. (2004) · Neuroscience · doi:10.1016/j.neuroscience.2003.12.026 · Cited by 89
Color Sensitivity of Cells Responsive to Complex Stimuli in the Temporal Cortex
Edwards, R., Xiao, D., Keysers, C., Földiák, P., & Perrett, D. I. (2003) · Journal of Neurophysiology · doi:10.1152/jn.00524.2002 · Cited by 80
Self-citationBarraclough, N. E., Xiao, D., Oram, M. W., & Perrett, D. I. (2006) · Progress in brain research · doi:10.1016/s0079-6123(06)54007-5 · Cited by 78
Self-citationLui, F., Gregory, K. M., Blanks, R. H. I., & Giolli, R. A. (1995) · The Journal of Comparative Neurology · doi:10.1002/cne.903630308 · Cited by 74
Cortex Controls Multisensory Depression in Superior Colliculus
Jiang, W., & Stein, B. E. (2003) · Journal of Neurophysiology · doi:10.1152/jn.00369.2003 · Cited by 71
Rapid serial visual presentation for the determination of n eural selectivity in area STSa
Földiák, P., Xiao, D., Keysers, C., Edwards, R., & Perrett, D. I. (2004) · Progress in brain research · doi:10.1016/s0079-6123(03)14407-x · Cited by 58
Self-citationOram, M. W., Perrett, D. I., & Hietanen, J. K. (1993) · Experimental Brain Research · doi:10.1007/bf00228696 · Cited by 53
Self-citationBell, A. H., Corneil, B. D., Munoz, D. P., & Meredith, M. A. (2003) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2003.02976.x · Cited by 45
Action perception and imitation: a tutorial
Bekkering, H., & Wohlschläger, A. (2002) · Oxford University Press eBooks · doi:10.1093/oso/9780198510697.003.0015 · Cited by 24
Cited by
Multisensory integration: current issues from the perspective of the single neuron
Stein, B. E., & Stanford, T. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2331 · Cited by 1,598
Is neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463
Understanding others' actions and goals by mirror and mentalizing systems: a meta-analysis
Van Overwalle, F., & Baetens, K. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.06.009 · Cited by 1,329
“The aIPS and PMC receive input from the superior temporal sulcus (STS), which contains polysensory neurons that respond to motion from different perceptual modalities (Barraclough et al., 2005).”
Understanding the recognition of facial identity and facial expression
Calder, A. J., & Young, A. W. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1724 · Cited by 948
“There is evidence that STS cells in macaques are sensitive to both visual and auditory components of animated biologically relevant stimuli, including facial signal”
BackgroundDriver, J., & Noesselt, T. (2008) · Neuron · doi:10.1016/j.neuron.2007.12.013 · Cited by 940
Metadata
- Authors
- Barraclough*, N. E., Xiao*, D., Baker, C. I., Oram, M. W., & Perrett, D. I.
- Venue
- Journal of Cognitive Neuroscience
- Publisher
- MIT Press - Journals
- Published
- 2005-03-01
- Volume
- 17
- Issue
- 3
- Pages
- 377-391
- ISSN
- 0898-929X (ISSN-L)
- 1530-8898
- OpenAlex
- W2166941808
- Cited by
- 332
- References
- 76
- FWCI
- 26.24
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.