Multiple Object Response Normalization in Monkey Inferotemporal Cortex
Open accessThe DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
The highest stages of the visual ventral pathway are commonly assumed to provide robust representation of object identity by disregarding confounding factors such as object position, size, illumination, and the presence of other objects (clutter). However, whereas neuronal responses in monkey inferotemporal cortex (IT) can show robust tolerance to position and size changes, previous work shows that responses to preferred objects are usually reduced by the presence of nonpreferred objects. More broadly, we do not yet understand multiple object representation in IT. In this study, we systematically examined IT responses to pairs and triplets of objects in three passively viewing monkeys across a broad range of object effectiveness. We found that, at least under these limited clutter conditions, a large fraction of the response of each IT neuron to multiple objects is reliably predicted as the average of its responses to the constituent objects in isolation. That is, multiple object responses depend primarily on the relative effectiveness of the constituent objects, regardless of object identity. This average effect becomes virtually perfect when populations of IT neurons are pooled. Furthermore, the average effect cannot simply be explained by attentional shifts but behaves as a primarily feedforward response property. Together, our observations are most consistent with mechanistic models in which IT neuronal outputs are normalized by summed synaptic drive into IT or spiking activity within IT and suggest that normalization mechanisms previously revealed at earlier visual areas are operating throughout the ventral visual stream.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Neural Mechanisms of Selective Visual Attention
Desimone, R., & Duncan, J. (1995) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.18.030195.001205 · Cited by 8,388 · cited 1×
“Thus, although previous studies have investigated how attention modulates processing of targets in the presence of distracters (Moran and Desimone, 1985; Maunsell, 1995; Connor et al., 1997; Chelazzi et al., 1998), here we seek to understand the " core " feedforward processing of multiple objects.”
BackgroundDistributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311556 · Cited by 4,710
Preface: Cerebral Cortex Has Come of Age
Goldman-Rakic, P. S., & Rakic, P. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1-a · Cited by 3,393
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309 · cited 2×
“Moreover , although some contemporary models of the ventral stream use MAX operations (Riesenhuber and Poggio, 1999a), even these models do not predict complete clutter tolerance in IT (Riesenhuber and Poggio, 1999b).”
BackgroundMethodMathematical Statistics and Data Analysis
Rabinowitz, L. (1989) · Technometrics · doi:10.1080/00401706.1989.10488573 · Cited by 3,116
Spikes: Exploring the Neural Code
Rieke, F. M., Warland, D., van Steveninck, R. R. D. R., & Bialek, W. (1996) · Medical Entomology and Zoology · Cited by 2,844
Selective Attention Gates Visual Processing in the Extrastriate Cortex
Moran, J., & Desimone, R. (1985) · Science · doi:10.1126/science.4023713 · Cited by 2,433 · cited 5×
“This does not rule out all possible explanations of the average effect that involve attention or imply that attention plays no role in IT responses to multiple objects [it can play a role under some conditions (Moran and Desimone, 1985; Chelazzi et al., 1998)].”
BackgroundInfluentialShadlen, M. N., & Newsome, W. T. (1998) · The Journal of Neuroscience · doi:10.1523/jneurosci.18-10-03870.1998 · Cited by 2,178 · cited 1×
“, 1997 ; Shadlen and Newsome , 1998 ) ; ( 2 ) markedly different than the value predicted by the attention hypothesis ( 1 .”
BackgroundNormalization of cell responses in cat striate cortex
Heeger, D. J. (1992) · Visual Neuroscience · doi:10.1017/s0952523800009640 · Cited by 1,835 · cited 3×
“This finding suggests that divisive normalization mechanisms analogous to those proposed to explain response rescaling in early visual stages (Heeger, 1992; Desimone and Duncan, 1995; Heeger et al., 1996; Carandini et al., 1997; Reynolds et al., 1999) and area MT (Recanzone et al.”
BackgroundMethodSoftky, W., & Koch, C. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-01-00334.1993 · Cited by 1,689 · cited 2×
“0 predicted by Poisson spiking ( Softky and Koch , 1993 ; Shadlen and Newsome , 1994 ; Rieke et al .”
BackgroundInferotemporal cortex and object vision
Tanaka, K. (1996) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.19.030196.000545 · Cited by 1,402 · cited 2×
“Electrophysiological studies show that IT neurons can be selective for complex objects while also being tolerant to some transformations (object position, scale, and pose) (for review, see Logothetis and Sheinberg , 1996; Tanaka, 1996).”
BackgroundLogothetis, N. K., & Sheinberg, D. L. (1996) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.19.030196.003045 · Cited by 1,351 · cited 1×
“Electrophysiological studies show that IT neurons can be selective for complex objects while also being tolerant to some transformations (object position, scale, and pose) (for review, see Logothetis and Sheinberg , 1996; Tanaka, 1996).”
BackgroundNoise, neural codes and cortical organization
Shadlen, M. N., & Newsome, W. T. (1994) · Current Opinion in Neurobiology · doi:10.1016/0959-4388(94)90059-0 · Cited by 1,252 · cited 2×
“0 predicted by Poisson spiking ( Softky and Koch , 1993 ; Shadlen and Newsome , 1994 ; Rieke et al .”
BackgroundCompetitive Mechanisms Subserve Attention in Macaque Areas V2 and V4
Reynolds, J. H., Chelazzi, L., & Desimone, R. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-05-01736.1999 · Cited by 1,236 · cited 1×
“Although some progress on understanding such processing has been made in area V4 (Reynolds et al., 1999; Gawne and Received May 20, 2005; revised July 22, 2005; accepted July 23, 2005.”
BackgroundAttentional modulation of visual motion processing in cortical areas MT and MST
Treue, S., & Maunsell, J. H. R. (1996) · Nature · doi:10.1038/382539a0 · Cited by 1,025 · cited 1×
“, 1999 ) or a specific target object ( Moran and Desimone , 1985 ; Treue and Maunsell , 1996 ; Chelazzi et al .”
BackgroundShort-term conceptual memory for pictures.
Potter, M. C. (1976) · Journal of Experimental Psychology: Human Learning & Memory · doi:10.1037//0278-7393.2.5.509 · Cited by 1,008
Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex
Carandini, M., Heeger, D. J., & Movshon, J. A. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-21-08621.1997 · Cited by 922 · cited 2×
“This finding suggests that divisive normalization mechanisms analogous to those proposed to explain response rescaling in early visual stages (Heeger, 1992; Desimone and Duncan, 1995; Heeger et al., 1996; Carandini et al., 1997; Reynolds et al., 1999) and area MT (Recanzone et al.”
BackgroundMethodNatural signal statistics and sensory gain control
Schwartz, O., & Simoncelli, E. P. (2001) · Nature Neuroscience · doi:10.1038/90526 · Cited by 835 · cited 2×
“, 1997 ; Schwartz and Simoncelli , 2001 ; Cavanaugh et al .”
BackgroundMethodNature and Interaction of Signals From the Receptive Field Center and Surround in Macaque V1 Neurons
Cavanaugh, J. R., Bair, W., & Movshon, J. A. (2002) · Journal of Neurophysiology · doi:10.1152/jn.00692.2001 · Cited by 778 · cited 1×
“The average effect could also arise if the output of each IT neuron was normalized by the total spiking activity of a broad population of IT cells, similar to divisive normalization models proposed to explain nonlinear behavior in early visual areas (Heeger, 1992; Heeger et al., 1996; Carandini et al., 1997; Schwartz…”
MethodResponses of Neurons in Inferior Temporal Cortex During Memory-Guided Visual Search
Chelazzi, L., Duncan, J., Miller, E. K., & Desimone, R. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.6.2918 · Cited by 726
Spatial Attention Effects in Macaque Area V4
Connor, C. E., Preddie, D. C., Gallant, J. L., & Van Essen, D. C. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-09-03201.1997 · Cited by 448 · cited 1×
“Thus, although previous studies have investigated how attention modulates processing of targets in the presence of distracters (Moran and Desimone, 1985; Maunsell, 1995; Connor et al., 1997; Chelazzi et al., 1998), here we seek to understand the " core " feedforward processing of multiple objects.”
BackgroundInteracting Roles of Attention and Visual Salience in V4
Reynolds, J. H., & Desimone, R. (2003) · Neuron · doi:10.1016/s0896-6273(03)00097-7 · Cited by 402
Generalization in vision and motor control
Poggio, T., & Bizzi, E. (2004) · Nature · doi:10.1038/nature03014 · Cited by 386
Keysers, C., Xiao, D.-K., Földiák, P., & Perrett, D. I. (2001) · Journal of Cognitive Neuroscience · doi:10.1162/089892901564199 · Cited by 335 · cited 1×
“At this presentation rate, IT neurons show robust object selectivity (Keysers et al., 2001), and this rate is consistent with that produced spontaneously by free-viewing monkey (DiCarlo and Maunsell, 2000).”
ResultThe Brain's Visual World: Representation of Visual Targets in Cerebral Cortex
Maunsell, J. H. R. (1995) · Science · doi:10.1126/science.270.5237.764 · Cited by 314 · cited 1×
“Thus, although previous studies have investigated how attention modulates processing of targets in the presence of distracters (Moran and Desimone, 1985; Maunsell, 1995; Connor et al., 1997; Chelazzi et al., 1998), here we seek to understand the " core " feedforward processing of multiple objects.”
BackgroundComputational models of cortical visual processing.
Heeger, D. J., Simoncelli, E. P., & Movshon, J. A. (1996) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.93.2.623 · Cited by 280 · cited 2×
“This finding suggests that divisive normalization mechanisms analogous to those proposed to explain response rescaling in early visual stages (Heeger, 1992; Desimone and Duncan, 1995; Heeger et al., 1996; Carandini et al., 1997; Reynolds et al., 1999) and area MT (Recanzone et al.”
BackgroundMethodSpatial sensitivity of macaque inferior temporal neurons
Op De Beeck, H., & Vogels, R. (2000) · The Journal of Comparative Neurology · doi:10.1002/1096-9861(20001030)426:4<505::aid-cne1>3.0.co;2-m · Cited by 278 · cited 3×
“…position, have a broad range of sizes (Op De Beeck and Vogels, 2000), and can often be small relative to the separation of our objects (Op De Beeck and Vogels, 2000; DiCarlo and Maunsell, 2003), we used these RF variations to ask whether there was any relationship between the averaging…”
BackgroundMethodInfluentialThe guidance of eye movements during active visual search
Motter, B. C., & Belky, E. J. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00349-0 · Cited by 250
Sheinberg, D. L., & Logothetis, N. K. (2001) · The Journal of Neuroscience · doi:10.1523/jneurosci.21-04-01340.2001 · Cited by 242 · cited 2×
“However, this idealized notion of IT is inconsistent with available data showing that IT responses are altered in cluttered scenes (Sheinberg and Logothetis, 2001; Rolls et al., 2003), and responses to object pairs are typically weaker than responses to isolated, preferred objects (Sato, 1989;…”
BackgroundMiller, E. K., Gochin, P. M., & Gross, C. G. (1993) · Brain Research · doi:10.1016/0006-8993(93)90187-r · Cited by 229 · cited 1×
“…IT responses are altered in cluttered scenes (Sheinberg and Logothetis, 2001; Rolls et al., 2003), and responses to object pairs are typically weaker than responses to isolated, preferred objects (Sato, 1989; Miller et al., 1993; Rolls and Tovee, 1995; Chelazzi et al., 1998; Missal et al., 1999).”
Background“Rapid Categorization of Natural Images by Rhesus Monkeys”
Fabre-Thorpe, M., Richard, G., & Thorpe, S. J. (1998) · NeuroReport · doi:10.1097/00001756-199801260-00023 · Cited by 217 · cited 1×
“2) that is constrained by IT latencies and behavioral reaction times to be most relevant for recognition tasks (Fabre-Thorpe et al., 1998; DiCarlo and Maunsell, 2000).”
BackgroundReading without saccadic eye movements
Rubin, G. S., & Turano, K. (1992) · Vision Research · doi:10.1016/0042-6989(92)90032-e · Cited by 217 · cited 1×
“…clutter tolerance even for brief presentation conditions (e.g., ϳ100 ms) without explicit attentional instruction (Potter, 1976; Intraub, 1980; Rubin and Turano, 1992), this suggests that top-down attention is not strictly required for robust recognition in clutter but that powerful,…”
BackgroundDiCarlo, J. J., & Maunsell, J. H. R. (2003) · Journal of Neurophysiology · doi:10.1152/jn.00358.2002 · Cited by 207 · cited 2×
“However, because IT neuronal RFs are not all centered at the same retinal position, have a broad range of sizes (Op De Beeck and Vogels, 2000), and can often be small relative to the separation of our objects (Op De Beeck and Vogels, 2000; DiCarlo and Maunsell, 2003), we used these RF variations to ask whether there w…”
MethodSelf-citationSpatial Summation in the Receptive Fields of MT Neurons
Britten, K. H., & Heuer, H. W. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-12-05074.1999 · Cited by 200 · cited 4×
“, 1997 ; Britten and Heuer , 1999 ; Heuer and Britten , 2002 ) .”
BackgroundInfluentialHow parallel is visual processing in the ventral pathway?
Rousselet, G. A., Thorpe, S. J., & Fabre-Thorpe, M. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.06.003 · Cited by 197 · cited 1×
“In this context, some have suggested that IT neurons should ideally be tolerant to visual clutter, i.e., their response to an effective object should be essentially unaffected by the presence of other, less-effective objects (Rousselet et al., 2003, 2004).”
BackgroundAre Cortical Models Really Bound by the “Binding Problem”?
Riesenhuber, M., & Poggio, T. (1999) · Neuron · doi:10.1016/s0896-6273(00)80824-7 · Cited by 191 · cited 2×
“Moreover , although some contemporary models of the ventral stream use MAX operations (Riesenhuber and Poggio, 1999a), even these models do not predict complete clutter tolerance in IT (Riesenhuber and Poggio, 1999b).”
BackgroundMethodThe Receptive Fields of Inferior Temporal Cortex Neurons in Natural Scenes
Rolls, E. T., Aggelopoulos, N. C., & Zheng, F. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-01-00339.2003 · Cited by 179 · cited 1×
“…notion of IT is inconsistent with available data showing that IT responses are altered in cluttered scenes (Sheinberg and Logothetis, 2001; Rolls et al., 2003), and responses to object pairs are typically weaker than responses to isolated, preferred objects (Sato, 1989; Miller et al.,…”
BackgroundPresentation rate and the representation of briefly glimpsed pictures in memory.
Intraub, H. (1980) · Journal of Experimental Psychology: Human Learning & Memory · doi:10.1037//0278-7393.6.1.1 · Cited by 176
Responses of MT and MST Neurons to One and Two Moving Objects in the Receptive Field
Recanzone, G. H., Wurtz, R. H., & Schwarz, U. (1997) · Journal of Neurophysiology · doi:10.1152/jn.1997.78.6.2904 · Cited by 171 · cited 1×
“, 1999) and area MT (Recanzone et al., 1997; Britten and Heuer, 1999) may be at work in IT.”
BackgroundRolls, E., & Tovee, M. (1995) · Experimental Brain Research · doi:10.1007/bf00241500 · Cited by 153 · cited 3×
“, 1993 ; Rolls and Tovee , 1995 ; Missal et al .”
BackgroundForm representation in monkey inferotemporal cortex is virtually unaltered by free viewing
DiCarlo, J. J., & Maunsell, J. H. R. (2000) · Nature Neuroscience · doi:10.1038/77722 · Cited by 148 · cited 3×
“…ar - guably most physiologically relevant because it is consistent with that spontaneously produced by the eye movements of free - viewing monkeys ( DiCarlo and Maunsell , 2000 ) , we wondered whether the average result would have been found if we had used more " standard " presentation conditions…”
ResultSelf-citationLampl, I., Ferster, D., Poggio, T., & Riesenhuber, M. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00060.2004 · Cited by 139 · cited 1×
“However, mechanisms that produce averaging behavior may not be the only ones mediating stimulus interactions, because some populations of cat V1 neurons (Lampl et al., 2004) and monkey V4 neurons (Gawne and Martin, 2002) have responses that are similar to the response of the best constituent stimulus of each pair (CCI…”
BackgroundContrast Dependence of Response Normalization in Area MT of the Rhesus Macaque
Heuer, H. W., & Britten, K. H. (2002) · Journal of Neurophysiology · doi:10.1152/jn.00255.2002 · Cited by 136 · cited 3×
“, 1997 ; Britten and Heuer , 1999 ; Heuer and Britten , 2002 ) .”
BackgroundResponses of Primate Visual Cortical V4 Neurons to Simultaneously Presented Stimuli
Gawne, T. J., & Martin, J. M. (2002) · Journal of Neurophysiology · doi:10.1152/jn.2002.88.3.1128 · Cited by 134 · cited 6×
“Specifically, as suggested by a previous V4 study (Gawne and Martin, 2002), we considered only object pair conditions in which the less effective objects in isolation evoked a response that was less than half the response evoked by the more effective object.”
BackgroundMethodResultInfluentialThe zone of focal attention during active visual search
Motter, B. C., & Belky, E. J. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00252-6 · Cited by 130
Sato, T. (1989) · Experimental Brain Research · doi:10.1007/bf00250563 · Cited by 124 · cited 2×
“Consistent with previous investigators ( Sato , 1989 ; Miller et al .”
BackgroundResultShelton, C. R. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008170818506 · Cited by 75 · cited 1×
“Cars were built using an algorithm (Shelton, 2000) that found corresponding points in each pair of threedimensional car prototypes and represented each car prototype as a vector of point coordinates.”
MethodShape Interactions in Macaque Inferior Temporal Neurons
Missal, M., Vogels, R., Li, C.-Y., & Orban, G. A. (1999) · Journal of Neurophysiology · doi:10.1152/jn.1999.82.1.131 · Cited by 73 · cited 1×
“…IT responses are altered in cluttered scenes (Sheinberg and Logothetis, 2001; Rolls et al., 2003), and responses to object pairs are typically weaker than responses to isolated, preferred objects (Sato, 1989; Miller et al., 1993; Rolls and Tovee, 1995; Chelazzi et al., 1998; Missal et al., 1999).”
BackgroundResponses of macaque inferior temporal neurons to overlapping shapes
Missal, M. (1997) · Cerebral Cortex · doi:10.1093/cercor/7.8.758 · Cited by 50
Taking the MAX from neuronal responses
Rousselet, G. A., Thorpe, S. J., & Fabre‐Thorpe, M. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00023-8 · Cited by 32 · cited 1×
“In this context, some have suggested that IT neurons should ideally be tolerant to visual clutter, i.e., their response to an effective object should be essentially unaffected by the presence of other, less-effective objects (Rousselet et al., 2003, 2004).”
Background
Cited by
Normalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906
“As predicted by such a model, there is evidence for suppression in ventral stream area V4 and the inferotemporal cortex ( IT ) when a non-preferred stimulus (or object) is presented with a preferred stimulu”
BackgroundHow does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790
“This contemporary view, that neuronal tolerance is the required and observed single unit phenomenology, has also been shown for less intuitive identity-preserving transformations such as the addition of clutter (Li et al., 2009; Zoccolan et al., 2005).”
InfluentialSelf-citationThe Normalization Model of Attention
Reynolds, J. H., & Heeger, D. J. (2009) · Neuron · doi:10.1016/j.neuron.2009.01.002 · Cited by 1,457
“…stream areas V4 and IT (Miller et al., 1993; Missal et al., 1997; Reynolds et al., 1999; Richmond et al., 1983; Rolls and Tovee, 1995; Sato, 1989; Zoccolan et al., 2005), and in dorsal stream visual cortical areas MT and MST (Heuer and Britten, 2002; Recanzone et al., 1997; Treue et al., 2000).”
BackgroundSpectral fingerprints of large-scale neuronal interactions
Siegel, M., Donner, T. H., & Engel, A. K. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3137 · Cited by 1,295
Untangling invariant object recognition
DiCarlo, J. J., & Cox, D. D. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.06.010 · Cited by 959
Self-citation
Metadata
- Authors
- Zoccolan, D., Cox, D. D., & DiCarlo, J. J.
- Venue
- The Journal of Neuroscience
- Publisher
- Society for Neuroscience
- Published
- 2005-09-07
- Volume
- 25
- Issue
- 36
- Pages
- 8150-8164
- ISSN
- 0270-6474 (ISSN-L)
- 1529-2401
- OpenAlex
- W2062906394
- Cited by
- 218
- References
- 52
- FWCI
- 3.74
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.