A Canonical Neural Circuit for Cortical Nonlinear Operations
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
A few distinct cortical operations have been postulated over the past few years, suggested by experimental data on nonlinear neural response across different areas in the cortex. Among these, the energy model proposes the summation of quadrature pairs following a squaring nonlinearity in order to explain phase invariance of complex V1 cells. The divisive normalization model assumes a gain-controlling, divisive inhibition to explain sigmoid-like response profiles within a pool of neurons. A gaussian-like operation hypothesizes a bell-shaped response tuned to a specific, optimal pattern of activation of the presynaptic inputs. A max-like operation assumes the selection and transmission of the most active response among a set of neural inputs. We propose that these distinct neural operations can be computed by the same canonical circuitry, involving divisive normalization and polynomial nonlinearities, for different parameter values within the circuit. Hence, this canonical circuit may provide a unifying framework for several circuit models, such as the divisive normalization and the energy models. As a case in point, we consider a feedforward hierarchical model of the ventral pathway of the primate visual cortex, which is built on a combination of the gaussian-like and max-like operations. We show that when the two operations are approximated by the circuit proposed here, the model is capable of generating selective and invariant neural responses and performing object recognition, in good agreement with neurophysiological data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Spatiotemporal energy models for the perception of motion
Adelson, E. H., & Bergen, J. R. (1985) · Journal of the Optical Society of America A · doi:10.1364/josaa.2.000284 · Cited by 3,442
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309 · cited 5×
“See Kouh (2007) and Serre et al. (2005) for a comprehensive description of the current, recently extended version of the model and Riesenhuber and Poggio (1999) for the original proposal.”
BackgroundMethodInfluentialSelf-citationSimplified neuron model as a principal component analyzer
Oja, E. (1982) · Journal of Mathematical Biology · doi:10.1007/bf00275687 · Cited by 2,309 · cited 2×
“This condition, also known as Oja’s rule (Oja, 1982), implies that the total synaptic weights are normalized or conserved.”
BackgroundNEURONAL CIRCUITS OF THE NEOCORTEX
Douglas, R. J., & Martin, K. A. C. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144152 · Cited by 1,886
Place units in the hippocampus of the freely moving rat
O'Keefe, J. (1976) · Experimental Neurology · doi:10.1016/0014-4886(76)90055-8 · Cited by 1,836
The Normalization Model of Attention
Reynolds, J. H., & Heeger, D. J. (2009) · Neuron · doi:10.1016/j.neuron.2009.01.002 · Cited by 1,457
Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit
Hahnloser, R. H. R., Sarpeshkar, R., Mahowald, M. A., Douglas, R. J., & Seung, H. S. (2000) · Nature · doi:10.1038/35016072 · Cited by 1,293
Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4
Reynolds, J. H., Chelazzi, L., & Desimone, R. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-05-01736.1999 · Cited by 1,236 · cited 1×
“…likely to constitute an important part of the motion selectivity in V1 and MT (Nowlan & Sejnowski, 1995; Rust, Schwartz, Movshon, & Simoncelli, 2005; Rust, Mante, Simoncelli, & Movshon, 2006) and of the attentional mechanisms (Lee, Itti, Koch, & Braun, 1999; Reynolds, Chelazzi, & Desimone, 1999).”
Gain Modulation from Background Synaptic Input
Chance, F. S., Abbott, L. F., & Reyes, A. D. (2002) · Neuron · doi:10.1016/s0896-6273(02)00820-6 · Cited by 1,012
Shape representation in the inferior temporal cortex of monkeys
Logothetis, N. K., Pauls, J., & Poggio, T. (1995) · Current Biology · doi:10.1016/s0960-9822(95)00108-4 · Cited by 963 · cited 5×
“…to the inferotemporal (IT) cortex, which is the last purely visual area in the hierarchy of the primate visual cortex, and in particular , Logothetis et al. (1995) have found some neurons in anterior IT, tuned to scaled, translated, and rotated (in depth) images of the preferred…”
BackgroundInfluentialSelf-citationA model of neuronal responses in visual area MT
Simoncelli, E. P., & Heeger, D. J. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00183-1 · Cited by 941 · cited 1×
“For example, a complex cell in V1 (or a motion-selective cell in MT, or a shape-selective cell in IT) may show a tuned response for a particular edge orientation (or velocity, or a face image), a sigmoidal response for the image contrast, and a translation-invariant response within its receptive field (Gross et al., 1…”
BackgroundNatural signal statistics and sensory gain control
Schwartz, O., & Simoncelli, E. P. (2001) · Nature Neuroscience · doi:10.1038/90526 · Cited by 835
Learning invariance from transformation sequences
Földiák, P. (1991) · Neural Computation · doi:10.1162/neco.1991.3.2.194 · Cited by 661 · cited 1×
“We also note that the invariance property can be learned by the similar Hebbian mechanism (Bartlett & Sejnowski, 1998; Földiák, 1991).”
BackgroundSummation and Division by Neurons in Primate Visual Cortex
Carandini, M., & Heeger, D. J. (1994) · Science · doi:10.1126/science.8191289 · Cited by 649 · cited 7×
“The energy model and many divisive normalization models (Adelson & Bergen, 1985; Carandini & Heeger, 1994; Rust et al., 2005) have used (half) squaring operations only.”
BackgroundMethodInfluentialHow MT cells analyze the motion of visual patterns
Rust, N. C., Mante, V., Simoncelli, E. P., & Movshon, J. A. (2006) · Nature Neuroscience · doi:10.1038/nn1786 · Cited by 516 · cited 4×
“As shown here and as considered by Rust et al. (2006), such " tuned " normalization can sharpen the tuning curves within the afferent input space, and it may be implemented by the stronger self-inhibiting synapses in Figure 1B.”
BackgroundMethodInfluentialA canonical microcircuit for neocortex
Douglas, R. J., Martin, K. A. C., & Whitteridge, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.480 · Cited by 514 · cited 3×
“As shown by several experimental studies (Douglas et al., 1989; Marino et al., 2005; Wang, Fujita, & Murayama, 2000), inhibitory mechanisms are critically involved in generating neural selectivity and invariance, and in our proposal, the input normalization by a pool cell or lateral inhibition…”
BackgroundShape Representation in Area V4: Position-Specific Tuning for Boundary Conformation
Pasupathy, A., & Connor, C. E. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.86.5.2505 · Cited by 483 · cited 2×
“Such translation-invariance properties are found in other parts of the primate visual cortex (Gallant et al., 1996; Ito et al., 1995; Logothetis et al., 1995; Pasupathy & Connor, 2001).”
BackgroundThe Contribution of Noise to Contrast Invariance of Orientation Tuning in Cat Visual Cortex
Anderson, J. S., Lampl, I., Gillespie, D. C., & Ferster, D. (2000) · Science · doi:10.1126/science.290.5498.1968 · Cited by 422 · cited 1×
“From the biophysical point of view, such different exponents may originate from noise and linear-threshold function between voltage and spike generation ( Anderson, Lampl, Gillespie, & Ferster, 2000; Miller & Troyer, 2002), where a wide distribution of polynomial exponents, as large as”
BackgroundFeedforward Mechanisms of Excitatory and Inhibitory Cortical Receptive Fields
Bruno, R. M., & Simons, D. J. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-24-10966.2002 · Cited by 354
Learning in spiking neural networks by reinforcement of stochastic synaptic transmission
Seung, H. S. (2003) · Neuron · doi:10.1016/s0896-6273(03)00761-x · Cited by 340 · cited 1×
“However, choosing the target values themselves may be unsupervised or subject to a hedonistic principle (Seung, 2003).”
BackgroundShunting Inhibition Does Not Have a Divisive Effect on Firing Rates
Holt, G. R., & Koch, C. (1997) · Neural Computation · doi:10.1162/neco.1997.9.5.1001 · Cited by 250
Representation of Concurrent Stimuli by Population Activity in Visual Cortex
Busse, L., Wade, A. R., & Carandini, M. (2009) · Neuron · doi:10.1016/j.neuron.2009.11.004 · Cited by 247
Neurophysiology of shape processing
Perrett, D. I., & Oram, M. W. (1993) · Image and Vision Computing · doi:10.1016/0262-8856(93)90011-5 · Cited by 213 · cited 2×
“As proposed by Fukushima and others (Fukushima, Miyake, & Ito, 1983; Hubel & Wiesel, 1962; Perrett & Oram, 1993; Riesenhuber & Poggio, 1999), a plausible feedforward mechanism for invariance is to pool from the afferent cells tuned to the transformed versions of a stimulus (e.g., translation,…”
BackgroundNeural Noise Can Explain Expansive, Power-Law Nonlinearities in Neural Response Functions
Miller, K. D., & Troyer, T. W. (2002) · Journal of Neurophysiology · doi:10.1152/jn.00425.2001 · Cited by 207 · cited 4×
“…the membrane potential V and the activity of the output neuron y are approximated by a linear relationship, or a polynomial " nonlinearity " with exponent 1 (Miller & Troyer, 2002) (note that other monotonic relationships can also generate gaussian-like tuning and approximate max-like behaviors).”
BackgroundInfluentialLampl, I., Ferster, D., Poggio, T., & Riesenhuber, M. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00060.2004 · Cited by 139 · cited 1×
“A few recent physiological experiments (Gawne & Martin, 2002; Lampl et al., 2004) have reported that when multiple stimuli are simultaneously presented, some neurons produce a max-like response.”
BackgroundSelf-citationResponses of Primate Visual Cortical V4 Neurons to Simultaneously Presented Stimuli
Gawne, T. J., & Martin, J. M. (2002) · Journal of Neurophysiology · doi:10.1152/jn.2002.88.3.1128 · Cited by 134
Cited by
Normalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906
How does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790
“Notably, both AND-like and OR-like computations can be formulated as variants of the NLNmodel class described above (Kouh and Poggio, 2008), illustrating the link to canonical cortical models (see inset in Figure 6).”
InfluentialThe Normalization Model of Attention
Reynolds, J. H., & Heeger, D. J. (2009) · Neuron · doi:10.1016/j.neuron.2009.01.002 · Cited by 1,457
“Shunting inhibition through lateral connections from other neurons in the cortical neighborhood has been proposed as one possible mechanism for normalization (Carandini and Heeger, 1994; Carandini et al., 1997; Kouh and Poggio, 2008).”
BackgroundInfluentialReconstructing Speech from Human Auditory Cortex
Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., Knight, R. T., & Chang, E. F. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001251 · Cited by 604
“Such amplitude-based coding schemes are broadly referred to as ‘‘energy models’’ [32,33].”
BackgroundPhoneme and word recognition in the auditory ventral stream
DeWitt, I., & Rauschecker, J. P. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1113427109 · Cited by 465
Metadata
- Authors
- Kouh, M., & Poggio, T.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2008-06
- Volume
- 20
- Issue
- 6
- Pages
- 1427-1451
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2148165961
- Cited by
- 162
- References
- 26
- FWCI
- 6.49
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.