A feedforward architecture accounts for rapid categorization
Open accessAbstract
Primates are remarkably good at recognizing objects. The level of performance of their visual system and its robustness to image degradations still surpasses the best computer vision systems despite decades of engineering effort. In particular, the high accuracy of primates in ultra rapid object categorization and rapid serial visual presentation tasks is remarkable. Given the number of processing stages involved and typical neural latencies, such rapid visual processing is likely to be mostly feedforward. Here we show that a specific implementation of a class of feedforward theories of object recognition (that extend the Hubel and Wiesel simple-to-complex cell hierarchy and account for many anatomical and physiological constraints) can predict the level and the pattern of performance achieved by humans on a rapid masked animal vs. non-animal categorization task.
AI snapshot
Written from the paper’s full text.
TL;DR
A specific feedforward hierarchical model of ventral-stream object recognition predicts human performance on a rapid masked animal vs. non-animal categorization task. The model matches human accuracy and image-by-image response patterns, supporting a mostly feedforward account of immediate recognition.
Key findings
- Human observers and the model showed similar overall accuracy across four animal categories: 80% correct for humans and 82% for the model, with no significant differences across categories.
- Image-by-image agreement between model and human classifications was high: correlations of 0.71, 0.84, 0.71, and 0.60 for head, close-body, medium-body, and far-body categories, respectively (P < 0.01).
- The model was robust to 90° and 180° image rotation without retraining, matching human performance patterns.
- In the SOA experiment, the model predicted human hit rate well between 50- and 80-ms SOA; for SOAs longer than 80 ms, humans outperformed the model, while the 50-ms SOA condition was only ≈5% lower than the no-mask ceiling.
Methods
- The model is a feedforward extension of the Hubel–Wiesel simple-to-complex cell hierarchy, with alternating S and C layers from V1 to IT and a supervised IT-to-PFC classification stage; it contained on the order of 10 million units, with intermediate stages shaped by unsupervised learning and only the top task-specific circuit trained on labeled examples.
- Human psychophysics used gray-value 256 × 256 pixel images (≈7° × 7° visual angle), 20-ms stimulus presentation, a 1/f noise backward mask of 80 ms, and a 50-ms SOA in the main experiment; participants were 18–35 years old, with n = 24 in the fixed 50-ms SOA experiment, n = 14 in the rotation experiment, and n = 21 in the variable-SOA experiment.
- The stimulus database contained 1,200 images: 600 animal and 600 non-animal images, with four balanced animal categories of 150 animals and 150 matching distractors each; model performance was assessed with d′ and a random-splits procedure repeated n = 20 times.
Limitations
- The paper states that as a feedforward model of the ventral stream, the architecture cannot account for everyday vision involving eye movements and top-down effects mediated by back projections, which are not implemented.
- The paper states that model performance decreases significantly as clutter increases, and preliminary results suggest top-down attention or back projections may be needed for the far-animal condition.
- The paper notes that part of the discrepancy between model and humans is likely due to the relatively small training set (300 animal and 300 non-animal images), and that humans outperform the model for SOAs longer than 80 ms, leaving open whether feedback effects explain this.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 70 cited sources checked · 0 supported · 0 flagged
- Findingown data§ Results
Both the model and human observers tend to produce similar responses, both correct and incorrect.
“Most importantly, both the model and human observers tend to produce similar responses (both correct and incorrect; see Fig. 3).”
Quote in the paperSubstantial claim- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Findingown data§ A feedforward architecture accounts for rapid categorization
A specific implementation of a class of feedforward theories of object recognition can predict the level and pattern of performance achieved by humans on a rapid masked animal vs. non-animal categorization task.
“Here we show that a specific implementation of a class of feedforward theories of object recognition (that extend the Hubel and Wiesel simple-tocomplex cell hierarchy and account for many anatomical and physiological constraints) can predict the level and the pattern of performance achieved by humans on a rapid masked animal vs. non-animal categorization task.”
Quote in the paperSubstantial claim- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Findingreasoning§ Discussion
The model mimics human-level performance for the 50-ms SOA condition.
“The model indeed mimics human-level performance for the 50-ms SOA condition.”
Quote in the paperSubstantial claim - Findingown data§ Discussion
When the amount of clutter present in the images increases, the performance of the model decreases significantly.
“However, when the amount of clutter present in the images increase, the performance of the model decreases significantly.”
Quote in the paperSubstantial claim - Findingreasoning§ Discussion
The architecture of Fig. 1 contains a total of 10^7 tuned units.
“As a result of this new learning stage, the architecture of Fig. 1 contains a total of 10 7 tuned units.”
Quote in the paperSubstantial claim
- Findingown data§ Results
The delay between the stimulus and the mask onset modulates the level of performance of the observers, improving gradually from the 20-ms SOA condition to the no-mask condition.
“As expected, the delay between the stimulus and the mask onset modulates the level of performance of the observers improving gradually from the 20-ms SOA condition to the no-mask condition (see SI Fig. 6).”
Quote in the paperNot a substantial claim - Findingown data§ Results
The model predicts human-level hit rate very well between the 50- and the 80-ms SOA conditions.
“The model predicts human-level hit rate very well between the 50- and the 80-ms SOA conditions.”
Quote in the paperSubstantial claim - Findingown data§ Results
For SOAs longer than 80 ms, human observers outperform the model.
“For SOAs longer than 80 ms, human observers outperform the model (the performance for the 50-ms SOA condition, however, is only ≈ 5% lower than the ceiling performance in the no-mask condition).”
Quote in the paperNot a substantial claim - Findingcitation§ Discussion
In agreement with the human data, the model is able to recognize objects with limited clutter.
“In agreement with the human data, the model is able to recognize objects with limited clutter (see ref. 39 for results on a large database of 101 object categories).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [39] Serre T, Wolf L, Bileschi S, Poggio T (2007) IEEETrans Pattern Anal Mach Intell 29:411-426. source
Not checked in the free run.
- Interpretationreasoning§ A feedforward architecture accounts for rapid categorization
By alternating between S layers of simple units and C layers of complex units, the model achieves a difficult tradeoff between selectivity and invariance.
“By alternating between S layers of simple units and C layers of complex units, the model achieves a difficult tradeoff between selectivity and invariance”
Quote in the paperSubstantial claim- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Premisecitation§ Results
Increasing the SOA on a similar animal-vs.-non-animal categorization task above 44 ms only has a minor effect on performance.
“It was found (31) that increasing the SOA on a similar animalvs. non-animalcategorization task above 44 ms only has a minor effect on performance (accuracy scores for longer SOA conditions were not significantly different).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [31] source
Not checked in the free run.
- Interpretationcitation§ A feedforward architecture accounts for rapid categorization
A core hierarchical feedforward architecture may be a reasonable starting point for a theory of visual cortex aiming to explain immediate recognition.
“Although this finding does not rule out local feedback loops within an area, it does suggest that a core hierarchical feedforward architecture may be a reasonable starting point for a theory of visual cortex aiming to explain immediate recognition, the initial phase of recognition before eye movements and high-level processes can play a role (10-13).”
Quote in the paperSubstantial claim - Premisecitation§ A feedforward architecture accounts for rapid categorization
The model shows good agreement with other data in V4 about the response of neurons to combinations of simple two-bar stimuli.
“It also shows good agreement (23) with other data in V4 (28) about the response of neurons to combinations of simple two-bar stimuli (within the receptive field of the S 2 units)”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [23]
Not checked in the free run.
- cites [28] source
Not checked in the free run.
- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Premisecitation§ A feedforward architecture accounts for rapid categorization
Readout from C2b units in the model predicted recent readout experiments in IT, showing very similar selectivity and invariance for the same set of stimuli.
“Readout from C 2b units in the model described here predicted (23) recent readout experiments in IT (9), showing very similar selectivity and invariance for the same set of stimuli.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [23]
Not checked in the free run.
- cites [9] source
Not checked in the free run.
- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Premisecitation§ A feedforward architecture accounts for rapid categorization
The model predicts, at the C1 and C2 levels, respectively, the max-like behavior of a subclass of complex cells in V1 and V4.
“For instance, the model predicts (23), at the C 1 and C 2 levels, respectively, the max-like behavior of a subclass of complex cells in V1 (26) and V4 (27).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [23]
Not checked in the free run.
- cites [26] source
Not checked in the free run.
- cites [27] source
Not checked in the free run.
- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Premisecitation§ A feedforward architecture accounts for rapid categorization
This class of models is qualitatively and quantitatively consistent with several properties of subpopulations of cells in V1, V4, IT, and PFC as well as fMRI and psychophysical data.
“This class of models seems to be qualitatively and quantitatively consistent with [and in some cases actually predicts (23)] several properties of subpopulations of cells in V1, V4, IT, and PFC (25) as well as fMRI and psychophysical data.”
- Also stated in: § A feedforward architecture accounts for rapid categorization · plain
“The model accounts for a set of basic facts about the cortical mechanisms of recognition that have been established over the last decades”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [23]
Not checked in the free run.
- cites [25] source
Not checked in the free run.
- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Also stated in: § A feedforward architecture accounts for rapid categorization · plain
- Premisereasoning§ A feedforward architecture accounts for rapid categorization
Given the number of processing stages involved and typical neural latencies, rapid visual processing is likely to be mostly feedforward.
“Given the number of processing stages involved and typical neural latencies, such rapid visual processing is likely to be mostly feedforward.”
Quote in the paperNot a substantial claim - Premisecitation§ A feedforward architecture accounts for rapid categorization
Activity of small neuronal populations in monkey IT, over very short time intervals (as small as 12.5 ms) and only about 100 ms after stimulus onset, contains surprisingly accurate and robust information supporting a variety of recognition tasks.
“Very recent data (9) show that the activity of small neuronal populations in monkey IT, over very short time intervals (as small as 12.5 ms) and only ≈ 100 ms after stimulus onset, contains surprisingly accurate and robust information supporting a variety of recognition tasks.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [9] source
Not checked in the free run.
- Interpretationreasoning§ A feedforward architecture accounts for rapid categorization
As a feedforward model of the ventral stream pathway, the architecture cannot account for everyday vision, which involves eye movements and top-down effects mediated by higher brain centers and extensive anatomical back projections.
“Of course as a feedforward model of the ventral stream pathway, the architecture of Fig. 1 cannot account for our everyday vision which involves eye movements and top-down effects, which are mediated by higher brain centers and the extensive anatomical back projections found throughout the visual cortex, and are not implemented in the present feedforward model.”
Quote in the paperSubstantial claim - Premisecitation§ A feedforward architecture accounts for rapid categorization
The hypothesis that basic processing of information is feedforward is supported most directly by the short time spans required for a selective response to appear in IT cells.
“The hypothesis that the basic processing of information is feedforward is supported most directly by the short time spans required for a selective response to appear in IT cells (8).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [8] source
Not checked in the free run.
- Premisecitation§ A feedforward architecture accounts for rapid categorization
Ultra rapid categorization is a task for which back projections are likely to be inactive.
“Thus, a natural paradigm for comparing the performance of human observers in an object-recognition task to that of a feedforward model of visual processing is ultra rapid categorization, a task for which back projections are likely to be inactive (30, 31).”
Quote in the paperSubstantial claim0 of 2 sources checked - Interpretationown data§ Results
Together with a lesion study performed on the model, the data suggest that the large, overall set of features from V2 to V4 and the posterior inferotemporal cortex underlies the human-like performance in this task.
“Together with the results of a ''lesion study'' performed on the model (see SI Fig. 4), the data suggest that it is the large, overall set of features from V2 to V4 and the posterior inferotemporal cortex that underlies such a human-like performance in this task.”
Quote in the paperSubstantial claim- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Interpretationcitation§ A feedforward architecture accounts for rapid categorization
Plausible biophysical circuits may implement the two key operations assumed by the theory within the time constraints of the experimental data.
“In addition, plausible biophysical circuits may implement the two key operations (5) assumed by the theory within the time constraints of the experimental data (8).”
Quote in the paperSubstantial claim0 of 2 sources checked- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Premisecitation§ A feedforward architecture accounts for rapid categorization
Some C2 units in the model show a tuning for boundary conformations consistent with recordings from V4.
“some of the C 2 units in the model show a tuning for boundary conformations which is consistent with recordings from V4 (29)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [29] source
Not checked in the free run.
- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Interpretationnone given§ Materials and Methods
S4 units become tuned to views of the target object by storing in their synaptic weights the pattern of activity of their afferents during a presentation of a particular exemplar.
“Like units in lower stages become tuned to patches of natural images (see SI Text ); S 4 units become tuned to views of the target object by storing in their synaptic weights the pattern of activity of their afferents during a presentation of a particular exemplar.”
Quote in the paperSubstantial claim - Study design and conductcitation§ Discussion
The parameters of the S1 and C1 model units were constrained by physiology data so that their tuning properties would agree with those of cortical simple and complex cells.
“For instance, the parameters (see SI Table 1) of the S 1 and C 1 model units were constrained by physiology data (1, 36, 37) so that their tuning properties would agree with those of cortical simple and complex cells (see SI Text ).”
Quote in the paperNot a substantial claim0 of 3 sources checked - Interpretationown data§ Discussion
This finding provides computational neuroscience support to the conjecture that a task-independent, unsupervised, developmental-like learning stage may exist in the ventral stream to generate a large dictionary of shape-tuned units with various degrees of selectivity and invariance from V1 to IT, consistent with recent data.
“This finding provides computational neuroscience support to the conjecture that a task-independent, unsupervised, developmental-like learning stage may exist in the ventral stream to generate a large dictionary of shape-tuned units with various degrees of selectivity and invariance from V1 to IT, consistent with recent data (52).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [52] Freedman D, Riesenhuber M, Poggio T, Miller E (2006) Cereb Cortex , in press.
Not checked in the free run.
- Interpretationown data§ Discussion
Under these conditions, the present feedforward model may provide a satisfactory description of information processing in the ventral stream of the visual cortex.
“This finding suggests that under these conditions, the present feedforward model may provide a satisfactory description of information processing in the ventral stream of the visual cortex.”
Quote in the paperSubstantial claim - Interpretationcitation§ Discussion
The model is remarkably robust to parameter values' detailed wiring and even exact form of the two basic operations and of the learning rule.
“In addition, the model is remarkably robust to parameter values' detailed wiring and even exact form of the two basic operations and of the learning rule (23).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23]
Not checked in the free run.
- Premisecitation§ Discussion
The later response includes recurrent processing, that is a modulation through back projections from higher to lower areas.
“Several studies ( SI Text ) have shown that this later response includes recurrent processing, that is a modulation through back projections from higher to lower areas.”
Quote in the paperSubstantial claim - Interpretationcitation§ Discussion
The new model implementation used in this study improves the original model in two significant ways.
“The new model implementation used in this study improves the original model (5) in two significant ways.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [5] source
Not checked in the free run.
- Interpretationcitation§ Discussion
Our results agree with several theories of visual processing that suggest that an initial feedforward sweep driven by bottom-up inputs builds a base representation that relies on a basic dictionary of generic features before more complex tasks or visual routines can take place through recurrent projections from higher areas.
“Our results indeed agree with several theories of visual processing that suggest that an initial feedforward sweep driven by bottom-up inputs builds a base representation that relies on a basic dictionary of generic features (11-13, 17, 41) before more complex tasks or visual routines can take place through recurrent projections from higher areas (20, 42, 43, 45).”
Quote in the paperSubstantial claim0 of 5 sources checked- cites [11] source
Not checked in the free run.
- cites [12] source
Not checked in the free run.
- cites [13] source
Not checked in the free run.
- cites [17] source
Not checked in the free run.
- cites [41] Evans K, Treisman A (2005) J Exp Psychol Hum Percept Perform 31:1476-1492. source
Not checked in the free run.
- Premisecitation§ Discussion
During masked stimulus presentations, the feedforward bottom-up components of cortical cells' response (the early response from response onset for a period lasting about the stimulus mask) remains essentially unaltered whereas the later response is interrupted.
“Previous physiological studies have shown that during masked stimulus presentations, the feedforward bottom-up components of cortical cells' response (i.e., the early response from response onset for a period lasting about the stimulus mask) remains essentially unaltered whereas the later response is interrupted (see SI Text ) (see also refs. 42, 43, and 44 for recent reviews).”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [42] Enns J, Lollo VD (2000) Trends Cognit Neurosci 4:345-351. source
Not checked in the free run.
- cites [43] Lamme V, Roelfsema P (2000) Trends Neurosci 23:571-579. source
Not checked in the free run.
- cites [44] Breitmeyer B, Ogmen H (2006) Visual Masking: Time Slices Through Conscious and Unconscious Vision (Oxford Univ Press, Oxford). source
Not checked in the free run.
- Interpretationcitation§ Discussion
The new model is closer to the anatomy and the physiology of the visual cortex in terms of quantitative parameter values.
“Additionally, the new model is closer to the anatomy and the physiology of the visual cortex in terms of quantitative parameter values.”
Quote in the paperNot a substantial claim - Interpretationown data§ Discussion
The decrease in model performance with increased clutter suggests a key role for the massive back projections found in the visual cortex.
“This suggests a key role for the massive back projections found in the visual cortex (46).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [46] Felleman DJ, van Essen DC (1991) Cereb Cortex 1:1-47. source
Not checked in the free run.
- Study design and conductnone given§ Results
The model was not retrained before being tested on the rotated images.
“The robustness of the model is particularly remarkable as it was not retrained before being tested on the rotated images.”
Quote in the paperSubstantial claim - Interpretationcitation§ Discussion
The major extension is a new unsupervised learning stage of the units in intermediate stages of the model.
“The major extension is a new unsupervised learning stage of the units in intermediate stages of the model (23, H20648 ).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [23]
Not checked in the free run.
- The authors state this as an assumption: The hierarchy of visual areas along the ventral stream of the visual cortex, from V1 to IT, builds a generic dictionary of shape-tuned units which provides a rich representation for task-specific categorization circuits in prefrontal areas.
- Interpretationreasoning§ Discussion
Significant top-down modulation should start for stimulus-mask interval ≈ 40-60 ms.
“Based on response latencies in the visual cortex ( SI Text ), we estimate that significant top-down modulation should start for stimulus-mask interval ≈ 40-60 ms ( SI Text ).”
Quote in the paperSubstantial claim - Interpretationcitation§ Discussion
The model accounts for the bypass routes from V2 to the posterior inferotemporal cortex and from V4 to the anterior inferotemporal cortex, unlike the original model.
“In addition to the main routes through the V4 to the IT cortex (4), the model also accounts for the bypass routes (38) from V2 to the posterior inferotemporal cortex and from V4 to the anterior inferotemporal cortex (Fig. 1) [unlike the original model (5)].”
Quote in the paperSubstantial claim0 of 3 sources checked - Interpretationcitation§ Discussion
After the initial unsupervised learning stage, only the task-specific circuits at the top level in the model, possibly corresponding to categorization units in PFC, have to be trained from a small set of labeled examples and in a task-specific manner for the 'mature' model to learn a categorization task.
“After this initial unsupervised learning stage, only the task-specific circuits at the top level in the model, possibly corresponding to categorization units in PFC (25), have to be trained from a small set of labeled examples and in a task-specific manner (see Materials and Methods ) for the ''mature'' model to learn a categorization task (e.g., animal vs. non-animal).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [25] source
Not checked in the free run.
- The authors state this as an assumption: The hierarchy of visual areas along the ventral stream of the visual cortex, from V1 to IT, builds a generic dictionary of shape-tuned units which provides a rich representation for task-specific categorization circuits in prefrontal areas.
- May combine two claims. 15
- Classified as probably not a substantial claim. 16
- The quote only partly states this claim. 1
May combine two claims.
Findingown data§ ResultsThe overall image-by-image correlation between the model and human observers is high: 0.71, 0.84, 0.71, and 0.60 for heads, close-body, medium-body, and far-body, respectively, with P < 0.01.
“The overall imageby-image correlation between the model and human observers is high (specifically 0.71, 0.84, 0.71, and 0.60 for heads, close-body, medium-body, and far-body, respectively, with P < 0.01).”
Quote in the paperSubstantial claim- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
May combine two claims.
Findingown data§ ResultsHuman observers and the model behave similarly: across all four animal categories, their levels of performance do not show significant differences, with overall correct = 80% for human observers and 82% for the model.
“Human observers and the model behave similarly: Across all four animal categories, their levels of performance do not show significant differences (with overall correct = 80% for human observers and 82% for the model).”
- Also stated in: § Results · plain
“Human observers and the model exhibit a very similar pattern of performance (measured with d ′ measure; see SI Text ).”
Quote in the paperSubstantial claim- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
- Also stated in: § Results · plain
May combine two claims.
Findingown data§ DiscussionOur main result is that a simple extension of the feedforward hierarchical architecture, suggested some 40 years ago by Hubel and Wiesel and reflecting the known physiology and anatomy of the visual cortex, correlates well with humans and exhibits comparable accuracy on a difficult (but rapid) recognition task.
“Nevertheless, our main result is that a simple extension of the feedforward hierarchical architecture, suggested some 40 years ago by Hubel and Wiesel and reflecting the known physiology and anatomy of the visual cortex, correlates well with humans and exhibits comparable accuracy on a difficult (but rapid) recognition task.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ ResultsThe level of performance of human observers reached a ceiling in the 80-ms SOA condition, except when the animal was camouflaged in the scene (far-body group).
“The level of performance of human observers reached a ceiling in the 80-ms SOA condition (except when the animal was camouflaged in the scene, i.e., far-body group).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown data§ ResultsThe model and human observers show a similar pattern of performance and are similarly robust to image rotation (90° and 180°).
“SI Fig. 5 shows indeed that the model ( Right ) and human observers ( Left ) show a similar pattern of performance and are similarly robust to image rotation.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone given§ ResultsPart of the discrepancy between the model and human observers is likely due to the relatively small set of examples used to train the model (300 animal and 300 non-animal images).
“Part of the discrepancy between the model and human observers is likely to be due to the relatively small set of examples used to train the model (300 animal and 300 non-animal images).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ ResultsThe accuracy of the human observers is well within the range of data previously obtained with go/no-go tasks on similar tasks.
“The accuracy of the human observers is well within the range of data previously obtained with go/no-go tasks on similar tasks (30, 31, 33).”
Quote in the paperNot a substantial claim0 of 3 sources checkedMay combine two claims.
Study design and conductnone given§ ResultsNo single model parameter was adjusted to fit the human data; all parameters apart from the supervised stage from IT to PFC were fixed before all tests by taking into account the physiology data from V1 to IT.
“It should be noted that no single model parameter was adjusted to fit the human data (all parameters apart from the supervised stage from IT to PFC were fixed before all tests by taking into account the physiology data from V1 to IT).”
Quote in the paperSubstantial claim- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
May combine two claims.
Premisecitation§ A feedforward architecture accounts for rapid categorizationFrom V1 to IT, there is an increase in invariance to position and scale and, in parallel, an increase in the size of the receptive fields and in the complexity of the optimal stimuli for the neurons.
“From V1 to IT, there is an increase in invariance to position and scale (1, 2, 4-6) and in parallel, an increase in the size of the receptive fields (2, 4) as well as in the complexity of the optimal stimuli for the neurons (2, 3, 7).”
Quote in the paperSubstantial claim0 of 7 sources checked- cites [1]
Not checked in the free run.
- cites [2] source
Not checked in the free run.
- cites [4] source
Not checked in the free run.
- cites [5] source
Not checked in the free run.
- cites [6] source
Not checked in the free run.
- cites [3] source
Not checked in the free run.
- cites [7] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ A feedforward architecture accounts for rapid categorizationPlasticity and learning are probably present at all stages and certainly at the level of IT and PFC.
“Finally, plasticity and learning are probably present at all stages and certainly at the level of IT (6) and PFC.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [6] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone given§ A feedforward architecture accounts for rapid categorizationThe model is a simplification and only accounts for the ventral stream of the visual cortex.
“Note that the model is a simplification and only accounts for the ventral stream of the visual cortex.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ A feedforward architecture accounts for rapid categorizationAnimal- vs. non-animal-recognition is a well established experiment.
“A well established experiment is an animal- vs. non-animal-recognition task (30-34).”
Quote in the paperNot a substantial claim0 of 5 sources checked- cites [30] source
Not checked in the free run.
- cites [31] source
Not checked in the free run.
- cites [32] Thorpe S, Fabre-Thorpe M (2001) Science 291:260-263. source
Not checked in the free run.
- cites [33] VanRullen R, Koch C (2003) J Cognit Neurosci 15:209-217. source
Not checked in the free run.
- cites [34] Rousselet G, Mace M, Fabre-Thorpe M (2003) J Vision 3:440-455. source
Not checked in the free run.
Probably not a substantial claim: may not pass S4 (based).
Interpretationnone given§ ResultsThe mask is expected to block significant top-down effects through the back projections.
“At the same time, we expect the mask to block significant top-down effects through the back projections (see Discussion and SI Text ).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductcitation§ A feedforward architecture accounts for rapid categorizationThe parameters of the S1 and C1 units were adjusted to match as closely as possible the tuning properties of V1 parafoveal simple and complex cells.
“The parameters of the S 1 and C 1 units (see SI Table 1) were adjusted so as to match as closely as possible the tuning properties of V1 parafoveal simple and complex cells (receptive field size, peak frequency, frequency, and orientation bandwidth; see ref. 24 for details).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [24]
Not checked in the free run.
- The authors state this as an assumption: One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ Materials and MethodsThe selectivity of neurons in IT depends on visual experience.
“This finding is consistent with a large body of data that suggests that the selectivity of neurons in IT depends on visual experience (see ref. 23 for a review).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [23]
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown data§ Materials and MethodsOther classifiers could be used; a linear SVM gave very similar results.
“Other classifiers could be used (a linear SVM gave very similar results).”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationcitation§ DiscussionThis class of feedforward models can duplicate the tuning properties of at least some cortical cells when probed with artificial stimuli and can handle the recognition of objects in the real world where objects may undergo drastic changes in appearance.
“Not only does this class of feedforward models seem to be able to duplicate the tuning properties of at least some cortical cells when probed with artificial stimuli, but it can also handle the recognition of objects in the real world (39, 40) where objects may undergo drastic changes in appearance (e.g., clutter, shape, illumination).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [39] Serre T, Wolf L, Bileschi S, Poggio T (2007) IEEETrans Pattern Anal Mach Intell 29:411-426. source
Not checked in the free run.
- cites [40] Serre T, Wolf L, Poggio T (2005) Proc IEEE Conf Comput Vision Pattern Recognit 2:994-1000.
Not checked in the free run.
May combine two claims. The quote only partly states this claim.
Interpretationreasoning§ DiscussionLearning in the new model proceeds in two independent stages: first, a slow developmental-like unsupervised learning stage where units from V1 to IT become adapted to the statistics of the natural environment; second, task-specific circuits at the top level are trained from a small set of labeled examples.
“Correspondingly, learning proceeds in two independent stages: First, during a slow developmental-like unsupervised learning stage, units from V1 to IT become adapted to the statistics of the natural environment (see SI Text for details).”
Quote in the paperSubstantial claim- The authors state this as an assumption: The hierarchy of visual areas along the ventral stream of the visual cortex, from V1 to IT, builds a generic dictionary of shape-tuned units which provides a rich representation for task-specific categorization circuits in prefrontal areas.
Classified as probably not a substantial claim.
Premisecitation§ ResultsRecent behavioral studies suggested that the animal categorization task can be performed very well by human observers on rotated images.
“Recent behavioral studies (34)** suggested that the animal categorization task can be performed very well by human observers on rotated images.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [34] Rousselet G, Mace M, Fabre-Thorpe M (2003) J Vision 3:440-455. source
Not checked in the free run.
May combine two claims.
Interpretationcitation§ DiscussionThe resulting dictionary is generic and universal in the sense that it can support several different recognition tasks and in particular, the recognition of many different object categories.
“The resulting dictionary is generic and universal in the sense that it can support several different recognition tasks (23) and in particular, the recognition of many different object categories.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [23]
Not checked in the free run.
- The authors state this as an assumption: The hierarchy of visual areas along the ventral stream of the visual cortex, from V1 to IT, builds a generic dictionary of shape-tuned units which provides a rich representation for task-specific categorization circuits in prefrontal areas.
Classified as probably not a substantial claim.
Premisecitation§ Materials and MethodsA recent study demonstrated that a linear classifier can read out with high accuracy and over extremely short time intervals (a single bin as short as 12.5 ms) object identity, object category, and other information from the activity of ≈ 100 neurons in IT.
“A recent study (9) demonstrated that a linear classifier can indeed read out with high accuracy and over extremely short time intervals (a single bin as short as 12.5 ms) object identity, object category, and other information (such as the position and size of the object) from the activity of ≈ 100 neurons in IT.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [9] source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitation§ Materials and MethodsThe random splits procedure used to train the PFC classification unit provides a good estimate of the expected error of a classifier.
“To train the PFC classification unit in the model, we used a random splits procedure, which has been shown to provide a good estimate of the expected error of a classifier (56).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [56] Devroye L, Laszlo G, Lugosi G (1996) A Probabilistic Theory of Pattern Recognition (Springer, New York). source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationcitation§ DiscussionBack projections may be important for visual awareness and beyond tasks such as visual categorization for perceptual organization and figure-ground segmentation or curve tracing.
“In addition, back projections may be important for visual awareness and beyond tasks such as visual categorization for perceptual organization and figure-ground segmentation (48-50) or curve tracing (51).”
Quote in the paperNot a substantial claim0 of 4 sources checked- cites [48] Roelfsema P, Lamme V, Spekreijse H, Bosch H (2002) J Cognit Neurosci 12:525-537. source
Not checked in the free run.
- cites [49] Lamme V, Zipser K, Spekreijse H (2002) J Cognit Neurosci 14:1044-1053. source
Not checked in the free run.
- cites [50] Lee T, Mumford D (2003) J Opt Soc Am 20:1434-1448. source
Not checked in the free run.
- cites [51] Roelfsema PR, Lamme VA, Spekreijse H (1998) Nature 395:376-381. source
Not checked in the free run.
Probably not a substantial claim: may not pass S4 (based).
Interpretationreasoning§ ResultsThe robustness of the model to image rotation is likely due to the fact that an image patch of a rotated animal is more similar to an image patch of an upright animal than to a non-animal.
“It is likely due to the fact that an image patch of a rotated animal is more similar to an image patch of an upright animal than to a non-animal.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationcitation§ DiscussionKey to the recognition performance of the model is the large number of tuned units across its hierarchical architecture, which is a direct consequence of learning from natural images and represents a redundant dictionary of fragment-like features that span a range of selectivities and invariances.
“Key to the recognition performance of the model is the large number of tuned units across its hierarchical architecture, which is a direct consequence of the learning from natural images and represent a redundant dictionary of fragment-like features (12, 17, 41) that span a range of selectivities and invariances.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [12] source
Not checked in the free run.
- cites [17] source
Not checked in the free run.
- cites [41] Evans K, Treisman A (2005) J Exp Psychol Hum Percept Perform 31:1476-1492. source
Not checked in the free run.
- The authors state this as an assumption: The hierarchy of visual areas along the ventral stream of the visual cortex, from V1 to IT, builds a generic dictionary of shape-tuned units which provides a rich representation for task-specific categorization circuits in prefrontal areas.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data§ Materials and MethodsWe obtained similar results with a more biologically plausible stochastic gradient learning approach using weight perturbations modified from ref. 57.
“To solve Eq. 1 , we used the nonbiological MATLABleft division operation for matrices, but we obtained similar results with a more biologically plausible stochastic gradient learning approach using weight perturbations modified from ref. 57, i.e., ( x i , y i ) pairs, where x i denotes the i th image in the training set and y i is its associated label (animal or non-animal).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [57] Sutton R, Barto A (1981) Psychol Rev 88:135-170. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown data§ Materials and MethodsThe error bars for the model in Fig. 3 correspond to the standard errors computed over n = 20 random runs.
“Note that the error bars for the model in Fig. 3 correspond to the standard errors computed over these n = 20 random runs.”
Quote in the paperNot a substantial claim
The authors state this as an assumption.
One of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.
“The theory assumes that one of the main functions of the ventral stream, just a part of the visual cortex, is to achieve a tradeoff between selectivity and invariance within a hierarchical architecture.”
The authors state this as an assumption.
The hierarchy of visual areas along the ventral stream of the visual cortex, from V1 to IT, builds a generic dictionary of shape-tuned units which provides a rich representation for task-specific categorization circuits in prefrontal areas.
“A key assumption in the new model is that the hierarchy of visual areas along the ventral stream of the visual cortex, from V1 to IT, builds a generic dictionary of shapetuned units which provides a rich representation for task-specific categorization circuits in prefrontal areas.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In I-theory, instead of training via gradient descent, image templates are stored in the weights of simple-complex cell networks while objects undergo transformations, similar to the use of stored templates in HMAX.
“Instead of training via gradient descent, image templates are stored in the weights of simple-complex cell networks while objects undergo transformations, similar to the use of stored templates in HMAX (Serre et al., 2007).”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Serre et al., 2007
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Advances in neural information processing systems 7
Cios, K. J., & Shields, M. E. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,089 · see citing works
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 19,308 · see citing works
A feature-integration theory of attention.
Treisman, A. M., & Gelade, G. (1980) · Cognitive Psychology · doi:10.1016/0010-0285(80)90005-5 · Cited by 12,366 · see citing works
The VideoToolbox software for visual psychophysics: transforming numbers into movies
Pelli, D. G. (1997) · Spatial Vision · doi:10.1163/156856897x00366 · Cited by 11,110 · see citing works
Distributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783 · see citing works
Receptive fields and functional architecture of monkey striate cortex
Hubel, D. H., & Wiesel, T. N. (1968) · The Journal of Physiology · doi:10.1113/jphysiol.1968.sp008455 · Cited by 6,520 · see citing works
Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759 · see citing works
Recognition-by-components: A theory of human image understanding.
Biederman, I. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.115 · Cited by 5,621 · see citing works
Detection Theory: A User's Guide
Macmillan, N. A., & Creelman, C. D. (1991) · Cited by 5,415 · see citing works
Ingle, D., Goodale, M. A., & Mansfield, R. J. W. (1982) · Medical Entomology and Zoology · Cited by 4,131 · see citing works
Speed of processing in the human visual system
Thorpe, S., Fize, D., & Marlot, C. (1996) · Nature · doi:10.1038/381520a0 · Cited by 3,538 · see citing works
Preface: Cerebral Cortex Has Come of Age
Goldman, P. S., & RAKIC, P. T. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1-a · Cited by 3,412 · see citing works
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309 · see citing works
Self-citationA Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., & Lugosi, G. (1996) · Stochastic Modelling and Applied Probability · doi:10.1007/978-1-4612-0711-5 · Cited by 3,274 · see citing works
The distinct modes of vision offered by feedforward and recurrent processing
Lamme, V. A. F., & Roelfsema, P. R. (2000) · Trends in Neurosciences · doi:10.1016/s0166-2236(00)01657-x · Cited by 2,338 · see citing works
Robust Object Recognition with Cortex-Like Mechanisms
Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.56 · Cited by 1,667 · see citing works
Self-citationSpatial frequency selectivity of cells in macaque visual cortex
De Valois, R. L., Albrecht, D. G., & Thorell, L. G. (1982) · Vision Research · doi:10.1016/0042-6989(82)90113-4 · Cited by 1,489 · see citing works
Hierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473 · see citing works
Toward a modern theory of adaptive networks: Expectation and prediction.
Sutton, R. S., & Barto, A. G. (1981) · Psychological Review · doi:10.1037/0033-295x.88.2.135 · Cited by 1,450 · see citing works
Inferotemporal cortex and object vision
Tanaka, K. (1996) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.19.030196.000545 · Cited by 1,402 · see citing works
Hochstein (2002) · doi:10.1016/s0896-6273(02)01091-7 · Cited by 1,397 · see citing works
Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4
Reynolds, J. H., Chelazzi, L., & Desimone, R. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-05-01736.1999 · Cited by 1,236 · see citing works
Categorical Representation of Visual Stimuli in the Primate Prefrontal Cortex
Freedman, D. J., Riesenhuber, M., Poggio, T., & Miller, E. K. (2001) · Science · doi:10.1126/science.291.5502.312 · Cited by 1,123 · see citing works
Self-citationShape representation in the inferior temporal cortex of monkeys
Logothetis, N. K., Pauls, J., & Poggio, T. (1995) · Current Biology · doi:10.1016/s0960-9822(95)00108-4 · Cited by 963 · see citing works
Self-citationOrganization and functions of cells responsive to faces in the temporal cortex
Perrett, D. I., Hietanen, J. K., Oram, M. W., & Benson, P. J. (1992) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1992.0003 · Cited by 882 · see citing works
Fast Readout of Object Identity from Macaque Inferior Temporal Cortex
Hung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. J. (2005) · Science · doi:10.1126/science.1117593 · Cited by 854 · see citing works
Self-citationStatistics of natural image categories
Torralba, A. B., & Oliva, A. (2003) · Network Computation in Neural Systems · doi:10.1088/0954-898x_14_3_302 · Cited by 845 · see citing works
Self-citationObject-based attention in the primary visual cortex of the macaque monkey
Roelfsema, P. R., Lamme, V. A. F., & Spekreijse, H. (1998) · Nature · doi:10.1038/26475 · Cited by 814 · see citing works
Kobatake, E., & Tanaka, K. (1994) · Journal of Neurophysiology · doi:10.1152/jn.1994.71.3.856 · Cited by 807 · see citing works
From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition
Schyns, P. G., & Oliva, A. (1994) · Psychological Science · doi:10.1111/j.1467-9280.1994.tb00500.x · Cited by 715 · see citing works
Self-citationEnns, J. T., & Di Lollo, V. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01520-5 · Cited by 684 · see citing works
Visual features of intermediate complexity and their use in classification
Ullman, S., Vidal-Naquet, M., & Sali, E. (2002) · Nature Neuroscience · doi:10.1038/nn870 · Cited by 600 · see citing works
Potter, M. C. (1975) · Science · doi:10.1126/science.1145183 · Cited by 594 · see citing works
Face-Selective Cells in the Temporal Cortex of Monkeys
Desimone, R. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.1 · Cited by 565 · see citing works
INVARIANT FACE AND OBJECT RECOGNITION IN THE VISUAL SYSTEM
Wallis, G. M., & Rolls, E. T. (1997) · Progress in Neurobiology · doi:10.1016/s0301-0082(96)00054-8 · Cited by 539 · see citing works
Shape Representation in Area V4: Position-Specific Tuning for Boundary Conformation
Pasupathy, A., & Connor, C. E. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.86.5.2505 · Cited by 483 · see citing works
Schiller, P. H., Finlay, B. L., & Volman, S. F. (1976) · Journal of Neurophysiology · doi:10.1152/jn.1976.39.6.1288 · Cited by 458 · see citing works
Visual Masking: Time Slices through Conscious and Unconscious Vision
Breitmeyer, B. G., & Öğmen, H. (2006) · Medical Entomology and Zoology · Cited by 396 · see citing works
Preattentive Object Files: Shapeless Bundles of Basic Features
WOLFE, J. M., & BENNETT, S. C. (1997) · Vision Research · doi:10.1016/s0042-6989(96)00111-3 · Cited by 349 · see citing works
Depth estimation from image structure
Torralba, A. B., & Oliva, A. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1033214 · Cited by 341 · see citing works
Self-citationMel, B. W. (1997) · Neural Computation · doi:10.1162/neco.1997.9.4.777 · Cited by 341 · see citing works
Breitmeyer, B., & Ogmen, H. (2006) · doi:10.1093/acprof:oso/9780198530671.001.0001 · Cited by 313 · see citing works
Seeking Categories in the Brain
Thorpe, S. J., & Fabre-Thorpe, M. (2001) · Science · doi:10.1126/science.1058249 · Cited by 300 · see citing works
Masking Interrupts Figure-Ground Signals in V1
Lamme, V. A. F., Zipser, K., & Spekreijse, H. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902320474490 · Cited by 285 · see citing works
Perception of Objects in Natural Scenes: Is It Really Attention Free?
Evans, K. K., & Treisman, A. (2005) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.31.6.1476 · Cited by 281 · see citing works
Figure—Ground Segregation in a Recurrent Network Architecture
Roelfsema, P. R., Lamme, V. A. F., Spekreijse, H., & Bosch, H. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/08989290260045756 · Cited by 254 · see citing works
Serre, T., Kouh, M., Cadieu, C. F., Knoblich, U., Kreiman, G., & Poggio, T. (2005) · Defense Technical Information Center (DTIC) · Cited by 245 · see citing works
Self-citationExperience-dependent sharpening of visual shape selectivity in inferior temporal cortex.
Freedman, D. J., Riesenhuber, M., Poggio, T., & Miller, E. (2005) · Cerebral Cortex · doi:10.1093/cercor/bhj100 · Cited by 241 · see citing works
Self-citationThe time course of visual processing: Backward masking and natural scene categorisation
Bacon-Macé, N., Macé, M. J.-M., Fabre-Thorpe, M., & Thorpe, S. J. (2005) · Vision Research · doi:10.1016/j.visres.2005.01.004 · Cited by 236 · see citing works
Biologically Motivated Computer Vision
Bülthoff, H., Poggio, T., Lee, S.-W., & Wallraven, C. (2002) · Lecture notes in computer science · doi:10.1007/3-540-36181-2 · Cited by 223 · see citing works
Self-citationIs it an animal? Is it a human face? Fast processing in upright and inverted natural scenes
Rousselet, G. A., Macé, M. J.-M., & Fabre-Thorpe, M. (2003) · Journal of Vision · doi:10.1167/3.6.5 · Cited by 215 · see citing works
Neurophysiology of shape processing
Perrett, D. I., & Oram, M. W. (1993) · Image and Vision Computing · doi:10.1016/0262-8856(93)90011-5 · Cited by 213 · see citing works
The modular organization of projections from areas V1 and V2 to areas V4 and TEO in macaques
Nakamura, H., Gattass, R., Desimone, R., & Ungerleider, L. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-09-03681.1993 · Cited by 211 · see citing works
Learning optimized features for hierarchical models of invariant object recognition.
Wersing, H., & Körner, E. (2003) · Neural Computation · doi:10.1162/089976603321891800 · Cited by 169 · see citing works
The implementation of visual routines
Roelfsema, P. R., Lamme, V. A. F., & Spekreijse, H. (2000) · Vision Research · doi:10.1016/s0042-6989(00)00004-3 · Cited by 162 · see citing works
Visual Selective Behavior Can Be Triggered by a Feed-Forward Process
VanRullen, R., & Koch, C. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321208141 · Cited by 140 · see citing works
Lampl, I., Ferster, D., Poggio, T., & Riesenhuber, M. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00060.2004 · Cited by 139 · see citing works
Self-citationResponses of Primate Visual Cortical V4 Neurons to Simultaneously Presented Stimuli
Gawne, T. J., & Martin, J. M. (2002) · Journal of Neurophysiology · doi:10.1152/jn.2002.88.3.1128 · Cited by 134 · see citing works
Gross, C. G. (1998) · The MIT Press eBooks · doi:10.7551/mitpress/1662.001.0001 · Cited by 100 · see citing works
A history of medicine: primitive and ancient medicine.
Prioreschi, P. (1991) · PubMed · Cited by 57 · see citing works
An integrated network for invariant visual detection and recognition
Amit, Y., & Mascaro, M. (2003) · Vision Research · doi:10.1016/s0042-6989(03)00306-7 · Cited by 52 · see citing works
Hyperfunctioning thyroid nodules.
Laurberg, P., Pedersen, K. M., Vestergaard, P., & Vestergaard, H. (1991) · PubMed · Cited by 21 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- DH Hubel, TN Wiesel J Phys 195, 215–243 (1968).
- T Serre, L Wolf, T Poggio Proc IEEE Conf Comput Vision Pattern Recognit 2, 994–1000 (2005).
- T Serre (MIT Cambridge MA PhD thesis. (2006).
- D Freedman, M Riesenhuber, T Poggio, E Miller Cereb Cortex, in press. (2006).
Cited by
Sorted by citations
Representational similarity analysis-connecting the branches of systems neuroscience
Kriegeskorte, N. (2008) · Frontiers in Systems Neuroscience · doi:10.3389/neuro.06.004.2008 · Cited by 3,355 · see citing works
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924 · see citing works
How does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790 · see citing works
Pessoa, L., & Adolphs, R. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2920 · Cited by 1,794 · see citing works
Using goal-driven deep learning models to understand sensory cortex
Yamins, D. L. K., & DiCarlo, J. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4244 · Cited by 1,621 · see citing works
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204 · see citing works
Pros and cons of GAN evaluation measures
Borji, A. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.10.009 · Cited by 963 · see citing works
Statistically optimal perception and learning: from behavior to neural representations
Fiser, J., Berkes, P., Orbán, G., & Lengyel, M. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.01.003 · Cited by 672 · see citing works
Can structure predict function in the human brain?
Honey, C. J., Thivierge, J.-P., & Sporns, O. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.01.071 · Cited by 629 · see citing works
GREENE, M., & OLIVA, A. (2009) · Cognitive Psychology · doi:10.1016/j.cogpsych.2008.06.001 · Cited by 467 · see citing works
Self-citationvan Kerkoerle, T., Self, M. W., & Roelfsema, P. R. (2017) · Nature Communications · doi:10.1038/ncomms13804 · Cited by 281 · see citing works
Linde-Domingo, J., Treder, M. S., Kerrén, C., & Wimber, M. (2019) · Nature Communications · doi:10.1038/s41467-018-08080-2 · Cited by 152 · see citing works
Prediction, Bayesian inference and feedback in speech recognition
Norris, D., McQueen, J. M., & Cutler, A. (2016) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2015.1081703 · Cited by 170 · see citing works
Performance vs. competence in human–machine comparisons
Firestone, C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905334117 · Cited by 150 · see citing works
A review of rapid serial visual presentation-based brain–computer interfaces
Lees, S., Dayan, N., Cecotti, H., McCullagh, P., Maguire, L., Lotte, F., & Coyle, D. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aa9817 · Cited by 149 · see citing works
Decoding the yellow of a gray banana
Bannert, M. M., & Bartels, A. (2013) · Current Biology · doi:10.1016/j.cub.2013.09.016 · Cited by 146 · see citing works
A social Bayesian brain: how social knowledge can shape visual perception
Otten, M., Seth, A. K., & Pinto, Y. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2016.05.002 · Cited by 124 · see citing works
Prior object-knowledge sharpens properties of early visual feature-detectors
Teufel, C., Dakin, S. C., & Fletcher, P. C. (2018) · Scientific Reports · doi:10.1038/s41598-018-28845-5 · Cited by 119 · see citing works
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119 · see citing works
Towards a Unified View on Pathways and Functions of Neural Recurrent Processing
Pennartz, C. M. A., Dora, S., Muckli, L., & Lorteije, J. A. M. (2019) · Trends in Neurosciences · doi:10.1016/j.tins.2019.07.005 · Cited by 73 · see citing works
“In addition to engaging in feedforward processing [1], primary sensory cortices receive feedback from higher cortical areas and thus partake in recurrent processing (RP) (see Glossary) [2,3], which may serve multiple functions (Figure 1).”
Computational mechanisms underlying cortical responses to the affordance properties of visual scenes
Bonner, M. F., & Epstein, R. A. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006111 · Cited by 94 · see citing works
Unsupervised learning of invariant representations
Anselmi, F., Leibo, J. Z., Rosasco, L., Mutch, J., Tacchetti, A., & Poggio, T. (2016) · Theoretical Computer Science · doi:10.1016/j.tcs.2015.06.048 · Cited by 88 · see citing works
Self-citationParkinson disease psychosis: from phenomenology to neurobiological mechanisms
Pagonabarraga, J., Bejr-Kasem, H., Martinez-Horta, S., & Kulisevsky, J. (2024) · Nature Reviews Neurology · doi:10.1038/s41582-023-00918-8 · Cited by 69 · see citing works
Higher performers upregulate brain signal variability in response to more feature-rich visual input
Garrett, D. D., Epp, S. M., Kleemeyer, M., Lindenberger, U., & Polk, T. A. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116836 · Cited by 48 · see citing works
Model metamers reveal divergent invariances between biological and artificial neural networks
Feather, J., Leclerc, G., Mądry, A., & McDermott, J. H. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01442-0 · Cited by 59 · see citing works
Generative models for active vision
Parr, T., Sajid, N., Da Costa, L., Mirza, M. B., & Friston, K. J. (2021) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2021.651432 · Cited by 46 · see citing works
Hierarchy of speech-driven spectrotemporal receptive fields in human auditory cortex
Venezia, J. H., Thurman, S. M., Richards, V. M., & Hickok, G. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.11.049 · Cited by 47 · see citing works
The brain of the beholder: honouring individual representational idiosyncrasies
Charest, I., & Kriegeskorte, N. (2015) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2014.1002505 · Cited by 27 · see citing works
Predictive coding with spiking neurons and feedforward gist signaling
Lee, K., Dora, S., Mejias, J. F., Bohte, S. M., & Pennartz, C. M. A. (2024) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2024.1338280 · Cited by 14 · see citing works
- Authors
- Serre, T., Oliva, A., & Poggio, T.
- Affiliations
- McGovern Institute for Brain Research (US)
- Massachusetts Institute of Technology (US)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2007-04-10
- Volume
- 104
- Issue
- 15
- Pages
- 6424-6429
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC1847457
- OpenAlex
- W2132172482
- Semantic Scholar
- 3ffed16d381216f976b193ae58c084598272697f
- Cited by
- 918
- References
- 63
- Citations by year
- 2026: 20 · 2025: 30 · 2024: 22 · 2023: 33 · 2022: 37 · 2021: 52 · 2020: 49 · 2019: 48 · 2018: 50 · 2017: 50 · 2016: 64 · 2015: 70 · 2014: 61 · 2013: 70 · 2012: 59
- FWCI
- 24.61
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Science Foundation (0546262, 0218506)
- Eugene McDermott Foundation
- Toyota Motor Corporation
- National Institutes of Health
- Advanced Research Projects Agency
- National Institute of Mental Health (1 P20MH66239-01A1, P20MH066239)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- feedforward architecture, rapid serial visual presentation, primate visual system, Hubel and Wiesel
- MeSH terms
- Adult, Humans, Models, Neurological, Pattern Recognition, Visual, Photic Stimulation, Psychophysics, Visual Perception, Recognition, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
