Using goal-driven deep learning models to understand sensory cortex
Abstract
Fueled by innovation in the computer vision and artificial intelligence communities, recent developments in computational neuroscience have used goal-driven hierarchical convolutional neural networks (HCNNs) to make strides in modeling neural single-unit and population responses in higher visual cortical areas. In this Perspective, we review the recent progress in a broader modeling context and describe some of the key technical innovations that have supported it. We then outline how the goal-driven HCNN approach can be used to delve even more deeply into understanding the development and organization of sensory cortical processing.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A shortcoming of the DNN approach is that it does not predict how an individual neuron causally contributes to behaviour.
“A shortcoming of this approach is that the DNN does not predict how an individual neuron causally contributes to behaviour, making it difficult to interpret the role of the neuron in the sensorimotor transformation.”
From Mapping model units to visual neurons reveals population code for social behaviour (2024) · cites this paper as [1]
This is a stronger criterion than typical evaluations of DNN models and neural activity, where a linear mapping from DNN features (~10,000 feature variables) to neural responses is fit.
“This is a stronger criterion than typical evaluations of DNN models and neural activity, where a linear mapping from DNN features (~10,000 feature variables) to neural responses is fit1.”
From Mapping model units to visual neurons reveals population code for social behaviour (2024) · cites this paper as [1]
Computational models that mimic brain information processing during perceptual, cognitive and control tasks are beginning to be developed and tested with brain and behavioral data.
“Computational models that mimic brain information processing during perceptual, cognitive and control tasks are beginning to be developed and tested with brain and behavioral data.”
From Cognitive computational neuroscience (2018) · cites this paper as [38]
Computational neuroscience has begun to test complex computational models that can explain high-level sensory and cognitive brain representations.
“Computational neuroscience has also begun to test complex computational models that can explain high-level sensory and cognitive brain representations37,38.”
From Cognitive computational neuroscience (2018) · cites this paper as [38]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
Distributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783 · see citing works
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 7,048 · see citing works
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691 · see citing works
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795 · see citing works
Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759 · see citing works
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · see citing works
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 5,037 · see citing works
Receptive fields of single neurones in the cat's striate cortex
Hubel, D. H., & Wiesel, T. N. (1959) · The Journal of Physiology · doi:10.1113/jphysiol.1959.sp006308 · Cited by 4,653 · see citing works
Convolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL · doi:10.5555/303568.303704 · Cited by 4,637 · see citing works
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451 · see citing works
Preface: Cerebral Cortex Has Come of Age
Goldman, P. S., & RAKIC, P. T. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1-a · Cited by 3,412 · see citing works
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309 · see citing works
Visual Feature Integration and the Temporal Correlation Hypothesis
Singer, W., & Gray, C. M. (1995) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.18.030195.003011 · Cited by 3,157 · see citing works
(1961) · Information and Control · doi:10.1016/s0019-9958(61)80064-8 · Cited by 2,904 · see citing works
(2007) · CaltechAUTHORS (California Institute of Technology) · Cited by 2,274 · see citing works
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924 · see citing works
Self-citationNormalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906 · see citing works
Bergstra, J., Yamins, D. L. K., & Cox, D. (2013) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 1,817 · see citing works
Self-citationContext-dependent computation by recurrent dynamics in prefrontal cortex
Mante, V., Sussillo, D., Shenoy, K. V., & Newsome, W. T. (2013) · Nature · doi:10.1038/nature12742 · Cited by 1,807 · see citing works
How does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790 · see citing works
Self-citationThe Unreasonable Effectiveness of Data
Halevy, A. Y., Norvig, P., & Pereira, F. M. B. (2009) · IEEE Intelligent Systems · doi:10.1109/mis.2009.36 · Cited by 1,776 · see citing works
Jones, J. P., & Palmer, L. A. (1987) · Journal of Neurophysiology · doi:10.1152/jn.1987.58.6.1233 · Cited by 1,658 · see citing works
Possible Principles Underlying the Transformations of Sensory Messages
Barlow, H. B. (2012) · The MIT Press eBooks · doi:10.7551/mitpress/9780262518420.003.0013 · Cited by 1,318 · see citing works
Spatio-temporal correlations and visual signalling in a complete neuronal population
Pillow, J. W., Shlens, J., Paninski, L., Sher, A., Litke, A. M., Chichilnisky, E. J., & Simoncelli, E. P. (2008) · Nature · doi:10.1038/nature07140 · Cited by 1,315 · see citing works
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204 · see citing works
Categorical Representation of Visual Stimuli in the Primate Prefrontal Cortex
Freedman, D. J., Riesenhuber, M., Poggio, T., & Miller, E. K. (2001) · Science · doi:10.1126/science.291.5502.312 · Cited by 1,123 · see citing works
The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070 · see citing works
Ullman, S. (1984) · Cognition · doi:10.1016/0010-0277(84)90023-4 · Cited by 1,049 · see citing works
Shape representation in the inferior temporal cortex of monkeys
Logothetis, N. K., Pauls, J., & Poggio, T. (1995) · Current Biology · doi:10.1016/s0960-9822(95)00108-4 · Cited by 963 · see citing works
Guclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961 · see citing works
Untangling invariant object recognition
DiCarlo, J. J., & Cox, D. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.06.010 · Cited by 948 · see citing works
Self-citationA feedforward architecture accounts for rapid categorization
Serre, T., Oliva, A., & Poggio, T. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0700622104 · Cited by 918 · see citing works
Choice-specific sequences in parietal cortex during a virtual-navigation decision task
Harvey, C. D., Coen, P., & Tank, D. W. (2012) · Nature · doi:10.1038/nature10918 · Cited by 916 · see citing works
Spatial summation in the receptive fields of simple cells in the cat's striate cortex.
Movshon, J. A., Thompson, I. D., & Tolhurst, D. J. (1978) · The Journal of Physiology · doi:10.1113/jphysiol.1978.sp012488 · Cited by 890 · see citing works
Deep Learning of Representations for Unsupervised and Transfer Learning.
Bengio, Y. (2011) · Cited by 885 · see citing works
Fast Readout of Object Identity from Macaque Inferior Temporal Cortex
Hung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. J. (2005) · Science · doi:10.1126/science.1117593 · Cited by 854 · see citing works
Self-citationCadieu, C. F., Hong, H., Yamins, D. L. K., Pinto, N., Ardila, D., Solomon, E. A., Majaj, N. J., & DiCarlo, J. J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003963 · Cited by 824 · see citing works
Self-citationMetamers of the ventral stream
Freeman, J., & Simoncelli, E. P. (2011) · Nature Neuroscience · doi:10.1038/nn.2889 · Cited by 667 · see citing works
Do We Know What the Early Visual System Does?
Carandini, M., Demb, J. B., Mante, V., Tolhurst, D. J., Dan, Y., Olshausen, B. A., Gallant, J. L., & Rust, N. C. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.3726-05.2005 · Cited by 586 · see citing works
Population coding of shape in area V4
Pasupathy, A., & Connor, C. E. (2002) · Nature Neuroscience · doi:10.1038/972 · Cited by 552 · see citing works
Incentivizing Exploration In Reinforcement Learning With Deep Predictive ModelsPreprint
Stadie, B. C., Levine, S., & Abbeel, P. (2015) · arXiv.org · doi:10.48550/arxiv.1507.00814 · Cited by 545 · see citing works
Why is Real-World Visual Object Recognition Hard?
Pinto, N., Cox, D., & DiCarlo, J. J. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040027 · Cited by 537 · see citing works
Self-citationHow MT cells analyze the motion of visual patterns
Rust, N. C., Mante, V., Simoncelli, E. P., & Movshon, J. A. (2006) · Nature Neuroscience · doi:10.1038/nn1786 · Cited by 516 · see citing works
The topography of high-order human object areas
Malach, R., Levy, I., & Hasson, U. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(02)01870-3 · Cited by 459 · see citing works
Neural responses to polar, hyperbolic, and Cartesian gratings in area V4 of the macaque monkey
Gallant, J. L., Connor, C. E., Rakshit, S., Lewis, J. W., & Essen, D. C. V. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.76.4.2718 · Cited by 443 · see citing works
Neural Representation and the Cortical Code
deCharms, R. C., & Zador, A. M. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.613 · Cited by 392 · see citing works
Rust, N. C., & DiCarlo, J. J. (2010) · Journal of Neuroscience · doi:10.1523/jneurosci.0179-10.2010 · Cited by 388 · see citing works
Self-citationA neural code for three-dimensional object shape in macaque inferotemporal cortex
Yamane, Y., Carlson, E. T., Bowman, K. C., Wang, Z., & Connor, C. E. (2008) · Nature Neuroscience · doi:10.1038/nn.2202 · Cited by 291 · see citing works
Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?
Khorrami, P., Paine, T. L., & Huang, T. S. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.12 · Cited by 285 · see citing works
Pinto, N., Doukhan, D., DiCarlo, J. J., & Cox, D. D. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000579 · Cited by 248 · see citing works
Self-citationResponses of striate cortex cells to grating and checkerboard patterns.
Valois, K. K. D., Valois, R. L. D., & Yund, E. W. (1979) · The Journal of Physiology · doi:10.1113/jphysiol.1979.sp012827 · Cited by 217 · see citing works
Majaj, N. J., Hong, H., Solomon, E. A., & DiCarlo, J. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5181-14.2015 · Cited by 193 · see citing works
Self-citationTrade-Off between Object Selectivity and Tolerance in Monkey Inferotemporal Cortex
Zoccolan, D., Kouh, M., Poggio, T., & DiCarlo, J. J. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.1897-07.2007 · Cited by 169 · see citing works
Self-citationAfraz, A., Boyden, E. S., & DiCarlo, J. J. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1423328112 · Cited by 165 · see citing works
Self-citationHulbert, J. C., & Norman, K. A. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhu284 · Cited by 147 · see citing works
Spectral Receptive Field Properties Explain Shape Selectivity in Area V4
David, S. V., Hayden, B. Y., & Gallant, J. L. (2006) · Journal of Neurophysiology · doi:10.1152/jn.00575.2006 · Cited by 133 · see citing works
Hoyle, G. (1984) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0001863x · Cited by 129 · see citing works
Signals in inferotemporal and perirhinal cortex suggest an untangling of visual target information
Pagan, M., Urban, L. S., Wohl, M. P., & Rust, N. C. (2013) · Nature Neuroscience · doi:10.1038/nn.3433 · Cited by 125 · see citing works
Yamins, D. L. K., Hong, H., Cadieu, C. F., & DiCarlo, J. J. (2013) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 117 · see citing works
Self-citationZhu, L., Lin, C., Huang, H., Chen, Y., & Yuille, A. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88688-4_56 · Cited by 94 · see citing works
Relating population-code representations between man, monkey, and computational models
Kriegeskorte, N. (2009) · Frontiers in Neuroscience · doi:10.3389/neuro.01.035.2009 · Cited by 92 · see citing works
Choudhary, S., Sloan, S., Fok, S., Neckar, A., Trautmann, E. M., Gao, P., Stewart, T., Eliasmith, C., & Boahen, K. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33269-2_16 · Cited by 89 · see citing works
Stimulus-invariant processing and spectrotemporal reverse correlation in primary auditory cortex
Klein, D. J., Simon, J. Z., Depireux, D. A., et al. (2006) · Journal of Computational Neuroscience · doi:10.1007/s10827-005-3589-4 · Cited by 67 · see citing works
Curvature Processing Dynamics in Macaque Area V4
Yau, J. M., Pasupathy, A., Brincat, S. L., & Connor, C. E. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhs004 · Cited by 65 · see citing works
Learning Sparse Image Codes using a Wavelet Pyramid Architecture
Olshausen, B. A., Sallee, P. A., & Lewicki, M. S. (2007) · Cited by 59 · see citing works
Trade-off between curvature tuning and position invariance in visual area V4
Sharpee, T. O., Kouh, M., & Reynolds, J. H. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1217479110 · Cited by 57 · see citing works
Understanding Brains: Details, Intuition, and Big Data
Marder, E. E. (2015) · PLoS Biology · doi:10.1371/journal.pbio.1002147 · Cited by 32 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623 · see citing works
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310 · see citing works
Artificial intelligence: A powerful paradigm for scientific research
Xu, Y., Liu, X., Cao, X., Huang, C., Liu, E., Qian, S., Liu, X., Wu, Y., Dong, F., Qiu, C.-W., et al. (2021) · The Innovation · doi:10.1016/j.xinn.2021.100179 · Cited by 1,726 · see citing works
Neuroscience-Inspired Artificial Intelligence
Hassabis, D., Kumaran, D., Summerfield, C., & Botvinick, M. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.011 · Cited by 1,557 · see citing works
What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior
Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018) · Neuron · doi:10.1016/j.neuron.2018.10.002 · Cited by 1,055 · see citing works
A deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027 · see citing works
Self-citationWhat Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated
Kumaran, D., Hassabis, D., & McClelland, J. L. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.05.004 · Cited by 736 · see citing works
Reinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667 · see citing works
Cognitive computational neuroscience
Kriegeskorte, N., & Douglas, P. K. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0210-5 · Cited by 535 · see citing works
Kell, A. J. E., Yamins, D. L. K., Shook, E. N., Norman-Haignere, S. V., & McDermott, J. H. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.044 · Cited by 544 · see citing works
Self-citationIf deep learning is the answer, what is the question?
Saxe, A., Nelli, S., & Summerfield, C. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-00395-8 · Cited by 397 · see citing works
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420 · see citing works
Hulse, B. K., Haberkern, H., Franconville, R., Turner-Evans, D., Takemura, S.-Y., Wolff, T., Noorman, M., Dreher, M., Dan, C., Parekh, R., et al. (2021) · eLife · doi:10.7554/elife.66039 · Cited by 392 · see citing works
Recurrence is required to capture the representational dynamics of the human visual system
Kietzmann, T. C., Spoerer, C. J., Sörensen, L. K. A., Cichy, R. M., Hauk, O., & Kriegeskorte, N. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905544116 · Cited by 406 · see citing works
Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks
Hasson, U., Nastase, S. A., & Goldstein, A. (2020) · Neuron · doi:10.1016/j.neuron.2019.12.002 · Cited by 387 · see citing works
Brains and algorithms partially converge in natural language processing
Caucheteux, C., & King, J.-R. (2022) · Communications Biology · doi:10.1038/s42003-022-03036-1 · Cited by 318 · see citing works
Mechanosensation and Adaptive Motor Control in Insects
Tuthill, J. C., & Wilson, R. I. (2016) · Current Biology · doi:10.1016/j.cub.2016.06.070 · Cited by 290 · see citing works
Deep reinforcement learning and its neuroscientific implications
Botvinick, M., Wang, J. X., Dabney, W., Miller, K. J., & Kurth-Nelson, Z. (2020) · Neuron · doi:10.1016/j.neuron.2020.06.014 · Cited by 269 · see citing works
Quilty-Dunn, J., Porot, N., & Mandelbaum, E. (2023) · Behavioral and Brain Sciences · doi:10.1017/s0140525x22002849 · Cited by 205 · see citing works
Deep neural networks in computational neuroscience
Kietzmann, T. C., McClure, P., & Kriegeskorte, N. (2019) · Oxford Research Encyclopedia of Neuroscience · doi:10.1093/acrefore/9780190264086.013.46 · Cited by 186 · see citing works
Meta-learning in natural and artificial intelligence
Wang, J. X. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2021.01.002 · Cited by 171 · see citing works
High-Level Prediction Signals in a Low-Level Area of the Macaque Face-Processing Hierarchy
Schwiedrzik, C. M., & Freiwald, W. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.09.007 · Cited by 136 · see citing works
Higgins, I., Chang, L., Langston, V., Hassabis, D., Summerfield, C., Tsao, D., & Botvinick, M. (2021) · Nature Communications · doi:10.1038/s41467-021-26751-5 · Cited by 162 · see citing works
Performance vs. competence in human–machine comparisons
Firestone, C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905334117 · Cited by 150 · see citing works
The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence
Watson, D. (2019) · Minds and Machines · doi:10.1007/s11023-019-09506-6 · Cited by 148 · see citing works
Empiricism without magic: transformational abstraction in deep convolutional neural networks
Buckner, C. (2018) · Synthese · doi:10.1007/s11229-018-01949-1 · Cited by 149 · see citing works
Potential downside of high initial visual acuity.
Vogelsang, L., Gilad-Gutnick, S., Ehrenberg, E., Yonas, A., Diamond, S., Held, R., & Sinha, P. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1800901115 · Cited by 130 · see citing works
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119 · see citing works
Deep learning in biomedical data science
Baldi, P. (2018) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-080917-013343 · Cited by 119 · see citing works
Structure learning and the posterior parietal cortex
Summerfield, C., Luyckx, F., & Sheahan, H. (2020) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2019.101717 · Cited by 118 · see citing works
Deep learning and the global workspace theory
VanRullen, R., & Kanai, R. (2021) · Trends in Neurosciences · doi:10.1016/j.tins.2021.04.005 · Cited by 128 · see citing works
Mind Reading and Writing: The Future of Neurotechnology
Roelfsema, P. R., Denys, D., & Klink, P. C. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.04.001 · Cited by 107 · see citing works
Kuzovkin, I., Vicente, R., Petton, M., Lachaux, J.-P., Baciu, M., Kahane, P., Rheims, S., Vidal, J. R., & Aru, J. (2018) · Communications Biology · doi:10.1038/s42003-018-0110-y · Cited by 104 · see citing works
A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping
Michaels, J. A., Schaffelhofer, S., Agudelo-Toro, A., & Scherberger, H. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2005087117 · Cited by 103 · see citing works
Conwell, C., Prince, J. S., Kay, K. N., Alvarez, G. A., & Konkle, T. (2024) · Nature Communications · doi:10.1038/s41467-024-53147-y · Cited by 100 · see citing works
Deep language algorithms predict semantic comprehension from brain activity
Caucheteux, C., Gramfort, A., & King, J.-R. (2022) · Scientific Reports · doi:10.1038/s41598-022-20460-9 · Cited by 91 · see citing works
Computational mechanisms underlying cortical responses to the affordance properties of visual scenes
Bonner, M. F., & Epstein, R. A. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006111 · Cited by 94 · see citing works
Deep Neural Networks for Modeling Visual Perceptual Learning
Wenliang, L. K., & Seitz, A. R. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.1620-17.2018 · Cited by 83 · see citing works
Abstract representations emerge naturally in neural networks trained to perform multiple tasks
Johnston, W. J., & Fusi, S. (2023) · Nature Communications · doi:10.1038/s41467-023-36583-0 · Cited by 89 · see citing works
Inferring neural activity before plasticity as a foundation for learning beyond backpropagation
Song, Y., Millidge, B., Salvatori, T., Lukasiewicz, T., Xu, Z., & Bogacz, R. (2024) · Nature Neuroscience · doi:10.1038/s41593-023-01514-1 · Cited by 85 · see citing works
How deep is the brain? The shallow brain hypothesis
Suzuki, M., Pennartz, C. M. A., & Aru, J. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00756-z · Cited by 79 · see citing works
Neural representational geometry underlies few-shot concept learning
Sorscher, B., Ganguli, S., & Sompolinsky, H. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2200800119 · Cited by 78 · see citing works
Generating meaning: active inference and the scope and limits of passive AI
Pezzulo, G., Parr, T., Cisek, P., Clark, A., & Friston, K. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.10.002 · Cited by 80 · see citing works
Cognition as a Window into Neuronal Population Space
Ruff, D. A., Ni, A. M., & Cohen, M. R. (2018) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-080317-061936 · Cited by 63 · see citing works
Lessons from infant learning for unsupervised machine learning
Zaadnoordijk, L., Besold, T. R., & Cusack, R. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00488-2 · Cited by 65 · see citing works
Tasks and their role in visual neuroscience
Kay, K., Bonnen, K., Denison, R. N., Arcaro, M. J., & Barack, D. L. (2023) · Neuron · doi:10.1016/j.neuron.2023.03.022 · Cited by 61 · see citing works
Multiple realizability and the spirit of functionalism
Cao, R. (2022) · Synthese · doi:10.1007/s11229-022-03524-1 · Cited by 58 · see citing works
Unveiling functions of the visual cortex using task-specific deep neural networks
Dwivedi, K., Bonner, M. F., Cichy, R. M., & Roig, G. (2021) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009267 · Cited by 52 · see citing works
Evolving the olfactory system with machine learning
Wang, P. Y., Sun, Y., Axel, R., Abbott, L. F., & Yang, G. R. (2021) · Neuron · doi:10.1016/j.neuron.2021.09.010 · Cited by 54 · see citing works
A model of egocentric to allocentric understanding in mammalian brains
Uria, B., Ibarz, B., Banino, A., Zambaldi, V., Kumaran, D., Hassabis, D., Barry, C., & Blundell, C. (2020) · doi:10.1101/2020.11.11.378141 · Cited by 48 · see citing works
Striatal action-value neurons reconsidered
Elber-Dorozko, L., & Loewenstein, Y. (2018) · eLife · doi:10.7554/elife.34248 · Cited by 53 · see citing works
Hulse, B. K., Haberkern, H., Franconville, R., Turner-Evans, D. B., Takemura, S., Wolff, T., Noorman, M., Dreher, M., Dan, C., Parekh, R., et al. (2020) · doi:10.1101/2020.12.08.413955 · Cited by 392 · see citing works
“…learning, a broader question concerns the level of understanding that is appropriate or even possible for the function of large and complex neural networks (Gao and Ganguli, 2015; Hasson et al., 2020; Lillicrap and Kording, 2019; Richards et al., 2019; Saxe et al., 2020; Yamins and DiCarlo, 2016).”
BackgroundMapping model units to visual neurons reveals population code for social behaviour
Cowley, B. R., Calhoun, A. J., Rangarajan, N., Ireland, E., Turner, M. H., Pillow, J. W., & Murthy, M. (2024) · Nature · doi:10.1038/s41586-024-07451-8 · Cited by 45 · see citing works
“Because DNNs are optimized via stochastic gradient descent, the training procedure of a DNN is not deterministic; different random initializations and different orderings of the training data may lead to DNNs with different prediction performances.”
BackgroundMethodInfluentialLearning probabilistic neural representations with randomly connected circuits
Maoz, O., Tkačik, G., Esteki, M. S., Kiani, R., & Schneidman, E. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912804117 · Cited by 44 · see citing works
Discovering cognitive strategies with tiny recurrent neural networks
Ji-An, L., Benna, M. K., & Mattar, M. G. (2025) · Nature · doi:10.1038/s41586-025-09142-4 · Cited by 64 · see citing works
Lappalainen, J. K., Tschopp, F. D., Prakhya, S., McGill, M., Nern, A., Shinomiya, K., Takemura, S.-Y., Gruntman, E., Macke, J. H., & Turaga, S. C. (2023) · doi:10.1101/2023.03.11.532232 · Cited by 31 · see citing works
Predicting individual neuron responses with anatomically constrained task optimization
Mano, O., Creamer, M. S., Badwan, B. A., & Clark, D. A. (2021) · Current Biology · doi:10.1016/j.cub.2021.06.090 · Cited by 24 · see citing works
The memory systems of the human brain and generative artificial intelligence
Rolls, E. T. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e31965 · Cited by 23 · see citing works
Universal dimensions of visual representation
Chen, Z., & Bonner, M. F. (2025) · Science Advances · doi:10.1126/sciadv.adw7697 · Cited by 16 · see citing works
A machine learning framework to optimize optic nerve electrical stimulation for vision restoration
Romeni, S., Zoccolan, D., & Micera, S. (2021) · Patterns · doi:10.1016/j.patter.2021.100286 · Cited by 17 · see citing works
Evolution of Brains and Computers: The Roads Not Taken
Solé, R., & Seoane, L. F. (2022) · Entropy · doi:10.3390/e24050665 · Cited by 14 · see citing works
Rudroff, T., Rainio, O., & Klén, R. (2024) · Brain Sciences · doi:10.3390/brainsci14111111 · Cited by 11 · see citing works
Universality of representation in biological and artificial neural networks
Hosseini, E., Casto, C., Zaslavsky, N., Conwell, C., Richardson, M., & Fedorenko, E. (2024) · doi:10.1101/2024.12.26.629294 · Cited by 10 · see citing works
Generating Meaning: Active Inference and the Scope and Limits of Passive AI
Pezzulo, G., Parr, T., Cisek, P., Clark, A., & Friston, K. (2023) · doi:10.31234/osf.io/8xgzv · Cited by 8 · see citing works
Better artificial intelligence does not mean better models of biology
Linsley, D., Feng, P., & Serre, T. (2026) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.11.016 · Cited by 7 · see citing works
A 7T fMRI dataset of synthetic images for out-of-distribution modeling of vision
Gifford, A. T., Cichy, R. M., Naselaris, T., & Kay, K. (2026) · Nature Communications · doi:10.1038/s41467-026-69345-9 · Cited by 6 · see citing works
DeKraker, J., Eichert, N., Hong, S.-J., Evans, A. C., & Bernhardt, B. C. (2026) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2026.106890 · Cited by 1 · see citing works
An, J., Yang, L., Wang, M., Lu, L., An, W., & Yang, E. (2026) · arXiv · doi:10.48550/arxiv.2603.26323
“As spatial reasoning becomes increasingly embedded in downstream systems, limited insight into internal representations hinders robustness assessment, interpretability, and principled comparison with biological cognition(Yamins and DiCarlo, 2016).”
- Authors
- Yamins, D. L. K., & DiCarlo, J. J.
- Affiliations
- McGovern Institute for Brain Research (US)
- Corresponding author
- Yamins, Daniel L K
- Venue
- Nature Neuroscience
- Published
- 2016-02-23
- Volume
- 19
- Issue
- 3
- Pages
- 356-365
- ISSN
- 1097-6256 (ISSN-L)
- 1546-1726
- OpenAlex
- W2274405424
- Cited by
- 1,621
- References
- 72
- Citations by year
- 2026: 133 · 2025: 193 · 2024: 188 · 2023: 164 · 2022: 167 · 2021: 183 · 2020: 197 · 2019: 173 · 2018: 115 · 2017: 82 · 2016: 26
- FWCI
- 13.86
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Nature Neuroscience (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Keywords
- hierarchical convolutional neural network, sensory cortex, visual cortex, neural population responses, single-unit responses, cortical development, cortical organization, computer vision algorithms
- MeSH terms
- Animals, Goals, Humans, Learning, Models, Neurological, Somatosensory Cortex, Visual Cortex, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review