Using artificial neural networks to ask ‘why’ questions of minds and brains
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Neuroscientists have long characterized the properties and functions of the nervous system, and are increasingly succeeding in answering how brains perform the tasks they do. But the question 'why' brains work the way they do is asked less often. The new ability to optimize artificial neural networks (ANNs) for performance on human-like tasks now enables us to approach these 'why' questions by asking when the properties of networks optimized for a given task mirror the behavioral and neural characteristics of humans performing the same task. Here we highlight the recent success of this strategy in explaining why the visual and auditory systems work the way they do, at both behavioral and neural levels.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
The fusiform face area: a module in human extrastriate cortex specialized for face perception
Kanwisher, N., McDermott, J., & Chun, M. M. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-11-04302.1997 · Cited by 7,738 · cited 1×
“…the way it is, with small regions selective for the speci fi c categories of faces [e.g., occipital face area (OFA), fusiform face area (FFA)] [35], places [parahippocampal place area (PPA)] [36], bodies [extrastriate body area (EBA)] [37], and words [visual word form area (VWFA)] [38]…”
BackgroundSelf-citationFukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759 · cited 1×
“In the fi eld of computational vision, hand-engineered models of visual object recognition have been proposed for decades, but their performance lagged behind human abilities [8 – 10].”
BackgroundRao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 1×
“(A) The illusory motion apparent in the static ‘rotating snake’ image was mirrored in PredNet [82], a network trained to predict the next frame in thousands of video sequences [83], suggesting an account of this illusion in terms of predictive coding [84].”
BackgroundThe distributed human neural system for face perception
Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01482-0 · Cited by 4,891 · cited 1×
“Within the domain of face perception, a classic view [46] holds that while face identity is processed by the ventral pathway (e.”
BackgroundHierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309
A cortical area selective for visual processing of the human body
Downing, P. E., Jiang, Y., Shuman, M., & Kanwisher, N. (2001) · Science · doi:10.1126/science.1063414 · Cited by 2,043
Self-citationCohen, L., Dehaene, S., Naccache, L., Lehéricy, S., Dehaene-Lambertz, G., Hénaff, M.-A., & Michel, F. (2000) · Brain · doi:10.1093/brain/123.2.291 · Cited by 1,954
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Yamins, D. L. K., Hong, H., Cadieu, C. F., Solomon, E. A., Seibert, D., & DiCarlo, J. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1403112111 · Cited by 1,924
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
A deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027 · cited 1×
“ANNs offer a solution to this problem for the case of optimization of minds and brains [7].”
BackgroundGuclu, U., & van Gerven, M. A. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.5023-14.2015 · Cited by 961
Task representations in neural networks trained to perform many cognitive tasks
Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T., & Wang, X.-J. (2019) · Nature Neuroscience · doi:10.1038/s41593-018-0310-2 · Cited by 549
Kell, A. J. E., Yamins, D. L. K., Shook, E. N., Norman-Haignere, S. V., & McDermott, J. H. (2018) · Neuron · doi:10.1016/j.neuron.2018.03.044 · Cited by 544 · cited 6×
“ANNs mirror the functional organization of the early visual cortex [30] and hierarchical organization of auditory cortex [16,17], show layer-wise correspondences between neural responses in primary versus higher-level regions of visual cortex [13 – 15], and brain-like functional differences between…”
InfluentialThe neural architecture of language: Integrative modeling converges on predictive processing
Schrimpf, M., Blank, I. A., Tuckute, G., Kauf, C., Hosseini, E. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2105646118 · Cited by 517
Self-citationKar, K., Kubilius, J., Schmidt, K., Issa, E. B., & DiCarlo, J. J. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0392-5 · Cited by 496 · cited 1×
“(iii) Architecture variations include the number and size of layers in the network, whether the network is purely feedforward or contains skip connections or recurrence [75,76], cell-type variability [77], and wiring costs [39].”
Shared computational principles for language processing in humans and deep language models
Goldstein, A., Zada, Z., Buchnik, E., Schain, M., Price, A., Aubrey, B., Nastase, S. A., Feder, A., Emanuel, D., Cohen, A., et al. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01026-4 · Cited by 447
Recurrence is required to capture the representational dynamics of the human visual system
Kietzmann, T. C., Spoerer, C. J., Sörensen, L. K. A., Cichy, R. M., Hauk, O., & Kriegeskorte, N. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905544116 · Cited by 406 · cited 1×
“(iii) Architecture variations include the number and size of layers in the network, whether the network is purely feedforward or contains skip connections or recurrence [75,76], cell-type variability [77], and wiring costs [39].”
BackgroundBrains and algorithms partially converge in natural language processing
Caucheteux, C., & King, J.-R. (2022) · Communications Biology · doi:10.1038/s42003-022-03036-1 · Cited by 318 · cited 1×
“Opportunities abound for extending this work into new domains, including language [18,19,63], navigation [64–67], motor control [68,69], and higherlevel cognition [70,71].”
BackgroundHiggins, I., Chang, L., Langston, V., Hassabis, D., Summerfield, C., Tsao, D., & Botvinick, M. (2021) · Nature Communications · doi:10.1038/s41467-021-26751-5 · Cited by 162 · cited 1×
“Single unit analyses (not illustrated) in networks and brains reveals the tuning properties of individual units, whether they are selective for meaningful perceptual attributes, forming disentangled representations [86], or exhibit mixed selectivity and distributed representations [87].”
BackgroundAn ecologically motivated image dataset for deep learning yields better models of human vision
Mehrer, J., Spoerer, C. J., Jones, E. C., Kriegeskorte, N., & Kietzmann, T. C. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2011417118 · Cited by 140
Individual differences among deep neural network models
Mehrer, J., Spoerer, C. J., Kriegeskorte, N., & Kietzmann, T. C. (2020) · Nature Communications · doi:10.1038/s41467-020-19632-w · Cited by 133 · cited 1×
“Better yet, researchers should kick the tires on network hyperparameters (e.g., using different random initializations [58]) to test the robustness of the emergence of the particular phenomenon in question.”
BackgroundBrain-like functional specialization emerges spontaneously in deep neural networks
Dobs, K., Martinez, J., Kell, A. J. E., & Kanwisher, N. (2022) · Science Advances · doi:10.1126/sciadv.abl8913 · Cited by 129 · cited 4×
“However, segregation of the highly heterogeneous visual category of food in both networks [44] and brains [72] suggests that this account is incomplete at best.”
BackgroundMethodInfluentialSelf-citationNasr, K., Viswanathan, P., & Nieder, A. (2019) · Science Advances · doi:10.1126/sciadv.aav7903 · Cited by 110 · cited 1×
“Analyses in this vein have revealed the spontaneous emergence of face-selective [88,89] and number-selective units [90,91] in ANNs.”
BackgroundIllusory Motion Reproduced by Deep Neural Networks Trained for Prediction
Watanabe, E., Kitaoka, A., Sakamoto, K., Yasugi, M., & Tanaka, K. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00345 · Cited by 101 · cited 1×
“(A) The illusory motion apparent in the static ‘ rotating snake ’ image was mirrored in PredNet [82], a network trained to predict the next frame in thousands of video sequences [83], suggesting an account of this illusion in terms of predictive coding [84].”
BackgroundRotational dynamics in motor cortex are consistent with a feedback controller
Kalidindi, H. T., Cross, K. P., Lillicrap, T. P., Omrani, M., Falotico, E., Sabes, P. N., & Scott, S. H. (2021) · eLife · doi:10.7554/elife.67256 · Cited by 83
Emergence of Visual Center-Periphery Spatial Organization in Deep Convolutional Neural Networks
Mohsenzadeh, Y., Mullin, C., Lahner, B., & Oliva, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-61409-0 · Cited by 31 · cited 2×
“One such study [42] trained a CNN on object and scene categorization and found that the network organized itself into units with a central image bias and units more selective for the background of images.”
Background
Cited by
Smart waste management: A paradigm shift enabled by artificial intelligence
Olawade, D. B., Fapohunda, O., Wada, O. Z., Usman, S. O., Ige, A. O., Ajisafe, O., & Oladapo, B. I. (2024) · Waste Management Bulletin · doi:10.1016/j.wmb.2024.05.001 · Cited by 320
The neuroconnectionist research programme
Doerig, A., Sommers, R. P., Seeliger, K., Richards, B., Ismael, J., Lindsay, G. W., Kording, K. P., Konkle, T., van Gerven, M. A. J., Kriegeskorte, N., et al. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00705-w · Cited by 233
Conwell, C., Prince, J. S., Kay, K. N., Alvarez, G. A., & Konkle, T. (2022) · doi:10.1101/2022.03.28.485868 · Cited by 160
Conwell, C., Prince, J. S., Kay, K. N., Alvarez, G. A., & Konkle, T. (2024) · Nature Communications · doi:10.1038/s41467-024-53147-y · Cited by 100
Meta-learned models of cognition
Binz, M., Dasgupta, I., Jagadish, A. K., Botvinick, M., Wang, J. X., & Schulz, E. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x23003266 · Cited by 38
Metadata
- Authors
- Kanwisher, N., Khosla, M., & Dobs, K.
- Venue
- Trends in Neurosciences
- Publisher
- Elsevier BV
- Published
- 2023-03
- Volume
- 46
- Issue
- 3
- Pages
- 240-254
- ISSN
- 0166-2236 (ISSN-L)
- OpenAlex
- W4317567565
- Cited by
- 182
- References
- 28
- FWCI
- 4.99
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.