The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Towards deep learning with segregated dendrites
Open accessAbstract
Deep learning has led to significant advances in artificial intelligence, in part, by adopting strategies motivated by neurophysiology. However, it is unclear whether deep learning could occur in the real brain. Here, we show that a deep learning algorithm that utilizes multi-compartment neurons might help us to understand how the neocortex optimizes cost functions. Like neocortical pyramidal neurons, neurons in our model receive sensory information and higher-order feedback in electrotonically segregated compartments. Thanks to this segregation, neurons in different layers of the network can coordinate synaptic weight updates. As a result, the network learns to categorize images better than a single layer network. Furthermore, we show that our algorithm takes advantage of multilayer architectures to identify useful higher-order representations—the hallmark of deep learning. This work demonstrates that deep learning can be achieved using segregated dendritic compartments, which may help to explain the morphology of neocortical pyramidal neurons.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“An additional issue that should be recognized is that the error rates which our network achieved were by no means as low as can be achieved with artificial neural networks, nor at human levels of performance (LeCun et al., 1998; Li et al., 2016).”
BackgroundMethod(2014) · Cited by 33,419
(2008) · Journal of Machine Learning Research · Cited by 33,051
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-citationAdvances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Neuronal Oscillations in Cortical Networks
Buzsáki, G., & Draguhn, A. (2004) · Science · doi:10.1126/science.1099745 · Cited by 6,576
Malenka, R. C., & Bear, M. F. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.012 · Cited by 3,968
(2013) · Cited by 3,740
Synaptic Plasticity and Memory: An Evaluation of the Hypothesis
Martin, S. J., Grimwood, P. D., & Morris, R. G. M. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.649 · Cited by 2,794 · cited 4×
“Hebbian learning models can produce representations that resemble the representations in the real brain (Zylberberg et al., 2011; Leibo et al., 2017) and they are backed up by decades of experimental findings (Malenka and Bear, 2004; Dan and Poo, 2004; Martin et al., 2000).”
BackgroundInfluentialUsing goal-driven deep learning models to understand sensory cortex
Yamins, D. L. K., & DiCarlo, J. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4244 · Cited by 1,621
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · Cognitive Science · doi:10.1111/j.1551-6708.1987.tb00862.x · Cited by 1,471 · cited 3×
“Perhaps the most biologically unrealistic component of backpropagation is the use of non-local “weight-transport” (Figure 1B) (Grossberg, 1987).”
BackgroundPfeffer, C. K., Xue, M., He, M., Huang, Z. J., & Scanziani, M. (2013) · Nature Neuroscience · doi:10.1038/nn.3446 · Cited by 1,353
Top-down influences on visual processing
Gilbert, C. D., & Li, W. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3476 · Cited by 1,224 · cited 2×
“Indeed, framing cortico-cortical feedback as a mechanism to predict or modulate incoming sensory activity is a more common way of viewing feedback signals in the neocortex (Larkum, 2013; Gilbert and Li, 2013; Zhang et al., 2014; Fiser et al., 2016; Leinweber et al., 2017).”
BackgroundDeep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204
A new cellular mechanism for coupling inputs arriving at different cortical layers
Larkum, M. E., Zhu, J. J., & Sakmann, B. (1999) · Nature · doi:10.1038/18686 · Cited by 1,180
Cortical interneurons that specialize in disinhibitory control
Pi, H.-J., Hangya, B., Kvitsiani, D., Sanders, J. I., Huang, Z. J., & Kepecs, A. (2013) · Nature · doi:10.1038/nature12676 · Cited by 1,136 · cited 3×
“Furthermore, distal dendrite targeting neurons can themselves can be rapidly inhibited in response to temporally precise neuromodulatory inputs (Pi et al., 2013; Pfeffer et al., 2013; Karnani et al., 2016; Hangya et al., 2015; Brombas et al., 2014).”
BackgroundSpike Timing-Dependent Plasticity of Neural Circuits
Dan, Y., & Poo, M.-M. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.007 · Cited by 1,040
(2007) · Cited by 937
Cadieu, C. F., Hong, H., Yamins, D. L. K., Pinto, N., Ardila, D., Solomon, E. A., Majaj, N. J., & DiCarlo, J. J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003963 · Cited by 824
Random synaptic feedback weights support error backpropagation for deep learning
Lillicrap, T. P., Cownden, D., Tweed, D. B., & Akerman, C. J. (2016) · Nature Communications · doi:10.1038/ncomms13276 · Cited by 812
Self-citationLong-range and local circuits for top-down modulation of visual cortex processing
Zhang, S., Xu, M., Kamigaki, T., Hoang Do, J. P., Chang, W.-C., Jenvay, S., Miyamichi, K., Luo, L., & Dan, Y. (2014) · Science · doi:10.1126/science.1254126 · Cited by 804
Disynaptic Inhibition between Neocortical Pyramidal Cells Mediated by Martinotti Cells
Silberberg, G., & Markram, H. (2007) · Neuron · doi:10.1016/j.neuron.2007.02.012 · Cited by 768
A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex
Larkum, M. (2013) · Trends in Neurosciences · doi:10.1016/j.tins.2012.11.006 · Cited by 755 · cited 1×
“Indeed, framing cortico-cortical feedback as a mechanism to modulate incoming sensory activity is a more common way of viewing feedback signals in the neocortex (Larkum, 2013; Gilbert and Li, 2013; Zhang et al., 2014; Fiser et al., 2016).”
BackgroundThe recent excitement about neural networks
Crick, F. (1989) · Nature · doi:10.1038/337129a0 · Cited by 714
Behavioral time scale synaptic plasticity underlies CA1 place fields
Bittner, K. C., Milstein, A. D., Grienberger, C., Romani, S., & Magee, J. C. (2017) · Science · doi:10.1126/science.aan3846 · Cited by 707
Synaptic integration in tuft dendrites of layer 5 pyramidal neurons: a new unifying principle
Larkum, M. E., Nevian, T., Sandler, M., Polsky, A., & Schiller, J. (2009) · Science · doi:10.1126/science.1171958 · Cited by 677
Conjunctive input processing drives feature selectivity in hippocampal CA1 neurons
Bittner, K. C., Grienberger, C., Vaidya, S. P., Milstein, A. D., Macklin, J. J., Suh, J., Tonegawa, S., & Magee, J. C. (2015) · Nature Neuroscience · doi:10.1038/nn.4062 · Cited by 540
SciNet: Lessons Learned from Building a Power-efficient Top-20 System and Data Centre
Loken, C., Gruner, D., Groer, L., Peltier, R., Bunn, N., Craig, M., Henriques, T., Dempsey, J., Yu, C.-H., Chen, J., et al. (2010) · Journal of Physics: Conference Series · doi:10.1088/1742-6596/256/1/012026 · Cited by 493
Sjöström, P. J., & Häusser, M. (2006) · Neuron · doi:10.1016/j.neuron.2006.06.017 · Cited by 439
Active cortical dendrites modulate perception
Takahashi, N., Oertner, T. G., Hegemann, P., & Larkum, M. E. (2016) · Science · doi:10.1126/science.aah6066 · Cited by 419
Central Cholinergic Neurons Are Rapidly Recruited by Reinforcement Feedback
Hangya, B., Ranade, S. P., Lorenc, M., & Kepecs, A. (2015) · Cell · doi:10.1016/j.cell.2015.07.057 · Cited by 399
Cortical gamma band synchronization through somatostatin interneurons
Veit, J., Hakim, R., Jadi, M. P., Sejnowski, T. J., & Adesnik, H. (2017) · Nature Neuroscience · doi:10.1038/nn.4562 · Cited by 381
A Sensorimotor Circuit in Mouse Cortex for Visual Flow Predictions
Leinweber, M., Ward, D. R., Sobczak, J. M., Attinger, A., & Keller, G. B. (2017) · Neuron · doi:10.1016/j.neuron.2017.08.036 · Cited by 375
Dendritic encoding of sensory stimuli controlled by deep cortical interneurons
Murayama, M., Pérez-Garci, E., Nevian, T., Bock, T., Senn, W., & Larkum, M. E. (2009) · Nature · doi:10.1038/nature07663 · Cited by 369
A Top-Down Cortical Circuit for Accurate Sensory Perception
Manita, S., Suzuki, T., Homma, C., Matsumoto, T., Odagawa, M., Yamada, K., Ota, K., Matsubara, C., Inutsuka, A., Sato, M., et al. (2015) · Neuron · doi:10.1016/j.neuron.2015.05.006 · Cited by 365
Experience-dependent spatial expectations in mouse visual cortex
Fiser, A., Mahringer, D., Oyibo, H. K., Petersen, A. V., Leinweber, M., & Keller, G. B. (2016) · Nature Neuroscience · doi:10.1038/nn.4385 · Cited by 332
Layer-specific modulation of neocortical dendritic inhibition during active wakefulness
Muñoz, W., Tremblay, R., Levenstein, D., & Rudy, B. (2017) · Science · doi:10.1126/science.aag2599 · Cited by 305
Learning Rules for Spike Timing-Dependent Plasticity Depend on Dendritic Synapse Location
Letzkus, J. J., Kampa, B. M., & Stuart, G. J. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2650-06.2006 · Cited by 301
Deep Neural Networks as a Computational Model for Human Shape Sensitivity
Kubilius, J., Bracci, S., & Op de Beeck, H. P. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004896 · Cited by 295
Towards Biologically Plausible Deep Learning
Bengio, Y., Lee, D.-H., Bornschein, J., Mesnard, T., & Lin, Z. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.04156 · Cited by 276
Dendritic spikes in apical dendrites of neocortical layer 2/3 pyramidal neurons
Larkum, M. E., Waters, J., Sakmann, B., & Helmchen, F. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.1717-07.2007 · Cited by 274
Dopamine neurons encode performance error in singing birds
Gadagkar, V., Puzerey, P. A., Chen, R., Baird-Daniel, E., Farhang, A. R., & Goldberg, J. H. (2016) · Science · doi:10.1126/science.aah6837 · Cited by 268
Opening Holes in the Blanket of Inhibition: Localized Lateral Disinhibition by VIP Interneurons
Karnani, M. M., Jackson, J., Ayzenshtat, I., Hamzehei Sichani, A., Manoocheri, K., Kim, S., & Yuste, R. (2016) · The Journal of Neuroscience · doi:10.1523/jneurosci.3646-15.2016 · Cited by 236
Learning by the Dendritic Prediction of Somatic Spiking
Urbanczik, R., & Senn, W. (2014) · Neuron · doi:10.1016/j.neuron.2013.11.030 · Cited by 230 · cited 4×
“Our model builds on previous neural networks research (Lee et al., 2015; Lillicrap et al., 2016) as well as computational studies of supervised learning in multi-compartment neurons (Urbanczik and Senn, 2014; Körding and König, 2001; Spratling and Johnson, 2006).”
BackgroundMethodResultInfluentialNeural Networks and Neuroscience-Inspired Computer Vision
Cox, D. D., & Dean, T. (2014) · Current Biology · doi:10.1016/j.cub.2014.08.026 · Cited by 223
Lee, D.-H., Zhang, S., Fischer, A., & Bengio, Y. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-23528-8_31 · Cited by 216
Zylberberg, J., Murphy, J. T., & DeWeese, M. R. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002250 · Cited by 201
How Important Is Weight Symmetry in Backpropagation?
Liao, Q., Leibo, J. Z., & Poggio, T. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10279 · Cited by 155
Calcium Spikes in Basal Dendrites of Layer 5 Pyramidal Neurons during Action Potential Bursts
Kampa, B. M., & Stuart, G. J. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.3062-05.2006 · Cited by 143
Supervised and Unsupervised Learning with Two Sites of Synaptic Integration
Kording, K. P., & Peter König, B. (2001) · Journal of Computational Neuroscience · doi:10.1023/a:1013776130161 · Cited by 126
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119
Chrna2-Martinotti cells synchronize layer 5 type A pyramidal cells via rebound excitation
Hilscher, M. M., Leão, R. N., Edwards, S. J., Leão, K. E., & Kullander, K. (2017) · PLoS Biology · doi:10.1371/journal.pbio.2001392 · Cited by 118
Cortical Region Interactions and the Functional Role of Apical Dendrites
Spratling, M. (2002) · Behavioral and Cognitive Neuroscience Reviews · doi:10.1177/1534582302001003003 · Cited by 108
Activity-Dependent Modulation of Layer 1 Inhibitory Neocortical Circuits by Acetylcholine
Brombas, A., Fletcher, L. N., & Williams, S. R. (2014) · Journal of Neuroscience · doi:10.1523/jneurosci.4470-13.2014 · Cited by 100
Extrastriate feedback to primary visual cortex in primates: a quantitative analysis of connectivity
Budd, J. M. L. (1998) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.1998.0396 · Cited by 99
Reinforcement learning in populations of spiking neuronsPreprint
Urbanczik, R., & Senn, W. (2008) · Nature Precedings · doi:10.1038/nn.2264 · Cited by 94
Context- and Output Layer-Dependent Long-Term Ensemble Plasticity in a Sensory Circuit
Yamada, Y., Bhaukaurally, K., Madarász, T. J., Pouget, A., Rodríguez, I., & Carleton, A. (2017) · Neuron · doi:10.1016/j.neuron.2017.02.006 · Cited by 93
European conference on machine learning and knowledge discovery in databases
Luttinen, J., Raiko, T., & Ilin, A. (2014) · Cited by 79
Mirrored STDP Implements Autoencoder Learning in a Network of Spiking Neurons
Burbank, K. S. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004566 · Cited by 75
Cortical Feedback Regulates Feedforward Retinogeniculate Refinement
Thompson, A. D., Picard, N., Min, L. Y., Fagiolini, M., & Chen, C. (2016) · Neuron · doi:10.1016/j.neuron.2016.07.040 · Cited by 75
A feedback model of perceptual learning and categorization
Spratling, M., & Johnson, M. H. (2006) · Visual Cognition · doi:10.1080/13506280500168562 · Cited by 58
Stability of the fittest: organizing learning through retroaxonal signals
Harris, K. D. (2008) · Trends in Neurosciences · doi:10.1016/j.tins.2007.12.002 · Cited by 48
Leibo, J. Z., Liao, Q., Anselmi, F., Freiwald, W. A., & Poggio, T. (2016) · Current Biology · doi:10.1016/j.cub.2016.10.015 · Cited by 48
Rules and mechanisms for efficient two-stage learning in neural circuits
Teşileanu, T., Ölveczky, B. P., & Balasubramanian, V. (2017) · eLife · doi:10.7554/elife.20944 · Cited by 33
Towards a Biologically Plausible BackpropPreprint
Scellier, B., & Bengio, Y. (2016) · arXiv (Cornell University) · Cited by 32
Depression-Biased Reverse Plasticity Rule Is Required for Stable Learning at Top-Down Connections
Burbank, K. S., & Kreiman, G. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002393 · Cited by 22
Cited by
Neftci, E. O., Mostafa, H., & Zenke, F. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2931595 · Cited by 1,538
A deep learning framework for neuroscience
Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., Clopath, C., Costa, R. P., de Berker, A., Ganguli, S., et al. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0520-2 · Cited by 1,027
Self-citationLillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J., & Hinton, G. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0277-3 · Cited by 908
Self-citationTang, J., Yuan, F., Shen, X., Wang, Z., Rao, M., He, Y., Sun, Y., Li, X., Zhang, W., Li, Y., et al. (2019) · Advanced Materials · doi:10.1002/adma.201902761 · Cited by 834
Synaptic Plasticity Forms and Functions
Magee, J. C., & Grienberger, C. (2020) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-090919-022842 · Cited by 806
Metadata
- Authors
- Guerguiev, J., Lillicrap, T. P., & Richards, B. A.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2017-12-05
- Volume
- 6
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W2529004582
- Cited by
- 371
- References
- 71
- FWCI
- 18.90
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.