Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Biological neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifelong learning. The interplay of these elements leads to the emergence of biological intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) employ simulated evolution in-silico to breed plastic neural networks with the aim to autonomously design and create learning systems. EPANN experiments evolve networks that include both innate properties and the ability to change and learn in response to experiences in different environments and problem domains. EPANNs' aims include autonomously creating learning systems, bootstrapping learning from scratch, recovering performance in unseen conditions, testing the computational advantages of particular neural components, and deriving hypotheses on the emergence of biological learning. Thus, EPANNs may include a large variety of different neuron types and dynamics, network architectures, plasticity rules, and other factors. While EPANNs have seen considerable progress over the last two decades, current scientific and technological advances in artificial neural networks are setting the conditions for radically new approaches and results. Exploiting the increased availability of computational resources and of simulation environments, the often challenging task of hand-designing learning neural networks could be replaced by more autonomous and creative processes. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and possible developments are presented.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-organized formation of topologically correct feature maps
Kohonen, T. (1982) · Biological Cybernetics · doi:10.1007/bf00337288 · Cited by 9,529
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · cited 1×
“Recurrent neural architectures based on long short-term memory (LSTM) allow very complex tasks to be solved through gradient descent training (Greff et al., 2015; Hochreiter and Schmidhuber, 1997) and have recently shown promise when combined with evolution (Rawal and Miikkulainen, 2016).”
BackgroundMcClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256 · cited 1×
“Inspiration for new approaches could come from complementary learning systems (McClelland et al., 1995; Kumaran et al., 2016) that humans seem to possess, which include fast and slow learning components.”
BackgroundPredictive reward signal of dopamine neurons
Schultz, W. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.1.1 · Cited by 4,564
Completely Derandomized Self-Adaptation in Evolution Strategies
Hansen, N., & Ostermeier, A. (2001) · Evolutionary Computation · doi:10.1162/106365601750190398 · Cited by 4,397
Nanoscale Memristor Device as Synapse in Neuromorphic Systems
Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., & Lu, W. (2010) · Nano Letters · doi:10.1021/nl904092h · Cited by 4,174 · cited 1×
“The progress of EPANNs will crucially depend on implementations that take advantage of the increased computational power of parallel computation with GPUs and neuromorphic hardware (Jo et al., 2010; Monroe, 2014).”
BackgroundWIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication
Jaeger, H., & Haas, H. (2004) · Science · doi:10.1126/science.1091277 · Cited by 3,811 · cited 1×
“(2007), showing that the evolved RNNs were more compact and resulted in better learning than randomly initialized echo state networks (Jaeger and Haas, 2004).”
BackgroundGenetic Algorithms and Machine Learning
Goldberg, D. E., & Holland, J. H. (1988) · Machine Learning · doi:10.1023/a:1022602019183 · Cited by 3,193 · cited 1×
“A central idea in evolutionary computation (Goldberg and Holland, 1988) is that evolutionary processes, similar to those that occurred in nature during the course of billions of years (Darwin, 1859; Dobzhansky, 1970), can be simulated with computer software.”
BackgroundLearning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150 · cited 1×
“Finding appropriate abstractions and representations, just as it was fundamental in the advances in deep learning to represent input spaces and hierarchical features (Bengio et al., 2013; Oquab et al., 2014), can also extend to representations of internal models, learning mechanisms, and genetic encodings, affecting t…”
BackgroundContinual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Bienenstock, E., Cooper, L., & Munro, P. (1982) · The Journal of Neuroscience · doi:10.1523/jneurosci.02-01-00032.1982 · Cited by 2,896 · cited 2×
“Variations have been proposed to include, e.g. terms to achieve stability (Oja, 1982; Bienenstock et al., 1982) and various constraints (Miller and Mackay, 1994), or more advanced update dynamics such as dual weights for fast and slow decay (Levy and Bairaktaris, 1995; Soltoggio, 2015).”
BackgroundHuman-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Simplified neuron model as a principal component analyzer
Oja, E. (1982) · Journal of Mathematical Biology · doi:10.1007/bf00275687 · Cited by 2,309 · cited 2×
“Variations have been proposed to include, e.g. terms to achieve stability (Oja, 1982; Bienenstock et al., 1982) and various constraints (Miller and Mackay, 1994), or more advanced update dynamics such as dual weights for fast and slow decay (Levy and Bairaktaris, 1995; Soltoggio, 2015).”
BackgroundThe plastic human brain cortex
Pascual-Leone, A., Amedi, A., Fregni, F., & Merabet, L. B. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144216 · Cited by 1,929 · cited 1×
“, 2010): it occurs primarily during development, but there is evidence that it continues well into adulthood (Pascual-Leone et al., 2005).”
BackgroundThe free-energy principle: a rough guide to the brain?
Friston, K. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2009.04.005 · Cited by 1,831
Hybrid computing using a neural network with dynamic external memory
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016) · Nature · doi:10.1038/nature20101 · Cited by 1,412
Schultz, W., Apicella, P., & Ljungberg, T. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-03-00900.1993 · Cited by 1,291
A mechanism for the Hebb and the anti-Hebb processes underlying learning and memory.
Lisman, J. (1989) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.86.23.9574 · Cited by 1,127 · cited 2×
“Those advances in the 1970s and 1980s lead to the mathematical formulation of synaptic learning dynamics (Lisman, 1989; Brown et al., 1990), e.g. the BCM rule, inspired by the cat’s visual cortex (Bienenstock et al., 1982), and the Oja’s rule (Oja, 1982).”
BackgroundGraves, A., Wayne, G., & Danihelka, I. (2014) · arXiv · doi:10.48550/arxiv.1410.5401 · Cited by 1,058 · cited 5×
“EPANNs may reach solutions in which memory evolved in different fashions, e.g., preserved as self-sustained neural activity, encoded by connection weights modified by plasticity rules, stored with an external memory (e.g. Neural Turing Machine), or a combination of these approaches.”
BackgroundMethodInfluentialStructural plasticity and memory
Lamprecht, R., & LeDoux, J. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1301 · Cited by 1,010 · cited 1×
“Biological plastic neural networks are also capable of structural plasticity, which creates new pathways among neurons (Lamprecht and LeDoux, 2004; Chklovskii et al., 2004; Russo et al., 2010): it occurs primarily during development, but there is evidence that it continues well into adulthood (Pascual-Leone et al.”
BackgroundToward detecting emotions in spoken dialogs
Lee, C. M., & Narayanan, S. S. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.838534 · Cited by 988
Local GABA Circuit Control of Experience-Dependent Plasticity in Developing Visual Cortex
Hensch, T. K., Fagiolini, M., Mataga, N., Stryker, M. P., Baekkeskov, S., & Kash, S. F. (1998) · Science · doi:10.1126/science.282.5393.1504 · Cited by 910 · cited 1×
“by input-output learning experiences throughout early life (Hensch et al., 1998; Kolb and Gibb, 2011).”
BackgroundCatastrophic forgetting, rehearsal and pseudorehearsal
ROBINS, A. (1995) · Connection Science · doi:10.1080/09540099550039318 · Cited by 833 · cited 1×
“Plasticity in neural networks, when continuously active, was also observed to cause catastrophic forgetting (Robins, 1995).”
BackgroundNeuromodulation: acetylcholine and memory consolidation
Hasselmo, M. E. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01365-0 · Cited by 809
What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated
Kumaran, D., Hassabis, D., & McClelland, J. L. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.05.004 · Cited by 759
Artificial evolution for computer graphics
Sims, K. (1991) · Proceedings of the 18th annual conference on Computer graphics and interactive techniques · doi:10.1145/122718.122752 · Cited by 741 · cited 1×
“In the context of genotype-to-phenotype mappings, Compositional Patterns Producing Networks (CPPNs) (Stanley, 2007), and also the previous work of Sims (1991), demonstrated that structured phenotypes can be generated through a function mapping without going through the dynamic developmental process…”
BackgroundNeuroevolution: from architectures to learning
Floreano, D., Dürr, P., & Mattiussi, C. (2008) · Evolutionary Intelligence · doi:10.1007/s12065-007-0002-4 · Cited by 734 · cited 2×
“Just as neuromodulation is one particular plasticity mechanism that enables effective reward learning, other principles may benefit particular types of learning behavior, as outlined in the review by (Floreano et al., 2008) and in a recent book by Downing (2015).”
BackgroundThe addicted synapse: mechanisms of synaptic and structural plasticity in nucleus accumbens
Russo, S. J., Dietz, D. M., Dumitriu, D., Morrison, J. H., Malenka, R. C., & Nestler, E. J. (2010) · Trends in Neurosciences · doi:10.1016/j.tins.2010.02.002 · Cited by 654
The Restless Engram: Consolidations Never End
Dudai, Y. (2012) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062111-150500 · Cited by 644 · cited 1×
“The consequence of learning is memory, both explicit and implicit (Anderson, 2013), and its consolidation (Dudai, 2012).”
BackgroundHasselmo, M., & Schnell, E. (1994) · The Journal of Neuroscience · doi:10.1523/jneurosci.14-06-03898.1994 · Cited by 466 · cited 1×
“Modulatory chemicals such as acetylcholine (ACh), norepinephrine (NE), serotonin (5-HT) and dopamine (DA) appear to regulate a large variety of neural functions, from arousal and behavior (Harris-Warrick and Marder, 1991; Hasselmo and Schnell, 1994; Marder, 1996; Katz, 1995; Katz and Frost, 1996), to pattern generatio…”
BackgroundTraining-induced structural changes in the adult human brain
Draganski, B., & May, A. (2008) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.02.015 · Cited by 437 · cited 1×
“Neuroscien-tific advances reviewed in Damasio (1999); LeDoux (2003); Pascual-Leone et al. (2005); Doidge (2007); Draganski and May (2008) reveal that fundamental sensations, motor patterns, associations, and ways of thinking are less instinctive and hardwired than previously thought.”
BackgroundIs Heterosynaptic modulation essential for stabilizing hebbian plasiticity and memory
Bailey, C. H., Giustetto, M., Huang, Y.-Y., Hawkins, R. D., & Kandel, E. R. (2000) · Nature Reviews Neuroscience · doi:10.1038/35036191 · Cited by 382 · cited 1×
“Neuromodulation (Marder and Thirumalai, 2002; Gu, 2002; Bailey et al., 2000) is a mechanism that mediates neural processes by means of modulatory signals.”
BackgroundMemory retention – the synaptic stability versus plasticity dilemma
Abraham, W. C., & Robins, A. (2005) · Trends in Neurosciences · doi:10.1016/j.tins.2004.12.003 · Cited by 353
Neural Programmer-Interpreters
Reed, S., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06279 · Cited by 347 · cited 2×
“…shown promise in the context of deep learning; methods such as the Neural Turing Machine (Graves et al., 2014) can learn simple algorithms such as sort, copy, etc. and approaches such as the Neural Programmer-Interface (Reed and de Freitas, 2015) can learn to represent and execute programs.”
MethodClassical conditioning in a simple withdrawal reflex in Aplysia californica.
Carew, T., Walters, E., & Kandel, E. (1981) · The Journal of Neuroscience · doi:10.1523/jneurosci.01-12-01426.1981 · Cited by 281 · cited 1×
“Walters and Byrne (1983) showed that a single neuron can perform associative learning such as classical conditioning, a class of learning that is observed in simple neural systems such as by the Aplysia (Carew et al., 1981).”
BackgroundSchreiweis, C., Bornschein, U., Burguière, E., Kerimoglu, C., Schreiter, S., Dannemann, M., Goyal, S., Rea, E., French, C. A., Puliyadi, R., et al. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1414542111 · Cited by 190
The role of metaplasticity mechanisms in regulating memory destabilization and reconsolidation
Finnie, P. S. B., & Nader, K. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.03.008 · Cited by 189 · cited 1×
“Metaplasticity and consolidation mechanisms were suggested to address this problem (Finnie and Nader, 2012; Soltoggio, 2015).”
BackgroundChimpanzee Intelligence Is Heritable
Hopkins, W. D., Russell, J. L., & Schaeffer, J. (2014) · Current Biology · doi:10.1016/j.cub.2014.05.076 · Cited by 150
Why Greatness Cannot Be Planned: The Myth of the Objective
Stanley, K. O., & Lehman, J. (2015) · doi:10.1007/978-3-319-15524-1 · Cited by 139
Self-citationOpen-Ended Evolution: Perspectives from the OEE Workshop in York
Taylor, T., Bedau, M., Channon, A., Ackley, D., Banzhaf, W., Beslon, G., Dolson, E., Froese, T., Hickinbotham, S., Ikegami, T., et al. (2016) · Artificial Life · doi:10.1162/artl_a_00210 · Cited by 111 · cited 1×
“Open-ended evoluti n seeks to set up experiments that can volve indefi itely m re complex solutions given sufficient computational power (Taylor et al., 2016).”
BackgroundSelf-citation
Cited by
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Meta-Learning in Neural Networks: A Survey
Hospedales, T., Antoniou, A., Micaelli, P., & Storkey, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05439 · Cited by 886
“However they have also been applied to learn learning rules [183], optimizers [184], architectures [27], [123] and data augmentation strategies [135] in supervised learning.”
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
Designing neural networks through neuroevolution
Stanley, K. O., Clune, J., Lehman, J., & Miikkulainen, R. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0006-z · Cited by 689
Self-citation
Metadata
- Authors
- Soltoggio, A., Stanley, K. O., & Risi, S.
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2018-12
- Volume
- 108
- Pages
- 48-67
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2605353576
- Cited by
- 121
- References
- 48
- FWCI
- 9.90
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.