The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Real-Time Neuroevolution in the NERO Video Game
Abstract
In most modern video games, character behavior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. Yet, if game characters could learn through interacting with the player, behavior could improve as the game is played, keeping it interesting. This paper introduces the real-time Neuroevolution of Augmenting Topologies (rtNEAT) method for evolving increasingly complex artificial neural networks in real time, as a game is being played. The rtNEAT method allows agents to change and improve during the game. In fact, rtNEAT makes possible an entirely new genre of video games in which the player trains a team of agents through a series of customized exercises. To demonstrate this concept, the Neuroevolving Robotic Operatives (NERO) game was built based on rtNEAT. In NERO, the player trains a team of virtual robots for combat against other players' teams. This paper describes results from this novel application of machine learning, and demonstrates that rtNEAT makes possible video games like NERO where agents evolve and adapt in real time. In the future, rtNEAT may allow new kinds of educational and training applications through interactive and adapting games.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Some Studies in Machine Learning Using the Game of Checkers
Samuel, A. L. (1959) · IBM Journal of Research and Development · doi:10.1147/rd.33.0210 · Cited by 4,443
Preservation of Duplicate Genes by Complementary, Degenerative Mutations
Force, A., Lynch, M., Pickett, F. B., Amores, A., Yan, Y.-L., & Postlethwait, J. (1999) · Genetics · doi:10.1093/genetics/151.4.1531 · Cited by 3,529
Evolving Neural Networks through Augmenting Topologies
Stanley, K. O., & Miikkulainen, R. (2002) · Evolutionary Computation · doi:10.1162/106365602320169811 · Cited by 3,258
Self-citationHandbook of Evolutionary Computation
(1997) · Institute of Physics eBooks · doi:10.1887/0750308958 · Cited by 2,990
Evolving artificial neural networks
Yao, X. (1999) · Proceedings of the IEEE · doi:10.1109/5.784219 · Cited by 2,981
Evolution strategies – A comprehensive introduction
Beyer, H.-G., & Schwefel, H.-P. (2002) · Natural Computing · doi:10.1023/a:1015059928466 · Cited by 2,790
Genetic algorithms with sharing for multimodal function optimization
Goldberg, D. E., & Richardson, J. T. (1987) · international conference on Genetic algorithms · Cited by 1,912
Zebrafish hox Clusters and Vertebrate Genome Evolution
Amores, A., Force, A., Yan, Y.-L., Joly, L., Amemiya, C., Fritz, A., Ho, R. K., Langeland, J., Prince, V., Wang, Y.-L., et al. (1998) · Science · doi:10.1126/science.282.5394.1711 · Cited by 1,736
Some studies in machine learning using the game of checkers
Samuel, A. L. (2000) · IBM Journal of Research and Development · doi:10.1147/rd.441.0206 · Cited by 1,439
An evolutionary algorithm that constructs recurrent neural networks
Angeline, P. J., Saunders, G. M., & Pollack, J. B. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.265960 · Cited by 1,012
Genetic Algorithms for Real Parameter Optimization
Wright, A. H. (1991) · Foundations of Genetic Algorithms · doi:10.1016/b978-0-08-050684-5.50016-1 · Cited by 998
Incremental Induction of Decision Trees
Utgoff, P. E. (1989) · Machine Learning · doi:10.1023/a:1022699900025 · Cited by 791
Homeotic genes and the evolution of arthropods and chordates
Carroll, S. B. (1995) · Nature · doi:10.1038/376479a0 · Cited by 758
Genetic evolution of the topology and weight distribution of neural networks
Maniezzo, V. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.265959 · Cited by 436
Human-Level AI's Killer Application: Interactive Computer Games
Laird, J. E., & van Lent, M. (2000) · National Conference on Artificial Intelligence · Cited by 416
Competitive Coevolution through Evolutionary Complexification
Stanley, K. O., & Miikkulainen, R. (2004) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1338 · Cited by 413
Self-citationAnalog computation via neural networks
Siegelmann, H. T., & Sontag, E. D. (1994) · Theoretical Computer Science · doi:10.1016/0304-3975(94)90178-3 · Cited by 355
Learning and Evolution in Neural Networks
Nolfi, S., Parisi, D., & Elman, J. L. (1994) · Adaptive Behavior · doi:10.1177/105971239400300102 · Cited by 345
Automatic Creation of an Autonomous Agent: Genetic Evolution of a Neural-Network Driven Robot
Floreano, D., & Francesco Mondada, C. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/3117.003.0061 · Cited by 333
Efficient reinforcement learning through symbiotic evolution
Moriarty, D. E., & Miikkulainen, R. (1996) · Machine Learning · doi:10.1007/bf00114722 · Cited by 325
Self-citationAutomatic Creation of an Autonomous Agent: Genetic Evolution of a Neural Network Driven Robot
Floreano, D., & Francesco Mondada, C. (1994) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-a-010111549 · Cited by 303
A Comparison between Cellular Encoding and Direct Encoding for Genetic Neural Networks
(1996) · Genetic Programming 1996 · doi:10.7551/mitpress/3242.003.0013 · Cited by 268
Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces
Santamaria, J. C., Sutton, R. S., & Ram, A. (1997) · Adaptive Behavior · doi:10.1177/105971239700600201 · Cited by 258
Creating High-Level Components with a Generative Representation for Body-Brain Evolution
Hornby, G. S., & Pollack, J. B. (2002) · Artificial Life · doi:10.1162/106454602320991837 · Cited by 225
Blondie24: Playing at the Edge of AI
Fogel, D. B. (2001) · Cited by 218
Genetic Reinforcement Learning for Neurocontrol Problems
Whitley, D., Dominic, S., Das, R., & Anderson, C. W. (1993) · Machine Learning · doi:10.1007/bf00993045 · Cited by 215
Efficient Reinforcement Learning Through Evolving Neural Network Topologies
Stanley, K. O., & Miikkulainen, R. P. (2002) · Cited by 215
Self-citationEvolving neural network controllers for unstable systems
Wieland, A. P. (2002) · IJCNN-91-Seattle International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1991.155416 · Cited by 184
Towards designing artificial neural networks by evolution
Yao, X., & Liu, Y. (1998) · Applied Mathematics and Computation · doi:10.1016/s0096-3003(97)10005-4 · Cited by 147
A Coevolutionary Approach to Learning Sequential Decision Rules
Potter, M. A., Jong, K. A. D., & Grefenstette, J. J. (1995) · Cited by 144
Robust non-linear control through neuroevolution
Gomez, F. J., & Miikkulainen, R. P. (2003) · Texas ScholarWorks (Texas Digital Library) · Cited by 143
Self-citationSymbiotic Evolution of Neural Networks in Sequential Decision Tasks
Moriarty, D. E. (1997) · Cited by 130
A self-learning evolutionary chess program
Fogel, D. B., Hays, T., Hahn, S. L., & Quon, J. (2004) · Proceedings of the IEEE · doi:10.1109/jproc.2004.837633 · Cited by 126
Designing application-specific neural networks using the structured genetic algorithm
Dasgupta, D., & McGregor, D. R. (2003) · doi:10.1109/cogann.1992.273946 · Cited by 125
Repeated structure and dissociation of genotypic and phenotypic complexity in artificial ontogeny
Bongard, J., & Pfeifer, R. (2001) · Cited by 123
Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor.
Zhang, B., & Mühlenbein, H. (1993) · Cited by 116
Genetic Reinforcement Learning for Neurocontrol Problems
Whitley, D., Dominic, S., Das, R., & Anderson, C. W. (1993) · Machine Learning · doi:10.1023/a:1022674030396 · Cited by 115
Using a genetic algorithm to tune first-person shooter bots
Cole, N., Louis, S. J., & Miles, C. E. (2004) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2004.1330849 · Cited by 113
Genetic set recombination and its application to neural network topology optimisation
Radcliffe, N. J. (1993) · Neural Computing and Applications · doi:10.1007/bf01411376 · Cited by 112
Active Guidance for a Finless Rocket Using Neuroevolution
Gomez, F. J., & Miikkulainen, R. P. (2003) · Lecture notes in computer science · doi:10.1007/3-540-45110-2_105 · Cited by 112
Self-citationDiscovering Complex Othello Strategies through Evolutionary Neural Networks
Moriarty, D. E., & Miikkulainen, R. P. (1995) · Connection Science · doi:10.1080/09540099550039228 · Cited by 111
Self-citationEvolving neural control systems
Saravanan, N., & Fogel, D. B. (1995) · IEEE Expert · doi:10.1109/64.393139 · Cited by 110
Increasing Genomic Complexity by Gene Duplication and the Origin of Vertebrates
Martin, A. P. (1999) · The American Naturalist · doi:10.1086/303231 · Cited by 100
Stanley, K. O., & Miikkulainen, R. P. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24855-2_130 · Cited by 85
Self-citationLearning procedural knowledge through observation
van Lent, M., & Laird, J. E. (2001) · International Conference on Knowledge Capture · doi:10.1145/500737.500765 · Cited by 71
Representation and Evolution of Neural Networks
Mandischer, M. (1993) · Artificial Neural Nets and Genetic Algorithms · doi:10.1007/978-3-7091-7533-0_93 · Cited by 67
Connectionist Theory Refinement: Genetically Searching the Space of Network Topologies
Opitz, D. W., & SHAVLIK, J. W. (1997) · Journal of Artificial Intelligence Research · doi:10.1613/jair.368 · Cited by 61
Machine learning in games: a survey
Fürnkranz, J. (2001) · Cited by 60
Parmee, I. C., & Bonham, C. R. (2000) · Artificial intelligence for engineering design analysis and manufacturing · doi:10.1017/s0890060400141010 · Cited by 57
2-D Pole Balancing with Recurrent Evolutionary Networks
Gomez, F. J., & Miikkulainen, R. P. (1998) · Perspectives in neural computing · doi:10.1007/978-1-4471-1599-1_63 · Cited by 51
Self-citationEvolving Neural Networks to Play Go.
Richards, N., Moriarty, D. E., Mcquesten, P. H., et al. (1997) · Cited by 46
Evolving Obstacle Avoidance Behavior in a Robot Arm
Moriarty, D. E., & Miikkulainen, R. P. (1996) · The MIT Press eBooks · doi:10.7551/mitpress/3118.003.0056 · Cited by 40
Self-citationGeisler, B. (2002) · Cited by 33
A 'Neural' Network that Learns to Play Backgammon
Tesauro, G. J., & B, T. J. S. A. (1987) · Cited by 25
Evolutionary ordered neural network with a linked-list encoding scheme
Lee, C., & Kim, J.-H. (2002) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/icec.1996.542680 · Cited by 13
Delta-Gann: A New Approach To Training Neural Networks Using Genetic Algorithms
Krishnan, R., & Ciesielski, V. B. (1994) · Cited by 4
Evolving Obstacle Avoidance Behavior in a Robot Arm
Maes, P., Matarić, M. J., Meyer, J.-A., Pollack, J. B., & Wilson, S. W. (1996) · Cited by 3
Cited by
Abandoning Objectives: Evolution Through the Search for Novelty Alone
Lehman, J., & Stanley, K. O. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00025 · Cited by 917
Self-citationA Hypercube-Based Encoding for Evolving Large-Scale Neural Networks
Stanley, K. O., D'Ambrosio, D. B., & Gauci, J. (2009) · Artificial Life · doi:10.1162/artl.2009.15.2.15202 · Cited by 780
Self-citationCompositional pattern producing networks: A novel abstraction of development
Stanley, K. O. (2007) · Genetic Programming and Evolvable Machines · doi:10.1007/s10710-007-9028-8 · Cited by 713
Self-citationDesigning neural networks through neuroevolution
Stanley, K. O., Clune, J., Lehman, J., & Miikkulainen, R. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0006-z · Cited by 689
Self-citationQuo vadis artificial intelligence?
Jiang, Y., Li, X., Luo, H., Yin, S., & Kaynak, O. (2022) · Discover Artificial Intelligence · doi:10.1007/s44163-022-00022-8 · Cited by 544
Metadata
- Authors
- Stanley, K. O., Bryant, B. D., & Miikkulainen, R.
- Venue
- IEEE Transactions on Evolutionary Computation
- Published
- 2005-12
- Volume
- 9
- Issue
- 6
- Pages
- 653-668
- ISSN
- 1089-778X (ISSN-L)
- OpenAlex
- W2140685838
- Cited by
- 390
- References
- 64
- FWCI
- 23.02
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.