Evolving large-scale neural networks for vision-based reinforcement learning
The DOI matches its Crossref record; 15 of the 16 references it cites were found in a registry or an index.
Abstract
The idea of using evolutionary computation to train artificial neural networks, or neuroevolution (NE), for reinforcement learning (RL) tasks has now been around for over 20 years. However, as RL tasks become more challenging, the networks required become larger, as do their genomes. But, scaling NE to large nets (i.e. tens of thousands of weights) is infeasible using direct encodings that map genes one-to-one to network components. In this paper, we scale-up our compressed network encoding where network weight matrices are represented indirectly as a set of Fourier-type coefficients, to tasks that require very-large networks due to the high-dimensionality of their input space. The approach is demonstrated successfully on two reinforcement learning tasks in which the control networks receive visual input: (1) a vision-based version of the octopus control task requiring networks with over 3 thousand weights, and (2) a version of the TORCS driving game where networks with over 1 million weights are evolved to drive a car around a track using video images from the driver's perspective.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
MACHINE PERCEPTION OF THREE-DIMENSIONAL SOLIDS,
Roberts, L. G. (1963) · Defense Technical Information Center (DTIC) · Cited by 1,551
Designing Neural Networks Using Genetic Algorithms with Graph Generation System
Kitnano, H. (1990) · Complex Systems · Cited by 612
Dynamic Model of the Octopus Arm. I. Biomechanics of the Octopus Reaching Movement
Yekutieli, Y., Sagiv-Zohar, R., Aharonov, R., Engel, Y., Hochner, B., & Flash, T. (2005) · Journal of Neurophysiology · doi:10.1152/jn.00684.2004 · Cited by 222
Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability
Schmidhuber, J. (1997) · Neural Networks · doi:10.1016/s0893-6080(96)00127-x · Cited by 184
Self-citationAccelerated Neural Evolution through Cooperatively Coevolved Synapses
GomezFaustino, SchmidhuberJürgen, & MiikkulainenRisto (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1390712 · Cited by 164
Self-citationGenerating large-scale neural networks through discovering geometric regularities
Gauci, J., & Stanley, K. O. (2007) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1276958.1277158 · Cited by 130
The 2009 Simulated Car Racing Championship
Loiacono, D., Lanzi, P. L., Togelius, J., Onieva, E., PELTA, D. A., Butz, M. V., Lonneker, T. D., Cardamone, L., Pérez-Liébana, D., Sáez, Y., et al. (2010) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2010.2050590 · Cited by 91
A novel generative encoding for exploiting neural network sensor and output geometry
D’Ambrosio, D. B., & Stanley, K. O. (2007) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1276958.1277155 · Cited by 87
Evolving neural networks in compressed weight space
Koutnik, J., Gomez, F., & Schmidhuber, J. (2010) · Proceedings of the 12th annual conference on Genetic and evolutionary computation · doi:10.1145/1830483.1830596 · Cited by 74
Self-citationAn adaptive 'broom balancer' with visual inputs
Tolat, & Widrow (1988) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/icnn.1988.23982 · Cited by 62
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Self-citationDeep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
“eschewed the commonly used backpropagation algorithm in favour of evolutionary algorithms [32, 19, 53], which are gradient-free policy search algorithms.”
InfluentialA Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
A Survey of Autonomous Driving: Common Practices and Emerging Technologies
Yurtsever, E., Lambert, J., Carballo, A., & Takeda, K. (2020) · IEEE Access · doi:10.1109/access.2020.2983149 · Cited by 1,889
“Furthermore, these design philosophies are realized with two alternative approaches: modular [15], [47], [51]–[58] or endto-end driving [59]–[67].”
BackgroundMethodResultInfluentialDeep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989385 · Cited by 1,424
Metadata
- Authors
- Koutník, J., Cuccu, G., Schmidhuber, J., & Gomez, F.
- Affiliations
- University of Applied Sciences and Arts of Southern Switzerland (CH)
- Venue
- Proceedings of the 15th annual conference on Genetic and evolutionary computation
- Publisher
- ACM
- Published
- 2013-07-06
- Pages
- 1061-1068
- OpenAlex
- W2038794597
- Cited by
- 130
- References
- 16
- Citations by year
- 2025: 2 · 2024: 2 · 2023: 10 · 2022: 9 · 2021: 15 · 2020: 18 · 2019: 9 · 2018: 19 · 2017: 19 · 2016: 12 · 2015: 8 · 2014: 7
- FWCI
- 5.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Genetic and Evolutionary Computation Conference (GECCO) (publishedVersion)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- neuroevolution, large-scale neural networks, vision-based reinforcement learning, high-dimensional input space
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evolving large-scale neural networks for vision-based reinforcement learning. https://app.cochat.ai/papers/10.1145/2463372.2463509
- The paper
- Koutník, J., Cuccu, G., Schmidhuber, J., & Gomez, F. (2013). Evolving large-scale neural networks for vision-based reinforcement learning. Proceedings of the 15th annual conference on Genetic and evolutionary computation. https://doi.org/10.1145/2463372.2463509
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.