Evolving large-scale neural networks for vision-based reinforcement learning
The DOI matches its Crossref record; 15 of the 16 references it cites were found in a registry or an index.
Abstract
The idea of using evolutionary computation to train artificial neural networks, or neuroevolution (NE), for reinforcement learning (RL) tasks has now been around for over 20 years. However, as RL tasks become more challenging, the networks required become larger, as do their genomes. But, scaling NE to large nets (i.e. tens of thousands of weights) is infeasible using direct encodings that map genes one-to-one to network components. In this paper, we scale-up our compressed network encoding where network weight matrices are represented indirectly as a set of Fourier-type coefficients, to tasks that require very-large networks due to the high-dimensionality of their input space. The approach is demonstrated successfully on two reinforcement learning tasks in which the control networks receive visual input: (1) a vision-based version of the octopus control task requiring networks with over 3 thousand weights, and (2) a version of the TORCS driving game where networks with over 1 million weights are evolved to drive a car around a track using video images from the driver's perspective.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
MACHINE PERCEPTION OF THREE-DIMENSIONAL SOLIDS,
Roberts, L. G. (1963) · Defense Technical Information Center (DTIC) · Cited by 1,551
Designing Neural Networks Using Genetic Algorithms with Graph Generation System
Kitnano, H. (1990) · Complex Systems · Cited by 612
Dynamic Model of the Octopus Arm. I. Biomechanics of the Octopus Reaching Movement
Yekutieli, Y., Sagiv-Zohar, R., Aharonov, R., Engel, Y., Hochner, B., & Flash, T. (2005) · Journal of Neurophysiology · doi:10.1152/jn.00684.2004 · Cited by 222
Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability
Schmidhuber, J. (1997) · Neural Networks · doi:10.1016/s0893-6080(96)00127-x · Cited by 181
Self-citationAccelerated Neural Evolution through Cooperatively Coevolved Synapses
GomezFaustino, SchmidhuberJürgen, & MiikkulainenRisto (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1390712 · Cited by 164
Self-citationGenerating large-scale neural networks through discovering geometric regularities
Gauci, J., & Stanley, K. O. (2007) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1276958.1277158 · Cited by 140
A novel generative encoding for exploiting neural network sensor and output geometry
D’Ambrosio, D. B., & Stanley, K. O. (2007) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1276958.1277155 · Cited by 97
The 2009 Simulated Car Racing Championship
Loiacono, D., Lanzi, P. L., Togelius, J., Onieva, E., PELTA, D. A., Butz, M. V., Lonneker, T. D., Cardamone, L., Pérez-Liébana, D., Sáez, Y., et al. (2010) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2010.2050590 · Cited by 91
Evolving neural networks in compressed weight space
Koutnik, J., Gomez, F., & Schmidhuber, J. (2010) · Proceedings of the 12th annual conference on Genetic and evolutionary computation · doi:10.1145/1830483.1830596 · Cited by 74
Self-citationAn adaptive 'broom balancer' with visual inputs
Tolat, & Widrow (1988) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/icnn.1988.23982 · Cited by 62
Cited by
Items 101–175 of 175
Evolving a TORCS Modular Fuzzy Driver Using Genetic Algorithms
Salem, M., Mora, A. M., Merelo, J. J., & García‐Sánchez, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-77538-8_24 · Cited by 11
Fine-tuning Deep RL with Gradient-Free Optimization
de Bruin, T., Kober, J., Tuyls, K., & Babuška, R. (2020) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2020.12.2240 · Cited by 11
Hu, C., Qiao, R., Gong, W., Yan, X., & Wang, L. (2022) · Memetic Computing · doi:10.1007/s12293-022-00375-8 · Cited by 10
Actor-Critic Reinforcement Learning with Neural Networks in Continuous Games
Leuenberger, G., & Wiering, M. (2018) · International Conference on Agents and Artificial Intelligence (ICAART) · doi:10.5220/0006556500530060 · Cited by 10
Monte-Carlo Tree Search for Simulated Car Racing.
Fischer, J., Falsted, N., Vielwerth, M., Togelius, J., & Risi, S. (2015) · IT University Of Copenhagen (IT University of Copenhagen) · Cited by 10
Bégin‐Drolet, A., Collin, S., Gosselin, J., & Ruel, J. (2018) · Journal of Process Control · doi:10.1016/j.jprocont.2017.11.007 · Cited by 9
DeepFoldit -- A Deep Reinforcement Learning Neural Network Folding ProteinsPreprint
Panou, D. N., & Reczko, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.03442 · Cited by 9
Gao, T., Li, J., & Mi, Q. (2023) · International Conference on Emerging Techniques in Computational Intelligence (ICETCI) · doi:10.1109/icetci58599.2023.10331190 · Cited by 7
Evolutionary Echo State Network: evolving reservoirs in the Fourier space
Basterrech, S., & Rubino, G. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892892 · Cited by 7
Risi, S., & Stanley, K. O. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17470 · Cited by 6
Quantum Deep Learning Triuniverse
McCoss, A. M. (2016) · Journal of Quantum Information Science · doi:10.4236/jqis.2016.64015 · Cited by 6
Hajewski, J., Oliveira, S., & Xing, X. (2020) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3377929.3390011 · Cited by 6
WiseBench: A Motion Planning Benchmarking Framework for Autonomous Vehicles
Ilievski, M. (2020) · UWSpace (University of Waterloo) · Cited by 6
Self Training Autonomous Driving Agent
Kotyan, S., Vargas, D. V., & Venkanna, U. (2019) · Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE) · doi:10.23919/sice.2019.8859883 · Cited by 5
Search-Based Exploration and Diagnosis of TOAD-GAN
Edwards, M., Jiang, M., & Togelius, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v17i1.18901 · Cited by 5
A robustness analysis of Deep Q Networks
Tow, A. W., Rowlands, S., Leitner, J., Suenderhauf, N., Milford, M. J., & Upcroft, B. (2016) · QUT ePrints (Queensland University of Technology) · Cited by 5
POET: An Evo-Devo Method to Optimize the Weights of Large Artificial Neural Networks
Fontana, A., Soltoggio, A., & Wróbel, B. (2014) · Conference on Artificial Life (ALIFE) · doi:10.1162/978-0-262-32621-6-ch073 · Cited by 5
Learning One Abstract Bit at a Time Through Self-invented Experiments Encoded as Neural Networks
Herrmann, V., Kirsch, L., & Schmidhuber, J. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-47958-8_16 · Cited by 5
Self-citationDeterministic Policy Gradients With General State TransitionsPreprint
Cai, Q., Pan, L., & Tang, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03708 · Cited by 5
Teaching UAVs to Race: End-to-End Regression of Agile Controls in SimulationPreprint
Müller, M., Casser, V., Smith, N., Michels, D. L., & Ghanem, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05884 · Cited by 5
Learn to Race: Sequential Actor-Critic Reinforcement Learning for Autonomous Racing
Liu, R., Zhuang, W., Tong, F., & Yin, G. (2023) · IEEE International Automated Vehicle Validation Conference (IAVVC) · doi:10.1109/iavvc57316.2023.10328086 · Cited by 4
Towards neuroimaging real-time driving using Convolutional Neural Networks
Musoles, C. F. (2016) · Computer Science and Electronic Engineering Conference · doi:10.1109/ceec.2016.7835901 · Cited by 4
The Foundations of Deep Learning with a Path Towards General Intelligence
Özkural, E. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-97676-1_16 · Cited by 4
A Deep Reinforcement Learning Approach for Autonomous Car Racing
Guo, F., & Wu, Z. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-23712-7_27 · Cited by 4
Permutation-Invariant Representation of Neural Networks with Neuron Embeddings
Zhou, R., Muise, C. J., & Hu, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-02056-8_19 · Cited by 4
Grinin, L. E., Леонидович, Г. А., & Леонидович, Г. И. (2023) · Философия и общество · doi:10.30884/jfio/2023.04.01 · Cited by 4
Utilization of Deep Reinforcement Learning for Saccadic-Based Object Visual Search
Kornuta, T., & Rocki, K. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-54042-9_56 · Cited by 4
Teaching UAVs to Race Using Sim4CVPreprint
Müller, M., Casser, V., Smith, N., Michels, D. L., & Ghanem, B. (2017) · arXiv (Cornell University) · Cited by 4
Learning and generalizing behaviors for robots from human demonstration
Fabisch, A. (2020) · Media (https://www.suub.uni-bremen.de/) · doi:10.26092/elib/382 · Cited by 4
Zhu, Q., Wang, S.-M., & Ni, Y. (2025) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2025.113495 · Cited by 3
Draft: Deep Learning in Neural Networks: An Overview
Idsia, L. (2014) · Cited by 3
Deep spectral Q‐learning with application to mobile health
Gao, Y., Shi, C., & Song, R. (2023) · Stat · doi:10.1002/sta4.564 · Cited by 3
Coupling Evolution and Information Theory for Autonomous Robotic Exploration
Zhang, G., & Sébag, M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10762-2_84 · Cited by 3
Automatic Driving Decision Algorithm Based on Multi-dimensional Deep Space-Time Network
Zhang, J., Yuan, J., Pan, H., Ma, Q., & Yu, Y. H. (2018) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-00828-4_8 · Cited by 3
A Comprehensive Survey on Real-Time Voltage Stability Assessment for Power Systems
Wadhwa, G., Kharb, A., Mishra, S., Kumar, M., & Srivastav, S. (2020) · IEEE International Conference on Industrial and Information Systems (ICIIS) · doi:10.1109/iciis51140.2020.9342706 · Cited by 3
Deep learning based approaches for imitation learning.
Hussein, A. M. (2018) · Open Access Institutional Repository at Robert Gordon University (Robert Gordon University) · Cited by 3
Driving-Policy Adaptive Safeguard for Autonomous Vehicles Using Reinforcement LearningPreprint
Cao, Z. L., Xu, S., Zhang, S., Peng, H., & Yang, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.01010 · Cited by 3
Papafotiou, T., Tsardoulias, E., Nikolaou, A., Papagiannitsi, A., Christodoulou, D., Gkountras, I., & Symeonidis, A. L. (2025) · Machines · doi:10.3390/machines13060514 · Cited by 2
Efficient Evolution of Variational Autoencoders
Hajewski, J., & Oliveira, S. (2021) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc51732.2021.9376167 · Cited by 2
Object Boundary Detection and Classification with Image-Level Labels
Koh, J. Y., Samek, W., Müller, K., & Binder, A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66709-6_13 · Cited by 2
Sample Efficient Deep Reinforcement Learning for Control
de Bruin, T. (2019) · Research Repository (Delft University of Technology) · doi:10.4233/uuid:f8faacb0-9a55-453d-97fd-0388a3c848ee · Cited by 2
Controlling the locomotion of quadruped robots with learning methods
Aractingi, M. (2023) · theses.fr (ABES) · doi:10.70675/d46a9cd8zd72cz4b1bzaa8ez71e0c6b58928 · Cited by 2
Salem, M., Mora, A. M., Merelo, J. J., & García‐Sánchez, P. (2018) · Communications in computer and information science · doi:10.1007/978-3-319-91479-4_20 · Cited by 1
Visual Search as a Probabilistic Sequential Decision Process in Software Autonomous System
Ghosh, A., & Huang, S. (2018) · IEEE SoutheastCon · doi:10.1109/secon.2018.8479127 · Cited by 1
MAIA: The role of innate behaviors when picking flowers in Minecraft with Q-learning
Siljebråt, H. (2015) · Lund University Publications Student Papers (Lund University) · Cited by 1
Learning as a Sampling Problem
Stadie, B. C. (2018) · eScholarship (California Digital Library) · Cited by 1
New parallel algorithms for support vector machines and neural architecture search
Hajewski, J., Zhu, X., Varadarajan, K., Yang, T., Oliveira, S., & Stewart, D. E. (2020) · doi:10.17077/etd.005377 · Cited by 1
Adversarial Robustness Testing of Deep Reinforcement Learning Based Automated Trading Software
Ataiefard, F. (2022) · doi:10.11575/prism/40285 · Cited by 1
Building Intelligent Autonomous Navigation Agents
Chaplot, D. S. (2021) · arXiv (Cornell University) · doi:10.1184/r1/19552297 · Cited by 1
Retaining Experience and Growing SolutionsPreprint
Ffrancon, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.01474
Is depth information and optical flow helpful for visual control?
Hansen, J., & Ebner, M. (2016) · Bio-Algorithms and Med-Systems · doi:10.1515/bams-2015-0044
Platform for Rapid Prototyping of AI Architectures
Hrosso, P., Knopp, J., Vítků, J., & Fedorčák, D. (2015) · ITAT
Zero Shot Learning for Semantic Boundary Detection - How Far Can We Get?Preprint
Koh, J. Y., Samek, W., Müller, K., & Binder, A. (2016) · arXiv (Cornell University)
Διαμαντής, Ε. (2018) · Dione (University of Piraeus)
Object Boundary Detection and Classification with Image-level LabelsPreprint
Koh, J. Y., Samek, W., Müller, K., & Binder, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.09187
Towards solving neural networks with optimization trajectory search
Parsenadze, L. T., Vargas, D. V., & Fujita, T. (2019) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3319619.3326796
Mladenović, M. N. (2020) · Zbornik radova Fakulteta tehničkih nauka u Novom Sadu · doi:10.24867/07be44mladenovic
Exploring the Training Data for Online Learning of Autonomous Driving in a Simulated Environment
Kindstedt, M. (2020) · DiVA (Digitala Vetenskapliga Arkivet)
Attention-Based Models For Deep Reinforcement Learning
Barati, E. (2019) · Human Biology
The Role of Bio-Inspired Modularity in General Learning
StClair, R. A., Hahn, W. E., & Barenholtz, E. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-93758-4_27
Issues, Challenges, and Progress of Autonomous Robotic Platoons
Gunagwera, A., & Zengin, A. T. (2022) · Advances in computational intelligence and robotics book series · doi:10.4018/978-1-6684-5381-0.ch010
An Enterprise Risk Prediction Model Combined Financial Data and News Based on Memory Network
Li, P. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-6901-0_107
Policy-Induced Self-Supervision Improves Representation Finetuning in Visual RLPreprint
Arnold, S. M. R., & Sha, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.06009
Using Deep RL to Improve the ACAS-Xu Policy: Concept Paper
Perotto, F. S. (2024) · International Conference on Cognitive Aircraft Systems (ICCAS) · doi:10.5220/0013023200004562
Water pressure optimisation for leakage management using deep reinforcement learning
Negm, A. S. (2024) · Lancaster EPrints (Lancaster University) · doi:10.17635/lancaster/thesis/2302
Tian, Y. (2024) · cIRcle (University of British Columbia) · doi:10.14288/1.0438656
Κατανομή πόρων σε ασύρματα δίκτυα πολλαπλής πρόσβασης διαίρεσης ρυθμού με χρήση ενισχυτικής μάθησης
Kapsalis, G. (2023) · DSpace - NTUA (National Technical University of Athens) · doi:10.26240/heal.ntua.24804
Neuro-Evolution: Representations, Efficiency and Development
Assunção, F. (2021) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Sistema de conducción autónoma aplicado a rotondas simulado en Gazebo y Controlado en ROS 2
Vicente Vidal, P. (2023) · Archivo Digital UPM (Universidad Politécnica de Madrid)
Application of reinforcement learning in electrical machine design
SarcheshmehPour, Y. (2023) · Aaltodoc (Aalto University)
Automatic Discovery and Diagnosis of Security and Safety Defects in Autonomous Driving Software
Wan, Z. (2025) · eScholarship (California Digital Library)
Exploratory Robotic Controllers : An Evolution and Information Theory Driven Approach
Zhang, G. (2015) · theses.fr (ABES) · doi:10.70675/be25a63az960fz47f4z803ezeee0a9e0d714
Vehicle localization based on scene perception
Liu, L. (2021) · theses.fr (ABES) · doi:10.70675/43424a72z2eedz41d7z9829z41cfa8ff911a
From Weights to Words: Representation in Neuroevolution and Explainable AI
Zhou, R. (2026) · QSpace (Queen's University Library)
Metadata
- Authors
- Koutník, J., Cuccu, G., Schmidhuber, J., & Gomez, F.
- Affiliations
- University of Applied Sciences and Arts of Southern Switzerland (CH)
- Venue
- Proceedings of the 15th annual conference on Genetic and evolutionary computation
- Publisher
- ACM
- Published
- 2013-07-06
- Pages
- 1061-1068
- OpenAlex
- W2038794597
- Cited by
- 130
- References
- 16
- Citations by year
- 2025: 2 · 2024: 2 · 2023: 10 · 2022: 9 · 2021: 15 · 2020: 18 · 2019: 9 · 2018: 19 · 2017: 19 · 2016: 12 · 2015: 8 · 2014: 7
- FWCI
- 5.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Genetic and Evolutionary Computation Conference (GECCO) (publishedVersion)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- neuroevolution, large-scale neural networks, vision-based reinforcement learning, high-dimensional input space
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.