Evolving large-scale neural networks for vision-based reinforcement learning
The DOI matches its Crossref record; 15 of the 16 references it cites were found in a registry or an index.
Abstract
The idea of using evolutionary computation to train artificial neural networks, or neuroevolution (NE), for reinforcement learning (RL) tasks has now been around for over 20 years. However, as RL tasks become more challenging, the networks required become larger, as do their genomes. But, scaling NE to large nets (i.e. tens of thousands of weights) is infeasible using direct encodings that map genes one-to-one to network components. In this paper, we scale-up our compressed network encoding where network weight matrices are represented indirectly as a set of Fourier-type coefficients, to tasks that require very-large networks due to the high-dimensionality of their input space. The approach is demonstrated successfully on two reinforcement learning tasks in which the control networks receive visual input: (1) a vision-based version of the octopus control task requiring networks with over 3 thousand weights, and (2) a version of the TORCS driving game where networks with over 1 million weights are evolved to drive a car around a track using video images from the driver's perspective.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
MACHINE PERCEPTION OF THREE-DIMENSIONAL SOLIDS,
Roberts, L. G. (1963) · Defense Technical Information Center (DTIC) · Cited by 1,551
Designing Neural Networks Using Genetic Algorithms with Graph Generation System
Kitnano, H. (1990) · Complex Systems · Cited by 612
Dynamic Model of the Octopus Arm. I. Biomechanics of the Octopus Reaching Movement
Yekutieli, Y., Sagiv-Zohar, R., Aharonov, R., Engel, Y., Hochner, B., & Flash, T. (2005) · Journal of Neurophysiology · doi:10.1152/jn.00684.2004 · Cited by 222
Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability
Schmidhuber, J. (1997) · Neural Networks · doi:10.1016/s0893-6080(96)00127-x · Cited by 181
Self-citationAccelerated Neural Evolution through Cooperatively Coevolved Synapses
GomezFaustino, SchmidhuberJürgen, & MiikkulainenRisto (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1390712 · Cited by 164
Self-citationGenerating large-scale neural networks through discovering geometric regularities
Gauci, J., & Stanley, K. O. (2007) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1276958.1277158 · Cited by 140
A novel generative encoding for exploiting neural network sensor and output geometry
D’Ambrosio, D. B., & Stanley, K. O. (2007) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1276958.1277155 · Cited by 97
The 2009 Simulated Car Racing Championship
Loiacono, D., Lanzi, P. L., Togelius, J., Onieva, E., PELTA, D. A., Butz, M. V., Lonneker, T. D., Cardamone, L., Pérez-Liébana, D., Sáez, Y., et al. (2010) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2010.2050590 · Cited by 91
Evolving neural networks in compressed weight space
Koutnik, J., Gomez, F., & Schmidhuber, J. (2010) · Proceedings of the 12th annual conference on Genetic and evolutionary computation · doi:10.1145/1830483.1830596 · Cited by 74
Self-citationAn adaptive 'broom balancer' with visual inputs
Tolat, & Widrow (1988) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/icnn.1988.23982 · Cited by 62
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Self-citationDeep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
“eschewed the commonly used backpropagation algorithm in favour of evolutionary algorithms [32, 19, 53], which are gradient-free policy search algorithms.”
InfluentialA Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
A Survey of Autonomous Driving: Common Practices and Emerging Technologies
Yurtsever, E., Lambert, J., Carballo, A., & Takeda, K. (2020) · IEEE Access · doi:10.1109/access.2020.2983149 · Cited by 1,889
“Furthermore, these design philosophies are realized with two alternative approaches: modular [15], [47], [51]–[58] or endto-end driving [59]–[67].”
BackgroundMethodResultInfluentialDeep reinforcement learning for robotic manipulation with asynchronous off-policy updates
Gu, S., Holly, E., Lillicrap, T., & Levine, S. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989385 · Cited by 1,424
A Survey on Deep Learning: Algorithms, Techniques, and Applications
Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., Reyes, M. P., Shyu, M.-L., Chen, S.-C., & Iyengar, S. S. (2019) · ACM Computing Surveys · doi:10.1145/3234150 · Cited by 1,403
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Imitation Learning: A Survey of Learning Methods
Hussein, A., Gaber, M. M., Elyan, E., & Jayne, C. (2018) · ACM Computing Surveys · doi:10.1145/3054912 · Cited by 1,060
“Similarly, Koutn´ık et al. [2013] use deep neural networks to learn from video streams in a car racing game.”
BackgroundEvolution Strategies as a Scalable Alternative to Reinforcement Learning
Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1703.03864 · Cited by 1,027
“Koutník et al. [2013, 2010] and Srivastava et al. [2012] have similarly applied an an ES method to RL problems with visual inputs, but where the policy was compressed in a number of different ways.”
MethodMultiagent cooperation and competition with deep reinforcement learning
Tampuu, A., Matiisen, T., Kodelja, D., Kuzovkin, I., Korjus, K., Aru, J., Aru, J., & Vicente, R. (2017) · PLOS ONE · doi:10.1371/journal.pone.0172395 · Cited by 877
Reinforcement learning for control: Performance, stability, and deep approximators
Buşoniu, L., de Bruin, T., Tolić, D., Kober, J., & Palunko, I. (2018) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2018.09.005 · Cited by 463
Playing FPS Games with Deep Reinforcement Learning
Lample, G., & Chaplot, D. S. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10827 · Cited by 452
Recurrent World Models Facilitate Policy Evolution
Ha, D., & Schmidhuber, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.01999 · Cited by 429
Self-citationVirtual to Real Reinforcement Learning for Autonomous Driving
Pan, X., You, Y., Wang, Z., & Lu, C. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.11 · Cited by 326
Learning to Navigate in Complex Environments
Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A. J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03673 · Cited by 323
Continuous Deep Q-Learning with Model-based Acceleration
Gu, S., Lillicrap, T., Sutskever, I., & Levine, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00748 · Cited by 304
Language Understanding for Text-based Games using Deep Reinforcement Learning
Narasimhan, K., Kulkarni, T., & Barzilay, R. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1001 · Cited by 285
Deep Reinforcement Learning: An Overview
Mousavi, S. S., Schukat, M., & Howley, E. (2018) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-319-56991-8_32 · Cited by 261
Deep Learning for Video Game Playing
Justesen, N., Bontrager, P., Togelius, J., & Risi, S. (2020) · IEEE Transactions on Games · doi:10.1109/tg.2019.2896986 · Cited by 257
Analyzing Classifiers: Fisher Vectors and Deep Neural Networks
Lapuschkin, S., Binder, A., Montavon, G., Muller, K.-R., & Samek, W. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.318 · Cited by 201
Deep reinforcement learning in transportation research: A review
Farazi, N. P., Zou, B., Ahamed, T., & Barua, L. (2021) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2021.100425 · Cited by 202
Ha, D., & Schmidhuber, J. (2018) · Zenodo · doi:10.5281/zenodo.1207631 · Cited by 187
Self-citationFrom Pixels to Torques: Policy Learning with Deep Dynamical Models
Wahlström, N., Schön, T. B., & Deisenroth, M. P. (2015) · arXiv (Cornell University) · doi:10.25561/21120 · Cited by 73
Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications
Müller, M., Casser, V., Lahoud, J., Smith, N., & Ghanem, B. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1073-7 · Cited by 178
Santana, E., & Hotz, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.01230 · Cited by 174
Neuroevolution in Games: State of the Art and Open Challenges
Risi, S., & Togelius, J. (2015) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2015.2494596 · Cited by 173
Annotated History of Modern AI and Deep LearningPreprint
Schmidhuber, J. (2022) · King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) · doi:10.48550/arxiv.2212.11279 · Cited by 175
Self-citationSurvey on Artificial Intelligence for Vehicles
Li, J., Cheng, H., Guo, H., & Qiu, S. (2018) · Automotive Innovation · doi:10.1007/s42154-018-0009-9 · Cited by 171
Human-like Autonomous Vehicle Speed Control by Deep Reinforcement Learning with Double Q-Learning
Zhang, Y. H., Sun, P., Yin, Y., Lin, L., & Wang, X. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500630 · Cited by 162
Brain-Inspired Cognitive Model With Attention for Self-Driving Cars
Chen, S., Zhang, S., Shang, J., Chen, B., & Zheng, N. (2017) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2017.2717451 · Cited by 160
Stable reinforcement learning with autoencoders for tactile and visual data
van Hoof, H., Chen, N., Karl, M., van der Smagt, P., & Peters, J. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759578 · Cited by 141
Schmidhuber, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.09249 · Cited by 138
Self-citationEvolutionary Reinforcement Learning: A Survey
Bai, H., Cheng, R., & Jin, Y. (2023) · Intelligent Computing · doi:10.34133/icomputing.0025 · Cited by 138
Evolving deep unsupervised convolutional networks for vision-based reinforcement learning
Koutník, J., Schmidhuber, J., & Gomez, F. J. (2014) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2576768.2598358 · Cited by 131
Self-citationTuray, T., & Vladimirova, T. (2022) · IEEE Access · doi:10.1109/access.2022.3147495 · Cited by 136
Ye, F., Zhang, S., Wang, P., & Chan, C. (2021) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv48863.2021.9575880 · Cited by 121
Query-Efficient Imitation Learning for End-to-End Autonomous Driving
Zhang, J., & Cho, K. (2016) · arXiv · doi:10.48550/arxiv.1605.06450 · Cited by 118
RUDDER: Return Decomposition for Delayed Rewards
Arjona-Medina, J. A., Gillhofer, M., Widrich, M., Unterthiner, T., Brandstetter, J., & Hochreiter, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07857 · Cited by 104
Query-Efficient Imitation Learning for End-to-End Simulated Driving
Zhang, J., & Cho, K. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10857 · Cited by 102
Playing FPS Games with Deep Reinforcement Learning
Lample, G., & Chaplot, D. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.05521 · Cited by 96
Ohn-Bar, E., Prakash, A., Behl, A., Chitta, K., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01131 · Cited by 95
A Gradient-Guided Evolutionary Approach to Training Deep Neural Networks
Yang, S., Tian, Y., He, C., Zhang, X., Tan, K. C., & Jin, Y. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3061630 · Cited by 85
Vision-Based Mobile Robotics Obstacle Avoidance With Deep Reinforcement Learning
Wenzel, P., Schön, T., Leal-Taixé, L., & Cremers, D. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9560787 · Cited by 77
Controlling the Solo12 quadruped robot with deep reinforcement learning
Aractingi, M., Léziart, P., Flayols, T., Pérez, J., Silander, T., & Souères, P. (2023) · Scientific Reports · doi:10.1038/s41598-023-38259-7 · Cited by 76
Some Considerations on Learning to Explore via Meta-Reinforcement Learning
Stadie, B. C., Yang, G., Houthooft, R., Chen, X., Duan, Y., Wu, Y., Abbeel, P., & Sutskever, I. (2018) · arXiv · doi:10.48550/arxiv.1803.01118 · Cited by 74
“ly, a flurry of new work in Deep Reinforcement Learning has provided the foundations for tackling RL problems that were previously thought intractable. This work includes: 1) (Mnih et al., 2015; 2016; Koutnk et al., 2013), which allow for discrete control in complex environments directly from raw images. 2) (Schulman e…”
BackgroundDeep reinforcement learning challenges and opportunities for urban water systems
Negm, A. S., Ma, X., & Aggidis, G. (2024) · Water Research · doi:10.1016/j.watres.2024.121145 · Cited by 70
Choudhury, S., Narayanan, B., Moret, M., Hatzimanikatis, V., & Miskovic, L. (2024) · Nature Catalysis · doi:10.1038/s41929-024-01220-6 · Cited by 67
Autoencoder-augmented neuroevolution for visual doom playing
Alvernaz, S., & Togelius, J. (2017) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2017.8080408 · Cited by 67
Learning End-to-End Autonomous Steering Model from Spatial and Temporal Visual Cues
Chi, L., & Mu, Y. (2017) · doi:10.1145/3132734.3132737 · Cited by 67
Wan, Z., Shen, J., Chuang, J., Xia, X., Garcia, J., Ma, J., & Chen, Q. A. (2022) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.24177 · Cited by 63
Imitation Learning:A Survey of Learning Methods
Hussein, A., Gaber, M. M., Elyan, E., & Jayne, C. (2017) · ACM Computing Surveys · doi:10.1145/3071073 · Cited by 62
Derivative-free reinforcement learning: a review
Qian, H., & Yu, Y. (2021) · Frontiers of Computer Science · doi:10.1007/s11704-020-0241-4 · Cited by 59
Learning to Play No-Press Diplomacy with Best Response Policy IterationPreprint
Anthony, T., Eccles, T., Tacchetti, A., Kramár, J., Gemp, I., Hudson, T. C., Porcel, N., Lanctot, M., Pérolat, J., Everett, R., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04635 · Cited by 57
Modeling of Magnetic Fields and Extended Objects for Localization Applications
Wahlström, N. (2015) · Linköping University Electronic Press eBooks · doi:10.3384/diss.diva-122396 · Cited by 52
Neuroevolution of self-interpretable agents
Tang, Y., Nguyen, D., & Ha, D. (2020) · Proceedings of the 2020 Genetic and Evolutionary Computation Conference · doi:10.1145/3377930.3389847 · Cited by 51
Liu, M., Zhao, F., Yin, J., Niu, J., & Liu, Y. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3066366 · Cited by 51
Lifted Relational Neural NetworksPreprint
Šír, G., Aschenbrenner, V., Železný, F., & Kuželka, O. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.05128 · Cited by 50
Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Driving
Yurtsever, E., Capito, L., Redmill, K., & Ozgune, U. (2020) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv47402.2020.9304735 · Cited by 47
CNN Based Traffic Sign Recognition for Mini Autonomous Vehicles
Satılmış, Y., Tufan, F., Şara, M., Karslı, M., Eken, S., & Sayar, A. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-99996-8_8 · Cited by 44
Autonomous Driving in Reality with Reinforcement Learning and Image TranslationPreprint
Xu, N., Tan, B., & Kong, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.05299 · Cited by 42
MADRaS : Multi Agent Driving Simulator
Santara, A., Rudra, S., Buridi, S. A., Kaushik, M., Naik, A., Kaul, B., & Ravindran, B. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12531 · Cited by 40
One Big Net For EverythingPreprint
Schmidhuber, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08864 · Cited by 38
Self-citationHuman-in-the-loop reinforcement learning
Liang, H., Lu, Y., Cheng, H., Tu, W., & Xu, M. (2017) · Chinese Automation Congress · doi:10.1109/cac.2017.8243575 · Cited by 36
Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programs
Najarro, E., Sudhakaran, S., & Risi, S. (2023) · Conference on Artificial Life (ALIFE) · doi:10.1162/isal_a_00697 · Cited by 36
Comparing driving behavior of humans and autonomous driving in a professional racing simulator
Remonda, A., Veas, E. E., & Luzhnica, G. (2021) · PLoS ONE · doi:10.1371/journal.pone.0245320 · Cited by 34
An Introduction to Lifelong Supervised LearningPreprint
Sodhani, S., Faramarzi, M., Mehta, S. V., Malviya, P., Abdelsalam, M., Janarthanan, J., & Chandar, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.04354 · Cited by 35
Enhancing Text-based Reinforcement Learning Agents with Commonsense KnowledgePreprint
Murugesan, K., Atzeni, M., Shukla, P., Sachan, M., Kapanipathi, P., & Talamadupula, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00811 · Cited by 32
HyperNCA: Growing Developmental Networks with Neural Cellular AutomataPreprint
Najarro, E., Sudhakaran, S., Glanois, C., & Risi, S. (2022) · IT University Of Copenhagen (IT University of Copenhagen) · doi:10.48550/arxiv.2204.11674 · Cited by 32
Fast DENSER: Efficient Deep NeuroEvolution
Assunção, F., Lourenço, N., Machado, P., & Ribeiro, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-16670-0_13 · Cited by 31
ReinforcementDriving: Exploring Trajectories and Navigation for Autonomous Vehicles
Liu, M., Zhao, F., Niu, J., & Liu, Y. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2960872 · Cited by 31
Teaching UAVs to Race: End-to-End Regression of Agile Controls in Simulation
Müller, M., Casser, V., Smith, N., Michels, D. L., & Ghanem, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11012-3_2 · Cited by 28
Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular DecompositionPreprint
Heydari, J., Saha, O., & Ganapathy, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09018 · Cited by 27
Do Artificial Reinforcement-Learning Agents Matter Morally?Preprint
Tomasik, B. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.8233 · Cited by 26
SAD-GAN: Synthetic Autonomous Driving using Generative Adversarial NetworksPreprint
Ghosh, A., Bhattacharya, B., & Chowdhury, S. B. R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.08788 · Cited by 26
Online Evolution of Deep Convolutional Network for Vision-Based Reinforcement Learning
Koutník, J., Schmidhuber, J., & Gomez, F. J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-08864-8_25 · Cited by 25
Self-citationA Wavelet-based Encoding for Neuroevolution
van Steenkiste, S., Koutník, J., Driessens, K., & Schmidhuber, J. (2016) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2908812.2908905 · Cited by 24
Self-citationGPLAC: Generalizing Vision-Based Robotic Skills Using Weakly Labeled Images
Singh, A., Yang, L., & Levine, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.623 · Cited by 22
UE4Sim: A Photo-Realistic Simulator for Computer Vision ApplicationsPreprint
Mueller, M. F., Casser, V., Lahoud, J., Smith, N., & Ghanem, B. (2017) · King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) · Cited by 22
Evolutionary Echo State Network: A neuroevolutionary framework for time series prediction
Basterrech, S., & Rubino, G. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110463 · Cited by 20
Learning to Generalize With Object-Centric Agents in the Open World Survival Game Crafter
Stanić, A., Tang, Y.-J., Ha, D., & Schmidhuber, J. (2022) · IEEE Transactions on Games · doi:10.1109/tg.2023.3276849 · Cited by 16
Self-citationA Restart-based Rank-1 Evolution Strategy for Reinforcement Learning
Chen, Z., Zhou, Y., He, X., & Jiang, S. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/295 · Cited by 19
Multi-Task End-to-End Self-Driving Architecture for CAV Platoons
Huch, S., Öngel, A., Betz, J., & Lienkamp, M. (2021) · Sensors · doi:10.3390/s21041039 · Cited by 18
Learning to Identify Critical States for Reinforcement Learning from Videos
Liu, H., Zhuge, M., Li, B., Wang, Y., Faccio, F., Ghanem, B., & Schmidhuber, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00187 · Cited by 18
Self-citationAutonomous vehicle control via deep reinforcement learning
Kardell, S., & Kuosku, M. (2017) · Chalmers Publication Library (Chalmers University of Technology) · Cited by 18
Exploiting Generalization in the Subspaces for Faster Model-Based Reinforcement Learning
Hashemzadeh, M., Hosseini, R., & Ahmadabadi, M. N. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2869978 · Cited by 17
Li, X., Wang, Y., Wang, K., Yan, L., & Wang, F. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500459 · Cited by 16
Liu, D., Zhao, J., Xi, A., Wang, C., Huang, X., Lai, K., & Liu, C. (2020) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2020.08641 · Cited by 15
Deep Learning and Reward Design for Reinforcement Learning
Guo, X. (2017) · Deep Blue (University of Michigan) · Cited by 15
Design Space of Behaviour Planning for Autonomous DrivingPreprint
Ilievski, M., Sedwards, S., Gaurav, A., Balakrishnan, A., Sarkar, A., Lee, J., Bouchard, F., Iaco, R. D., & Czarnecki, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.07931 · Cited by 15
End-to-End Adaptive Cruise Control Based on Timing Network
Cao, Z. L., Yang, D., Jiang, K., Wang, T., Jiao, X., & Xiao, Z. (2018) · Lecture notes in electrical engineering · doi:10.1007/978-981-10-8506-2_56 · Cited by 14
An Evolutionary Approach to Variational Autoencoders
Hajewski, J., & Oliveira, S. (2020) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc47524.2020.9031239 · Cited by 14
Reinforcement Learning for Multimodal Foundation Models: A Survey
Wu, W., Gao, C., Chen, J., Lin, Q., Meng, Q.-W., Zhang, Y., Qiu, Y., Zhou, H., & Shou, M. Z. (2026) · ACM Computing Surveys · doi:10.1145/3837760 · Cited by 13
Mapping Ensembles of Trees to Sparse, Interpretable Multilayer Perceptron Networks
Rodríguez-Salas, D., Mürschberger, N., Ravikumar, N., Seuret, M., & Maier, A. (2020) · SN Computer Science · doi:10.1007/s42979-020-00268-y · Cited by 13
Cognitive map-based model: Toward a developmental framework for self-driving cars
Chen, S., Shang, J., Zhang, S., & Zheng, N. (2017) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2017.8317627 · Cited by 13
Ali, M. H., Hamdan, A., & Alareeni, B. A. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-69221-6_12 · Cited by 13
Deep Learning: Our Miraculous Year 1990-1991Preprint
Schmidhuber, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05744 · Cited by 13
Self-citationDistributed Evolution of Deep Autoencoders
Hajewski, J., Oliveira, S., & Xing, X. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-80119-9_6 · Cited by 12
Information Optimization and Transferable State Abstractions in Deep Reinforcement Learning
Gomez, D., Quijano, N., & Giraldo, L. F. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3200726 · Cited by 12
Learning Useful Representations of Recurrent Neural Network Weight MatricesPreprint
Herrmann, V., Faccio, F., & Schmidhuber, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.11998 · Cited by 12
Self-citationA Review of Causal Decision Making
Ge, L., Cai, H., Wan, R., Xu, Y., & Song, R. (2026) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.21001 · Cited by 12
Metadata
- Authors
- Koutník, J., Cuccu, G., Schmidhuber, J., & Gomez, F.
- Affiliations
- University of Applied Sciences and Arts of Southern Switzerland (CH)
- Venue
- Proceedings of the 15th annual conference on Genetic and evolutionary computation
- Publisher
- ACM
- Published
- 2013-07-06
- Pages
- 1061-1068
- OpenAlex
- W2038794597
- Cited by
- 130
- References
- 16
- Citations by year
- 2025: 2 · 2024: 2 · 2023: 10 · 2022: 9 · 2021: 15 · 2020: 18 · 2019: 9 · 2018: 19 · 2017: 19 · 2016: 12 · 2015: 8 · 2014: 7
- FWCI
- 5.81
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Genetic and Evolutionary Computation Conference (GECCO) (publishedVersion)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- neuroevolution, large-scale neural networks, vision-based reinforcement learning, high-dimensional input space
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.