Quality Diversity: A New Frontier for Evolutionary Computation
Open accessAbstract
While evolutionary computation and evolutionary robotics take inspiration from nature, they have long focused mainly on problems of performance optimization. Yet evolution in nature can be interpreted as more nuanced than a process of simple optimization. In particular, natural evolution is a divergent search that optimizes locally within each niche as it simultaneously diversifies. This tendency to discover both quality and diversity at the same time differs from many of the conventional algorithms of machine learning, and also thereby suggests a different foundation for inferring the approach of greatest potential for evolutionary algorithms. In fact, several recent evolutionary algorithms called quality diversity (QD) algorithms(e.g. novelty search with local competition and MAP-Elites) have drawn inspiration from this more nuanced view, aiming to fill a space of possibilities with the best possible example of each type of achievable behavior. The result is a new class of algorithms that return an archive of diverse, high-quality behaviors in a single run. The aim in this paper is to study the application of QD algorithms in challenging environments (in particular complex mazes) to establish their best practices for ambitious domains in the future. In addition to providing insight into cases when QD succeeds and fails, a new approach is investigated that hybridizes multiple views of behaviors (called behavior characterizations) in the same run, which succeeds in overcoming some of the challenges associated with searching for QD with respect to a behavior characterization that is not necessarily sufficient for generating both quality and diversity at the same time.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Under Quality-Diversity optimization, the search space is divided into behavioral niches to evolve diverse solutions, a paradigm that challenged the dominant approach of training a single agent, as it proved more robust in real-world settings.
“under Quality-Diversity optimization (Pugh et al., 2016), the search space is divided into behavioral niches to evolve diverse solutions, a paradigm that challenged the dominant approach of training a single agent, as it proved more robust in real-world settings (Cully et al., 2015).”
From Dynamics of niche construction in adaptable populations evolving in diverse environments (2023) · cites this paper as Pugh et al., 2016
The clustering approach aims to preserve diversity and is related to Lexicase and fitness uniform optimization.
“This approach, which aims to preserve diversity55,56, is related to Lexicase57 in that both approaches consider a set of test cases for scoring an individual, and it is related to fitness uniform optimization58, which also clusters individuals on the basis of their fitness value; however, we sample the clusters on the basis of their score instead of uniformly, as detailed next.”
From Mathematical discoveries from program search with large language models (2024) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421 · see citing works
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053 · see citing works
Genetic algorithms in search, optimization, and machine learning
(1989) · Choice Reviews Online · doi:10.5860/choice.27-0936 · Cited by 49,169 · see citing works
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565 · see citing works
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,501 · see citing works
Evolving Neural Networks through Augmenting Topologies
Stanley, K. O., & Miikkulainen, R. P. (2002) · Evolutionary Computation · doi:10.1162/106365602320169811 · Cited by 3,269 · see citing works
Self-citationDeep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258 · see citing works
Artificial Intelligence through Simulated Evolution
Fogel, L. J., Owens, A. J., & Walsh, M. J. (1966) · International Linear Collider · Cited by 2,814 · see citing works
Pattern recognition and machine learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5091 · Cited by 2,782 · see citing works
Genetic algorithms with sharing for multimodal function optimization
Goldberg, D. E., & Richardson, J. T. (1987) · international conference on Genetic algorithms · Cited by 1,891 · see citing works
Evolutionary computation: Comments on the history and current state
Back, T., Hammel, U., & Schwefel, H.-P. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585888 · Cited by 1,536 · cited 1× · see citing works
“The casting of evolution as an algorithm for optimization naturally pitted evolutionary computation against the many subfields of machine learning invested in optimization, leading to conflicts and critiques, sometimes portraying evolutionary computation as ad hoc, unprincipled, less effective than other more theoreti…”
MethodNolfi, S., & Floreano, D. (2000) · The MIT Press eBooks · doi:10.7551/mitpress/2889.001.0001 · Cited by 1,531 · see citing works
Abandoning Objectives: Evolution Through the Search for Novelty Alone
Lehman, J., & Stanley, K. O. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00025 · Cited by 1,216 · see citing works
Self-citationKirschner, M., & Gerhart, J. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.15.8420 · Cited by 1,129 · see citing works
Simon, H. A. (1957) · Cited by 1,086 · see citing works
Robots that can adapt like animals
Cully, A., Clune, J., Tarapore, D., & Mouret, J.-B. (2015) · Nature · doi:10.1038/nature14422 · Cited by 979 · see citing works
Compositional pattern producing networks: A novel abstraction of development
Stanley, K. O. (2007) · Genetic Programming and Evolvable Machines · doi:10.1007/s10710-007-9028-8 · Cited by 714 · see citing works
Self-citationEvolution and Optimum Seeking: The Sixth Generation
Schwefel, H. (1993) · John Wiley & Sons, Inc. eBooks · Cited by 653 · see citing works
A Taxonomy for Artificial Embryogeny
Stanley, K. O., & Miikkulainen, R. P. (2003) · Artificial Life · doi:10.1162/106454603322221487 · Cited by 426 · see citing works
Self-citationExplorations in Evolutionary Robotics
Cliff, D., Husbands, P., & Harvey, I. R. (1993) · Adaptive Behavior · doi:10.1177/105971239300200104 · Cited by 415 · see citing works
Illuminating search spaces by mapping elitesPreprint
Mouret, J.-B., & Clune, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.04909 · Cited by 407 · see citing works
Exploiting Open-Endedness to Solve Problems Through the Search for Novelty
Lehman, J., & Stanley, K. O. (2008) · Cited by 393 · see citing works
Self-citationEvolving a diversity of virtual creatures through novelty search and local competition
Lehman, J., & Stanley, K. O. (2011) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2001576.2001606 · Cited by 391 · see citing works
Self-citationEvolutionary Robotics: What, Why, and Where to
Doncieux, S., Bredèche, N., Mouret, J.-B., & Eiben, A. E. (2015) · Frontiers in Robotics and AI · doi:10.3389/frobt.2015.00004 · Cited by 278 · see citing works
Encouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study
Mouret, J.-B., & Doncieux, S. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00048 · Cited by 254 · see citing works
Creating High-Level Components with a Generative Representation for Body-Brain Evolution
Hornby, G. S., & Pollack, J. B. (2002) · Artificial Life · doi:10.1162/106454602320991837 · Cited by 226 · see citing works
Evolving modular genetic regulatory networks
Bongard, J. (2003) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2002.1004528 · Cited by 185 · see citing works
Revising the evolutionary computation abstraction
Lehman, J., & Stanley, K. O. (2010) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1830483.1830503 · Cited by 182 · see citing works
Self-citationThe advantages of generative grammatical encodings for physical design
Hornby, G. S., & Pollack, J. B. (2002) · Congress on Evolutionary Computation · doi:10.1109/cec.2001.934446 · Cited by 176 · see citing works
Novelty-Based Multiobjectivization
Mouret, J.-B. (2011) · Studies in computational intelligence · doi:10.1007/978-3-642-18272-3_10 · Cited by 139 · see citing works
Confronting the Challenge of Quality Diversity
Pugh, J. K., Soros, L. B., Szerlip, P., & Stanley, K. O. (2015) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2739480.2754664 · Cited by 135 · see citing works
Self-citationEvolution of swarm robotics systems with novelty search
Gomes, J., Urbano, P., & Christensen, A. L. (2013) · Swarm Intelligence · doi:10.1007/s11721-013-0081-z · Cited by 124 · see citing works
Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity
Mouret, J.-B., & Doncieux, S. (2009) · 2009 IEEE Congress on Evolutionary Computation · doi:10.1109/cec.2009.4983077 · Cited by 106 · see citing works
OPEN-ENDED ARTIFICIAL EVOLUTION
Standish, R. K. (2003) · International Journal of Computational Intelligence and Applications · doi:10.1142/s1469026803000914 · Cited by 93 · see citing works
Nguyen, A. M., Yosinski, J., & Clune, J. (2015) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2739480.2754703 · Cited by 92 · see citing works
Devising Effective Novelty Search Algorithms
Gomes, J., Mariano, P., & Christensen, A. L. (2015) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2739480.2754736 · Cited by 80 · see citing works
Evolving plastic neural networks with novelty search
Risi, S., Hughes, C. E., & Stanley, K. O. (2010) · Adaptive Behavior · doi:10.1177/1059712310379923 · Cited by 75 · see citing works
Self-citationHow novelty search escapes the deceptive trap of learning to learn
Risi, S., Vanderbleek, S. D., Hughes, C. E., & Stanley, K. O. (2009) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1569901.1569923 · Cited by 75 · see citing works
Self-citationBehavioral repertoire learning in robotics
Cully, A., & Mouret, J.-B. (2013) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2463372.2463399 · Cited by 70 · see citing works
On the deleterious effects of a priori objectives on evolution and representation
Woolley, B. G., & Stanley, K. O. (2011) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2001576.2001707 · Cited by 64 · see citing works
Self-citationEvolvability Is Inevitable: Increasing Evolvability without the Pressure to Adapt
Lehman, J., & Stanley, K. O. (2013) · PLoS ONE · doi:10.1371/journal.pone.0062186 · Cited by 58 · see citing works
Self-citationTransforming Exploratory Creativity with DeLeNoX
Liapis, A., Martínez, H. P., Togelius, J., & Yannakakis, G. N. (2021) · IT University Of Copenhagen (IT University of Copenhagen) · doi:10.48550/arxiv.2103.11715 · Cited by 52 · see citing works
Generic behaviour similarity measures for evolutionary swarm robotics
Gomes, J., & Christensen, A. L. (2013) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2463372.2463398 · Cited by 48 · see citing works
Critical factors in the performance of novelty search
Kistemaker, S., & Whiteson, S. (2011) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2001576.2001708 · Cited by 41 · see citing works
Searching for novel clustering programs
Naredo, E., & Trujillo, L. (2013) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2463372.2463505 · Cited by 39 · see citing works
Enhancements to constrained novelty search
Liapis, A., Yannakakis, G. N., & Togelius, J. (2013) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2463372.2463416 · Cited by 38 · see citing works
Creative Generation of 3D Objects with Deep Learning and Innovation Engines.
Lehman, J., Risi, S., & Clune, J. (2016) · IT University Of Copenhagen (IT University of Copenhagen) · Cited by 38 · see citing works
Speciation in Behavioral Space for Evolutionary Robotics
Trujillo, L., Gustavo Olague, A., Lutton, É., de Vega, F. F., Dozal, L., & Clemente, E. (2011) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-011-9542-z · Cited by 31 · see citing works
Single-unit pattern generators for quadruped locomotion
Morse, G., Risi, S., Snyder, C. R., & Stanley, K. O. (2013) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2463372.2463461 · Cited by 30 · see citing works
Self-citationMind As Machine: A History of Cognitive Science Two-Volume Set
Boden, M. (2006) · Choice Reviews Online · doi:10.5860/choice.44-6202 · Cited by 29 · see citing works
Automatically Discovering Properties that Specify the Latent Behavior of UML Models
Goldsby, H. J., & Cheng, B. H. C. (2010) · Cited by 27 · see citing works
Novelty search creates robots with general skills for exploration
Velez, R., & Clune, J. (2014) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2576768.2598225 · Cited by 26 · see citing works
Discovering Several Robot Behaviors through Speciation
Trujillo, L., Gustavo Olague, A., Lutton, É., & de Vega, F. F. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-78761-7_17 · Cited by 25 · see citing works
Confronting the challenge of learning a flexible neural controller for a diversity of morphologies
Risi, S., & Stanley, K. O. (2013) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2463372.2463397 · Cited by 25 · see citing works
Self-citationSearching for novel regression functions
Martínez, Y., Naredo, E., Trujillo, L., & Galván, E. (2013) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2013.6557548 · Cited by 24 · see citing works
Novelty-Based Fitness: An Evaluation under the Santa Fe Trail
Doucette, J. A., & Heywood, M. I. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-12148-7_5 · Cited by 23 · see citing works
Solving Deceptive Tasks in Robot Body-Brain Co-evolution by Searching for Behavioral Novelty
Krčah, P. (2011) · Intelligent systems reference library · doi:10.1007/978-3-642-23363-0_7 · Cited by 23 · see citing works
Novelty of behaviour as a basis for the neuro-evolution of operant reward learning
Soltoggio, A., & Jones, B. (2009) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1569901.1569925 · Cited by 21 · see citing works
Automatically Discovering Properties That Specify the Latent Behavior of UML Models
Goldsby, H. J., & Cheng, B. H. C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-16145-2_22 · Cited by 16 · see citing works
Indirectly Encoded Sodarace for Artificial Life
Szerlip, P., & Stanley, K. O. (2013) · doi:10.7551/978-0-262-31709-2-ch033 · Cited by 12 · see citing works
Self-citationUnsupervised Feature Learning through Divergent Discriminative Feature Accumulation
Szerlip, P., Morse, G., Pugh, J. K., & Stanley, K. O. (2015) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v29i1.9601 · Cited by 11 · see citing works
Self-citationSolving deceptive tasks in robot body-brain co-evolution by searching for behavioral novelty
Krčah, P. (2010) · International Conference on Intelligent Systems Design and Applications (ISDA) · doi:10.1109/isda.2010.5687250 · Cited by 10 · see citing works
Towards Directed Open-Ended Search by a Novelty Guided Evolution Strategy
Graening, L., Aulig, N., & Olhofer, M. (2010) · Springer eBooks · doi:10.1007/978-3-642-15871-1_8 · Cited by 7 · see citing works
Why Evolutionary Robotics Will Matter
Stanley, K. O. (2011) · Studies in computational intelligence · doi:10.1007/978-3-642-18272-3_3 · Cited by 5 · see citing works
Self-citationEnhancements to Constrained Novelty Search Two-Population Novelty Search for Generating Game Content
Liapis, A., Yannakakis, G. N., Togelius, J., & 2013, F. A. C. O. G. A. E. C. (. (2013) · IT University Of Copenhagen (IT University of Copenhagen) · Cited by 3 · see citing works
Flagged first, then by citations
3 of the 68 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Bedau, M. (2008). "The arrow of complexity hypothesis (abstract), " in Proceedings of the Eleventh International Conference on Artificial Life (Alife XI), eds S. Bullock, J. Noble, R. Watson, and M. Bedau (Cambridge, MA: MIT Press), 750.
- Boden, M. (2006). Mind as Machine: A History of Cognitive Science. USA: Oxford University Press.
- De Jong, K. A. (2002). Evolutionary Computation: A Unified Perspective. Cambridge, MA: MIT Press.
- Holland, J. H. (1975). Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. Ann Arbor, MI: University of Michigan Press.
- Mahfoud, S. W. (1995). Niching Methods for Genetic Algorithms. Ph.D. thesis, University of Illinois at Urbana-Champaign, Urbana, IL.
- Mitchell, T. M. (1997). Machine Learning. New York, NY: McGraw-Hill.
Cited by
Sorted by citations
Designing neural networks through neuroevolution
Stanley, K. O., Clune, J., Lehman, J., & Miikkulainen, R. P. (2018) · Nature Machine Intelligence · doi:10.1038/s42256-018-0006-z · Cited by 702 · see citing works
Self-citationDifferential Evolution: A Recent Review Based on State-of-the-Art Works
Ahmad, M. F., Isa, N. A. M., Lim, W. H., & Ang, K. M. (2022) · Alexandria Engineering Journal · doi:10.1016/j.aej.2021.09.013 · Cited by 579 · see citing works
Such, F. P., Madhavan, V., Conti, E., Lehman, J., Stanley, K. O., & Clune, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06567 · Cited by 689 · see citing works
Self-citationExploration in deep reinforcement learning: A survey
Ladosz, P., Weng, L., Kim, M., & Oh, H. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.003 · Cited by 525 · see citing works
“More detailed reviews of exploration in evolution strategies can be found in [78] and [79].”
BackgroundMathematical discoveries from program search with large language models
Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J. S., Wang, P., Fawzi, O., et al. (2024) · Nature · doi:10.1038/s41586-023-06924-6 · Cited by 544 · see citing works
“This approach, which aims to preserve diversity 55,56 , is related to Lexicase 57 in that both approaches consider a set of test cases for scoring an individual, and it is related to fitness uniform optimization 58 , which also clusters individuals on the basis of their fitness value; however, we…”
BackgroundXavier, M. S., Tawk, C., Zolfagharian, A., Pinskier, J., Howard, D., Young, T. R., Lai, J., Harrison, S. M., Yong, Y. K., Bodaghi, M., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3179589 · Cited by 440 · see citing works
Explainability in deep reinforcement learning
Heuillet, A., Couthouis, F., & Díaz-Rodríguez, N. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106685 · Cited by 337 · see citing works
Quality and Diversity Optimization: A Unifying Modular Framework
Cully, A., & Demiris, Y. (2018) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2017.2704781 · Cited by 258 · see citing works
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., & Clune, J. (2021) · RePEc: Research Papers in Economics · Cited by 201 · see citing works
Self-citationJiang, Y., Salley, D., Sharma, A., Keenan, G., Mullin, M., & Cronin, L. (2022) · Science Advances · doi:10.1126/sciadv.abo2626 · Cited by 194 · see citing works
Evolving embodied intelligence from materials to machines
Howard, D., Eiben, A. E., Kennedy, D. F., Mouret, J.-B., Valencia, P., & Winkler, D. A. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0009-9 · Cited by 164 · see citing works
A Survey on Policy Search Algorithms for Learning Robot Controllers in a Handful of Trials
Chatzilygeroudis, K., Vassiliades, V., Stulp, F., Calinon, S., & Mouret, J.-B. (2020) · IEEE Transactions on Robotics · doi:10.1109/tro.2019.2958211 · Cited by 160 · see citing works
Vassiliades, V., Chatzilygeroudis, K. I., & Mouret, J.-B. (2017) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2017.2735550 · Cited by 152 · see citing works
Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap
Wu, X., Wu, S.-H., Wu, J., Feng, L., & Tan, K. C. (2024) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2024.3506731 · Cited by 177 · see citing works
Covariance matrix adaptation for the rapid illumination of behavior space
Fontaine, M. C., Togelius, J., Nikolaidis, S., & Hoover, A. K. (2020) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3377930.3390232 · Cited by 125 · see citing works
Procedural Content Generation through Quality Diversity
Gravina, D., Khalifa, A., Liapis, A., Togelius, J., & Yannakakis, G. N. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848053 · Cited by 123 · see citing works
Stacking-based multi-objective evolutionary ensemble framework for prediction of diabetes mellitus
Singh, N., & Singh, P. (2019) · Journal of Applied Biomedicine · doi:10.1016/j.bbe.2019.10.001 · Cited by 119 · see citing works
Wang, R., Lehman, J., Clune, J., & Stanley, K. O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01753 · Cited by 166 · see citing works
Self-citationA GAN-Based Surrogate Model for Instantaneous Urban Wind Flow Prediction
Kastner, P., & Dogan, T. (2023) · Building and Environment · doi:10.1016/j.buildenv.2023.110384 · Cited by 113 · see citing works
Conti, E., Madhavan, V., Such, F. P., Lehman, J., Stanley, K. O., & Clune, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06560 · Cited by 108 · see citing works
Self-citationDeep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey
Majid, A. Y., Majid, A. Y., Saaybi, S., van Rietbergen, T., François-Lavet, V., Prasad, R. R. V., & Verhoeven, C. J. M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264540 · Cited by 108 · see citing works
Reset-free Trial-and-Error Learning for Robot Damage Recovery
Chatzilygeroudis, K. I., Vassiliades, V., & Mouret, J.-B. (2017) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2017.11.010 · Cited by 104 · see citing works
Evolutionary Reinforcement Learning: A Survey
Bai, H., Cheng, R., & Jin, Y. (2023) · Intelligent Computing · doi:10.34133/icomputing.0025 · Cited by 101 · see citing works
Quality-Diversity Optimization: A Novel Branch of Stochastic Optimization
Chatzilygeroudis, K. I., Cully, A., Vassiliades, V., & Mouret, J.-B. (2021) · Springer optimization and its applications · doi:10.1007/978-3-030-66515-9_4 · Cited by 93 · see citing works
A substrate-independent framework to characterize reservoir computers
Dale, M., Miller, J. F., Stepney, S., & Trefzer, M. A. (2019) · Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rspa.2018.0723 · Cited by 85 · see citing works
Safe mutations for deep and recurrent neural networks through output gradients
Lehman, J., Chen, J., Clune, J., & Stanley, K. O. (2018) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3205455.3205473 · Cited by 85 · see citing works
Self-citationDiscovering the elite hypervolume by leveraging interspecies correlation
Vassiliades, V., & Mouret, J.-B. (2018) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3205455.3205602 · Cited by 76 · see citing works
Evolution Through Large Models
Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., & Stanley, K. O. (2024) · Genetic and Evolutionary Computation · doi:10.1007/978-981-99-3814-8_11 · Cited by 74 · see citing works
Self-citationPolicy gradient assisted MAP-Elites
Nilsson, O., & Cully, A. (2021) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3449639.3459304 · Cited by 71 · see citing works
Quality diversity for multi-task optimization
Mouret, J.-B., & Maguire, G. (2020) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3377930.3390203 · Cited by 69 · see citing works
Mapping hearthstone deck spaces through MAP-elites with sliding boundaries
Fontaine, M. C., Lee, S., Soros, L. B., Silva, F. D. M., Togelius, J., & Hoover, A. K. (2019) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3321707.3321794 · Cited by 67 · see citing works
Self-citationA Systematic Literature Review of the Successors of “NeuroEvolution of Augmenting Topologies”
Papavasileiou, E., Cornelis, J. P., & Jansen, B. (2020) · Evolutionary Computation · doi:10.1162/evco_a_00282 · Cited by 65 · see citing works
Evolution of Repertoire-Based Control for Robots With Complex Locomotor Systems
Duarte, M., Gomes, J., Oliveira, S. M., & Christensen, A. L. (2017) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2017.2722101 · Cited by 65 · see citing works
Empowering Quality Diversity in Dungeon Design with Interactive Constrained MAP-Elites
Alvarez, A., Dahlskog, S., Font, J., & Togelius, J. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848022 · Cited by 64 · see citing works
ES is more than just a traditional finite-difference approximator
Lehman, J., Chen, J., Clune, J., & Stanley, K. O. (2018) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3205455.3205474 · Cited by 64 · see citing works
Self-citationOn the automated, evolutionary design of neural networks: past, present, and future
Baldominos, A., Sáez, Y., & Isasi, P. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04160-6 · Cited by 61 · see citing works
Mouret, J.-B., & Chatzilygeroudis, K. I. (2017) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3067695.3082052 · Cited by 61 · see citing works
Combining Evolution and Deep Reinforcement Learning for Policy Search: A Survey
Sigaud, O. (2022) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3569096 · Cited by 59 · see citing works
Doncieux, S., Laflaquière, A., & Coninx, A. (2019) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3321707.3321752 · Cited by 55 · see citing works
Data-Efficient Design Exploration through Surrogate-Assisted Illumination
Gaier, A., Asteroth, A., & Mouret, J.-B. (2018) · Evolutionary Computation · doi:10.1162/evco_a_00231 · Cited by 54 · see citing works
Reservoir computing quality: connectivity and topology
Dale, M., O’Keefe, S., Sebald, A., Stepney, S., & Trefzer, M. A. (2020) · Natural Computing · doi:10.1007/s11047-020-09823-1 · Cited by 53 · see citing works
Wang, R., Lehman, J., Rawal, A., Zhi, J., Li, Y., Clune, J., & Stanley, K. O. (2020) · arXiv · doi:10.48550/arxiv.2003.08536 · Cited by 53 · see citing works
Self-citationA survey on Evolutionary Reinforcement Learning algorithms
Zhu, Q., Wu, X., Lin, Q., Ma, L., Li, J., Ming, Z., & Chen, J. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126628 · Cited by 52 · see citing works
Fast Online Adaptation in Robotics through Meta-Learning Embeddings of Simulated Priors
Kaushik, R., Anne, T., & Mouret, J.-B. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341462 · Cited by 50 · see citing works
Gaier, A., Asteroth, A., & Mouret, J.-B. (2017) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3071178.3071282 · Cited by 48 · see citing works
Brant, J. C., & Stanley, K. O. (2017) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3071178.3071186 · Cited by 47 · see citing works
Self-citationDiscovering diverse athletic jumping strategies
Yin, Z., Yang, Z., van de Panne, M., & Yin, K. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459817 · Cited by 48 · see citing works
Baba is Y’all: Collaborative Mixed-Initiative Level Design
Charity, M., Khalifa, A., & Togelius, J. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231807 · Cited by 45 · see citing works
Alvarez, A., Dahlskog, S., Font, J., & Togelius, J. (2020) · IEEE Transactions on Games · doi:10.1109/tg.2020.3046133 · Cited by 41 · see citing works
Diversity policy gradient for sample efficient quality-diversity optimization
Pierrot, T., Macé, V., Chalumeau, F., Flajolet, A., Cideron, G., Beguir, K., Cully, A., Sigaud, O., & Perrin, N. (2022) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3512290.3528845 · Cited by 62 · see citing works
Bayesian Optimization with Automatic Prior Selection for Data-Efficient Direct Policy Search
Pautrat, R., Chatzilygeroudis, K. I., & Mouret, J.-B. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463197 · Cited by 41 · see citing works
Discovering Representations for Black-box Optimization
Gaier, A., Asteroth, A., & Mouret, J.-B. (2020) · Publication Server of Bonn-Rhein-Sieg University of Applied Sciences (Bonn-Rhein-Sieg University of Applied Sciences) · Cited by 41 · see citing works
Stanley, K. O. (2019) · Artificial Life · doi:10.1162/artl_a_00294 · Cited by 38 · see citing works
Self-citationHierarchical behavioral repertoires with unsupervised descriptors
Cully, A., & Demiris, Y. (2018) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3205455.3205571 · Cited by 37 · see citing works
Diverse Agents for Ad-Hoc Cooperation in Hanabi
Canaan, R., Togelius, J., Nealen, A., & Menzel, S. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8847944 · Cited by 37 · see citing works
Fast and stable MAP-Elites in noisy domains using deep grids
Flageat, M., & Cully, A. (2020) · Conference on Artificial Life (ALIFE) · doi:10.1162/isal_a_00316 · Cited by 37 · see citing works
Discovering representations for black-box optimization
Gaier, A., Asteroth, A., & Mouret, J.-B. (2020) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3377930.3390221 · Cited by 36 · see citing works
Empowering Quality Diversity in Dungeon Design with Interactive Constrained MAP-Elites
Alvarez, A., Dahlskog, S., Font, J., & Togelius, J. (2019) · Malmö University Publications (Malmö University) · Cited by 37 · see citing works
MAP-Elites Enables Powerful Stepping Stones and Diversity for Modular Robotics
Nordmoen, J., Veenstra, F., Ellefsen, K. O., & Glette, K. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.639173 · Cited by 36 · see citing works
Unsupervised Behavior Discovery With Quality-Diversity Optimization
Grillotti, L., & Cully, A. (2022) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2022.3159855 · Cited by 36 · see citing works
Reality-Assisted Evolution of Soft Robots through Large-Scale Physical Experimentation: A Review
Howison, T., Hauser, S., Hughes, J., & Fumiya Iida, A. (2020) · Artificial Life · doi:10.1162/artl_a_00330 · Cited by 35 · see citing works
Open-endedness in synthetic biology: A route to continual innovation for biological design
Stock, M., & Gorochowski, T. E. (2024) · Science Advances · doi:10.1126/sciadv.adi3621 · Cited by 33 · see citing works
Kim, S., Coninx, A., & Doncieux, S. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103710 · Cited by 34 · see citing works
Adaptive Prior Selection for Repertoire-Based Online Adaptation in Robotics
Kaushik, R., Desreumaux, P., & Mouret, J.-B. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2019.00151 · Cited by 34 · see citing works
Differentiable Quality DiversityPreprint
Fontaine, M. C., & Nikolaidis, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03894 · Cited by 34 · see citing works
Regenerating Soft Robots Through Neural Cellular Automata
Horibe, K., Walker, K., & Risi, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72812-0_3 · Cited by 33 · see citing works
Policy manifold search: exploring the manifold hypothesis for diversity-based neuroevolution
Rakicevic, N., Cully, A., & Kormushev, P. (2021) · Proceedings of the Genetic and Evolutionary Computation Conference · doi:10.1145/3449639.3459320 · Cited by 32 · see citing works
Recent trends in robot learning and evolution for swarm robotics
Kuckling, J. (2023) · Frontiers in Robotics and AI · doi:10.3389/frobt.2023.1134841 · Cited by 33 · see citing works
Approximating gradients for differentiable quality diversity in reinforcement learning
Tjanaka, B., Fontaine, M. C., Togelius, J., & Nikolaidis, S. (2022) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3512290.3528705 · Cited by 36 · see citing works
Illuminating diverse neural cellular automata for level generation
Earle, S., Snider, J., Fontaine, M. C., Nikolaidis, S., & Togelius, J. (2022) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3512290.3528754 · Cited by 31 · see citing works
Unsupervised Learning and Exploration of Reachable Outcome Space
Paolo, G., Laflaquière, A., Coninx, A., & Doncieux, S. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196819 · Cited by 30 · see citing works
From Machine Learning to Artificial General Intelligence: A Roadmap and Implications
Obaid, O. I. (2023) · Mesopotamian Journal of Big Data · doi:10.58496/mjbd/2023/012 · Cited by 29 · see citing works
Quality Diversity Through Surprise
Gravina, D., Liapis, A., & Yannakakis, G. N. (2018) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2018.2877215 · Cited by 30 · see citing works
Aerodynamic Design Exploration through Surrogate-Assisted Illumination
Gaier, A., Asteroth, A., & Mouret, J.-B. (2017) · AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (MAO) · doi:10.2514/6.2017-3330 · Cited by 29 · see citing works
Qin, H., Xiang, Y., Liu, F., Han, Y., & Wang, Y. (2025) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2025.101849 · Cited by 28 · see citing works
Sarkar, A., & Cooper, S. (2021) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3472538.3472545 · Cited by 28 · see citing works
Prototype Discovery Using Quality-Diversity
Hagg, A., Asteroth, A., & Bäck, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99253-2_40 · Cited by 28 · see citing works
Wang, T., Peng, X., Wang, T., Liu, T., & Xu, D. (2024) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2024.101475 · Cited by 27 · see citing works
Procedural Dungeon Generation: A Survey
Viana, B. M. F., & dos Santos, S. R. (2021) · Journal on Interactive Systems · doi:10.5753/jis.2021.999 · Cited by 27 · see citing works
Urquhart, N. B., & Hart, E. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99253-2_39 · Cited by 26 · see citing works
On the Versatile Uses of Partial Distance Correlation in Deep Learning
Zhen, X., Meng, Z., Chakraborty, R., & Singh, V. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_19 · Cited by 26 · see citing works
Deep surrogate assisted MAP-elites for automated hearthstone deckbuilding
Zhang, Y., Fontaine, M. C., Hoover, A. K., & Nikolaidis, S. (2022) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3512290.3528718 · Cited by 25 · see citing works
Novelty search makes evolvability inevitable
Doncieux, S., Paolo, G., Laflaquière, A., & Coninx, A. (2020) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3377930.3389840 · Cited by 26 · see citing works
Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Survey
Liu, Z., Xu, X., Qiao, P., & Li, D. (2024) · ACM Computing Surveys · doi:10.1145/3703453 · Cited by 25 · see citing works
Osa, T., Tangkaratt, V., & Sugiyama, M. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.04.009 · Cited by 24 · see citing works
Illuminating Mario Scenes in the Latent Space of a Generative Adversarial Network
Fontaine, M. C., Liu, R., Khalifa, A., Modi, J., Togelius, J., Hoover, A. K., & Nikolaidis, S. (2021) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v35i7.16740 · Cited by 46 · see citing works
Huang, L. J., Zhang, C., Fu, Y. Z., Jiang, Y., He, E., Qi, M., Du, M., Kong, X., Cheng, J. J., Cronin, L., et al. (2025) · Journal of the American Chemical Society · doi:10.1021/jacs.5c05916 · Cited by 25 · see citing works
On the use of quality diversity algorithms for the traveling thief problem
Nikfarjam, A., Neumann, A., & Neumann, F. (2022) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3512290.3528752 · Cited by 23 · see citing works
Efficient Novelty Search Through Deep Reinforcement Learning
Shi, L., Li, S., Zheng, Q., Yao, M., & Pan, G. (2020) · IEEE Access · doi:10.1109/access.2020.3008735 · Cited by 23 · see citing works
Empirical analysis of PGA-MAP-Elites for Neuroevolution in Uncertain Domains
Flageat, M., Chalumeau, F., & Cully, A. (2023) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3577203 · Cited by 22 · see citing works
The Emergence of Canalization and Evolvability in an Open-Ended, Interactive Evolutionary System
Huizinga, J., Stanley, K. O., & Clune, J. (2018) · Artificial Life · doi:10.1162/artl_a_00263 · Cited by 21 · see citing works
Self-citationJegorova, M., Doncieux, S., & Hospedales, T. M. (2019) · doi:10.1109/icdl-epirob44920.2019 · Cited by 22 · see citing works
QED: Using Quality-Environment-Diversity to Evolve Resilient Robot Swarms
Bossens, D. M., & Tarapore, D. (2020) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2020.3036578 · Cited by 22 · see citing works
Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents
Zhang, J., Hu, S., Lu, C., Lange, R., & Clune, J. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i3.15063 · Cited by 34 · see citing works
Matheron, G., Perrin, N., & Sigaud, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61616-8_24 · Cited by 21 · see citing works
Multi-objective Genetic Programming for Explainable Reinforcement Learning
Videau, M., Leite, A., Teytaud, O., & Schoenauer, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-02056-8_18 · Cited by 20 · see citing works
AmbieGen: A search-based framework for autonomous systems testing
Humeniuk, D. V., Khomh, F., & Antoniol, G. (2023) · Science of Computer Programming · doi:10.1016/j.scico.2023.102990 · Cited by 20 · see citing works
A population-based approach for multi-agent interpretable reinforcement learning
Crespi, M., Ferigo, A., Custode, L. L., & Iacca, G. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110758 · Cited by 20 · see citing works
Learning highly diverse robot throwing movements through quality diversity search
Kim, S., & Doncieux, S. (2017) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3067695.3082463 · Cited by 20 · see citing works
Gajewski, A., Clune, J., Stanley, K. O., & Lehman, J. (2019) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3321707.3321876 · Cited by 20 · see citing works
Self-citation
- Authors
- Pugh, J. K., Soros, L. B., & Stanley, K. O.
- Affiliations
- University of Central Florida (US)
- Corresponding author
- Stanley, Kenneth O.
- Venue
- Frontiers in Robotics and AI
- Publisher
- Frontiers Media SA
- Published
- 2016-07-12
- Volume
- 3
- ISSN
- 2296-9144 (ISSN-L)
- OpenAlex
- W2462548332
- Cited by
- 514
- References
- 68
- Citations by year
- 2026: 43 · 2025: 54 · 2024: 63 · 2023: 68 · 2022: 72 · 2021: 51 · 2020: 73 · 2019: 44 · 2018: 26 · 2017: 19 · 2016: 1
- FWCI
- 24.71
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Frontiers in Robotics and AI (publishedVersion, cc-by)
- Journal of International Crisis and Risk Communication Research (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (IIS-1421925)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Metaheuristic Optimization Algorithms Research
- Physical Sciences › Engineering › Mechanical Engineering › Modular Robots and Swarm Intelligence
- Keywords
- quality diversity, evolutionary algorithms, novelty search, MAP-Elites, niche diversity, evolutionary robotics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
