CoChat
Sign Up
Cited by 258Evolutionary Algorithms and ApplicationsMetaheuristic Optimization Algorithms ResearchAdvanced Multi-Objective Optimization Algorithms

Quality and Diversity Optimization: A Unifying Modular Framework

Open access
Cully, Antoine, Demiris, Yiannis·2018·IEEE Transactions on Evolutionary Computation
DOI 10.1109/tevc.2017.2704781
Open this paper in CoChat — free account

Abstract

The optimization of functions to find the best solution according to one or several objectives has a central role in many engineering and research fields. Recently, a new family of optimization algorithms, named quality-diversity (QD) optimization, has been introduced, and contrasts with classic algorithms. Instead of searching for a single solution, QD algorithms are searching for a large collection of both diverse and high-performing solutions. The role of this collection is to cover the range of possible solution types as much as possible, and to contain the best solution for each type. The contribution of this paper is threefold. First, we present a unifying framework of QD optimization algorithms that covers the two main algorithms of this family (multidimensional archive of phenotypic elites and the novelty search with local competition), and that highlights the large variety of variants that can be investigated within this family. Second, we propose algorithms with a new selection mechanism for QD algorithms that outperforms all the algorithms tested in this paper. Lastly, we present a new collection management that overcomes the erosion issues observed when using unstructured collections. These three contributions are supported by extensive experimental comparisons of QD algorithms on three different experimental scenarios.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tevc.2017.2704781

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Frameworks unifying quality and diversity optimization can guarantee the diversity of generated solutions that may later serve in robotics to learn behaviorally diverse policies.

    “Frameworks unifying quality and diversity optimization can guarantee the diversity of generated solutions that may later serve in robotics to learn behaviorally diverse policies [73].”

    From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [73]

    The abstract (p. 1) presents a unifying quality-diversity framework, and the Introduction (p. 1) links collections of different behaviors to robotics. Definition II.1 (p. 4) says these algorithms aim to produce diverse solutions; it does not guarantee that the generated solutions will be diverse. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 43

  1. [1][1] A. Antoniou and W.-S. Lu, Practical optimization: algorithms and engineering applications . Springer Science & Business Media, 2007. -
  2. [2][2] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, 'Learning representations by back-propagating errors,' Cognitive modeling , vol. 5, no. 3, p. 1, 1988. -
  3. [4][4] A. Cully, J. Clune, D. Tarapore, and J.-B. Mouret, 'Robots that can adapt like animals,' Nature , vol. 521, no. 7553, pp. 503-507, 2015. -
  4. [6][6] J. C. Spall, Introduction to stochastic search and optimization: estimation, simulation, and control . John Wiley & Sons, 2005, vol. 65. -
  5. [7][7] H. Lipson and J. B. Pollack, 'Automatic design and manufacture of robotic lifeforms,' Nature , vol. 406, no. 6799, pp. 974-978, 2000. -
  6. [8][8] M. Schmidt and H. Lipson, 'Distilling free-form natural laws from experimental data,' science , vol. 324, no. 5923, pp. 81-85, 2009. -
  7. [9][9] A. E. Eiben and J. Smith, 'From evolutionary computation to the evolution of things,' Nature , vol. 521, no. 7553, pp. 476-482, 2015. -
  8. [10][10] J. Bongard, V. Zykov, and H. Lipson, 'Resilient machines through continuous self-modeling,' Science , vol. 314, no. 5802, 2006. -
  9. [11][11] S. Koos, J.-B. Mouret, and S. Doncieux, 'The transferability approach: Crossing the reality gap in evolutionary robotics,' Evolutionary Computation, IEEE Transactions on , vol. 17, no. 1, pp. 122-145, 2013. -
  10. [12][12] Y. Demiris, L. Aziz-Zadeh, and J. Bonaiuto, 'Information processing in the mirror neuron system in primates and machines,' Neuroinformatics , vol. 12, no. 1, pp. 63-91, 2014. -
  11. [13][13] A. Cully and J.-B. Mouret, 'Behavioral repertoire learning in robotics,' in Proceedings of the 15th annual conference on Genetic and Evolutionary Computation . ACM, 2013, pp. 175-182. -
  12. [14][14] J. Lehman and K. O. Stanley, 'Evolving a diversity of virtual creatures through novelty search and local competition,' in Proceedings of the 13th annual conference on Genetic and Evolutionary Computation . ACM, 2011, pp. 211-218. -
  13. [15][15] J.-B. Mouret and J. Clune, 'Illuminating search spaces by mapping elites,' arXiv preprint arXiv:1504.04909 , 2015. - · arXiv
  14. [16][16] J. K. Pugh, L. Soros, P. A. Szerlip, and K. O. Stanley, 'Confronting the challenge of quality diversity,' in Proceedings of the 2015 on Genetic and Evolutionary Computation Conference . ACM, 2015, pp. 967-974. -
  15. [18][18] J. Lehman and K. O. Stanley, 'Abandoning objectives: Evolution through the search for novelty alone,' Evolutionary Computation , vol. 19, no. 2, pp. 189-223, 2011. -
  16. [19][19] D. E. Goldberg and J. Richardson, 'Genetic algorithms with sharing for multimodal function optimization,' in Genetic algorithms and their applications: Proceedings of the Second International Conference on Genetic Algorithms . Hillsdale, NJ: Lawrence Erlbaum, 1987, pp. 41-49. -
  17. [20][20] S. W. Mahfoud, 'Niching methods for genetic algorithms,' Urbana , vol. 51, no. 95001, pp. 62-94, 1995. -
  18. [21][21] G. Singh and K. Deb Dr, 'Comparison of multi-modal optimization algorithms based on evolutionary algorithms,' in Proceedings of the 8th annual conference on Genetic and Evolutionary Computation . ACM, 2006, pp. 1305-1312. -
  19. [22][22] X. Yin and N. Germay, 'A fast genetic algorithm with sharing scheme using cluster analysis methods in multimodal function optimization,' in Artificial neural nets and genetic algorithms . Springer, 1993. -
  20. [23][23] A. P´ etrowski, 'A clearing procedure as a niching method for genetic algorithms,' in Evolutionary Computation, 1996., Proceedings of IEEE International Conference on . IEEE, 1996, pp. 798-803. -
  21. [25][25] N. Kohl and P. Stone, 'Policy gradient reinforcement learning for fast quadrupedal locomotion,' in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) , vol. 3. IEEE, 2004, pp. 2619-2624. -
  22. [26][26] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, 'A fast and elitist multiobjective genetic algorithm: Nsga-ii,' Evolutionary Computation, IEEE Transactions on , vol. 6, no. 2, pp. 182-197, 2002. -
  23. [27][27] C. Maestre, A. Cully, C. Gonzales, and S. Doncieux, 'Bootstrapping interactions with objects from raw sensorimotor data: a novelty search based approach,' in Development and Learning and Epigenetic Robotics (ICDL-EpiRob), 2015 Joint IEEE International Conference on . IEEE, 2015, pp. 7-12. -
  24. [28][28] F. Benureau and P.-Y. Oudeyer, 'Behavioral diversity generation in autonomous exploration through reuse of past experience,' Frontiers in Robotics and AI , vol. 3, p. 8, 2016. -
  25. [30][30] J. Clune, J.-B. Mouret, and H. Lipson, 'The evolutionary origins of modularity,' Proceedings of the Royal Society of London B: Biological Sciences , vol. 280, no. 1755, 2013. -
  26. [31][31] A. Cully, 'Creative adaptation through learning,' Ph.D. dissertation, Universit´ e Pierre et Marie Curie, 2015. -
  27. [32][32] M. Duarte, J. Gomes, S. M. Oliveira, and A. L. Christensen, 'Evorbc: Evolutionary repertoire-based control for robots with arbitrary locomotion complexity,' in Proceedings of the 25th annual conference on Genetic and Evolutionary Computation . ACM, 2016. -
  28. [33][33] A. Nguyen, J. Yosinski, and J. Clune, 'Deep neural networks are easily fooled: High confidence predictions for unrecognizable images,' in Conference on Computer Vision and Pattern Recognition . IEEE, 2015. -
  29. [34][34] A. M. Nguyen, J. Yosinski, and J. Clune, 'Innovation engines: Automated creativity and improved stochastic optimization via deep learning,' in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation . ACM, 2015, pp. 959-966. -
  30. [35][35] V. Vassiliades, K. Chatzilygeroudis, and J.-B. Mouret, 'Scaling up map-elites using centroidal voronoi tessellations,' arXiv preprint arXiv:1610.05729 , 2016. - · arXiv
  31. [36][36] D. Smith, L. Tokarchuk, and G. Wiggins, 'Rapid phenotypic landscape exploration through hierarchical spatial partitioning,' in International Conference on Parallel Problem Solving from Nature . Springer, 2016, pp. 911-920. -
  32. [37][37] J. Lehman and R. Miikkulainen, 'Enhancing divergent search through extinction events,' in Proceedings of the 2015 on Genetic and Evolutionary Computation Conference . ACM, 2015, pp. 951-958. -
  33. [38][38] D. Tarapore and J.-B. Mouret, 'Evolvability signatures of generative encodings: beyond standard performance benchmarks,' Information Sciences , vol. 313, pp. 43-61, 2015. -
  34. [39][39] D. Tarapore, J. Clune, A. Cully, and J.-B. Mouret, 'How do different encodings influence the performance of the map-elites algorithm?' in Genetic and Evolutionary Computation Conference , 2016. -
  35. [40][40] J.-B. Mouret and S. Doncieux, 'Encouraging behavioral diversity in evolutionary robotics: An empirical study,' Evolutionary Computation , vol. 20, no. 1, pp. 91-133, 2012. -
  36. [41][41] M. Laumanns, L. Thiele, K. Deb, and E. Zitzler, 'Combining convergence and diversity in evolutionary multiobjective optimization,' Evolutionary Computation , vol. 10, no. 3, pp. 263-282, 2002. -
  37. [43][43] P.-Y. Oudeyer, F. Kaplan, and V. V. Hafner, 'Intrinsic motivation systems for autonomous mental development,' Evolutionary Computation, IEEE Transactions on , vol. 11, no. 2, pp. 265-286, 2007. -
  38. [44][44] J. Lehman, S. Risi, and J. Clune, 'Creative generation of 3d objects with deep learning and innovation engines,' in Proceedings of the 7th International Conference on Computational Creativity , 2016. -
  39. [45][45] M. Pigliucci, 'Is evolvability evolvable?' Nature Reviews Genetics , vol. 9, no. 1, pp. 75-82, 2008. -
  40. [46][46] L. Altenberg et al. , 'The evolution of evolvability in genetic programming,' Advances in genetic programming , vol. 3, pp. 47-74, 1994. -
  41. [47][47] J.-B. Mouret and S. Doncieux, 'Sferes v2: Evolvin'in the multi-core world,' in Evolutionary Computation (CEC), 2010 IEEE Congress on . IEEE, 2010, pp. 1-8. -
  42. [48][48] J. P. Shaffer, 'Multiple hypothesis testing,' Annual review of psychology , vol. 46, no. 1, pp. 561-584, 1995. -
  43. [49][49] A. Alexandrescu, Modern C++ design: generic programming and design patterns applied . Addison-Wesley, 2001.

Cited by

Sorted by citations

Authors
Cully, A., & Demiris, Y.
Affiliations
Imperial College London (GB)
ORCID
Venue
IEEE Transactions on Evolutionary Computation
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2018-04
Volume
22
Issue
2
Pages
245-259
ISSN
  • 1089-778X (ISSN-L)
  • 1941-0026
DOI
10.1109/tevc.2017.2704781
OpenAlex
W2702283731
Cited by
258
References
2
Citations by year
2026: 18 · 2025: 30 · 2024: 35 · 2023: 46 · 2022: 35 · 2021: 28 · 2020: 33 · 2019: 18 · 2018: 12 · 2017: 3
FWCI
10.77
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Evolutionary Computation (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • Spiral (Imperial College London) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
EU Horizon2020 Project PAL (643783-RIA)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Metaheuristic Optimization Algorithms Research
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Advanced Multi-Objective Optimization Algorithms
Keywords
solution diversity, selection mechanisms, collection management, multi-objective optimization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review