The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
On metrics for comparing nondominated sets
Abstract
Evolutionary multiobjective optimization (EMO) boasts a proliferation of algorithms and benchmark problems. We need principled ways to compare the performance of different EMO algorithms, but this is complicated by the fact that the result of an EMO run is not a single scalar value, but a collection of vectors forming a nondominated set. Various metrics for nondominated sets have been suggested. We compare several, using the framework of 'outperformance relations' (Hansen and Jaszkiewicz, 1998). This enables us to criticize and contrast a variety of published metrics, leading to some recommendations on which seem most useful in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Comparison of Multiobjective Evolutionary Algorithms: Empirical Results
Zitzler, E., Deb, K., & Thiele, L. (2000) · Evolutionary Computation · doi:10.1162/106365600568202 · Cited by 5,697
A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimization: NSGA-II
Deb, K., Agrawal, S., Pratap, A., & Meyarivan, T. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45356-3_83 · Cited by 5,289
Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy
Knowles, J. D., & Corne, D. W. (2000) · Evolutionary Computation · doi:10.1162/106365600568167 · Cited by 2,286
Self-citationEvolutionary algorithms for multiobjective optimization: methods and applications
Zitzler, E. (1999) · doi:10.3929/ethz-a-003856832 · Cited by 1,965
Multiobjective evolutionary algorithms: classifications, analyses, and new innovations
Lamont, G. B., & Veldhuizen, D. A. V. (1999) · Cited by 1,212
Fault Tolerant Design Using Single and Multicriteria Genetic Algorithm Optimization.
Schott, J. R. (1995) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 1,015
PESA-II: region-based selection in evolutionary multiobjective optimization
Corne, D., Jerram, N. R., Knowles, J. D., & Oates, M. J. (2001) · Cited by 1,011
Self-citationCzyzżak, P., & Jaszkiewicz, A. (1998) · Journal of Multi‐Criteria Decision Analysis · doi:10.1002/(sici)1099-1360(199801)7:1<34::aid-mcda161>3.0.co;2-6 · Cited by 793
Evaluating the quality of approximations to the non-dominated set
Hansen, M. P., & Jaszkiewicz, A. (1998) · Cited by 301
Cited by
Performance assessment of multiobjective optimizers: an analysis and review
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C. M., & da Fonseca, V. G. (2003) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2003.810758 · Cited by 4,330
Pymoo: Multi-Objective Optimization in Python
Blank, J., & Deb, K. (2020) · IEEE Access · doi:10.1109/access.2020.2990567 · Cited by 2,344
Indicator-Based Selection in Multiobjective Search
Zitzler, E., & Künzli, S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30217-9_84 · Cited by 2,259
A review of multiobjective test problems and a scalable test problem toolkit
Huband, S., Hingston, P., Barone, L., & While, L. (2006) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.861417 · Cited by 2,090
HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization
Bader, J., & Zitzler, E. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00009 · Cited by 2,081
Knowles, J. (2006) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.851274 · Cited by 1,254
Self-citationThe balance between proximity and diversity in multiobjective evolutionary algorithms
Bosman, P. A. N., & Thierens, D. (2003) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2003.810761 · Cited by 1,154
A Comprehensive Review on NSGA-II for Multi-Objective Combinatorial Optimization Problems
Verma, S., Pant, M., & Snasel, V. (2021) · IEEE Access · doi:10.1109/access.2021.3070634 · Cited by 974
Evolutionary many-objective optimization: A short review
Ishibuchi, H., Tsukamoto, N., & Nojima, Y. (2008) · 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) · doi:10.1109/cec.2008.4631121 · Cited by 914
Covariance Matrix Adaptation for Multi-objective Optimization
Igel, C., Hansen, N., & Roth, S. (2007) · Evolutionary Computation · doi:10.1162/evco.2007.15.1.1 · Cited by 788
Ishibuchi, H., Yoshida, T., & Murata, T. (2003) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2003.810752 · Cited by 773
Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels
Emmerich, M. T. M., Giannakoglou, K. C., & Naujoks, B. (2006) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.859463 · Cited by 759
Borg: An Auto-Adaptive Many-Objective Evolutionary Computing Framework
Hadka, D., & Reed, P. (2013) · Evolutionary Computation · doi:10.1162/evco_a_00075 · Cited by 717
A mayfly optimization algorithm
Zervoudakis, K., & Tsafarakis, S. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106559 · Cited by 692
Maier, H. R., Kapelan, Z., Kasprzyk, J., Kollat, J., Matott, L. S., Cunha, M. C., Dandy, G. C., Gibbs, M. S., Keedwell, E., Marchi, A., et al. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2014.09.013 · Cited by 657
Deb, K., Mohan, M., & Mishra, S. (2005) · Evolutionary Computation · doi:10.1162/106365605774666895 · Cited by 637
An EMO Algorithm Using the Hypervolume Measure as Selection Criterion
Emmerich, M., Beume, N., & Naujoks, B. (2005) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-31880-4_5 · Cited by 604
A Competitive-Cooperative Coevolutionary Paradigm for Dynamic Multiobjective Optimization
Goh, C.-K., & Tan, K. C. (2009) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2008.920671 · Cited by 592
Evolutionary multiobjective optimization in water resources: The past, present, and future
Reed, P. M., Hadka, D., Herman, J. D., Kasprzyk, J. R., & Kollat, J. B. (2013) · Advances in Water Resources · doi:10.1016/j.advwatres.2012.01.005 · Cited by 541
Schutze, O., Esquivel, X., Lara, A., & Coello, C. A. C. (2012) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2011.2161872 · Cited by 511
Performance Scaling of Multi-objective Evolutionary Algorithms
Khare, V., Yao, X., & Deb, K. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-36970-8_27 · Cited by 453
On the Evolutionary Optimization of Many Conflicting Objectives
Purshouse, R. C., & Fleming, P. J. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2007.910138 · Cited by 451
Performance metrics in multi-objective optimization
Riquelme, N., Von Lucken, C., & Baran, B. (2015) · 2015 Latin American Computing Conference (CLEI) · doi:10.1109/clei.2015.7360024 · Cited by 444
Binary differential evolution with self-learning for multi-objective feature selection
Zhang, Y., Gong, D.-W., Gao, X.-Z., Tian, T., & Sun, X.-Y. (2020) · Information Sciences · doi:10.1016/j.ins.2019.08.040 · Cited by 429
Quality Evaluation of Solution Sets in Multiobjective Optimisation: A Survey
Li, M., & Yao, X. (2020) · ACM Computing Surveys · doi:10.1145/3300148 · Cited by 342
Consistencies and Contradictions of Performance Metrics in Multiobjective Optimization
Jiang, S., Ong, Y.-S., Zhang, J., & Feng, L. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2307319 · Cited by 314
Quality Assessment of Pareto Set Approximations
Zitzler, E., Knowles, J., & Thiele, L. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88908-3_14 · Cited by 313
Self-citationPerformance assessment of the metaheuristic optimization algorithms: an exhaustive review
Halim, A. H., Ismail, I., & Das, S. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09906-6 · Cited by 293
Decomposition-Based Algorithms Using Pareto Adaptive Scalarizing Methods
Wang, R., Zhang, Q., & Zhang, T. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2016.2521175 · Cited by 283
A critical survey of performance indices for multi-objective optimisation
Okabe, T., Jin, Y., & Sendhoff, B. (2004) · The 2003 Congress on Evolutionary Computation, 2003. CEC '03. · doi:10.1109/cec.2003.1299759 · Cited by 278
Techniques for highly multiobjective optimisation: some nondominated points are better than others
Corne, D. W., & Knowles, J. D. (2007) · Proceedings of the 9th annual conference on Genetic and evolutionary computation · doi:10.1145/1276958.1277115 · Cited by 273
Self-citationCarbon-efficient scheduling of flow shops by multi-objective optimization
Ding, J.-Y., Song, S., & Wu, C. (2016) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.05.019 · Cited by 264
Diversity Management in Evolutionary Many-Objective Optimization
Adra, S. F., & Fleming, P. J. (2011) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2010.2058117 · Cited by 262
A Set-Based Genetic Algorithm for Interval Many-Objective Optimization Problems
Gong, D., Sun, J., & Miao, Z. (2018) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2016.2634625 · Cited by 261
Considerations in engineering parallel multiobjective evolutionary algorithms
Van Veldhuizen, D. A., Zydallis, J. B., & Lamont, G. B. (2003) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2003.810751 · Cited by 260
Giuliani, M., Castelletti, A., Pianosi, F., Mason, E., & Reed, P. M. (2016) · Journal of Water Resources Planning and Management · doi:10.1061/(asce)wr.1943-5452.0000570 · Cited by 257
Benchmarking in Optimization: Best Practice and Open Issues
Bartz-Beielstein, T., Doerr, C., Berg, D. V. D., Bossek, J., Chandrasekaran, S., Eftimov, T., Fischbach, A., Kerschke, P., La Cava, W., Lopez-Ibanez, M., et al. (2020) · arXiv · doi:10.48550/arxiv.2007.03488 · Cited by 214
Metadata
- Authors
- Knowles, J., & Corne, D.
- Venue
- Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600)
- Publisher
- IEEE
- Published
- 2003-06-25
- Volume
- 1
- Pages
- 711-716
- OpenAlex
- W1521316934
- Cited by
- 560
- References
- 9
- FWCI
- 26.83
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.