A critical survey of performance indices for multi-objective optimisation
The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
A large number of methods for solving multiobjective optimisation (MOO) problems have been developed. To compare these methods rigorously, or to measure the performance of a particular MOO algorithm quantitatively, a variety of performance indices (PIs) have been proposed. We provide an overview of the various PIs and attempts to categorise them into a certain number of classes according to their properties. Comparative studies have been conducted using a group of artificial solution sets and a group of solution sets obtained by various MOO solvers to show the advantages and disadvantages of the PIs. The comparative studies show that many PIs may be misleading in that they fail to truly reflect the quality of solution sets. Thus, it may not be a good practice to evaluate the performance of MOO solvers based on PIs only.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053 · cited 1×
“Distance-based. The PI has been extended to include spread information, which is now termed index [Deb01, Deb02 ].”
BackgroundMulti-Objective Optimization Using Evolutionary Algorithms
Deb, K., & Kalyanmoy, D. (2001) · Andalas University Repository (Andalas University) · Cited by 17,500
Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach
Zitzler, E., & Thiele, L. (1999) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.797969 · Cited by 8,798
Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms
Srinivas, N., & Deb, K. (1994) · Evolutionary Computation · doi:10.1162/evco.1994.2.3.221 · Cited by 6,935 · cited 1×
“Niche-based. A PI called Chi-Square-Like Deviation Measure has been suggested [Deb89, Sri94 ].”
BackgroundComparison of Multiobjective Evolutionary Algorithms: Empirical Results
Zitzler, E., Deb, K., & Thiele, L. (2000) · Evolutionary Computation · doi:10.1162/106365600568202 · Cited by 5,697
Evolutionary Algorithms for Solving Multi-Objective Problems
(2007) · Genetic and Evolutionary Computation Series · doi:10.1007/978-0-387-36797-2 · Cited by 5,686
A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimization: NSGA-II
Deb, K., Agrawal, S., Pratap, A., & Meyarivan, T. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45356-3_83 · Cited by 5,289
Bäck, T. (1996) · Oxford University Press eBooks · doi:10.1093/oso/9780195099713.001.0001 · Cited by 5,113
Performance assessment of multiobjective optimizers: an analysis and review
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C. M., & da Fonseca, V. G. (2003) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2003.810758 · Cited by 4,330
Evolutionary Algorithms for Solving Multi-Objective Problems
Coello Coello, C. A., Van Veldhuizen, D. A., & Lamont, G. B. (2002) · Genetic Algorithms and Evolutionary Computation · doi:10.1007/978-1-4757-5184-0 · Cited by 2,663
Cited by
A review of multiobjective test problems and a scalable test problem toolkit
Huband, S., Hingston, P., Barone, L., & While, L. (2006) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.861417 · Cited by 2,090
A faster algorithm for calculating hypervolume
While, L., Hingston, P., Barone, L., & Huband, S. (2006) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.851275 · Cited by 1,100
Evolutionary many-objective optimization: A short review
Ishibuchi, H., Tsukamoto, N., & Nojima, Y. (2008) · 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) · doi:10.1109/cec.2008.4631121 · Cited by 914
Performance metrics in multi-objective optimization
Riquelme, N., Von Lucken, C., & Baran, B. (2015) · 2015 Latin American Computing Conference (CLEI) · doi:10.1109/clei.2015.7360024 · Cited by 444
Quality Evaluation of Solution Sets in Multiobjective Optimisation: A Survey
Li, M., & Yao, X. (2020) · ACM Computing Surveys · doi:10.1145/3300148 · Cited by 342
Consistencies and Contradictions of Performance Metrics in Multiobjective Optimization
Jiang, S., Ong, Y.-S., Zhang, J., & Feng, L. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2307319 · Cited by 314
Performance assessment of the metaheuristic optimization algorithms: an exhaustive review
Halim, A. H., Ismail, I., & Das, S. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09906-6 · Cited by 293
Green scheduling of a two-machine flowshop: Trade-off between makespan and energy consumption
Mansouri, S. A., Aktas, E., & Besikci, U. (2016) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.08.064 · Cited by 289
Solving Rotated Multi-objective Optimization Problems Using Differential Evolution
Iorio, A. W., & Li, X. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30549-1_74 · Cited by 263
A Survey on Modeling and Optimizing Multi-Objective Systems
Cho, J.-H., Wang, Y., Chen, I.-R., Chan, K. S., & Swami, A. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2698366 · Cited by 203
“[156] conduct a survey”
Diversity Comparison of Pareto Front Approximations in Many-Objective Optimization
Li, M., Yang, S., & Liu, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2310651 · Cited by 172
Laszczyk, M., & Myszkowski, P. B. (2019) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2019.04.001 · Cited by 79
Metadata
- Authors
- Okabe, T., Jin, Y., & Sendhoff, B.
- Venue
- The 2003 Congress on Evolutionary Computation, 2003. CEC '03.
- Publisher
- IEEE
- Published
- 2004-07-09
- Volume
- 2
- Pages
- 878-885
- OpenAlex
- W2156679542
- Cited by
- 278
- References
- 24
- FWCI
- 8.16
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.