The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
A review of multiobjective test problems and a scalable test problem toolkit
Abstract
When attempting to better understand the strengths and weaknesses of an algorithm, it is important to have a strong understanding of the problem at hand. This is true for the field of multiobjective evolutionary algorithms (EAs) as it is for any other field. Many of the multiobjective test problems employed in the EA literature have not been rigorously analyzed, which makes it difficult to draw accurate conclusions about the strengths and weaknesses of the algorithms tested on them. In this paper, we systematically review and analyze many problems from the EA literature, each belonging to the important class of real-valued, unconstrained, multiobjective test problems. To support this, we first introduce a set of test problem criteria, which are in turn supported by a set of definitions. Our analysis of test problems highlights a number of areas requiring attention. Not only are many test problems poorly constructed but also the important class of nonseparable problems, particularly nonseparable multimodal problems, is poorly represented. Motivated by these findings, we present a flexible toolkit for constructing well-designed test problems. We also present empirical results demonstrating how the toolkit can be used to test an optimizer in ways that existing test suites do not
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053
Multi-Objective Optimization Using Evolutionary Algorithms
Deb, K., & Kalyanmoy, D. (2001) · Andalas University Repository (Andalas University) · Cited by 17,328
No free lunch theorems for optimization
Wolpert, D. H., & Macready, W. G. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585893 · Cited by 14,237
Goldberg, D. E. (1989) · Medical Entomology and Zoology · Cited by 10,109
Comparison of Multiobjective Evolutionary Algorithms: Empirical Results
Zitzler, E., Deb, K., & Thiele, L. (2000) · Evolutionary Computation · doi:10.1162/106365600568202 · Cited by 5,697
Performance assessment of multiobjective optimizers: an analysis and review
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C. M., & da Fonseca, V. G. (2003) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2003.810758 · Cited by 4,330
Randomization, Bootstrap and Monte Carlo Methods in Biology
Manly, B. F. J. (2018) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315273075 · Cited by 4,194
Multiple Objective Optimization with Vector Evaluated Genetic Algorithms
Schaffer, J. D. (1985) · international conference on Genetic algorithms · Cited by 3,326
Handbook of Evolutionary Computation
(1997) · Institute of Physics eBooks · doi:10.1887/0750308958 · Cited by 2,990
Genetic Algorithms in Search, Optimization & Machine Learning
Goldberg, D. E. (1989) · Medical Entomology and Zoology · Cited by 2,304
Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy
Knowles, J. D., & Corne, D. W. (2000) · Evolutionary Computation · doi:10.1162/106365600568167 · Cited by 2,286
An Overview of Evolutionary Algorithms in Multiobjective Optimization
Fonseca, C. M., & Fleming, P. J. (1995) · Evolutionary Computation · doi:10.1162/evco.1995.3.1.1 · Cited by 2,266
Scalable Test Problems for Evolutionary Multiobjective Optimization
Deb, K., Thiele, L., Laumanns, M., & Zitzler, E. (2005) · Advanced Information and Knowledge Processing · doi:10.1007/1-84628-137-7_6 · Cited by 2,010
Scalable multi-objective optimization test problems
Deb, K., Thiele, L., Laumanns, M., & Zitzler, E. (2003) · Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600) · doi:10.1109/cec.2002.1007032 · Cited by 1,627
Combining Convergence and Diversity in Evolutionary Multiobjective Optimization
Laumanns, M., Thiele, L., Deb, K., & Zitzler, E. (2002) · Evolutionary Computation · doi:10.1162/106365602760234108 · Cited by 1,487
Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses
Niknian, M. (1995) · Technometrics · doi:10.1080/00401706.1995.10484341 · Cited by 1,462
Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems
Deb, K. (1999) · Evolutionary Computation · doi:10.1162/evco.1999.7.3.205 · Cited by 1,451
Evolutionary multi-objective optimization: a historical view of the field
Coello Coello, C. A. (2006) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2006.1597059 · Cited by 1,446
Randomization and Monte Carlo Methods in Biology
Manly, B. F. J. (1991) · Springer eBooks · doi:10.1007/978-1-4899-2995-2 · Cited by 1,233
Multiobjective evolutionary algorithms: classifications, analyses, and new innovations
Lamont, G. B., & Veldhuizen, D. A. V. (1999) · Cited by 1,212
A multi-objective genetic local search algorithm and its application to flowshop scheduling
Ishibuchi, H., & Murata, T. (1998) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.704576 · Cited by 1,019
SPEA2: Improving the Strength Pareto Evolutionary Algorithm For Multiobjective Optimization
Zitzler, E., Giannakoglou, K., Tsahalis, D., Periaux, J., Papailiou, K., (eds, T. F., Ler, E. Z., Laumanns, M., & Thiele, L. (2002) · Cited by 1,010
A variant of evolution strategies for vector optimization
Kursawe, F. (2006) · Lecture Notes in Computer Science · doi:10.1007/bfb0029752 · Cited by 597
Good, P. (2000) · Springer Series in Statistics · doi:10.1007/978-1-4757-3235-1 · Cited by 573
On metrics for comparing nondominated sets
Knowles, J., & Corne, D. (2003) · Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600) · doi:10.1109/cec.2002.1007013 · Cited by 560
Handbook of Evolutionary Computation
(1997) · CRC Press eBooks · doi:10.1201/9780367802486 · Cited by 413
A Scalable Multi-objective Test Problem Toolkit
Huband, S., Barone, L., While, L., & Hingston, P. (2005) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-31880-4_20 · Cited by 384
Self-citationOn the performance assessment and comparison of stochastic multiobjective optimizers
Fonseca, C. M., & Fleming, P. J. (1996) · Lecture Notes in Computer Science · doi:10.1007/3-540-61723-x_1022 · Cited by 378
A critical survey of performance indices for multi-objective optimisation
Okabe, T., Jin, Y., & Sendhoff, B. (2004) · The 2003 Congress on Evolutionary Computation, 2003. CEC '03. · doi:10.1109/cec.2003.1299759 · Cited by 278
Finding Acceptable Solutions in the Pareto-Optimal Range using Multiobjective Genetic Algorithms
Bentley, P. J., & Wakefield, J. P. (1998) · Soft Computing in Engineering Design and Manufacturing · doi:10.1007/978-1-4471-0427-8_25 · Cited by 248
Jin, Y., Olhofer, M., & Sendhoff, B. (2007) · Cited by 211
Inferential Performance Assessment of Stochastic Optimisers and the Attainment Function
da Fonseca, V. G., Fonseca, C. M., & Hall, A. O. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44719-9_15 · Cited by 204
Multiobjective evolutionary algorithm for the optimization of noisy combustion processes
Büche, D., Stoll, P., Dornberger, R., & Koumoutsakos, P. (2002) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcb.2002.804372 · Cited by 155
On Test Functions for Evolutionary Multi-objective Optimization
Okabe, T., Jin, Y., Olhofer, M., et al. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30217-9_80 · Cited by 115
A multi-sexual genetic algorithm for multiobjective optimization
Lis, J., & Eiben, A. E. (2002) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/icec.1997.592269 · Cited by 114
An evolution strategy with probabilistic mutation for multi-objective optimisation
Huband, S., Hingston, P., While, L., & Barone, L. (2004) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2003.1299373 · Cited by 84
Self-citationDiversity assessment of Pareto optimal solution sets: an entropy approach
Farhang-Mehr, A., & Azarm, S. (2003) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2002.1007015 · Cited by 76
Shaw, K. J., Nortcliffe, A., Thompson, M., Love, J., Fleming, P. J., & Fonseca, C. M. (2003) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.1999.781905 · Cited by 56
Agent-based evolutionary multiobjective optimisation
Socha, K., & Kisiel‐Dorohinicki, M. (2003) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2002.1006218 · Cited by 43
Designing Crushers With A Multi-objective Evolutionary Algorithm
Barone, L., While, L., & Hingston, P. (2002) · Cited by 15
Self-citation
Cited by
Platemo: a matlab platform for evolutionary multi-objective optimization [educational forum]
Tian, Y., Cheng, R., Zhang, X., & Jin, Y. (2017) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2017.2742868 · Cited by 2,777
Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II
Li, H., & Zhang, Q. (2009) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2008.925798 · Cited by 2,543
Multiobjective evolutionary algorithms: A survey of the state of the art
Zhou, A., Qu, B.-Y., Li, H., Zhao, S.-Z., Suganthan, P. N., & Zhang, Q. (2011) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2011.03.001 · Cited by 2,255
HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization
Bader, J., & Zitzler, E. (2011) · Evolutionary Computation · doi:10.1162/evco_a_00009 · Cited by 2,081
A Reference Vector Guided Evolutionary Algorithm for Many-Objective Optimization
Cheng, R., Jin, Y., Olhofer, M., & Sendhoff, B. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2016.2519378 · Cited by 1,760
Metadata
- Authors
- Huband, S., Hingston, P., Barone, L., & While, L.
- Venue
- IEEE Transactions on Evolutionary Computation
- Published
- 2006-10
- Volume
- 10
- Issue
- 5
- Pages
- 477-506
- ISSN
- 1089-778X (ISSN-L)
- OpenAlex
- W2143185749
- Cited by
- 2,090
- References
- 40
- FWCI
- 20.16
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.