The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Survey of quality measures for multi-objective optimization: Construction of complementary set of multi-objective quality measures
Abstract
Abstract In recent years interest in multiobjective optimization has flourished. Many Quality Measures (QM) have been developed to allow comparison of results gained by many methods. Unfortunately significant amount of various QMs along with the lack of imposed taxonomy have caused vagueness in the naming conventions. Hence a cohesive taxonomy is proposed that allows for classification of both existing and future QMs. This paper additionally provides thorough description of recently used QMs while attempting to unify the nomenclature. Advantages and disadvantages are shown along with the various features of the measures in given problem – as an example Multi–Skill Resource Constrained Project Scheduling Problem is given. Finally, a complementary set of QMs is proposed that can create a meaningful comparison of obtained multiobjective solutions to a multiobjective problem. Supplementary measures are proposed for specialized applications and open issues in the field are identified.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, many real-life problems can be characterized as multi-objective problems involving multiple conflicting objectives that should be considered simultaneously [51, 52].”
From Applications and Challenges of Artificial Intelligence in Space Missions (2024) · cites this paper as [52]
Citations
Cites
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053
Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach
Zitzler, E., & Thiele, L. (1999) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.797969 · Cited by 8,798
Comparison of Multiobjective Evolutionary Algorithms: Empirical Results
Zitzler, E., Deb, K., & Thiele, L. (2000) · Evolutionary Computation · doi:10.1162/106365600568202 · Cited by 5,809
Multiobjective optimization using evolutionary algorithms — A comparative case study
Zitzler, E., & Thiele, L. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0056872 · Cited by 2,262
Evolutionary algorithms for multiobjective optimization: methods and applications
Zitzler, E. (1999) · doi:10.3929/ethz-a-003856832 · Cited by 1,974
Multiobjective evolutionary algorithms: classifications, analyses, and new innovations
Lamont, G. B., & Veldhuizen, D. A. V. (1999) · Cited by 1,212
The balance between proximity and diversity in multiobjective evolutionary algorithms
Bosman, P. A. N., & Thierens, D. (2003) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2003.810761 · Cited by 1,154
Evolutionary many-objective optimization: A short review
Ishibuchi, H., Tsukamoto, N., & Nojima, Y. (2008) · 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) · doi:10.1109/cec.2008.4631121 · Cited by 914
A New Dominance Relation-Based Evolutionary Algorithm for Many-Objective Optimization
Yuan, Y., Xu, H., Wang, B., & Yao, X. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2015.2420112 · Cited by 752
Mirjalili, S., Jangir, P., & Saremi, S. (2017) · Applied Intelligence · doi:10.1007/s10489-016-0825-8 · Cited by 746
Schutze, O., Esquivel, X., Lara, A., & Coello, C. A. C. (2012) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2011.2161872 · Cited by 511
Proceedings of the 36th International Conference on Software Engineering
(2014) · doi:10.1145/2568225 · Cited by 477
Performance metrics in multi-objective optimization
Riquelme, N., Von Lucken, C., & Baran, B. (2015) · 2015 Latin American Computing Conference (CLEI) · doi:10.1109/clei.2015.7360024 · Cited by 444
Test Problems for Large-Scale Multiobjective and Many-Objective Optimization
Cheng, R., Jin, Y., Olhofer, M., & sendhoff, B. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2600577 · Cited by 427
A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers
Knowles, J. D., Thiele, L., & Zitzler, E. (2006) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000023822 · Cited by 409
Zhou, A., Jin, Y., Zhang, Q., Sendhoff, B., & Tsang, E. P. K. (2006) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2006.1688406 · Cited by 350
Hamdy, M., Nguyen, A.-T., & Hensen, J. J. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2016.03.035 · Cited by 341
Consistencies and Contradictions of Performance Metrics in Multiobjective Optimization
Jiang, S., Ong, Y.-S., Zhang, J., & Feng, L. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2307319 · Cited by 314
A critical survey of performance indices for multi-objective optimisation
Okabe, T., Jin, Y., & Sendhoff, B. (2004) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2003.1299759 · Cited by 297
Behavior of Multiobjective Evolutionary Algorithms on Many-Objective Knapsack Problems
Ishibuchi, H., Akedo, N., & Nojima, Y. (2015) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2014.2315442 · Cited by 288
Metrics for Quality Assessment of a Multiobjective Design Optimization Solution Set
Wu, J., & Azarm, S. (2001) · Journal of Mechanical Design · doi:10.1115/1.1329875 · Cited by 274
Multiobjective GAs, Quantitative Indices, and Pattern Classification
Bandyopadhyay, S., Pal, S. K., & Aruna, B. (2004) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2004.834438 · Cited by 210
On the properties of the R2 indicator
Brockhoff, D., Wagner, T., & Trautmann, H. (2012) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2330163.2330230 · Cited by 180
Diversity Comparison of Pareto Front Approximations in Many-Objective Optimization
Li, M., Yang, S., & Liu, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2310651 · Cited by 174
Pasandideh, S. H. R., Niaki, S. T. A., & Asadi, K. (2014) · Information Sciences · doi:10.1016/j.ins.2014.08.068 · Cited by 173
Wang, S., Ali, S., Yue, T., Li, Y., & Liaaen, M. C. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884880 · Cited by 115
A fast hypervolume driven selection mechanism for many-objective optimisation problems
Rostami, S., & Neri, F. (2016) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2016.12.002 · Cited by 110
Karthikeyan, S., Asokan, P., Savarimuthu, N., et al. (2015) · International Journal of Bio-Inspired Computation · doi:10.1504/ijbic.2015.073165 · Cited by 100
Performance measures for dynamic multi-objective optimisation algorithms
Helbig, M., & Engelbrecht, A. P. (2013) · Information Sciences · doi:10.1016/j.ins.2013.06.051 · Cited by 99
Myszkowski, P. B., Olech, Ł. P., Laszczyk, M., & Skowroński, M. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2017.10.014 · Cited by 91
Self-citationEvolutionary algorithms for multi-objective optimization: performance assessments and comparisons
Tan, K. C., Lee, T. H., & Khor, E. F. (2002) · Congress on Evolutionary Computation · doi:10.1109/cec.2001.934296 · Cited by 86
13th AIAA/ISSMO Multidisciplinary Analysis Optimization Conference
(2010) · doi:10.2514/mmao10 · Cited by 80
Spread Assessment for Evolutionary Multi-Objective Optimization
Li, M., & Zheng, J. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01020-0_20 · Cited by 76
Laszczyk, M., & Myszkowski, P. B. (2019) · Information Sciences · doi:10.1016/j.ins.2019.01.002 · Cited by 74
Self-citationA region division based diversity maintaining approach for many-objective optimization
Pan, L., He, C., Tian, Y., Su, Y., & Zhang, X. (2017) · Integrated Computer-Aided Engineering · doi:10.3233/ica-170542 · Cited by 72
Fonseca, C. M., da Fonseca, V. G., & Paquete, L. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-31880-4_18 · Cited by 67
Collette, Y., & Siarry, P. (2003) · Computers & Operations Research · doi:10.1016/j.cor.2003.08.017 · Cited by 61
PSFGA: Parallel processing and evolutionary computation for multiobjective optimisation
Negro, F. J. D. T., Ortega, J., Ros, E., Mota, S., Paechter, B., & Martı́n, J. E. (2004) · Parallel Computing · doi:10.1016/j.parco.2003.12.012 · Cited by 57
Bezerra, L. C. T., López‐Ibáñez, M., & Stützle, T. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54157-0_3 · Cited by 53
An immune multi-objective optimization algorithm with differential evolution inspired recombination
Qi, Y., Hou, Z., Yin, M., Sun, H., & Huang, J. (2015) · Applied Soft Computing · doi:10.1016/j.asoc.2015.01.012 · Cited by 52
U-measure: A quality measure for multiobjective programming
Leung, Y.-W., & Wang, Y. (2003) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2003.817059 · Cited by 52
A new distribution metric for comparing Pareto optimal solutions
Zheng, K., Yang, R., Xu, H., & Hu, J. (2016) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-016-1469-3 · Cited by 46
Farhang-Mehr, A., & Azarm, S. (2003) · Journal of Mechanical Design · doi:10.1115/1.1623186 · Cited by 45
Myszkowski, P. B., Laszczyk, M., Nikulin, I., & Skowroński, M. (2018) · Soft Computing · doi:10.1007/s00500-017-2997-5 · Cited by 40
Self-citationS-metric based multi-objective fireworks algorithm
Liu, L., Zheng, S., & Tan, Y. (2015) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2015.7257033 · Cited by 34
Solving the Bi-objective Traveling Thief Problem with Multi-objective Evolutionary Algorithms
Blank, J., Deb, K., & Mostaghim, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54157-0_4 · Cited by 34
Multi-objective optimization problem under fuzzy rule constraints using particle swarm optimization
Chakraborty, D., Guha, D., & Dutta, B. (2015) · Soft Computing · doi:10.1007/s00500-015-1639-z · Cited by 27
A New Approach on Many Objective Diversity Measurement
Mostaghim, S., & Teich, J. (2005) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000051614 · Cited by 25
DIVIDED RANGE GENETIC ALGORITHMS IN MULTIOBJECTIVE OPTIMIZATION PROBLEMS
Hiroyasu, T., Miki, M., & Watanabe, S. (1999) · Cited by 24
Lizárraga, G., Hernández-Aguirre, A., & Botello‐Rionda, S. (2008) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1389095.1389227 · Cited by 23
The bi-objective quadratic multiple knapsack problem: Model and heuristics
Chen, Y., & Hao, J. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2016.01.014 · Cited by 18
Myszkowski, P. B., Laszczyk, M., & Lichodij, J. (2017) · Annals of Computer Science and Information Systems · doi:10.15439/2017f317 · Cited by 18
Self-citationR 2-IBMOLS applied to a practical case of the multiobjective knapsack problem
Chabane, B., Basseur, M., & Hao, J. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.11.007 · Cited by 17
A Study of the Convergence Characteristics of Multiobjective Evolutionary Algorithms
Goel, T., & Stander, N. (2010) · AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (MAO) · doi:10.2514/6.2010-9233 · Cited by 16
Cai, X., Cheng, X., Fan, Z., Goodman, E. D., & Wang, L. (2015) · Soft Computing · doi:10.1007/s00500-015-1921-0 · Cited by 15
Rubio‐Largo, Á., Zhang, Q., & Vega-Rodríguez, M. A. (2014) · Soft Computing · doi:10.1007/s00500-014-1239-3 · Cited by 12
Fast Pareto Front Approximation for Cloud Instance Pool Optimization
Oprescu, A., Filip, A. (., & Kielmann, T. (2015) · doi:10.1145/2739482.2764720 · Cited by 2
Hassan, M. H., Makdyssiian, L., & Bilal, W. H. (2017) · Modern Applied Science · doi:10.5539/mas.v11n5p52 · Cited by 2
Research works printed but not linked: 1
- Wu, 2001, Metrics for quality assessment of a multiobjective design optimization solution set, J. Mech. Des., 123, 18, 10.1115/1.1329875 · DOI · Paper page
Cited by
Applications and Challenges of Artificial Intelligence in Space Missions
Oche, P. A., Ewa, G. A., & Ibekwe, N. (2024) · IEEE Access · doi:10.1109/access.2021.3132500 · Cited by 68
“However, many real-life problems can be characterized as multi-objective problems involving multiple conflicting objectives that should be considered simultaneously [51, 52].”
Background
Metadata
- Authors
- Laszczyk, M., & Myszkowski, P. B.
- Affiliations
- Wrocław University of Science and Technology (PL)
- Corresponding author
- Myszkowski, Paweł B.
- ORCID
- Myszkowski, Paweł B.
- Venue
- Swarm and Evolutionary Computation
- Publisher
- Elsevier BV
- Published
- 2019-08
- Volume
- 48
- Pages
- 109-133
- ISSN
- 2210-6502 (ISSN-L)
- OpenAlex
- W2932401226
- Cited by
- 79
- References
- 61
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 7 · 2023: 9 · 2022: 15 · 2021: 17 · 2020: 10 · 2019: 4
- FWCI
- 7.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- Locations
- Swarm and Evolutionary Computation (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Advanced Multi-Objective Optimization Algorithms
- Social Sciences › Decision Sciences › Management Science and Operations Research › Resource-Constrained Project Scheduling
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Keywords
- multi-objective optimization, quality metrics, comparison of solutions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
