The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies
Abstract
Machine learning is inherently a multiobjective task. Traditionally, however, either only one of the objectives is adopted as the cost function or multiple objectives are aggregated to a scalar cost function. This can be mainly attributed to the fact that most conventional learning algorithms can only deal with a scalar cost function. Over the last decade, efforts on solving machine learning problems using the Pareto-based multiobjective optimization methodology have gained increasing impetus, particularly due to the great success of multiobjective optimization using evolutionary algorithms and other population-based stochastic search methods. It has been shown that Pareto-based multiobjective learning approaches are more powerful compared to learning algorithms with a scalar cost function in addressing various topics of machine learning, such as clustering, feature selection, improvement of generalization ability, knowledge extraction, and ensemble generation. One common benefit of the different multiobjective learning approaches is that a deeper insight into the learning problem can be gained by analyzing the Pareto front composed of multiple Pareto-optimal solutions. This paper provides an overview of the existing research on multiobjective machine learning, focusing on supervised learning. In addition, a number of case studies are provided to illustrate the major benefits of the Pareto-based approach to machine learning, e.g., how to identify interpretable models and models that can generalize on unseen data from the obtained Pareto-optimal solutions. Three approaches to Pareto-based multiobjective ensemble generation are compared and discussed in detail. Finally, potentially interesting topics in multiobjective machine learning are suggested.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Multi-Objective Optimization Using Evolutionary Algorithms
Deb, K., & Kalyanmoy, D. (2001) · Andalas University Repository (Andalas University) · Cited by 17,328
Estimating the Number of Clusters in a Data Set Via the Gap Statistic
Tibshirani, R., Walther, G., & Hastie, T. (2001) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00293 · Cited by 6,205
Model Selection and Multimodel Inference
(2003) · Technometrics · doi:10.1198/tech.2003.s146 · Cited by 5,449
A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimization: NSGA-II
Deb, K., Agrawal, S., Pratap, A., & Meyarivan, T. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45356-3_83 · Cited by 5,289
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260
A direct adaptive method for faster backpropagation learning: the RPROP algorithm
Riedmiller, M., & Braun, H. (2002) · IEEE International Conference on Neural Networks · doi:10.1109/icnn.1993.298623 · Cited by 3,917
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
Evolving artificial neural networks
Yao, X. (1999) · Proceedings of the IEEE · doi:10.1109/5.784219 · Cited by 2,981
Model-Based Gaussian and Non-Gaussian Clustering
Banfield, J. D., & Raftery, A. E. (1993) · Biometrics · doi:10.2307/2532201 · Cited by 2,395
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy
Kuncheva, L. I., & Whitaker, C. J. (2003) · Machine Learning · doi:10.1023/a:1022859003006 · Cited by 2,335 · cited 1×
“…in [52], functional diversity , which measures the average Euclidean distances among the outputs of two neural networks, mutual information between the output of two networks, and the Yule's Q statistics [59], which measures the correlation of the errors made by two models, are also considered.”
MethodIntroduction to Machine Learning
(2019) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789811201967_0001 · Cited by 1,639
Regularization Theory and Neural Networks Architectures
Girosi, F., Jones, M., & Poggio, T. (1995) · Neural Computation · doi:10.1162/neco.1995.7.2.219 · Cited by 1,349
Das, I., & Dennis, J. E. (1997) · Structural Optimization · doi:10.1007/bf01197559 · Cited by 1,208
(1998) · The MIT Press eBooks · doi:10.7551/mitpress/5704.001.0001 · Cited by 1,114
Diversity creation methods: a survey and categorisation
Brown, G., Wyatt, J., Harris, R., & Yao, X. (2005) · Information Fusion · doi:10.1016/j.inffus.2004.04.004 · Cited by 985
Ensemble learning via negative correlation
Liu, Y., & Yao, X. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00073-8 · Cited by 712
ROC Graphs: Notes and Practical Considerations for Data Mining Researchers
Fawcett, T. E. (2003) · Cited by 632
Fuzzy modeling of high-dimensional systems: complexity reduction and interpretability improvement
Jin, Y. (2000) · IEEE Transactions on Fuzzy Systems · doi:10.1109/91.842154 · Cited by 480
Self-citationEmpirical evaluation of the improved Rprop learning algorithms
Igel, C., & Hüsken, M. (2003) · Neurocomputing · doi:10.1016/s0925-2312(01)00700-7 · Cited by 433
Ishibuchi, H., Murata, T., & Türkşen, I. B. (1997) · Fuzzy Sets and Systems · doi:10.1016/s0165-0114(96)00098-x · Cited by 379
Reducing Local Optima in Single-Objective Problems by Multi-objectivization
Knowles, J. D., Watson, R. A. R., & Corne, D. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44719-9_19 · Cited by 345
Generating Accurate and Diverse Members of a Neural-Network Ensemble
Opitz, D. W., & SHAVLIK, J. W. (1995) · Cited by 279
Generating concise and accurate classification rules for breast cancer diagnosis
Setiono, R. (2000) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(99)00041-x · Cited by 250
Multi-Objective Machine Learning
Jin, Y. (2006) · Studies in computational intelligence · doi:10.1007/3-540-33019-4 · Cited by 249
Self-citationEnsemble Learning Using Decorrelated Neural Networks
ROSEN, B. E. (1996) · Connection Science · doi:10.1080/095400996116820 · Cited by 248
Jin, Y., Olhofer, M., & Sendhoff, B. (2007) · Cited by 211
Self-citationCOVNET: a cooperative coevolutionary model for evolving artificial neural networks
Garcia-Pedrajas, N., Hervas-Martinez, C., & Munoz-Perez, J. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.810618 · Cited by 211
Multi-criteria Reinforcement Learning
Gábor, Z., Kalmár, Z., & Szepesvári, C. (1998) · Cited by 205
Wang, H., Kwong, S., Jin, Y., Wei, W., & Man, K. F. (2005) · Fuzzy Sets and Systems · doi:10.1016/j.fss.2004.07.013 · Cited by 175
Self-citationExtracting Rules from Artificial Neural Networks with Distributed Representations
Thrun, S. (1994) · Cited by 175
Neural network regularization and ensembling using multi-objective evolutionary algorithms
Jin, Y., Okabe, T., & Sendhoff, B. (2004) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2004.1330830 · Cited by 165
Self-citationSpeeding Up Backpropagation Using Multiobjective Evolutionary Algorithms
Abbass, H. A. (2003) · Neural Computation · doi:10.1162/089976603322385126 · Cited by 153
Evolving hybrid ensembles of learning machines for better generalisation
Chandra, A., & Yao, X. (2006) · Neurocomputing · doi:10.1016/j.neucom.2005.12.014 · Cited by 144
Classes of network connectivity and dynamics
Sporns, O., & Tononi, G. (2001) · Complexity · doi:10.1002/cplx.10015 · Cited by 138
Advanced Fuzzy Systems Design and Applications
Jin, Y. (2003) · Studies in fuzziness and soft computing · doi:10.1007/978-3-7908-1771-3 · Cited by 132
Self-citationMulti-class ROC analysis from a multi-objective optimisation perspective
Everson, R., & Fieldsend, J. E. (2005) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.016 · Cited by 122
Evolutionary model selection in unsupervised learning
Kim, Y., Street, W. N., & Menczer, F. (2002) · Intelligent Data Analysis · doi:10.3233/ida-2002-6605 · Cited by 122
Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization
Abbass, H. A. (2004) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2003.1299928 · Cited by 121
Exploiting the Trade-off — The Benefits of Multiple Objectives in Data Clustering
Handl, J., & Knowles, J. D. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-31880-4_38 · Cited by 116
A universal theorem on learning curves
Амари, Ш.-И. (1993) · Neural Networks · doi:10.1016/0893-6080(93)90013-m · Cited by 106
Advanced Fuzzy Systems Design and Applications
Jin (2003) · Cited by 103
Pareto Evolutionary Neural Networks
Fieldsend, J. E., & Singh, S. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.841794 · Cited by 101
Improving generalization of MLPs with multi-objective optimization
Teixeira, R. D. A., Braga, A. P., Takahashi, R. H. C., & Saldanha, R. R. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00327-1 · Cited by 99
DIVACE: Diverse and Accurate Ensemble Learning Algorithm
Chandra, A., & Yao, X. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-28651-6_91 · Cited by 95
Kupinski, M. A., & Anastasio, M. A. (1999) · IEEE Transactions on Medical Imaging · doi:10.1109/42.796281 · Cited by 87
Parameter Incremental Learning Algorithm for Neural Networks
Wan, S. H., & Banta, L. E. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.880581 · Cited by 77
Multi-objective Model Selection for Support Vector Machines
Igel, C. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-31880-4_37 · Cited by 74
Multi-Objective Machine Learning
Jin, Y. (2006) · Springer eBooks · doi:10.1007/11399346 · Cited by 74
Self-citationEvolving optimal feature extraction using multi-objective genetic programming
Zhang, Y., & Rockett, P. I. (2005) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1068009.1068143 · Cited by 67
Parameter Convergence and Learning Curves for Neural Networks
Fine, T. L., & Mukherjee, S. (1999) · Neural Computation · doi:10.1162/089976699300016647 · Cited by 48
Feature selection for ensembles:a hierarchical multi-objective genetic algorithm approach
Oliveira, L. S., Sabourin, R., Bortolozzi, F., & Suen, C. Y. (2005) · Proceedings of the International Conference on Document Analysis and Recognition · doi:10.1109/icdar.2003.1227748 · Cited by 43
EVOLUTIONARY MULTI-OBJECTIVE OPTIMISATION OF NEURAL NETWORKS FOR FACE DETECTION
Wiegand, S., Igel, C., & Handmann, U. (2004) · International Journal of Computational Intelligence and Applications · doi:10.1142/s1469026804001288 · Cited by 41
Jin, Y., Sendhoff, B., & Körner, E. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-31880-4_52 · Cited by 38
Self-citationTeo, J., & Abbass, H. A. (2004) · Evolutionary Computation · doi:10.1162/1063656041774974 · Cited by 36
Generalization Improvement in Multi-Objective Learning
Graening, L., Jin, Y., & Sendhoff, B. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.1716772 · Cited by 35
Self-citationOn-line learning with minimal degradation in feedforward networks
de Angulo, V. R., & Carme Torras, A. (1995) · IEEE Transactions on Neural Networks · doi:10.1109/72.377971 · Cited by 32
Multi-objective programming in SVMs
Bi, J. (2003) · Cited by 28
Structural Risk Minimization on Decision Trees Using an Evolutionary Multiobjective Optimization
Kim, D.-E. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24650-3_32 · Cited by 27
Catastrophic forgetting in simple networks: an analysis of the pseudorehearsal solution
Frean, M. R., & Robins, A. V. (1999) · Network Computation in Neural Systems · doi:10.1088/0954-898x/10/3/302 · Cited by 21
Alleviating Catastrophic Forgetting via Multi-Objective Learning
Jin, Y., & Sendhoff, B. (2006) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.1716554 · Cited by 20
Self-citationHierarchical Rank Density Genetic Algorithm for Radial-Basis Function Neural Network Design
Yen, G. G., & Lu, H. (2003) · International Journal of Computational Intelligence and Applications · doi:10.1142/s1469026803000975 · Cited by 19
Parameter identification with weightless regularization
Furukawa, T. (2001) · International Journal for Numerical Methods in Engineering · doi:10.1002/nme.215 · Cited by 17
Extraction Of Symbolic Rules From Artificial Neural Networks
Kamruzzaman, S. M., & Islam, M. M. (2007) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1071004 · Cited by 8
ERANN: An Algorithm to Extract Symbolic Rules from Trained Artificial Neural Networks
Kamruzzaman, S. M., Hamid, M. A., & Sarkar, A. J. (2012) · IETE Journal of Research · doi:10.4103/0377-2063.96181 · Cited by 2
Connectivity in real and evolved associative memories
Adams, R., Calcraft, L., & Davey, N. (2006) · University of Hertfordshire Research Archive (University of Hertfordshire) · Cited by 1
Cited by
Surrogate-assisted evolutionary computation: Recent advances and future challenges
Jin, Y. (2011) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2011.05.001 · Cited by 1,514
Self-citationExplainable artificial intelligence: A survey
Dosilovic, F. K., Brcic, M., & Hlupic, N. (2018) · 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2018.8400040 · Cited by 1,147
Methods of integrating data to uncover genotype-phenotype interactions
Ritchie, M. D., Holzinger, E. R., Li, R., Pendergrass, S. A., & Kim, D. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3868 · Cited by 1,085
Science Requirements and Detector Concepts for the Electron-Ion Collider
Abdul Khalek, R., Accardi, A., Adam, J., Adamiak, D., Akers, W., Albaladejo, M., Al-bataineh, A., Alexeev, M. G., Ameli, F., Antonioli, P., et al. (2022) · Nuclear Physics A · doi:10.1016/j.nuclphysa.2022.122447 · Cited by 913
Metaheuristic design of feedforward neural networks: A review of two decades of research
Ojha, V. K., Abraham, A., & Snášel, V. (2017) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2017.01.013 · Cited by 606
Metadata
- Authors
- Jin, Y., & Sendhoff, B.
- Venue
- IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)
- Published
- 2008-05
- Volume
- 38
- Issue
- 3
- Pages
- 397-415
- ISSN
- 1094-6977 (ISSN-L)
- OpenAlex
- W2112299196
- Cited by
- 455
- References
- 69
- FWCI
- 21.34
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.