The DOI matches its Crossref record; 137 of the 137 references it cites were found in a registry or an index.
Automated Algorithm Selection: Survey and Perspectives
Abstract
It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different algorithms. This is particularly pronounced for computationally hard problems, where in most cases, no single algorithm defines the state of the art; instead, there is a set of algorithms with complementary strengths. This performance complementarity can be exploited in various ways, one of which is based on the idea of selecting, from a set of given algorithms, for each problem instance to be solved the one expected to perform best. The task of automatically selecting an algorithm from a given set is known as the per-instance algorithm selection problem and has been intensely studied over the past 15 years, leading to major improvements in the state of the art in solving a growing number of discrete combinatorial problems, including propositional satisfiability and AI planning. Per-instance algorithm selection also shows much promise for boosting performance in solving continuous and mixed discrete/continuous optimisation problems. This survey provides an overview of research in automated algorithm selection, ranging from early and seminal works to recent and promising application areas. Different from earlier work, it covers applications to discrete and continuous problems, and discusses algorithm selection in context with conceptually related approaches, such as algorithm configuration, scheduling, or portfolio selection. Since informative and cheaply computable problem instance features provide the basis for effective per-instance algorithm selection systems, we also provide an overview of such features for discrete and continuous problems. Finally, we provide perspectives on future work in the area and discuss a number of open research challenges.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,156
No free lunch theorems for optimization
Wolpert, D. H., & Macready, W. G. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585893 · Cited by 14,237
Pattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000
The Origins of Order: Self-Organization and Selection in Evolution
Kauffman, S. A. (1993) · doi:10.1093/oso/9780195079517.001.0001 · Cited by 7,855
Comparison of Multiobjective Evolutionary Algorithms: Empirical Results
Zitzler, E., Deb, K., & Thiele, L. (2000) · Evolutionary Computation · doi:10.1162/106365600568202 · Cited by 5,697
Evolutionary Algorithms for Solving Multi-Objective Problems
(2007) · Genetic and Evolutionary Computation Series · doi:10.1007/978-0-387-36797-2 · Cited by 5,686
European journal of operational research
(1990) · Technological Forecasting and Social Change · doi:10.1016/0040-1625(90)90023-o · Cited by 5,365
An Effective Heuristic Algorithm for the Traveling-Salesman Problem
Lin, S., & Kernighan, B. W. (1973) · Operations Research · doi:10.1287/opre.21.2.498 · Cited by 3,821
Detecting Novel Associations in Large Data Sets
Reshef, D. N., Reshef, Y. A., Finucane, H. K., Grossman, S. R., McVean, G., Turnbaugh, P. J., Lander, E. S., Mitzenmacher, M., & Sabeti, P. C. (2011) · Science · doi:10.1126/science.1205438 · Cited by 3,339
Rokach, L. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-009-9124-7 · Cited by 2,657
Sequential Model-Based Optimization for General Algorithm Configuration
Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25566-3_40 · Cited by 2,469
Self-citationA review of multiobjective test problems and a scalable test problem toolkit
Huband, S., Hingston, P., Barone, L., & While, L. (2006) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.861417 · Cited by 2,090
Scalable Test Problems for Evolutionary Multiobjective Optimization
Deb, K., Thiele, L., Laumanns, M., & Zitzler, E. (2005) · Advanced Information and Knowledge Processing · doi:10.1007/1-84628-137-7_6 · Cited by 2,010
kernlab- An S4 Package for Kernel Methods in R
Karatzoglou, A., Smola, A., Hornik, K., & Zeileis, A. (2004) · Journal of Statistical Software · doi:10.18637/jss.v011.i09 · Cited by 1,786
The irace package: Iterated racing for automatic algorithm configuration
López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., & Stützle, T. (2016) · Operations Research Perspectives · doi:10.1016/j.orp.2016.09.002 · Cited by 1,761
An effective implementation of the Lin–Kernighan traveling salesman heuristic
Helsgaun, K. (2000) · European Journal of Operational Research · doi:10.1016/s0377-2217(99)00284-2 · Cited by 1,630
Density-Based Clustering in Spatial Databases: The Algorithm GDBSCAN and Its Applications
Sander, J., Ester, M., Kriegel, H.-P., & Xu, X. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009745219419 · Cited by 1,495
Knowledge Representation, Reasoning and Declarative Problem Solving
Baral, C. (2003) · Cambridge University Press eBooks · doi:10.1017/cbo9780511543357 · Cited by 1,413
Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms
Thornton, C., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487629 · Cited by 1,359
Self-citationHyper-heuristics: a survey of the state of the art
Burke, E. K., Gendreau, M., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., & Qu, R. (2013) · Journal of the Operational Research Society · doi:10.1057/jors.2013.71 · Cited by 1,263
The Fast Downward Planning System
Helmert, M. (2006) · Cited by 1,222
Multiobjective evolutionary algorithms: classifications, analyses, and new innovations
Lamont, G. B., & Veldhuizen, D. A. V. (1999) · Cited by 1,212
The Algorithm Selection Problem
Rice, J. R. (1976) · Advances in Computers · doi:10.1016/s0065-2458(08)60520-3 · Cited by 1,195
Handbook of Satisfiability: Second Edition
(2021) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/faia336 · Cited by 1,038
Foundations of genetic algorithms. 2
Booker, L. B. (1994) · Biosystems · doi:10.1016/0303-2647(94)90009-4 · Cited by 1,018
Multiobjective optimization Test Instances for the CEC 2009 Special Session and Competition
Zhang, Q., Zhou, A., Zhao, S.-Z., et al. (2009) · Cited by 951
No Free Lunch Theorems for SearchPreprint
Wolpert, D. H., & Macready, W. G. (1995) · RePEc: Research Papers in Economics · Cited by 927
ParamILS: An Automatic Algorithm Configuration Framework
Hutter, F., Hoos, H. H., Leyton-Brown, K., & Stuetzle, T. (2009) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2861 · Cited by 882
Self-citationSparse multinomial logistic regression: fast algorithms and generalization bounds
Krishnapuram, B., Carin, L., Figueiredo, M. A. T., & Hartemink, A. J. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.127 · Cited by 877
SATzilla: Portfolio-based Algorithm Selection for SAT
Xu, L., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2490 · Cited by 857
Self-citationScatter Search and Local NLP Solvers: A Multistart Framework for Global Optimization
Ugray, Z., Lasdon, L., Plummer, J., Glover, F., Kelly, J., & Martí, R. (2007) · INFORMS Journal on Computing · doi:10.1287/ijoc.1060.0175 · Cited by 726
Auto-WEKA: Automatic Model Selection and Hyperparameter Optimization in WEKA
Kotthoff, L., Thornton, C., Hoos, H. H., Hutter, F., & Leyton-Brown, K. (2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5_4 · Cited by 650
Self-citationProceedings of the 5th International Conference on Genetic Algorithms
Forrest, S. (1993) · Cited by 609
Cross-disciplinary perspectives on meta-learning for algorithm selection
Smith-Miles, K. A. (2009) · ACM Computing Surveys · doi:10.1145/1456650.1456656 · Cited by 552
An instance-weighting method to induce cost-sensitive trees
Ting, K. M. (2002) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2002.1000348 · Cited by 504
Gomes, C. P., & Selman, B. (2001) · Artificial Intelligence · doi:10.1016/s0004-3702(00)00081-3 · Cited by 461
Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions
Hansen, N., Ros, R., & Auger, A. (2009) · Cited by 414
General k-opt submoves for the Lin–Kernighan TSP heuristic
Helsgaun, K. (2009) · Mathematical Programming Computation · doi:10.1007/s12532-009-0004-6 · Cited by 414
Automated Planning: Theory and Practice
Long, D. (2005) · Industrial Robot: An International Journal · doi:10.1108/ir.2005.04932eae.001 · Cited by 408
Algorithm runtime prediction: Methods & evaluation
Hutter, F., Xu, L., Hoos, H. H., & Leyton-Brown, K. (2014) · Artificial Intelligence · doi:10.1016/j.artint.2013.10.003 · Cited by 395
Self-citationAn Economics Approach to Hard Computational Problems
Huberman, B. A., Lukose, R. M., & Hogg, T. (1997) · Science · doi:10.1126/science.275.5296.51 · Cited by 390
The Traveling Salesman Problem: A Computational Study
Cook, W. J., Applegate, D. L., Bixby, R. E., & Chvátal, V. (2011) · Princeton University Press eBooks · doi:10.1515/9781400841103 · Cited by 390
Exploratory landscape analysis
Mersmann, O., Bischl, B., Trautmann, H., Preuss, M., Weihs, C., & Rudolph, G. (2011) · Proceedings of the 13th annual conference on Genetic and evolutionary computation · doi:10.1145/2001576.2001690 · Cited by 389
Self-citationStochastic global optimization methods part II: Multi level methods
Rinnooy Kan, A. H. G., & Timmer, G. T. (1987) · Mathematical Programming · doi:10.1007/bf02592071 · Cited by 379
Proceedings of the 6th International Conference on Genetic Algorithms
Eshelman, L. J. (1995) · Cited by 355
Proceedings of the 10th European conference on artificial intelligence
Siklóssy, L., & Tepandi, J. (1993) · Engineering Applications of Artificial Intelligence · doi:10.1016/0952-1976(93)90070-e · Cited by 327
Planning Through Stochastic Local Search and Temporal Action Graphs in LPG
Gerevini, A., Saetti, A., & Serina, I. (2003) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1183 · Cited by 318
A survey of techniques for characterising fitness landscapes and some possible ways forward
Malan, K. M., & Engelbrecht, A. P. (2013) · Information Sciences · doi:10.1016/j.ins.2013.04.015 · Cited by 280
Bischl, B., Lang, M., Kotthoff, L., Schratz, P., Schiffner, J., Richter, J., Jones, Z., Casalicchio, G., Gallo, M., & Binder, M. (2013) · CRAN: Contributed Packages · doi:10.32614/cran.package.mlr · Cited by 277
Algorithm Selection for Combinatorial Search Problems: A Survey
Kotthoff, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-50137-6_7 · Cited by 270
Tušar, T., & Filipič, B. (2015) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2014.2313407 · Cited by 248
(2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9 · Cited by 236
Machine Learning for Data Streams: with Practical Examples in MOA
Bifet, A., Gavaldà, R., Holmes, G., & Pfahringer, B. (2018) · The MIT Press eBooks · doi:10.7551/mitpress/10654.001.0001 · Cited by 236
ASlib: A benchmark library for algorithm selection
Bischl, B., Kerschke, P., Kotthoff, L., Lindauer, M., Malitsky, Y., Fréchette, A., Hoos, H., Hutter, F., Leyton-Brown, K., Tierney, K., et al. (2016) · Artificial Intelligence · doi:10.1016/j.artint.2016.04.003 · Cited by 214
Self-citationA Comprehensive Survey on Fitness Landscape Analysis
Pitzer, E., & Affenzeller, M. (2012) · Studies in Computational Intelligence · doi:10.1007/978-3-642-23229-9_8 · Cited by 209
Muñoz, M. A., Sun, Y., Kirley, M., & Halgamuge, S. K. (2015) · Information Sciences · doi:10.1016/j.ins.2015.05.010 · Cited by 208
On Density-Based Data Streams Clustering Algorithms: A Survey
Amini, A., Wah, T. Y., & Saboohi, H. (2014) · Journal of Computer Science and Technology · doi:10.1007/s11390-014-1416-y · Cited by 200
Information Characteristics and the Structure of Landscapes
Vassilev, V. K., Fogarty, T. C., & Miller, J. F. (2000) · Evolutionary Computation · doi:10.1162/106365600568095 · Cited by 198
ISAC – Instance-Specific Algorithm Configuration
Kadıoğlu, S., Malitsky, Y., Sellmann, M., & Tierney, K. (2010) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-60750-606-5-751 · Cited by 192
Multiobjective genetic algorithms with application to control engineering problems.
Fonseca, C. M. (1995) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 192
Barrier Trees of Degenerate Landscapes 1
Christoph Flamm, A., Hofacker, I. L., Stadler, P. F., & Wolfinger, M. T. (2008) · Cited by 187
Understanding Random SAT: Beyond the Clauses-to-Variables Ratio
Nudelman, E., Leyton-Brown, K., Hoos, H. H., Devkar, A., & Shoham, Y. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30201-8_33 · Cited by 183
Self-citationFitness Landscapes and Evolvability
Smith, T., Husbands, P., & O'Shea, M. (2002) · Evolutionary Computation · doi:10.1162/106365602317301754 · Cited by 179
Plan Constraints and Preferences in PDDL3
Gerevini, A., & Long, D. P. (2006) · Cited by 177
Exploratory Landscape Analysis of Continuous Space Optimization Problems Using Information Content
Munoz, M. A., Kirley, M., & Halgamuge, S. K. (2015) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2014.2302006 · Cited by 173
Kerschke, P., & Trautmann, H. (2019) · Evolutionary Computation · doi:10.1162/evco_a_00236 · Cited by 173
Self-citationThe dispersion metric and the CMA evolution strategy
Lunacek, M., & Whitley, D. (2006) · Proceedings of the 8th annual conference on Genetic and evolutionary computation · doi:10.1145/1143997.1144085 · Cited by 171
COCO: A Platform for Comparing Continuous Optimizers in a Black-Box SettingPreprint
Hansen, N., Auger, A., Mersmann, O., Tušar, T., & Brockhoff, D. (2016) · HAL · Cited by 170
On the Futility of Blind Search: An Algorithmic View of “No Free Lunch”
Culberson, J. C. (1998) · Evolutionary Computation · doi:10.1162/evco.1998.6.2.109 · Cited by 164
Algorithm Selection and Scheduling
Kadıoğlu, S., Malitsky, Y., Sabharwal, A., Samulowitz, H., & Sellmann, M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23786-7_35 · Cited by 156
Instance spaces for machine learning classification
Muñoz, M. A., Villanova, L., Baatar, D., & Smith‐Miles, K. (2017) · Machine Learning · doi:10.1007/s10994-017-5629-5 · Cited by 151
Zchaff2004: An Efficient SAT Solver
Mahajan, Y. S., Fu, Z., & Malik, S. (2005) · Lecture notes in computer science · doi:10.1007/11527695_27 · Cited by 147
The online performance estimation framework: heterogeneous ensemble learning for data streams
van Rijn, J. N., Holmes, G., Pfahringer, B., & Vanschoren, J. (2017) · Machine Learning · doi:10.1007/s10994-017-5686-9 · Cited by 135
Stadler, P. F. (2007) · Springer eBooks · doi:10.1007/3-540-45692-9_10 · Cited by 135
The International SAT Solver Competitions
Järvisalo, M., Berre, D. L., Roussel, O., & Simon, L. (2012) · AI Magazine · doi:10.1609/aimag.v33i1.2395 · Cited by 127
AutoFolio: An Automatically Configured Algorithm Selector
Lindauer, M., Hoos, H. H., Hutter, F., & Schaub, T. H. (2015) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4726 · Cited by 125
Self-citationQuantifying ruggedness of continuous landscapes using entropy
Malan, K. M., & Engelbrecht, A. P. (2009) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2009.4983112 · Cited by 121
Kerschke, P., & Trautmann, H. (2019) · Studies in classification, data analysis, and knowledge organization · doi:10.1007/978-3-030-25147-5_7 · Cited by 113
Self-citationThe 2014 International Planning Competition: Progress and Trends
Vallati, M., Chrpa, L., Grześ, M., McCluskey, T. L., Roberts, M., & Sanner, S. (2015) · AI Magazine · doi:10.1609/aimag.v36i3.2571 · Cited by 112
Contemporary Evolution Strategies
Bäck, T., Foussette, C., & Krause, P. (2013) · Natural computing series · doi:10.1007/978-3-642-40137-4 · Cited by 106
Detecting Funnel Structures by Means of Exploratory Landscape Analysis
Kerschke, P., Preuß, M., Wessing, S., & Trautmann, H. (2015) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2739480.2754642 · Cited by 101
Self-citationDiscovering the suitability of optimisation algorithms by learning from evolved instances
Smith‐Miles, K., & Hemert, J. I. V. (2011) · Annals of Mathematics and Artificial Intelligence · doi:10.1007/s10472-011-9230-5 · Cited by 99
Evaluating Component Solver Contributions to Portfolio-Based Algorithm Selectors
Xu, L., Hutter, F., Hoos, H. H., & Leyton‐Brown, K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-31612-8_18 · Cited by 96
Self-citationMersmann, O., Bischl, B., Trautmann, H., Wagner, M., Bossek, J., & Neumann, F. (2013) · Annals of Mathematics and Artificial Intelligence · doi:10.1007/s10472-013-9341-2 · Cited by 96
Self-citationPer instance algorithm configuration of CMA-ES with limited budget
Belkhir, N., Dréo, J., Pierre Savéant, B., & Schoenauer, M. (2017) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3071178.3071343 · Cited by 96
A self-adaptive multi-engine solver for quantified Boolean formulas
Pulina, L., & Tacchella, A. (2008) · Constraints · doi:10.1007/s10601-008-9051-2 · Cited by 91
Leveraging TSP Solver Complementarity through Machine Learning
Kerschke, P., Kotthoff, L., Bossek, J., Hoos, H. H., & Trautmann, H. (2017) · Evolutionary Computation · doi:10.1162/evco_a_00215 · Cited by 91
Self-citationA Lifelong Learning Hyper-heuristic Method for Bin Packing
Sim, K., Hart, E., & Paechter, B. (2014) · Evolutionary Computation · doi:10.1162/evco_a_00121 · Cited by 85
Multiagent Optimization System for Solving the Traveling Salesman Problem (TSP)
Xie, X., & Liu, J. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2006910 · Cited by 84
Optimizing Data Stream Representation: An Extensive Survey on Stream Clustering Algorithms
Carnein, M., & Trautmann, H. (2019) · Business & Information Systems Engineering · doi:10.1007/s12599-019-00576-5 · Cited by 81
Self-citationModel-based genetic algorithms for algorithm configuration
Ansótegui, C., Malitsky, Y., Samulowitz, H., Sellmann, M., & Tierney, K. (2015) · International Conference on Artificial Intelligence · Cited by 80
claspfolio2: Advances in Algorithm Selection for Answer Set Programming
Hoos, H. H., Lindauer, M., & Schaub, T. H. (2014) · Theory and Practice of Logic Programming · doi:10.1017/s1471068414000210 · Cited by 79
Self-citationA case study of algorithm selection for the traveling thief problem
Wagner, M., Lindauer, M., Mısır, M., Nallaperuma, S., & Hutter, F. (2017) · Journal of Heuristics · doi:10.1007/s10732-017-9328-y · Cited by 79
Population-based Algorithm Portfolios with automated constituent algorithms selection
Tang, K., Peng, F., Chen, G., & Yao, X. (2014) · Information Sciences · doi:10.1016/j.ins.2014.03.105 · Cited by 78
An evaluation of sequential model-based optimization for expensive blackbox functions
Hutter, F., Hoos, H. H., & Leyton‐Brown, K. (2013) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/2464576.2501592 · Cited by 78
Self-citationOpenML: A Collaborative Science Platform
van Rijn, J. N., Bischl, B., Torgo, L., Gao, B., Umaashankar, V., Fischer, S., Winter, P., Wiswedel, B., Berthold, M. R., & Vanschoren, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40994-3_46 · Cited by 77
AClib: A Benchmark Library for Algorithm Configuration
Hutter, F., López‐Ibáñez, M., Fawcett, C., Lindauer, M., Hoos, H. H., Leyton‐Brown, K., & Stützle, T. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-09584-4_4 · Cited by 73
Self-citationFonseca, C. M., da Fonseca, V. G., & Paquete, L. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-31880-4_18 · Cited by 67
Muñoz, M. A., & Smith‐Miles, K. (2016) · Evolutionary Computation · doi:10.1162/evco_a_00194 · Cited by 67
Improving the State of the Art in Inexact TSP Solving Using Per-Instance Algorithm Selection
Kotthoff, L., Kerschke, P., Hoos, H. H., & Trautmann, H. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19084-6_18 · Cited by 66
Self-citationProceedings of the 5th International Conference on Parallel Problem Solving from Nature
Eiben, A. E., Bäck, T., Schoenauer, M., & Schwefel, H. (1998) · Cited by 64
Exploiting Competitive Planner Performance
Howe, A. E., Dahlman, E., Hansen, C., Scheetz, M., & von Mayrhauser, A. (2000) · Lecture notes in computer science · doi:10.1007/10720246_5 · Cited by 62
Bi-population CMA-ES agorithms with surrogate models and line searches
Loshchilov, I., Schoenauer, M., & Sébag, M. (2013) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/2464576.2482696 · Cited by 61
Analyzing Search Topology Without Running Any Search: On the Connection Between Causal Graphs and h+
Hoffmann, J. (2011) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3276 · Cited by 59
Algorithm Selection on Data Streams
van Rijn, J. N., Holmes, G., Pfahringer, B., & Vanschoren, J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11812-3_28 · Cited by 58
Evolving Combinatorial Problem Instances That Are Difficult to Solve
Hemert, J. I. V. (2006) · Evolutionary Computation · doi:10.1162/evco.2006.14.4.433 · Cited by 54
Quantifying Variable Interactions in Continuous Optimization Problems
Sun, Y., Kirley, M., & Halgamuge, S. K. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2016.2599164 · Cited by 45
An Empirical Comparison of Stream Clustering Algorithms
Carnein, M., Assenmacher, D., & Trautmann, H. (2017) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/3075564.3078887 · Cited by 44
Self-citationAlgorithm Selection as a Bandit Problem with Unbounded Losses
Gagliolo, M., & Schmidhuber, J. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13800-3_7 · Cited by 42
Stability of Controllers for Gaussian Process Dynamics
Vinogradska, J., Bischoff, B., Nguyen-Tuong, D., & Peters, J. (2017) · Max Planck Digital Library · Cited by 40
On composing an algorithm portfolio
Yuen, S. Y., & Zhang, X. (2015) · Memetic Computing · doi:10.1007/s12293-015-0159-9 · Cited by 36
Technical Communications of the 26th International Conference on Logic Programming
Janhunen, T. (2010) · Cited by 35
An Experimental Study of Adaptive Capping in irace
Cáceres, L. P., López‐Ibáñez, M., Hoos, H. H., & Stützle, T. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-69404-7_17 · Cited by 33
Self-citationThe Algorithm Selection Problem on the Continuous Optimization Domain
Muñoz, M. A., Kirley, M., & Halgamuge, S. K. (2012) · Studies in computational intelligence · doi:10.1007/978-3-642-32378-2_6 · Cited by 32
Benchmarking the local metamodel CMA-ES on the noiseless BBOB'2013 test bed
Auger, A., Brockhoff, D., & Hansen, N. (2013) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/2464576.2482701 · Cited by 32
Cohen, D. A. (2010) · Principles and Practice of Constraint Programming · Cited by 32
Fukunaga, A. S. (2002) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2000.870802 · Cited by 26
Analysing and characterising optimization problems using length scale
Morgan, R., & Gallagher, M. R. (2015) · Soft Computing · doi:10.1007/s00500-015-1878-z · Cited by 26
Feature Based Algorithm Configuration: A Case Study with Differential Evolution
Belkhir, N., Dréo, J., Pierre Savéant, B., & Schoenauer, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45823-6_15 · Cited by 25
On the selection of fitness landscape analysis metrics for continuous optimization problems
Sun, Y., Halgamuge, S. K., Kirley, M., & Muñoz, M. A. (2014) · International Conference on Information and Automation for Sustainability (ICIAfS) · doi:10.1109/iciafs.2014.7069635 · Cited by 23
Algorithm configuration data mining for CMA evolution strategies
van Rijn, S., Wang, H., van Stein, B., & Bäck, T. (2017) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3071178.3071205 · Cited by 22
Portfolio-Based Algorithm Selection for Circuit QBFs
Hoos, H. H., Peitl, T., Slivovsky, F., & Szeider, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-98334-9_13 · Cited by 17
Self-citationProceedings of the 10th international conference on Artificial Evolution
Collet, P., Monmarché, N., Legrand, P., Schoenauer, M., & Lutton, É. (2005) · Cited by 17
Boosting Sequential Solver Portfolios: Knowledge Sharing and Accuracy Prediction
Malitsky, Y., Sabharwal, A., Samulowitz, H., & Sellmann, M. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-44973-4_17 · Cited by 15
Parameterization of state-of-the-art performance indicators
Kerschke, P., Bossek, J., & Trautmann, H. (2018) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3205651.3208233 · Cited by 15
Self-citationStatic and Dynamic Portfolio Methods for Optimal Planning: An Empirical Analysis
Rizzini, M., Fawcett, C., Vallati, M., Gerevini, A., & Hoos, H. H. (2017) · International Journal of Artificial Intelligence Tools · doi:10.1142/s0218213017600065 · Cited by 14
Self-citationApplying Machine Learning Techniques to ASP Solving
Maratea, M., Pulina, L., & Ricca, F. (2012) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.iclp.2012.37 · Cited by 13
Cimatti, A., & Sebastiani, R. (2012) · Cited by 12
Proceedings of the Twenty-Fourth International Conference on Automated Planning and Scheduling
Chien, S., Fern, A., & Ruml, W. (2014) · AAAI Press eBooks · Cited by 11
Poursoltan, S., & Neumann, F. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-26555-1_38 · Cited by 5
Self-citationKerschke, P. (2017) · URN:NBN Resolver for Germany & Switzerland (German National Library) · Cited by 2
Self-citation
Cited by
Karimi-Mamaghan, M., Mohammadi, M., Meyer, P., Karimi-Mamaghan, A. M., & Talbi, E.-G. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.04.032 · Cited by 462
Osaba, E., Villar-Rodriguez, E., Del Ser, J., Nebro, A. J., Molina, D., LaTorre, A., Suganthan, P. N., Coello Coello, C. A., & Herrera, F. (2021) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2021.100888 · Cited by 406
Benchmarking in Optimization: Best Practice and Open Issues
Bartz-Beielstein, T., Doerr, C., Berg, D. V. D., Bossek, J., Chandrasekaran, S., Eftimov, T., Fischbach, A., Kerschke, P., La Cava, W., Lopez-Ibanez, M., et al. (2020) · arXiv · doi:10.48550/arxiv.2007.03488 · Cited by 214
Self-citationA review on the self and dual interactions between machine learning and optimisation
Song, H., Triguero, I., & Özcan, E. (2019) · Progress in Artificial Intelligence · doi:10.1007/s13748-019-00185-z · Cited by 101
Learning Performance-Improving Code Edits
Shypula, A., Madaan, A., Zeng, Y., Alon, U., Gardner, J., Hashemi, M., Neubig, G., Ranganathan, P., Bastani, O., & Yazdanbakhsh, A. (2023) · arXiv · doi:10.48550/arxiv.2302.07867 · Cited by 146
“…2022; Kumar et al., 2022), auto-vectorize code (Nuzman et al., 2006; Mendis et al., 2019), optimize GPU code (Liou et al., 2020; Cummins et al., 2021), and automatically select algorithms (Kotthoff, 2016; Kerschke et al., 2019). and room at the top (Leiserson et al., 2020; Sherry & Thompson, 2021).”
Method
Metadata
- Authors
- Kerschke, P., Hoos, H. H., Neumann, F., & Trautmann, H.
- Venue
- Evolutionary Computation
- Publisher
- MIT Press
- Published
- 2019-03
- Volume
- 27
- Issue
- 1
- Pages
- 3-45
- ISSN
- 1063-6560 (ISSN-L)
- 1530-9304
- OpenAlex
- W2898873842
- Cited by
- 435
- References
- 137
- FWCI
- 18.39
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.