The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
ParamILS: An Automatic Algorithm Configuration Framework
Open accessAbstract
The identification of performance-optimizing parameter settings is an important part of the development and application of algorithms. We describe an automatic framework for this algorithm configuration problem. More formally, we provide methods for optimizing a target algorithms performance on a given class of problem instances by varying a set of ordinal and/or categorical parameters. We review a family of local-search-based algorithm configuration procedures and present novel techniques for accelerating them by adaptively limiting the time spent for evaluating individual configurations. We describe the results of a comprehensive experimental evaluation of our methods, based on the configuration of prominent complete and incomplete algorithms for SAT. We also present what is, to our knowledge, the first published work on automatically configuring the CPLEX mixed integer programming solver. All the algorithms we considered had default parameter settings that were manually identified with considerable effort. Nevertheless, using our automated algorithm configuration procedures, we achieved substantial and consistent performance improvements.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Design and Analysis of Computer Experiments
Santner, T. J., Williams, B. J., & Notz, W. I. (2003) · Springer Series in Statistics · doi:10.1007/978-1-4757-3799-8 · Cited by 2,170
Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control
Spall, J. C. (2003) · Wiley eBooks · doi:10.1002/0471722138 · Cited by 1,890
Stochastic Local Search: Foundations & Applications
Hoos, H. H., & St�tzle, T. (2004) · Medical Entomology and Zoology · Cited by 1,227
Self-citationA new method for solving hard satisfiability problems
Selman, B., Levesque, H. J., & Mitchell, D. G. M. (1992) · Cited by 1,156
SATzilla: Portfolio-based Algorithm Selection for SAT
Xu, L., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2490 · Cited by 857
Self-citationMinimizing conflicts: a heuristic repair method for constraint satisfaction and scheduling problems
Minton, S., Johnston, M. D., Philips, A. B., & Laird, P. (1992) · Artificial Intelligence · doi:10.1016/0004-3702(92)90007-k · Cited by 843
A Racing Algorithm for Configuring Metaheuristics
Birattari, M., Stützle, T., Paquete, L., & Varrentrapp, K. (2002) · Cited by 498
Reactive Search and Intelligent Optimization
Battiti, R., Brunato, M., & Mascia, F. (2008) · Operations research, computer science. Interface series · doi:10.1007/978-0-387-09624-7 · Cited by 279
Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation
Maron, O., & Moore, A. (2018) · doi:10.1184/r1/6555050 · Cited by 265
A theoretician’s guide to the experimental analysis of algorithms
Johnson, D. (2002) · DIMACS Series in Discrete Mathematics and Theoretical Computer Science · doi:10.1090/dimacs/059/11 · Cited by 261
Cited by
Sequential Model-Based Optimization for General Algorithm Configuration
Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25566-3_40 · Cited by 2,469
Self-citationThe irace package: Iterated racing for automatic algorithm configuration
López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., & Stützle, T. (2016) · Operations Research Perspectives · doi:10.1016/j.orp.2016.09.002 · Cited by 1,761
“evolutionary algorithms [22] and local search [15].”
Self-citationImproving the search performance of SHADE using linear population size reduction
Tanabe, R., & Fukunaga, A. S. (2014) · 2014 IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2014.6900380 · Cited by 1,625
Automated Machine Learning: Methods, Systems, Challenges
(2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5 · Cited by 1,485
Scalable Nearest Neighbor Algorithms for High Dimensional Data
Muja, M., & Lowe, D. G. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2321376 · Cited by 1,407
Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms
Thornton, C., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487629 · Cited by 1,359
Self-citationHyper-heuristics: a survey of the state of the art
Burke, E. K., Gendreau, M., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., & Qu, R. (2013) · Journal of the Operational Research Society · doi:10.1057/jors.2013.71 · Cited by 1,263
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges
Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann, T., Becker, M., Boulesteix, A., et al. (2023) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1484 · Cited by 930
Auto-WEKA: Automatic Model Selection and Hyperparameter Optimization in WEKA
Kotthoff, L., Thornton, C., Hoos, H. H., Hutter, F., & Leyton-Brown, K. (2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5_4 · Cited by 650
Self-citationParameter tuning for configuring and analyzing evolutionary algorithms
Eiben, A. E., & Smit, S. K. (2011) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2011.02.001 · Cited by 622
Proceedings of the Genetic and Evolutionary Computation Conference
(2022) · doi:10.1145/3512290 · Cited by 546
From evolutionary computation to the evolution of things
Eiben, A. E., & Smith, J. (2015) · Nature · doi:10.1038/nature14544 · Cited by 496
Benchmark and Survey of Automated Machine Learning Frameworks
Zöller, M.-A., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11854 · Cited by 448
Automated Algorithm Selection: Survey and Perspectives
Kerschke, P., Hoos, H. H., Neumann, F., & Trautmann, H. (2019) · Evolutionary Computation · doi:10.1162/evco_a_00242 · Cited by 435
Self-citationCosta, E. B., Fonseca, B., Santana, M. A., de Araújo, F. F., & Rego, J. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.01.047 · Cited by 418
Osaba, E., Villar-Rodriguez, E., Del Ser, J., Nebro, A. J., Molina, D., LaTorre, A., Suganthan, P. N., Coello Coello, C. A., & Herrera, F. (2021) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2021.100888 · Cited by 406
Bayesian Optimization in a Billion Dimensions via Random Embeddings
Wang, Z., Hutter, F., Zoghi, M., Matheson, D., & De Feitas, N. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4806 · Cited by 390
Self-citationA Survey of Automatic Parameter Tuning Methods for Metaheuristics
Huang, C., Li, Y., & Yao, X. (2020) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2019.2921598 · Cited by 371
A review of automatic selection methods for machine learning algorithms and hyper-parameter values
Luo, G. (2016) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-016-0125-6 · Cited by 357
An easy-to-use real-world multi-objective optimization problem suite
Tanabe, R., & Ishibuchi, H. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106078 · Cited by 324
Learning to Branch in Mixed Integer Programming
Khalil, E., Le Bodic, P., Song, L., Nemhauser, G., & Dilkina, B. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10080 · Cited by 298
Ant Colony Optimization for Mixed-Variable Optimization Problems
Liao, T., Socha, K., Montes de Oca, M. A., Stutzle, T., & Dorigo, M. (2014) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2013.2281531 · Cited by 270
Algorithm Selection for Combinatorial Search Problems: A Survey
Kotthoff, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-50137-6_7 · Cited by 270
Performance-influence models for highly configurable systems
Siegmund, N., Grebhahn, A., Apel, S., & Kästner, C. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786845 · Cited by 267
Benchmarking in Optimization: Best Practice and Open Issues
Bartz-Beielstein, T., Doerr, C., Berg, D. V. D., Bossek, J., Chandrasekaran, S., Eftimov, T., Fischbach, A., Kerschke, P., La Cava, W., Lopez-Ibanez, M., et al. (2020) · arXiv · doi:10.48550/arxiv.2007.03488 · Cited by 214
SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
Lindauer, M., Eggensperger, K., Feurer, M., Biedenkapp, A., Deng, D., Benjamins, C., Ruhopf, T., Sass, R., & Hutter, F. (2021) · arXiv · doi:10.48550/arxiv.2109.09831 · Cited by 195
Self-citationVillanueva Zacarias, A. G., Reimann, P., & Mitschang, B. (2018) · Procedia CIRP · doi:10.1016/j.procir.2018.03.215 · Cited by 26
Metadata
- Authors
- Hutter, F., Hoos, H. H., Leyton-Brown, K., & Stuetzle, T.
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2009-10-30
- Volume
- 36
- Pages
- 267-306
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W2950680102
- Cited by
- 882
- References
- 32
- FWCI
- 22.77
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.