The DOI matches its DataCite record; 34 of the 38 references it cites were found in a registry or an index.
SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
Open accessAbstract
Algorithm parameters, in particular hyperparameters of machine learning algorithms, can substantially impact their performance. To support users in determining well-performing hyperparameter configurations for their algorithms, datasets and applications at hand, SMAC3 offers a robust and flexible framework for Bayesian Optimization, which can improve performance within a few evaluations. It offers several facades and pre-sets for typical use cases, such as optimizing hyperparameters, solving low dimensional continuous (artificial) global optimization problems and configuring algorithms to perform well across multiple problem instances. The SMAC3 package is available under a permissive BSD-license at https://github.com/automl/SMAC3.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent studies have shown that other optimisation algorithms are suitable for the AutoMoDe family (e.g., simulated annealing 65 and sequential model-based algorithm configuration 66,67 ).”
From Automatic design of stigmergy-based behaviours for robot swarms (2024) · cites this paper as [67]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070
(2012) · Cited by 8,009
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Taking the Human Out of the Loop: A Review of Bayesian Optimization
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & de Freitas, N. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2494218 · Cited by 6,195
THOMPSON, W. R. (1933) · Biometrika · doi:10.1093/biomet/25.3-4.285 · Cited by 2,712
Sequential Model-Based Optimization for General Algorithm Configuration
Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25566-3_40 · Cited by 2,469
Self-citationInformation-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. (2012) · IEEE Transactions on Information Theory · doi:10.48550/arxiv.0912.3995 · Cited by 2,100
The irace package: Iterated racing for automatic algorithm configuration
López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., & Stützle, T. (2016) · Operations Research Perspectives · doi:10.1016/j.orp.2016.09.002 · Cited by 1,761
DEAP: evolutionary algorithms made easy
Fortin, F.-A., Rainville, F.-M. D., Gardner, M.-A., Parizeau, M., & Gagné, C. (2012) · Cited by 1,690
Research works printed but not linked: 10
- E. Bakshy, L. Dworkin, B. Karrer, K. Kashin, B. Letham, A. Murthy, and S. Singh. Ae: A domain-agnostic platform for adaptive experimentation. In Proceedings of the international conference on Neural Information Processing Systems , pages 1-8, 2018.
- K. Eggensperger, P. M¨ uller, N. Malik, M. Feurer, R. Sass, A. Klein, N. Awad, Marius Lindauer, and Frank Hutter. HPOBench: A collection of reproducible multi-fidelity benchmark problems for hpo. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks , 2021.
- S. Falkner, A. Klein, and F. Hutter. BOHB: Robust and efficient hyperparameter optimization at scale. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning (ICML'18) , volume 80, pages 1437-1446. Proceedings of Machine Learning Research, 2018.
- M. Feurer and F. Hutter. Hyperparameter optimization. In F. Hutter, L. Kotthoff, and J. Vanschoren, editors, Automated Machine Learning: Methods, Systems, Challenges , volume 5 of The Springer Series on Challenges in Machine Learning , chapter 1, pages 3-38. Springer, 2019. Available for free at http://automl.org/book.
- K. Kandasamy, K. Vysyaraju, W. Neiswanger, B. Paria, C. Collins, J. Schneider, B. Poczos, and E. Xing. Tuning hyperparameters without grad students: Scalable and robust Bayesian optimisation with Dragonfly. Journal of Machine Learning Research , 21(81): 1-27, 2020.
- M. Lindauer, K. Eggensperger, M. Feurer, A. Biedenkapp, J. Marben, P. M¨ uller, and F. Hutter. BOAH: A Tool Suite for Multi-Fidelity Bayesian Optimization & Analysis of Hyperparameters. arXiv:1908.06756 [cs.LG] , 2019a. · arXiv
- J. Mockus, V. Tiesis, and A. Zilinskas. The application of Bayesian methods for seeking the extremum. Towards Global Optimization , 2(117-129), 1978.
- N. Srinivas, A. Krause, S. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In J. F¨ urnkranz and T. Joachims, editors, Proceedings of the 27th International Conference on Machine Learning (ICML'10) , pages 1015-1022. Omnipress, 2010.
- R. Turner, D. Eriksson, M. McCourt, J. Kiili, E. Laaksonen, Z. Xu, and I. Guyon. Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020. In H. Escalante and K. Hofmann, editors, NeurIPS 2020 Competition and Demonstration Track , volume 133 of Proceedings of Machine Learning Research , pages 3-26. PMLR, 2020.
- A. Zela, T. Elsken, T. Saikia, Y. Marrakchi, T. Brox, and F. Hutter. Understanding and robustifying differentiable architecture search. In Proceedings of the International Conference on Learning Representations (ICLR'20) , 2020. Published online: iclr.cc .
Cited by
Sorted by citations
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges
Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann, T., Becker, M., Boulesteix, A., et al. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1484 · Cited by 1,262
Self-citationSelf-Driving Laboratories for Chemistry and Materials Science
Tom, G., Schmid, S. P., Baird, S. G., Cao, Y., Darvish, K., Hao, H., Lo, S., Pablo-García, S., Rajaonson, E. M., Skreta, M., et al. (2024) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00055 · Cited by 707
A systematic review of hyperparameter optimization techniques in Convolutional Neural Networks
Raiaan, M. A. K., Sakib, S., Fahad, N. M., Mamun, A. A., Rahman, M. A., Shatabda, S., & Mukta, M. S. H. (2024) · Decision Analytics Journal · doi:10.1016/j.dajour.2024.100470 · Cited by 341
Amin, M. M., Cambria, E., & Schuller, B. W. (2023) · IEEE Intelligent Systems · doi:10.1109/mis.2023.3254179 · Cited by 162
An improved hyperparameter optimization framework for AutoML systems using evolutionary algorithms
Vincent, A. M., & Jidesh, P. (2023) · Scientific Reports · doi:10.1038/s41598-023-32027-3 · Cited by 188
Metadata
- Authors
- Lindauer, M., Eggensperger, K., Feurer, M., Biedenkapp, A., Deng, D., Benjamins, C., Ruhopf, T., Sass, R., & Hutter, F.
- Affiliations
- Leibniz University Hannover (DE)
- University of Freiburg (DE)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2109.09831
- OpenAlex
- W3199199693
- Cited by
- 195
- References
- 38
- Citations by year
- 2026: 1 · 2025: 46 · 2024: 60 · 2023: 64 · 2022: 22 · 2021: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- European Commission
- Deutsche Forschungsgemeinschaft (INST 37/935-1, 37/935-1, INST~37/935-1 FUGG, 37/935-1 FUGG)
- Eberhard Karls Universität Tübingen
- Generalitat de Catalunya (10.13039/501100011033, 13039/501100011033, 501100011033, AEI/10.13039/501100011033, MCIN /AEI/10.13039/501100011033, PID2019)
- Agència de Gestió d'Ajuts Universitaris i de Recerca (AEI/10.13039/501100011033, MCIN/AEI/10.13039/501100011033/, 10.13039/501100011033, 13039/501100011033)
- Agencia Estatal de Investigación (AEI//10.13039/501100011033/, 501100011033, 10.13039/501100011033, MCIN/AEI/10.13039/501 100 011 033, 13039, .13039/501100011033, MCIN / AEI/10, 10.13039, PID2019-109137GB-C21, AEI/10, PID2019)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Artificial Intelligence › Metaheuristic Optimization Algorithms Research
- Keywords
- Bayesian optimization, hyperparameter optimization, automated machine learning, global optimization, multi-fidelity optimization, algorithm configuration, hyperparameter search
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
