The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Dealing with categorical and integer-valued variables in Bayesian Optimization with Gaussian processes
Open accessAbstract
Some optimization problems are characterized by an objective that is very expensive, that lacks an analytical expression, and whose evaluations can be contaminated by noise. Bayesian Optimization (BO) methods can be used to solve these problems efficiently. BO relies on a probabilistic model of the objective, which is typically a Gaussian process (GP). This model is used to compute an acquisition function that estimates the expected utility (for solving the optimization problem) of evaluating the objective at each potential new point. A problem with GPs is, however, that they assume real-valued input variables and cannot easily deal with categorical or integer-valued values. Common methods to account for these variables, before evaluating the objective, include assuming they are real and then using a one-hot encoding, for categorical variables, or rounding to the closest integer, for integer-valued variables. We show that this leads to suboptimal results and introduce a novel approach to tackle categorical or integer-valued input variables within the context of BO with GPs. Several synthetic and real-world experiments support our hypotheses and show that our approach outperforms the results of standard BO using GPs on problems with categorical or integer-valued input variables.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Kernel Methods for Pattern Analysis
Shawe-Taylor, J., & Cristianini, N. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9780511809682 · Cited by 6,587
(2019) · Handbook of Machine Learning · doi:10.1142/9789811205675_0004 · Cited by 6,270
Taking the Human Out of the Loop: A Review of Bayesian Optimization
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & de Freitas, N. (2016) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2494218 · Cited by 6,049
(2011) · Cited by 3,939
Encyclopedia of Machine Learning
(2010) · Springer eBooks · doi:10.1007/978-0-387-30164-8 · Cited by 3,465
Cited by
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges
Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann, T., Becker, M., Boulesteix, A., et al. (2023) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1484 · Cited by 930
Bayesian Optimization for Adaptive Experimental Design: A Review
Greenhill, S., Rana, S., Gupta, S., Vellanki, P., & Venkatesh, S. (2020) · IEEE Access · doi:10.1109/access.2020.2966228 · Cited by 520
Sun, D., Wen, H., Wang, D., & Xu, J. (2020) · Geomorphology · doi:10.1016/j.geomorph.2020.107201 · Cited by 480
Recent Advances in Bayesian Optimization
Wang, X., Jin, Y., Schmitt, S., & Olhofer, M. (2023) · ACM Computing Surveys · doi:10.1145/3582078 · Cited by 477
Benchmark and Survey of Automated Machine Learning Frameworks
Zöller, M.-A., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11854 · Cited by 448
Hyper-Parameter Optimization: A Review of Algorithms and Applications
Yu, T., & Zhu, H. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.05689 · Cited by 437
Metadata
- Authors
- Garrido-Merchán, E. C., & Hernández-Lobato, D.
- Venue
- Neurocomputing
- Publisher
- Elsevier BV
- Published
- 2020-03
- Volume
- 380
- Pages
- 20-35
- ISSN
- 0925-2312 (ISSN-L)
- OpenAlex
- W2802235618
- Cited by
- 291
- References
- 34
- FWCI
- 19.50
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.