PreprintCited by 6,691Advanced Bandit Algorithms ResearchMachine Learning and Data ClassificationMachine Learning and Algorithms

The DOI matches its DataCite record; 23 of the 23 references it cites were found in a registry or an index.

Practical Bayesian Optimization of Machine Learning Algorithms

Open access

Snoek, Jasper, Larochelle, Hugo, Adams, Ryan P.·2012·arXiv

DOI 10.48550/arxiv.1206.2944arXiv 1206.2944

Open this paper in CoChat — free account Open PDF

Abstract

Machine learning algorithms frequently require careful tuning of model hyperparameters, regularization terms, and optimization parameters. Unfortunately, this tuning is often a "black art" that requires expert experience, unwritten rules of thumb, or sometimes brute-force search. Much more appealing is the idea of developing automatic approaches which can optimize the performance of a given learning algorithm to the task at hand. In this work, we consider the automatic tuning problem within the framework of Bayesian optimization, in which a learning algorithm's generalization performance is modeled as a sample from a Gaussian process (GP). The tractable posterior distribution induced by the GP leads to efficient use of the information gathered by previous experiments, enabling optimal choices about what parameters to try next. Here we show how the effects of the Gaussian process prior and the associated inference procedure can have a large impact on the success or failure of Bayesian optimization. We show that thoughtful choices can lead to results that exceed expert-level performance in tuning machine learning algorithms. We also describe new algorithms that take into account the variable cost (duration) of learning experiments and that can leverage the presence of multiple cores for parallel experimentation. We show that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization on a diverse set of contemporary algorithms including latent Dirichlet allocation, structured SVMs and convolutional neural networks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1206.2944

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Untitled work
(2012) · Cited by 8,009
Bayesian calibration of computer models
Kennedy, M. C., & O'Hagan, A. (2001) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00294 · Cited by 4,222
Untitled work
(2011) · Cited by 3,939
Multi-column deep neural networks for image classification
Ciresan, D. C., Meier, U., & Schmidhuber, J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248110 · Cited by 3,880
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement LearningPreprint
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,568
Sequential Model-Based Optimization for General Algorithm Configuration
Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25566-3_40 · Cited by 2,469
A Taxonomy of Global Optimization Methods Based on Response Surfaces
Jones, D. R. (2001) · Journal of Global Optimization · doi:10.1023/a:1012771025575 · Cited by 2,160
Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. (2012) · IEEE Transactions on Information Theory · doi:10.48550/arxiv.0912.3995 · Cited by 2,100

Show all 23 references

Research works printed but not linked: 2

Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In ICML , 2010.
Kevin Miller, M. Pawan Kumar, Benjamin Packer, Danny Goodman, and Daphne Koller. Max-margin minentropy models. In AISTATS , 2012.

Sorted by citations

Items 501–600 of 1,026

Model inversion via multi-fidelity Bayesian optimization: a new paradigm for parameter estimation in haemodynamics, and beyond
Perdikaris, P. G., & Karniadakis, G. E. (2016) · Journal of The Royal Society Interface · doi:10.1098/rsif.2015.1107 · Cited by 107
Deep Reinforcement Learning Supervised Autonomous Exploration in Office Environments
Zhu, D., Li, T., Ho, D., Wang, C., & Meng, M. Q. ‐. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8463213 · Cited by 109
Data-Efficient Autotuning With Bayesian Optimization: An Industrial Control Study
Neumann-Brosig, M., Marco, A., Schwarzmann, D., & Trimpe, S. (2019) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2018.2886159 · Cited by 107
Fast hyperparameter tuning using Bayesian optimization with directional derivatives
Joy, T. T., Rana, S., Gupta, S., & Venkatesh, S. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106247 · Cited by 107
Microscaler: Automatic Scaling for Microservices with an Online Learning Approach
Yu, G., Chen, P., & Zheng, Z. (2019) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws.2019.00023 · Cited by 107
Testing Scenario Library Generation for Connected and Automated Vehicles: An Adaptive Framework
Feng, S., Feng, Y., Sun, H., Zhang, Y., & Liu, H. X. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3023668 · Cited by 107
A Generative Neural Network for Maximizing Fitness and Diversity of Synthetic DNA and Protein Sequences
Linder, J., Bogard, N., Rosenberg, A. B., & Seelig, G. (2020) · Cell Systems · doi:10.1016/j.cels.2020.05.007 · Cited by 107
Explainable long-term building energy consumption prediction using QLattice
Wenninger, S., Kaymakci, C., & Wiethe, C. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.118300 · Cited by 111
Stochastic deep collocation method based on neural architecture search and transfer learning for heterogeneous porous media
Guo, H., Zhuang, X., Chen, P., Alajlan, N., & Rabczuk, T. (2022) · Engineering With Computers · doi:10.1007/s00366-021-01586-2 · Cited by 112
Efficient Methods for Natural Language Processing: A Survey
Treviso, M., Lee, J.-U., Ji, T., van Aken, B., Cao, Q., Ciosici, M. R., Hassid, M., Heafield, K., Hooker, S., Raffel, C., et al. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00577 · Cited by 156
Optimizing Millions of Hyperparameters by Implicit DifferentiationPreprint
Lorraine, J., Vicol, P. A., & Duvenaud, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.02590 · Cited by 126
Blind Hyperspectral Unmixing Using Autoencoders: A Critical Comparison
Palsson, B., Sveinsson, J. R., & Ulfarsson, M. O. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3140154 · Cited by 107
Bayesian Optimization for Broadband High-Efficiency Power Amplifier Designs
Chen, P., Merrick, B. M., & Brazil, T. J. (2015) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2015.2495360 · Cited by 106
Implementing AutoML in Educational Data Mining for Prediction Tasks
Tsiakmaki, M., Kostopoulos, G., Kotsiantis, S. B., & Ragos, O. (2019) · Applied Sciences · doi:10.3390/app10010090 · Cited by 108
kGCN: a graph-based deep learning framework for chemical structures
Kojima, R., Ishida, S., Ohta, M., Iwata, H., Honma, T., & Okuno, Y. (2020) · Journal of Cheminformatics · doi:10.1186/s13321-020-00435-6 · Cited by 108
Deep learning of immune cell differentiation
Maslova, A., Ramírez, R. N., Ma, K., Schmutz, H., Wang, C., Fox, C., Ng, B., Benoist, C. O., Mostafavi, S., & Immunological Genome Project (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2011795117 · Cited by 103
Quantum transport evidence of Weyl fermions in an epitaxial ferromagnetic oxide
Takiguchi, K., Wakabayashi, Y. K., Irie, H., Krockenberger, Y., Otsuka, T., Sawada, H., Nikolaev, S. A., Das, H., Tanaka, M., Taniyasu, Y., et al. (2020) · RePEc: Research Papers in Economics · Cited by 109
A machine learning approach for predicting heat transfer characteristics in micro-pin fin heat sinks
Kim, K., Lee, H., Kang, M., Lee, G., Jung, K., Kharangate, C. R., Asheghi, M., Goodson, K. E., & Lee, H. (2022) · International Journal of Heat and Mass Transfer · doi:10.1016/j.ijheatmasstransfer.2022.123087 · Cited by 111
Lithium-ion battery state of health estimation using a hybrid model based on a convolutional neural network and bidirectional gated recurrent unit
Mazzi, Y., Sassi, H. B., & Errahimi, F. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107199 · Cited by 108
GEFCom2012 hierarchical load forecasting: Gradient boosting machines and Gaussian processes
Lloyd, J. R. (2013) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2013.07.002 · Cited by 105
A machine learning based personalized system for driving state recognition
Yi, D., Su, J., Liu, C., Quddus, M., & Chen, W. (2019) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2019.05.042 · Cited by 105
Transfer learning with convolutional neural networks for cancer survival prediction using gene-expression data
López-García, G., Jerez, J. M., Franco, L., & Veredas, F. J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0230536 · Cited by 110
An Automated Machine Learning (AutoML) Method of Risk Prediction for Decision-Making of Autonomous Vehicles
Shi, X., Wong, Y. D., Chai, C., & Li, M. Z. F. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3002419 · Cited by 105
Bayesian optimization algorithm based support vector regression analysis for estimation of shear capacity of FRP reinforced concrete members
Alam, M. S., Sultana, N. N., & Hossain, S. M. Z. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107281 · Cited by 105
k-Nearest Neighbor Learning with Graph Neural Networks
Kang, S. (2021) · Mathematics · doi:10.3390/math9080830 · Cited by 105
Perception model of surrounding rock geological conditions based on TBM operational big data and combined unsupervised-supervised learning
Yin, X. S., Liu, Q., Huang, X., & Pan, Y. (2021) · Tunnelling and Underground Space Technology · doi:10.1016/j.tust.2021.104285 · Cited by 105
Deep Feature Learning for EEG RecordingsPreprint
Stober, S., Sternin, A., Owen, A. M., & Grahn, J. A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04306 · Cited by 105
Evolutionary many-tasking based on biocoenosis through symbiosis: A framework and benchmark problems
Liaw, R.-T., & Ting, C.-K. (2017) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2017.7969579 · Cited by 104
Complex-Valued Neural Networks With Nonparametric Activation Functions
Scardapane, S., Vaerenbergh, S. V., Hussain, A., & Uncini, A. (2018) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2018.2872600 · Cited by 116
Hyperparameter Tuning of Machine Learning Algorithms Using Response Surface Methodology: A Case Study of ANN, SVM, and DBN
Pannakkong, W., Thiwa-anont, K., Singthong, K., Parthanadee, P., & Buddhakulsomsiri, J. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/8513719 · Cited by 119
Wheat leaf rust detection at canopy scale under different LAI levels using machine learning techniques
Azadbakht, M., Ashourloo, D., Aghighi, H., Radiom, S., & Alimohammadi, A. (2018) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.11.016 · Cited by 103
An Uncertainty-Aware Approach to Optimal Configuration of Stream Processing Systems
Jamshidi, P., & Casale, G. (2016) · Proceedings - International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots.2016.17 · Cited by 104
System identification of ship dynamic model based on Gaussian process regression with input noise
Xue, Y., Liu, Y., Ji, C. W., Xue, G., & Huang, S. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107862 · Cited by 103
I Choose You: Automated Hyperparameter Tuning for Deep Learning-Based Side-Channel Analysis
Wu, L., Perin, G., & Picek, S. (2022) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2022.3218372 · Cited by 103
Convolutional Neural Networks-Based Lung Nodule Classification: A Surrogate-Assisted Evolutionary Algorithm for Hyperparameter Optimization
Zhang, M., Li, H., Pan, S., Lyu, J., Ling, S., & Su, S. W. (2021) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2021.3060833 · Cited by 102
A Fast Kriging-Assisted Evolutionary Algorithm Based on Incremental Learning
Zhan, D., & Xing, H. (2021) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2021.3067015 · Cited by 104
A Lightweight CNN Based on Transfer Learning for COVID-19 Diagnosis
Zhang, X., Zhou, J., Sun, W. J., & Jha, S. K. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.024589 · Cited by 104
Exploring exploratory programming
Beth Kery, M., & Myers, B. A. (2017) · 2017 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2017.8103446 · Cited by 123
Deep learning for real-time robust facial expression recognition on a smartphone
Song, I., Kim, H., & Jeon, P. B. (2014) · International Conference on Consumer Electronics · doi:10.1109/icce.2014.6776135 · Cited by 102
Hilbert maps: scalable continuous occupancy mapping with stochastic gradient descent
Ramos, F. T., & Ott, L. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.002 · Cited by 102
Long‐Time Predictive Modeling of Nonlinear Dynamical Systems Using Neural Networks
Pan, S., & Duraisamy, K. (2018) · Complexity · doi:10.1155/2018/4801012 · Cited by 102
OBOE
Yang, C., Akimoto, Y., Kim, D. W., & Udell, M. R. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330909 · Cited by 102
Active Learning for Deep Gaussian Process Surrogates
Sauer, A., Gramacy, R. B., & Higdon, D. (2021) · Technometrics · doi:10.1080/00401706.2021.2008505 · Cited by 102
Bayesian Hyperparameter Optimization for Deep Neural Network-Based Network Intrusion Detection
Masum, M., Shahriar, H., Haddad, H. M., Faruk, M. J. H., Valero, M., Khan, M. A., Rahman, M. A., Adnan, M. I., Cuzzocrea, A., & Wu, F. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671576 · Cited by 102
Human-in-the-Loop Optimization of Exoskeleton Assistance Via Online Simulation of Metabolic Cost
Gordon, D., McGreavy, C., Christou, A., & Vijayakumar, S. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2021.3133137 · Cited by 102
BananaSqueezeNet: A very fast, lightweight convolutional neural network for the diagnosis of three prominent banana leaf diseases
Bhuiyan, M. A. B., Abdullah, H. M., Arman, S. E., Rahman, S. S., & Mahmud, K. A. (2023) · Smart Agricultural Technology · doi:10.1016/j.atech.2023.100214 · Cited by 113
Artificial-intelligence-led revolution of construction materials: From molecules to Industry 4.0
Wang, X. Q., Chen, P., Chow, C. L., & Lau, D. (2023) · Matter · doi:10.1016/j.matt.2023.04.016 · Cited by 111
A new CNN-LSTM architecture for activity recognition employing wearable motion sensor data: Enabling diverse feature extraction
Koşar, E., & Barshan, B. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106529 · Cited by 108
Predictive Entropy Search for Bayesian Optimization with Unknown ConstraintsPreprint
Hernández-Lobato, J. M., Gelbart, M. A., Hoffman, M. D., Adams, R. P., & Zoubin Ghahramani, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.05312 · Cited by 102
Self-citation
A review on the self and dual interactions between machine learning and optimisation
Song, H., Triguero, I., & Özcan, E. (2019) · Progress in Artificial Intelligence · doi:10.1007/s13748-019-00185-z · Cited by 101
Bayesian optimization of pump operations in water distribution systems
Candelieri, A., Perego, R., & Archetti, F. (2018) · Journal of Global Optimization · doi:10.1007/s10898-018-0641-2 · Cited by 101
Tuning hyperparameters of a SVM-based water demand forecasting system through parallel global optimization
Candelieri, A., Giordani, I., Archetti, F., Баркалов, К. А., Meyerov, I. B., Polovinkin, A., Sysoyev, A., & Zolotykh, N. Y. (2018) · Computers & Operations Research · doi:10.1016/j.cor.2018.01.013 · Cited by 101
Early detection of Parkinson’s disease through patient questionnaire and predictive modelling
Prashanth, R., & Roy, S. D. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.09.008 · Cited by 105
Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series
Pelletier, C., Webb, G. I., & Petitjean, F. (2019) · Remote Sensing · doi:10.3390/rs11050523 · Cited by 122
Intact metabolite spectrum mining by deep learning in proton magnetic resonance spectroscopy of the brain
Lee, H. H., & Kim, H. (2019) · Magnetic Resonance in Medicine · doi:10.1002/mrm.27727 · Cited by 110
Impact of Hyperparameter Tuning on Machine Learning Models in Stock Price Forecasting
Hoque, K. E., & Aljamaan, H. (2021) · IEEE Access · doi:10.1109/access.2021.3134138 · Cited by 123
Optimized Random Forest Model for Remaining Useful Life Prediction of Experimental Bearings
Alfarizi, M. G., Tajiani, B., Vatn, J., & Yin, S. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3206339 · Cited by 108
Fault detection and classification in oil wells and production/service lines using random forest
Marins, M. A., Barros, B. D., Santos, I., Barrionuevo, D. C., Vargas, R. E. V., Prego, T. D. M., Lima, A. A. D., de Campos, M. L. R., da Silva, E. A. B., & Netto, S. L. (2020) · Journal of Petroleum Science and Engineering · doi:10.1016/j.petrol.2020.107879 · Cited by 100
Defect Prediction With Semantics and Context Features of Codes Based on Graph Representation Learning
Xu, J., Wang, F., & Ai, J. (2020) · IEEE Transactions on Reliability · doi:10.1109/tr.2020.3040191 · Cited by 100
An inquiry into machine learning-based automatic configuration tuning services on real-world database management systems
Aken, D. V., Yang, D., Brillard, S., Fiorino, A., Zhang, B., Bilien, C., & Pavlo, A. (2021) · Proceedings of the VLDB Endowment · doi:10.14778/3450980.3450992 · Cited by 100
Applicability of machine learning to a crack model in concrete bridges
Okazaki, Y., Okazaki, S., Asamoto, S., & Chun, P. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12532 · Cited by 101
Forecasting, hindcasting and feature selection of ocean waves via recurrent and sequence-to-sequence networks
Pirhooshyaran, M., & Snyder, L. (2020) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.107424 · Cited by 107
A Novel Machine Learning Approach to Estimate Grapevine Leaf Nitrogen Concentration Using Aerial Multispectral Imagery
Moghimi, A., Pourreza, A., Zuniga-Ramirez, G., Williams, L. E., & Fidelibus, M. W. (2020) · Remote Sensing · doi:10.3390/rs12213515 · Cited by 99
Robustness Gym: Unifying the NLP Evaluation Landscape
Goel, K., Rajani, N. F., Vig, J., Taschdjian, Z., Bansal, M., & Ré, C. (2021) · doi:10.18653/v1/2021.naacl-demos.6 · Cited by 145
Escaping local minima with local derivative-free methods: a numerical investigation
Cartis, C., Roberts, L., & Sheridan-Methven, O. (2021) · Optimization · doi:10.1080/02331934.2021.1883015 · Cited by 100
Intelligent rockburst prediction model with sample category balance using feedforward neural network and Bayesian optimization
Li, D., Liu, Z., Xiao, P., Zhou, J., & Armaghani, D. J. (2022) · Underground Space · doi:10.1016/j.undsp.2021.12.009 · Cited by 99
IoT data analytics in dynamic environments: From an automated machine learning perspective
Yang, L., & Shami, A. A. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105366 · Cited by 100
Improving Alzheimer’s Disease and Brain Tumor Detection Using Deep Learning with Particle Swarm Optimization
Ibrahim, R. F., Ghnemat, R., & Al-Haija, Q. S. A. (2023) · AI · doi:10.3390/ai4030030 · Cited by 99
3D-Printing and Machine Learning Control of Soft Ionic Polymer-Metal Composite Actuators
Carrico, J. D., Hermans, T., Kim, K. J., & Leang, K. K. (2019) · Scientific Reports · doi:10.1038/s41598-019-53570-y · Cited by 98
Assessment of stacked unidirectional and bidirectional long short-term memory networks for electricity load forecasting
Atef, S., & Eltawil, A. (2020) · Electric Power Systems Research · doi:10.1016/j.epsr.2020.106489 · Cited by 108
Machine Learning and Natural Language Processing for Prediction of Human Factors in Aviation Incident Reports
Madeira, T., Melício, R., Valério, D., & Santos, L. F. F. M. (2021) · Aerospace · doi:10.3390/aerospace8020047 · Cited by 98
Bayesian optimization package: PHYSBO
Motoyama, Y., Tamura, R., Yoshimi, K., Terayama, K., Ueno, T., & Tsuda, K. (2022) · Computer Physics Communications · doi:10.1016/j.cpc.2022.108405 · Cited by 98
Evolutionary Reinforcement Learning: A Survey
Bai, H., Cheng, R., & Jin, Y. (2023) · Intelligent Computing · doi:10.34133/icomputing.0025 · Cited by 112
Exact Gaussian Processes on a Million Data PointsPreprint
Wang, K. A., Pleiss, G., Gardner, J. R., Tyree, S., Weinberger, K. Q., & Wilson, A. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08114 · Cited by 98
Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with DragonflyPreprint
Kandasamy, K., Vysyaraju, K. R., Neiswanger, W., Paria, B., Collins, C. R., Schneider, J., Póczos, B., & Xing, E. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.06694 · Cited by 98
SVM kernel based on particle swarm optimized vector and Bayesian optimized SVM in atmospheric particulate matter forecasting
Kouziokas, G. N. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106410 · Cited by 98
DNN-Opt: An RL Inspired Optimization for Analog Circuit Sizing using Deep Neural Networks
Budak, A. F., Bhansali, P., Liu, B., Sun, N., Pan, D. Z., & Kashyap, C. V. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586139 · Cited by 97
Hybrid Feature-Based Disease Detection in Plant Leaf Using Convolutional Neural Network, Bayesian Optimized SVM, and Random Forest Classifier
Singh, A. K., Sreenivasu, S. V. N., Mahalaxmi, U. S. B. K., Sharma, H. D., Patil, D. D., & Asenso, E. (2022) · Journal of Food Quality · doi:10.1155/2022/2845320 · Cited by 108
Retracted
CustOmics: A versatile deep-learning based strategy for multi-omics integration
Benkirane, H., Pradat, Y., Michiels, S., & Cournède, P.-H. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010921 · Cited by 109
Scale-Invariant Convolutional Neural NetworksPreprint
Xu, Y., Xiao, T., Zhang, J., Yang, K., & Zhang, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.6369 · Cited by 115
Active tactile object exploration with Gaussian processes
Yi, Z., Calandra, R., Veiga, F. F., van Hoof, H., Hermans, T., Zhang, Y., & Peters, J. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759723 · Cited by 96
A deep learning based multitask model for network-wide traffic speed prediction
Zhang, K., Zheng, L., Liu, Z., & Jia, N. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.10.097 · Cited by 98
COSE: Configuring Serverless Functions using Statistical Learning
Akhtar, N., Raza, A., Ishakian, V., & Matta, I. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155363 · Cited by 96
Solid particle erosion rate predictions through LSBoost
Zhang, Y., & Xu, X. (2021) · Powder Technology · doi:10.1016/j.powtec.2021.04.072 · Cited by 96
Evolutionary design of neural network architectures: a review of three decades of research
Ünal, H. T., & Başçi̇ftçi̇, F. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10049-5 · Cited by 108
Comparison of models for predicting winter individual thermal comfort based on machine learning algorithms
Yang, B., Li, X., Liu, Y., Chen, L., Guo, R., Wang, F., & Yan, K. (2022) · Building and Environment · doi:10.1016/j.buildenv.2022.108970 · Cited by 96
Deep Spatio-Temporal Graph Network with Self-Optimization for Air Quality Prediction
Jin, X., Wang, Z.-Y., Kong, J., Bai, Y., Su, T., Ma, H., & Chakrabarti, P. (2023) · Entropy · doi:10.3390/e25020247 · Cited by 96
Meta-Learning Update Rules for Unsupervised Representation LearningPreprint
Metz, L., Maheswaranathan, N., Cheung, B., & Sohl‐Dickstein, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.00222 · Cited by 129
Autotune
Koch, P. N., Golovidov, O., Gardner, S., Wujek, B. A., Griffin, J., & Xu, Y. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3219837 · Cited by 95
ShEMO: a large-scale validated database for Persian speech emotion detection
Nezami, O. M., Lou, P. J., & Karami, M. (2018) · Language Resources and Evaluation · doi:10.1007/s10579-018-9427-x · Cited by 97
On the optimization of the support vector machine regression hyperparameters setting for gas sensors array applications
Laref, R., Losson, E., Sava, A., & Siadat, M. (2018) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2018.11.011 · Cited by 95
Dynamic reliability analysis using the extended support vector regression (X-SVR)
Feng, J., Liu, L., Wu, D., Li, G., Beer, M., & Gao, W. (2019) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2019.02.027 · Cited by 95
AutoML for Multi-Label Classification: Overview and Empirical Evaluation
Wever, M., Tornede, A., Mohr, F., & Hüllermeier, E. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3051276 · Cited by 95
Exploring Serverless Computing for Neural Network Training
Feng, L., Kudva, P. N., Silva, D. D., & Hu, J. (2018) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud.2018.00049 · Cited by 94
Ekya: Continuous Learning of Video Analytics Models on Edge Compute ServersPreprint
Bhardwaj, R., Xia, Z., Ananthanarayanan, G., Jiang, J., Karianakis, N., Shu, Y., Hsieh, K., Bahl, V., & Stoica, I. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10557 · Cited by 95
Optimizing DTW-based audio-to-MIDI alignment and matching
Raffel, C., & Ellis, D. P. W. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7471641 · Cited by 98
World Seaborne Trade in Real Time: A Proof of Concept for Building AIS-based Nowcasts from Scratch
Cerdeiro, D. A., Komáromi, A., Liu, Y., & Saeed, M. (2020) · IMF Working Paper · doi:10.5089/9781513544106.001 · Cited by 99
Machine learning bioactive compound solubilities in supercritical carbon dioxide
Zhang, Y., & Xu, X. (2021) · Chemical Physics · doi:10.1016/j.chemphys.2021.111299 · Cited by 93
Benchmarking deep learning-based models on nanophotonic inverse design problems
Ma, T., Tobah, M., Wang, H., & Guo, L. J. (2022) · Opto-Electronic Science · doi:10.29026/oes.2022.210012 · Cited by 102
Performance evaluation of deep feature learning for RGB-D image/video classification
Shao, L., Cai, Z., Liu, L., & Lü, K. (2017) · Information Sciences · doi:10.1016/j.ins.2017.01.013 · Cited by 92

Authors: Snoek, J., Larochelle, H., & Adams, R. P.
Affiliations: University of Toronto (CA)
Université de Sherbrooke (CA)
Harvard University (US)
ORCID: Adams, Ryan P.
Venue: arXiv
Publisher: arXiv
Published: 2012
ISSN: 2331-8422 (ISSN-L)
DOI: 10.48550/arxiv.1206.2944
arXiv: 1206.2944
OpenAlex: W2131241448
Cited by: 6,691
References: 23
Citations by year: 2026: 62 · 2025: 630 · 2024: 779 · 2023: 712 · 2022: 720 · 2021: 997 · 2020: 864 · 2019: 680 · 2018: 465 · 2017: 289 · 2016: 212 · 2015: 144 · 2014: 94 · 2013: 34 · 2012: 9
Type: posted content
Language: en
Open access: green
arxiv.org
Licence: arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations: arXiv (Cornell University) (submittedVersion)
Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
arXiv (Cornell University)
Funders: Defense Advanced Research Projects Agency
Natural Sciences and Engineering Research Council of Canada
Topics: Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Keywords: Bayesian optimization, hyperparameter optimization, Gaussian process priors, latent Dirichlet allocation, structured support vector machines, convolutional neural networks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Practical Bayesian Optimization of Machine Learning Algorithms

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks