Practical Bayesian Optimization of Machine Learning Algorithms
Open accessAbstract
Machine learning algorithms frequently require careful tuning of model hyperparameters, regularization terms, and optimization parameters. Unfortunately, this tuning is often a "black art" that requires expert experience, unwritten rules of thumb, or sometimes brute-force search. Much more appealing is the idea of developing automatic approaches which can optimize the performance of a given learning algorithm to the task at hand. In this work, we consider the automatic tuning problem within the framework of Bayesian optimization, in which a learning algorithm's generalization performance is modeled as a sample from a Gaussian process (GP). The tractable posterior distribution induced by the GP leads to efficient use of the information gathered by previous experiments, enabling optimal choices about what parameters to try next. Here we show how the effects of the Gaussian process prior and the associated inference procedure can have a large impact on the success or failure of Bayesian optimization. We show that thoughtful choices can lead to results that exceed expert-level performance in tuning machine learning algorithms. We also describe new algorithms that take into account the variable cost (duration) of learning experiments and that can leverage the presence of multiple cores for parallel experimentation. We show that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization on a diverse set of contemporary algorithms including latent Dirichlet allocation, structured SVMs and convolutional neural networks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Validation accuracy can also guide hyperparameter and architecture search; both uses reduce the validation set to a single number.
“Validation accuracy can also guide hyperparameter and architecture search (Snoek et al., 2012;Zoph & Le, 2017); both uses reduce the validation set to a single number.”
From VERSE: Verified Self-Evolving Optimizer for Agent Harnesses (2026) · cites this paper as Snoek et al., 2012
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705 · see citing works
(2012) · Cited by 8,009 · see citing works
Bayesian calibration of computer models
Kennedy, M. C., & O'Hagan, A. (2001) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00294 · Cited by 4,222 · see citing works
(2011) · Cited by 3,939 · see citing works
Multi-column deep neural networks for image classification
Ciresan, D. C., Meier, U., & Schmidhuber, J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248110 · Cited by 3,760 · see citing works
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,568 · see citing works
Sequential Model-Based Optimization for General Algorithm Configuration
Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25566-3_40 · Cited by 2,469 · see citing works
A Taxonomy of Global Optimization Methods Based on Response Surfaces
Jones, D. R. (2001) · Journal of Global Optimization · doi:10.1023/a:1012771025575 · Cited by 2,160 · see citing works
Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting
Srinivas, N., Krause, A., Kakade, S. M., & Seeger, M. (2012) · IEEE Transactions on Information Theory · doi:10.48550/arxiv.0912.3995 · Cited by 2,100 · see citing works
Online Learning for Latent Dirichlet Allocation
Hoffman, M. D., Blei, D. M., & Bach, F. (2010) · Cited by 1,359 · see citing works
Self-Paced Learning for Latent Variable Models
Kumar, M., Packer, B., & Koller, D. (2010) · neural information processing systems · Cited by 1,204 · see citing works
A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise
Kushner, H. J. (1964) · Journal of Basic Engineering · doi:10.1115/1.3653121 · Cited by 1,103 · see citing works
Multi-task Gaussian Process Prediction
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2007) · Cited by 916 · see citing works
Learning structural SVMs with latent variables
Yu, C., & Joachims, T. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553523 · Cited by 743 · see citing works
Convergence rates of efficient global optimization algorithmsPreprint
Bull, A. D. (2011) · arXiv (Cornell University) · doi:10.5555/1953048.2078198 · Cited by 454 · see citing works
On Random Weights and Unsupervised Feature Learning
Saxe, A., Koh, P. W., Chen, Z., Bhand, M., Suresh, B., & Ng, A. Y. (2011) · Cited by 353 · see citing works
Semiparametric Latent Factor Models
Teh, Y. W., Seeger, M., & Jordan, M. I. (2005) · Cited by 193 · see citing works
Slice sampling covariance hyperparameters of latent Gaussian modelsPreprint
Murray, I., & Adams, R. P. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1006.0868 · Cited by 163 · see citing works
Self-citationSelecting Receptive Fields in Deep Networks
Coates, A. (2024) · doi:10.57702/4h3zav33 · Cited by 138 · see citing works
Adaptive MCMC with Bayesian Optimization
Mahendran, N., Wang, Z., Hamze, F., & de Freitas, N. (2012) · Cited by 54 · see citing works
Dealing with asynchronicity in parallel Gaussian Process based global optimization
Ginsbourger, D., Januševskis, J., & Riche, R. L. (2011) · HAL · Cited by 50 · see citing works
Miller, K., Pawan, M., Packer, K. B., Goodman, D., & Koller, D. (2012) · HAL · Cited by 26 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In ICML , 2010.
- Kevin Miller, M. Pawan Kumar, Benjamin Packer, Danny Goodman, and Daphne Koller. Max-margin minentropy models. In AISTATS , 2012.
Cited by
Sorted by citations
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · see citing works
Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2018.10.045 · Cited by 20,384 · see citing works
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332 · see citing works
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · see citing works
Larian Studios Dev Team (2024) · doi:10.1145/3292500.3330701 · Cited by 8,618 · see citing works
“Spearmint [2] and GPyOpt use Gaussian Processes, and Hyperopt [1] employs tree-structured Parzen estimator (TPE) [6].”
MethodShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · see citing works
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,631 · see citing works
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · see citing works
“Similarly to (Hern´andez-Lobato & Adams, 2015) we use Bayesian optimisation (BO, (Snoek et al., 2012; Snoek & authors, 2015)) over validation log-likelihood to find optimal τ , and set the prior length-scale to 10 − 2 for most datasets based on the range of the data.”
MethodTaking the Human Out of the Loop: A Review of Bayesian Optimization
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & de Freitas, N. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2494218 · Cited by 6,195 · see citing works
Self-citationA Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3084827 · Cited by 5,273 · see citing works
“Some commonly used algorithms and toolkits include bayesian optimization [111], Nevergrad [112], and ZOOpt [113].”
MethodNeural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,091 · see citing works
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · see citing works
On hyperparameter optimization of machine learning algorithms: Theory and practice
Yang, L., & Shami, A. A. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.061 · Cited by 3,347 · see citing works
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 2,720 · see citing works
Randaugment: Practical automated data augmentation with a reduced search space
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00359 · Cited by 3,223 · see citing works
Generating Sentences from a Continuous Space
Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A., Jozefowicz, R., & Bengio, S. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1002 · Cited by 2,061 · see citing works
Probabilistic machine learning and artificial intelligence
Ghahramani, Z. (2015) · Nature · doi:10.1038/nature14541 · Cited by 1,995 · see citing works
Progressive Neural Architecture Search
Liu, C., Zoph, B., Neumann, M., Shlens, J., Wei, H., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., & Murphy, K. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_2 · Cited by 1,946 · see citing works
Bergstra, J., Yamins, D. L. K., & Cox, D. (2013) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 1,853 · see citing works
Deep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,785 · see citing works
AutoML: A survey of the state-of-the-art
He, X. J., Zhao, K., & Chu, X. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106622 · Cited by 1,778 · see citing works
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules.
Gómez‐Bombarelli, R., Wei, J. N., Duvenaud, D., Hernández-Lobato, J. M., Sánchez-Lengeling, B., Sheberla, D., Aguilera‐Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru‐Guzik, A. (2018) · Cambridge University Engineering Department Publications Database · doi:10.17863/cam.34134 · Cited by 1,813 · see citing works
Self-citationFrom Word Embeddings To Document Distances
Kusner, M. J., Sun, Y., Kolkin, N., & Weinberger, K. Q. (2015) · PolyPublie (École Polytechnique de Montréal) · Cited by 1,695 · see citing works
Using goal-driven deep learning models to understand sensory cortex
Yamins, D. L. K., & DiCarlo, J. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4244 · Cited by 1,621 · see citing works
Reynolds averaged turbulence modelling using deep neural networks with embedded invariance
Ling, J., Kurzawski, A. J., & Templeton, J. A. (2016) · Journal of Fluid Mechanics · doi:10.1017/jfm.2016.615 · Cited by 1,632 · see citing works
Unke, O. T., Chmiela, S., Sauceda, H. E., Gastegger, M., Poltavsky, I., Schütt, K. T., Tkatchenko, A., & Müller, K.-R. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.0c01111 · Cited by 1,647 · see citing works
PCANet: A Simple Deep Learning Baseline for Image Classification?
Chan, T.-H., Jia, K., Gao, S., Lu, J., Zeng, Z., & Ma, Y. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2475625 · Cited by 1,598 · see citing works
Deep learning-based electroencephalography analysis: a systematic review
Roy, Y., Banville, H., Albuquerque, I., Gramfort, A., Falk, T. H., & Faubert, J. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab260c · Cited by 1,532 · see citing works
Hutter, F., Kotthoff, L., & Vanschoren, J. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-05318-5 · Cited by 1,552 · see citing works
Deep learning in bioinformatics
Min, S., Lee, B., & Yoon, S. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw068 · Cited by 1,514 · see citing works
Deep Learning Applications in Medical Image Analysis
Ker, J., Wang, L., Rao, J. P., & Lim, C. C. T. (2017) · IEEE Access · doi:10.1109/access.2017.2788044 · Cited by 1,521 · see citing works
Deep learning for computational biology
Angermueller, C., Pärnamaa, T., Parts, L., & Stegle, O. (2016) · Molecular Systems Biology · doi:10.15252/msb.20156651 · Cited by 1,473 · see citing works
Optuna: A Next-generation Hyperparameter Optimization FrameworkPreprint
Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. (2019) · arXiv (Cornell University) · doi:10.4230/oasics.dx.2024.30 · Cited by 1,513 · see citing works
A Tutorial on Bayesian OptimizationPreprint
Frazier, P. I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.02811 · Cited by 1,423 · see citing works
Recurrent Neural Networks for Time Series Forecasting: Current status and future directions
Hewamalage, H., Bergmeir, C., & Bandara, K. (2021) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2020.06.008 · Cited by 1,371 · see citing works
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., & Ed, H. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3220007 · Cited by 1,329 · see citing works
(2015) · neural information processing systems · Cited by 1,305 · see citing works
Large-Scale Evolution of Image ClassifiersPreprint
Real, E., Moore, S., Selle, A. P., Saxena, S., Suematsu, Y. I. L., Tan, J., Le, Q. V., & Kurakin, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01041 · Cited by 1,372 · see citing works
Physics-informed neural networks for high-speed flows
Mao, Z., Jagtap, A. D., & Karniadakis, G. E. (2019) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2019.112789 · Cited by 1,287 · see citing works
Improving deep neural networks for LVCSR using rectified linear units and dropout
Dahl, G. E., Sainath, T. N., & Hinton, G. E. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639346 · Cited by 1,310 · see citing works
Deng, L., Hinton, G. E., & Kingsbury, B. E. D. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,222 · see citing works
Kelley, D. R., Snoek, J., & Rinn, J. L. (2016) · Genome Research · doi:10.1101/gr.200535.115 · Cited by 1,048 · see citing works
Self-citationMachine-learning-guided directed evolution for protein engineering
Yang, K., Wu, Z., & Arnold, F. H. (2019) · Nature Methods · doi:10.1038/s41592-019-0496-6 · Cited by 1,085 · see citing works
Gómez-Bombarelli, R., Aguilera-Iparraguirre, J., Hirzel, T. D., Duvenaud, D., Maclaurin, D., Blood-Forsythe, M. A., Chae, H. S., Einzinger, M., Ha, D.-G., Wu, T., et al. (2016) · Nature Materials · doi:10.1038/nmat4717 · Cited by 1,060 · see citing works
Self-citationXiong, Z., Wang, D., Liu, X., Zhong, F., Wan, X., Li, X., Li, Z. S., Luo, X., Chen, K., Jiang, H., et al. (2019) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.9b00959 · Cited by 1,083 · see citing works
Bayesian reaction optimization as a tool for chemical synthesis
Shields, B. J., Stevens, J., Li, J., Parasram, M., Damani, F., Alvarado, J. I. M., Janey, J. M., Adams, R. P., & Doyle, A. G. (2021) · Nature · doi:10.1038/s41586-021-03213-y · Cited by 1,063 · see citing works
Self-citationAuto-WEKA: combined selection and hyperparameter optimization of classification algorithms
Thornton, C., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487629 · Cited by 1,359 · see citing works
Speech emotion recognition using deep 1D & 2D CNN LSTM networks
Zhao, J., Mao, X., & Chen, L. (2019) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2018.08.035 · Cited by 1,044 · see citing works
Hyperopt: a Python library for model selection and hyperparameter optimization
Bergstra, J., Komer, B., Eliasmith, C., Yamins, D., & Cox, D. D. (2015) · Computational Science & Discovery · doi:10.1088/1749-4699/8/1/014008 · Cited by 1,002 · see citing works
ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks
Ding, X., Guo, Y., Ding, G., & Han, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00200 · Cited by 979 · see citing works
Compressing Neural Networks with the Hashing Trick
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv · doi:10.48550/arxiv.1504.04788 · Cited by 974 · see citing works
MRIQC: Advancing the automatic prediction of image quality in MRI from unseen sites
Esteban, O., Birman, D., Schaer, M., Koyejo, O. O., Poldrack, R. A., & Gorgolewski, K. J. (2017) · PLOS ONE · doi:10.1371/journal.pone.0184661 · Cited by 920 · see citing works
Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges
Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann, T., Becker, M., Boulesteix, A., et al. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1484 · Cited by 1,118 · see citing works
Auto-Keras: An Efficient Neural Architecture Search System
Jin, H., Song, Q., & Hu, X. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330648 · Cited by 963 · see citing works
Hyperband: a novel bandit-based approach to hyperparameter optimization
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2017) · Journal of Machine Learning Research · Cited by 930 · see citing works
Science Requirements and Detector Concepts for the Electron-Ion Collider
Khalek, R. A., Accardi, A., Adam, J., Adamiak, D., Akers, W., Albaladejo, M., Al-bataineh, A., Alexeev, M., Ameli, F., Antonioli, P., et al. (2022) · Nuclear Physics A · doi:10.1016/j.nuclphysa.2022.122447 · Cited by 1,133 · see citing works
A Survey on Distributed Machine Learning
Verbraeken, J., Wolting, M., Katzy, J., Kloppenburg, J., Verbelen, T., & Rellermeyer, J. S. (2020) · ACM Computing Surveys · doi:10.1145/3377454 · Cited by 902 · see citing works
Zhang, W., Wu, C., Zhong, H., Li, Y., & Wang, L. (2020) · Geoscience Frontiers · doi:10.1016/j.gsf.2020.03.007 · Cited by 913 · see citing works
Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey
Nguyen, G., Dlugolinský, Š., Bobák, M., Tran, V., García, Á. L., Heredia, I., Malík, P., & Hluchý, L. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-018-09679-z · Cited by 924 · see citing works
Lookman, T., Balachandran, P. V., Xue, D., & Yuan, R. (2019) · npj Computational Materials · doi:10.1038/s41524-019-0153-8 · Cited by 873 · see citing works
Automated machine learning: Review of the state-of-the-art and opportunities for healthcare
Waring, J., Lindvall, C., & Umeton, R. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101822 · Cited by 834 · see citing works
Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms
Bergstra, J., Yamins, D., & Cox, D. (2013) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-8b375195-003 · Cited by 784 · see citing works
A boosted decision tree approach using Bayesian hyper-parameter optimization for credit scoring
Xia, Y., Liu, C., Li, Y., & Liu, N. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.02.017 · Cited by 786 · see citing works
A tutorial survey of architectures, algorithms, and applications for deep learning
Deng, L. (2014) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2013.9 · Cited by 737 · see citing works
AI in Medical Imaging Informatics: Current Challenges and Future Directions
Panayides, A. S., Amini, A., Filipovic, N. D., Sharma, A., Tsaftaris, S. A., Young, A., Foran, D., Do, N., Golemati, S., Kurc, T., et al. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2991043 · Cited by 713 · see citing works
Applications of Deep Learning in Biomedicine
Mamoshina, P., Vieira, A., Lane, E., & Zhavoronkov, A. (2016) · Molecular Pharmaceutics · doi:10.1021/acs.molpharmaceut.5b00982 · Cited by 701 · see citing works
Scalable Bayesian Optimization Using Deep Neural NetworksPreprint
Snoek, J., Rippel, O., Swersky, K., Kiros, R., Satish, N., Sundaram, N., Patwary, M. A., Prabhat, & Adams, R. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.05700 · Cited by 702 · see citing works
Self-citationJing, L., Zhao, M., Li, P., & Xu, X. (2017) · Measurement · doi:10.1016/j.measurement.2017.07.017 · Cited by 698 · see citing works
Predicting the Price of Bitcoin Using Machine Learning
McNally, S. P., Roche, J., & Caton, S. J. (2018) · Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP) · doi:10.1109/pdp2018.2018.00060 · Cited by 690 · see citing works
A review on deep convolutional neural networks
Aloysius, N., & Geetha, M. K. (2017) · doi:10.1109/iccsp.2017.8286426 · Cited by 687 · see citing works
Auto-WEKA: Automatic Model Selection and Hyperparameter Optimization in WEKA
Kotthoff, L., Thornton, C., Hoos, H. H., Hutter, F., & Leyton‐Brown, K. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-05318-5_4 · Cited by 671 · see citing works
Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis
Fu, Y., Jung, A. W., Viñas, R., González, S., Vöhringer, H. S., Shmatko, A., Yates, L. R., Jimenez‐Liñan, M., Moore, L., & Gerstung, M. (2020) · Nature Cancer · doi:10.1038/s43018-020-0085-8 · Cited by 636 · see citing works
Tensorizing Neural NetworksPreprint
Novikov, A., Podoprikhin, D., Osokin, A., & Vetrov, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06569 · Cited by 656 · see citing works
Challenges in deploying machine learning: A survey of case studies
Paleyes, A., Urma, R.-G., & Lawrence, N. D. (2023) · ACM Computing Surveys · doi:10.1145/3533378 · Cited by 638 · see citing works
Evaluation of a Tree-based Pipeline Optimization Tool for Automating Data Science
Olson, R. S., Bartley, N., Urbanowicz, R. J., & Moore, J. H. (2016) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2908812.2908918 · Cited by 630 · see citing works
Deep learning of binary hash codes for fast image retrieval
Lin, K., Yang, H., Hsiao, J.-H., & Chen, C. (2015) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301269 · Cited by 617 · see citing works
Generative Moment Matching NetworksPreprint
Li, Y., Swersky, K., & Zemel, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02761 · Cited by 644 · see citing works
Alaa, A. M., Bolton, T. R., Angelantonio, E. D., Rudd, J. H. F., & van der Schaar, M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0213653 · Cited by 623 · see citing works
Autoreject: Automated artifact rejection for MEG and EEG data
Jas, M., Engemann, D. A., Bekhti, Y., Raimondo, F., & Gramfort, A. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.06.030 · Cited by 611 · see citing works
BEHRT: Transformer for Electronic Health Records
Li, Y., Rao, S., Solares, J. R. A., Hassaine, A., Ramakrishnan, R., Canoy, D., Zhu, Y., Rahimi, K., & Salimi-Khorshidi, G. (2020) · Scientific Reports · doi:10.1038/s41598-020-62922-y · Cited by 595 · see citing works
Automatic tuning of hyperparameters using Bayesian optimization
Victoria, A. H., & Maragatham, G. (2020) · Evolving Systems · doi:10.1007/s12530-020-09345-2 · Cited by 593 · see citing works
Tune: A Research Platform for Distributed Model Selection and TrainingPreprint
Liaw, R., Liang, E., Nishihara, R., Moritz, P., Gonzalez, J. E., & Stoica, I. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.05118 · Cited by 593 · see citing works
González, S., García, S., Ser, J. D., Rokach, L., & Herrera, F. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.07.007 · Cited by 595 · see citing works
Mathematics for Machine Learning
Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108679930 · Cited by 583 · see citing works
A systematic review on overfitting control in shallow and deep neural networks
Bejani, M. M., & Ghatee, M. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09975-1 · Cited by 598 · see citing works
Demystifying Parallel and Distributed Deep Learning
Ben‐Nun, T., & Hoefler, T. (2019) · ACM Computing Surveys · doi:10.1145/3320060 · Cited by 747 · see citing works
Machine learning for aerial image labeling
Hinton, G. E., & Mnih, V. (2013) · TSpace (University of Toronto) · Cited by 722 · see citing works
Efficient Architecture Search by Network Transformation
Cai, H., Chen, T., Zhang, W., Yu, Y. H., & Wang, Y.-X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11709 · Cited by 594 · see citing works
Golovin, D., Solnik, B., Moitra, S., Kochanski, G., Karro, J. E., & Sculley, D. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098043 · Cited by 585 · see citing works
Sequential regulatory activity prediction across chromosomes with convolutional neural networks
Kelley, D. R., Reshef, Y. A., Bileschi, M., Belanger, D., McLean, C. Y., & Snoek, J. (2018) · Genome Research · doi:10.1101/gr.227819.117 · Cited by 537 · see citing works
Self-citationA genetic programming approach to designing convolutional neural network architectures
Suganuma, M., Shirakawa, S., & Nagao, T. (2017) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3071178.3071229 · Cited by 551 · see citing works
Khan, A. A., Chaudhari, O., & Chandra, R. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122778 · Cited by 620 · see citing works
Random Search and Reproducibility for Neural Architecture SearchPreprint
Li, L., & Talwalkar, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07638 · Cited by 534 · see citing works
Bayesian Optimization for Adaptive Experimental Design: A Review
Greenhill, S., Rana, S., Gupta, S., Vellanki, P., & Venkatesh, S. (2020) · IEEE Access · doi:10.1109/access.2020.2966228 · Cited by 526 · see citing works
Machine learning and soil sciences: a review aided by machine learning tools
Padarian, J., Minasny, B., & McBratney, A. B. (2020) · SOIL · doi:10.5194/soil-6-35-2020 · Cited by 524 · see citing works
Probabilistic Backpropagation for Scalable Learning of Bayesian Neural NetworksPreprint
Hernández-Lobato, J. M., & Adams, R. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.05336 · Cited by 514 · see citing works
Self-citationArtificial Intelligence Applied to Battery Research: Hype or Reality?
Lombardo, T., Duquesnoy, M., El-Bouysidy, H., Årén, F., Gallo‐Bueno, A., Jørgensen, P. B., Bhowmik, A., Demortière, A., Ayerbe, E., Alcaide, F., et al. (2021) · Chemical Reviews · doi:10.1021/acs.chemrev.1c00108 · Cited by 511 · see citing works
Markovics, D., & Mayer, M. J. (2022) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2022.112364 · Cited by 524 · see citing works
Shin, H.-C., Orton, M. R., Collins, D. J., Doran, S. J., & Leach, M. O. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.277 · Cited by 488 · see citing works
- Authors
- Snoek, J., Larochelle, H., & Adams, R. P.
- Affiliations
- University of Toronto (CA)
- Université de Sherbrooke (CA)
- Harvard University (US)
- ORCID
- Adams, Ryan P.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2012
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1206.2944
- OpenAlex
- W2131241448
- Cited by
- 6,691
- References
- 23
- Citations by year
- 2026: 62 · 2025: 630 · 2024: 779 · 2023: 712 · 2022: 720 · 2021: 997 · 2020: 864 · 2019: 680 · 2018: 465 · 2017: 289 · 2016: 212 · 2015: 144 · 2014: 94 · 2013: 34 · 2012: 9
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- Bayesian optimization, hyperparameter optimization, Gaussian process priors, latent Dirichlet allocation, structured support vector machines, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
