The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Hyperparameter Optimization: Comparing Genetic Algorithm against Grid Search and Bayesian Optimization
Abstract
The performance of machine learning algorithms are affected by several factors, some of these factors are related to data quantity, quality, or its features. Another element is the choice of an appropriate algorithm to solve the problem and one major influence is the parameter configuration based on the problem specification. Parameters in machine learning can be classified in two types: (1) model parameters that are internal, configurable, and its value can be estimated from data such as weights of a deep neural network; and (2) hyperparameters, which are external and its values can not be estimated from data such as the learning rate for the training of a neural network. Hyperparameter values may be specified by a practitioner or using a heuristic, or parameter values obtained from other problems can be used etc., however, the best values of these parameters are identified when the algorithm has the highest accuracy, and these could be achieved by tuning the parameters. The main goal of this paper is to conduct a comparison study between different algorithms that are used in the optimization process in order to find the best hyperparameter values for the neural network. The algorithms applied are grid search algorithm, bayesian algorithm, and genetic algorithm. Different evaluation measures are used to conduct this comparison such as accuracy and running time.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
(2012) · Cited by 8,009
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026683 · Cited by 7,999 · cited 1×
“Different domains have applied classification, including bioinformatics [1] image processing, [2], text classification [3], [4], and language processing [5].”
MethodAn overview of gradient descent optimization algorithms
Ruder, S. (2016) · arXiv · doi:10.48550/arxiv.1609.04747 · Cited by 4,962 · cited 1×
“The following optimizers are listed below; more details are available in [16].”
Background(2011) · Cited by 3,939
Mean Absolute Percentage Error for regression models
de Myttenaere, A., Golden, B., Le Grand, B., & Rossi, F. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.12.114 · Cited by 1,502 · cited 1×
“• Mean Absolute Percentage Error (MAPE): as given in [17], MAPE is the most common measure used to predict errors, and works perfectly if there are no extreme values in the data.”
BackgroundEncyclopedia of Machine Learning and Data Mining
(2017) · Springer eBooks · doi:10.1007/978-1-4899-7687-1 · Cited by 914 · cited 10×
“5) Area Under the Curve (AUC) [15]: AUC is an experimental computation of the classification process performance based on the area under an Receiver Operating Characteristics (ROC) curve.”
MethodInfluentialData Mining and Analysis: Fundamental Concepts and Algorithms
Zaki, M. J., & Meira, W. (2014) · Cited by 780
Mohammed, R., Rawashdeh, J., & Abdullah, M. (2020) · 2020 11th International Conference on Information and Communication Systems (ICICS) · doi:10.1109/icics49469.2020.239556 · Cited by 684 · cited 2×
“Unfortunately a direct comparison with past research results cannot be made, however, putting it into perspective in [21] the same data set was investigated.”
MethodResultA review of automatic selection methods for machine learning algorithms and hyper-parameter values
Luo, G. (2016) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-016-0125-6 · Cited by 357 · cited 2×
“A machine learning model has two types of parameters [6]:”
MethodResampling Methods for Meta-Model Validation with Recommendations for Evolutionary Computation
Bischl, B., Mersmann, O., Trautmann, H., & Weihs, C. (2012) · Evolutionary Computation · doi:10.1162/evco_a_00069 · Cited by 186 · cited 1×
“An alternative solution [7] involves the use of hyperparameter-tuning strategies that are data dependent, which try to minimize the expected generalization error of the algorithm over a hyperparameter search space of considered candidate configurations, usually by evaluating predictions on”
Method
Research works printed but not linked: 1
- pérez-cruz, 2018, Divergence Estimation of Continuous Distributions, proceedings of IEEE international symposium on Information Theory, 1666
Cited by
Grid search hyperparameter tuning in additive manufacturing processes
Ogunsanya, M., Isichei, J., & Desai, S. S. (2023) · Manufacturing Letters · doi:10.1016/j.mfglet.2023.08.056 · Cited by 152
End-to-end deep learning framework for printed circuit board manufacturing defect classification
Bhattacharya, A., & Cloutier, S. G. (2022) · Scientific Reports · doi:10.1038/s41598-022-16302-3 · Cited by 167
“For fine-tuning the hyperparameters of our proposed model, we have used a genetic algorithm-based approach 64 .”
MethodJiang, X., & Xu, C. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11195772 · Cited by 137
“Bayesian optimization and genetic algorithm are both considered as a type of sequential optimization method, with which the existing results will influence the subsequent model hyperparameter selection [57].”
MethodA Survey of Privacy Risks and Mitigation Strategies in the Artificial Intelligence Life Cycle
Shahriar, S., Allana, S., Hazratifard, S. M., & Dara, R. (2023) · IEEE Access · doi:10.1109/access.2023.3287195 · Cited by 114
Automated asphalt pavement damage rate detection based on optimized GA-CNN
Li, J., Liu, T., Wang, X., & Jian-you, Y. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104180 · Cited by 112
Metadata
- Authors
- Alibrahim, H., & Ludwig, S. A.
- Affiliations
- North Dakota State University (US)
- ORCID
- Ludwig, Simone A.
- Venue
- 2021 IEEE Congress on Evolutionary Computation (CEC)
- Publisher
- IEEE
- Published
- 2021-06-28
- Pages
- 1551-1559
- OpenAlex
- W3191950921
- Cited by
- 489
- References
- 19
- Citations by year
- 2026: 79 · 2025: 168 · 2024: 120 · 2023: 86 · 2022: 35 · 2021: 1
- FWCI
- 27.95
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Congress on Evolutionary Computation (CEC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Metaheuristic Optimization Algorithms Research
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- hyperparameter optimization, genetic algorithm, grid search, Bayesian optimization, neural network hyperparameters, algorithm comparison, execution time, classification accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Hyperparameter Optimization: Comparing Genetic Algorithm against Grid Search and Bayesian Optimization. https://app.cochat.ai/papers/10.1109/cec45853.2021.9504761
- The paper
- Alibrahim, H., & Ludwig, S. A. (2021). Hyperparameter Optimization: Comparing Genetic Algorithm against Grid Search and Bayesian Optimization. 2021 IEEE Congress on Evolutionary Computation (CEC). https://doi.org/10.1109/cec45853.2021.9504761
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.