Optimizing ensemble weights and hyperparameters of machine learning models for regression problems
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Aggregating multiple learners through an ensemble of models aim to make better predictions by capturing the underlying distribution of the data more accurately. Different ensembling methods, such as bagging, boosting, and stacking/blending, have been studied and adopted extensively in research and practice. While bagging and boosting focus more on reducing variance and bias, respectively, stacking approaches target both by finding the optimal way to combine base learners. In stacking with the weighted average, ensembles are created from weighted averages of multiple base learners. It is known that tuning hyperparameters of each base learner inside the ensemble weight optimization process can produce better performing ensembles. To this end, an optimization-based nested algorithm that considers tuning hyperparameters as well as finding the optimal weights to combine ensembles (Generalized Weighted Ensemble with Internally Tuned Hyperparameters (GEM-ITH)) is designed. Besides, Bayesian search was used to speed-up the optimizing process and a heuristic was implemented to generate diverse and well-performing base learners. The algorithm is shown to be generalizable to real data sets through analyses with ten publicly available data sets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Wolpert, D. H. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80023-1 · Cited by 7,951
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260
Boosting a Weak Learning Algorithm by Majority
Freund, Y. (1995) · Information and Computation · doi:10.1006/inco.1995.1136 · Cited by 1,674
The elements of statistical learning: Data mining, inference, and prediction
Franklin, J. (2005) · The Mathematical Intelligencer · doi:10.1007/bf02985802 · Cited by 1,400 · cited 3×
“The third term, Var(ε), in Equation (3) is called irreducible error, which is the variance of the noise term in the true underlying function (f(x)) and cannot be reduced by any model (Hastie et al. 2005).”
BackgroundMethodInfluential
Metadata
- Authors
- Shahhosseini, M., Hu, G., & Pham, H.
- Venue
- Machine Learning with Applications
- Publisher
- Elsevier BV
- Published
- 2022-03
- Volume
- 7
- Pages
- 100251
- ISSN
- 2666-8270 (ISSN-L)
- OpenAlex
- W2967602207
- Cited by
- 206
- References
- 9
- FWCI
- 13.09
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Optimizing ensemble weights and hyperparameters of machine learning models for regression problems. https://app.cochat.ai/papers/10.1016/j.mlwa.2022.100251
- The paper
- Shahhosseini, M., Hu, G., & Pham, H. (2022). Optimizing ensemble weights and hyperparameters of machine learning models for regression problems. Machine Learning with Applications. https://doi.org/10.1016/j.mlwa.2022.100251
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.