The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
The Evolution of Boosting Algorithms: From Machine Learning to Statistical Modelling
Open accessAbstract
Summary: Background: The concept of boosting emerged from the field of machine learning. The basic idea is to boost the accuracy of a weak classifying tool by combining various instances into a more accurate prediction. This general concept was later adapted to the field of statistical modelling. Nowadays, boosting algorithms are often applied to estimate and select predictor effects in statistical regression models. Objectives: This review article attempts to highlight the evolution of boosting algorithms from machine learning to statistical modelling. Methods: We describe the AdaBoost algorithm for classification as well as the two most prominent statistical boosting approaches, gradient boosting and likelihood-based boosting for statistical modelling. We highlight the methodological background and present the most common software implementations. Results: Although gradient boosting and likelihood-based boosting are typically treated separately in the literature, they share the same methodological roots and follow the same fundamental concepts. Compared to the initial machine learning algorithms, which must be seen as black-box prediction schemes, they result in statistical models with a straight-forward interpretation. Conclusions: Statistical boosting algorithms have gained substantial interest during the last decade and offer a variety of options to address important research questions in modern biomedicine.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
John Lu, Z. Q. (2010) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2010.00646_6.x · Cited by 12,390
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Koyak, R. A., Hastie, T. J., & Tibshirani, R. J. (1991) · Journal of the American Statistical Association · doi:10.2307/2290538 · Cited by 7,596
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
Regression and time series model selection in small samples
HURVICH, C. M., & TSAI, C.-L. (1989) · Biometrika · doi:10.1093/biomet/76.2.297 · Cited by 6,586
Rosenwald, A., Wright, G., Chan, W. C., Connors, J. M., Campo, E., Fisher, R. I., Gascoyne, R. D., Muller-Hermelink, H. K., Smeland, E. B., Giltnane, J. M., et al. (2002) · New England Journal of Medicine · doi:10.1056/nejmoa012914 · Cited by 3,829
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants
Bauer, E., & Kohavi, R. (1999) · Machine Learning · doi:10.1023/a:1007515423169 · Cited by 2,638
Ensemble Methods: Foundations and Algorithms
Zhou, Z.-H. (2012) · doi:10.1201/b12207 · Cited by 2,534
Boosting the margin: a new explanation for the effectiveness of voting methods
Bartlett, P., Freund, Y., Lee, W. S., & Schapire, R. E. (1998) · The Annals of Statistics · doi:10.1214/aos/1024691352 · Cited by 2,306
Ensemble Methods: Foundations and Algorithms
Zhou, Z. (2012) · Cited by 2,051
Pattern Recognition and Machine Learning
Maindonald, J. (2007) · Journal of Statistical Software · doi:10.18637/jss.v017.b05 · Cited by 1,349
Rätsch, G., Onoda, T., & Müller, K.-R. (2001) · Machine Learning · doi:10.1023/a:1007618119488 · Cited by 1,299
Arcing classifier (with discussion and a rejoinder by the author)
Breiman, L. (1998) · The Annals of Statistics · doi:10.1214/aos/1024691079 · Cited by 1,105
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1987) · Information Processing Letters · doi:10.1016/0020-0190(87)90114-1 · Cited by 1,054
Boosting With the L2Loss: Regression and Classification
Bühlmann, P., & Yu, B. (2003) · Journal of the American Statistical Association · doi:10.1198/016214503000125 · Cited by 845
Model Selection and the Principle of Minimum Description Length
Hansen, M. H., & Yu, B. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753168398 · Cited by 788
Overfitting and undercomputing in machine learning
Dietterich, T. (1995) · ACM Computing Surveys · doi:10.1145/212094.212114 · Cited by 757
Boosting Algorithms: Regularization, Prediction and Model Fitting
Bühlmann, P., & Hothorn, T. (2007) · Statistical Science · doi:10.1214/07-sts242 · Cited by 726
Regression, Prediction and Shrinkage
Copas, J. B. (1983) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1983.tb01258.x · Cited by 612
gbm: Generalized Boosted Regression Models
Ridgeway, G., & Developers, G. (2003) · CRAN: Contributed Packages · doi:10.32614/cran.package.gbm · Cited by 555
Prediction Games and Arcing Algorithms
Breiman, L. (1999) · Neural Computation · doi:10.1162/089976699300016106 · Cited by 530
An Introduction to Boosting and Leveraging
Meir, R., & Rätsch, G. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-36434-x_4 · Cited by 447
Boosting: Foundations and Algorithms
Schapire, R. E., & Freund, Y. (2012) · The MIT Press eBooks · doi:10.7551/mitpress/8291.001.0001 · Cited by 417
Boosting for high-dimensional linear models
Bühlmann, P. (2006) · The Annals of Statistics · doi:10.1214/009053606000000092 · Cited by 410
The weighted majority algorithm
Littlestone, N., & Warmuth, M. K. (1989) · 30th Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.1989.63487 · Cited by 295
Cryptographic limitations on learning Boolean formulae and finite automata
Kearns, M. J., & Valiant, L. G. (1993) · Lecture notes in computer science · doi:10.1007/3-540-56483-7_21 · Cited by 276
Crytographic limitations on learning Boolean formulae and finite automata
Kearns, M., & Valiant, L. G. (1989) · Proceedings of the twenty-first annual ACM symposium on Theory of computing - STOC '89 · doi:10.1145/73007.73049 · Cited by 266
Model-based boosting in R: a hands-on tutorial using the R package mboost
Hofner, B., Mayr, A., Robinzonov, N., & Schmid, M. (2014) · Computational Statistics · doi:10.1007/s00180-012-0382-5 · Cited by 258
Self-citationThe strength of weak learnability
Schapire, R. E. (1989) · 30th Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.1989.63451 · Cited by 257
Boosting in the limit: maximizing the margin of learned ensembles
Grove, A. J., & Schuurmans, D. (1998) · Cited by 231
Boosting: Foundations and Algorithms
Schapire, R. E. (2013) · Kybernetes · doi:10.1108/03684921311295547 · Cited by 231
Allowing for mandatory covariates in boosting estimation of sparse high-dimensional survival models
Binder, H., & Schumacher, M. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-14 · Cited by 230
Self-citationHow boosting the margin can also boost classifier complexity
Reyzin, L., & Schapire, R. E. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143939 · Cited by 212
Generalized Additive Modeling with Implicit Variable Selection by Likelihood‐Based Boosting
Tutz, G., & Binder, H. (2006) · Biometrics · doi:10.1111/j.1541-0420.2006.00578.x · Cited by 208
Self-citationEvidence Contrary to the Statistical View of Boosting
Mease, D., & Wyner, A. J. (2008) · ScholarlyCommons (University of Pennsylvania) · doi:10.5555/1390681.1390687 · Cited by 184
Identifying Risk Factors for Severe Childhood Malnutrition by Boosting Additive Quantile Regression
Fenske, N., Kneib, T., & Hothorn, T. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.ap09272 · Cited by 179
Regularized ROC method for disease classification and biomarker selection with microarray data
Ma, S., & Huang, J. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti724 · Cited by 168
Boosting a weak learning algorithm by majority
Freund, Y. (1990) · Colt Proceedings 1990 · doi:10.1016/b978-1-55860-146-8.50019-9 · Cited by 163
Gene Expression Profiling Predicts the Development of Oral Cancer
Saintigny, P., Zhang, L., Fan, Y.-H., El‐Naggar, A. K., Papadimitrakopoulou, V. A., Feng, L., Lee, J. J., Kim, E. S., Hong, W. K., & Mao, L. (2011) · Cancer Prevention Research · doi:10.1158/1940-6207.capr-10-0155 · Cited by 162
Boosting additive models using component-wise P-Splines
Schmid, M. C., & Hothorn, T. (2008) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2008.09.009 · Cited by 144
Self-citationPartial Cox regression analysis for high-dimensional microarray gene expression data
Li, H., & Gui, J. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth900 · Cited by 144
Mayr, A. J., & Schmid, M. C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0084483 · Cited by 136
Self-citationTutz, G., & Binder, H. (2006) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.11.041 · Cited by 135
Self-citationKruppa, J., Liu, Y., Biau, G., Kohler, M., König, I. R., Malley, J. D., & Ziegler, A. (2014) · Biometrical Journal · doi:10.1002/bimj.201300068 · Cited by 97 · cited 1×
“Applications include the areas of disease diagnostics [18], genome-wide association studies [19] or the estimation of patient-specific risks and probabilities [20, 21].”
BackgroundLi, H., & Luan, Y. (2005) · Computer applications in the biosciences · doi:10.1093/bioinformatics/bti324 · Cited by 88
Improved Prediction of Body Fat by Measuring Skinfold Thickness, Circumferences, and Bone Breadths
García, A. L., Wagner, K., Hothorn, T., Koebnick, C., Zunft, H.-J. F., & Trippo, U. (2005) · Obesity Research · doi:10.1038/oby.2005.67 · Cited by 83
Extending Statistical Boosting: An Overview of Recent Methodological Developments
Binder, H., Gefeller, O., Schmid, M. C., & Mayr, A. J. (2014) · Methods of Information in Medicine · doi:10.3414/me13-01-0123 · Cited by 59
Self-citationHothorn, T., Bühlmann, P., Kneib, T., Schmid, M. C., & Hofner, B. (2006) · doi:10.32614/cran.package.mboost · Cited by 52
Self-citationComment: Boosting Algorithms: Regularization, Prediction and Model Fitting
Hastie, T. J. (2007) · Statistical Science · doi:10.1214/07-sts242a · Cited by 36
A Fast Estimate for the Population Recombination Rate Based on Regression
Lin, K., Futschik, A., & Li, H. (2013) · Genetics · doi:10.1534/genetics.113.150201 · Cited by 36
Comment: Boosting Algorithms: Regularization, Prediction and Model Fitting
Hastie, T. (2010) · Cited by 36
A new formula for optimized weight estimation in extreme fetal macrosomia (≥4500 g)
Faschingbauer, F., Dammer, U., Siemer, J., Schild, R., Schmid, M., Beckmann, M., & Kehl, S. (2013) · Ultraschall in der Medizin - European Journal of Ultrasound · doi:10.1055/s-0033-1355065 · Cited by 2
Self-citation(1996)
Research works printed but not linked: 12
- Schmid, 2012, Methods Inf Med, 51, 150, 10.1055/s-0038-1627042 · doi:10.1055/s-0038-1627042
- Faschingbauer, 2012, European Journal of Ultrasound, 33, 480
- 10.1002/sim.5490 · DOI
- Schapire, 1990, Machine Learning, 5, 197
- Ridgeway, 1999, Computing Science and Statistics, 31, 172
- Breiman, 1996, Machine Learning, 24, 123
- Mease, 2008, The Journal of Machine Learning Research, 9, 131
- 10.1214/07-STS242REJ · DOI
- Bühlmann, 2008, Journal of Machine Learning Research, 9, 187
- 10.3414/ME11-02-0021 · doi:10.3414/me11-02-0021
- 10.3414/ME11-02-0030 · DOI
- Bühlmann, 2007, Journal of Machine Learning Research, 7, 1001
Cited by
Luo, W., Phung, D., Tran, T., Gupta, S., Rana, S., Karmakar, C., Shilton, A., Yearwood, J., Dimitrova, N., Ho, T. B., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5870 · Cited by 988
Artificial Intelligence in Cardiovascular Imaging
Dey, D., Slomka, P. J., Leeson, P., Comaniciu, D., Shrestha, S., Sengupta, P. P., & Marwick, T. H. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.12.054 · Cited by 680
Machine learning prediction in cardiovascular diseases: a meta-analysis
Krittanawong, C., Virk, H. U. H., Bangalore, S., Wang, Z., Johnson, K. W., Pinotti, R., Zhang, H., Kaplin, S., Narasimhan, B., Kitai, T., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-72685-1 · Cited by 515
A comprehensive evaluation of ensemble learning for stock-market prediction
Nti, I. K., Adekoya, A. F., & Weyori, B. A. (2020) · Journal of Big Data · doi:10.1186/s40537-020-00299-5 · Cited by 329
Artificial intelligence and echocardiography
Alsharqi, M., Woodward, W. J., Mumith, J. A., Markham, D. C., Upton, R., & Leeson, P. (2018) · Echo Research & Practice · doi:10.1530/erp-18-0056 · Cited by 262
Metadata
- Authors
- Binder, H., Gefeller, O., Schmid, M., & Mayr, A.
- Venue
- Methods of Information in Medicine
- Publisher
- Georg Thieme Verlag KG
- Published
- 2014
- Volume
- 53
- Issue
- 06
- Pages
- 419-427
- ISSN
- 0026-1270 (ISSN-L)
- 2511-705X
- OpenAlex
- W4292334794
- Cited by
- 412
- References
- 60
- FWCI
- 15.47
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.