Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 1,814

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

CatBoost for big data: an interdisciplinary review

Open access
Hancock, John T., Khoshgoftaar, Taghi M.·2020·Journal of Big Data
DOI 10.1186/s40537-020-00369-8

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Gradient Boosted Decision Trees (GBDT’s) are a powerful tool for classification and regression tasks in Big Data. Researchers should be familiar with the strengths and weaknesses of current implementations of GBDT’s in order to use them effectively and make successful contributions. CatBoost is a member of the family of GBDT machine learning ensemble techniques. Since its debut in late 2018, researchers have successfully used CatBoost for machine learning studies involving Big Data. We take this opportunity to review recent research on CatBoost as it relates to Big Data, and learn best practices from studies that cast CatBoost in a positive light, as well as studies where CatBoost does not outshine other techniques, since we can learn lessons from both types of scenarios. Furthermore, as a Decision Tree based algorithm, CatBoost is well-suited to machine learning tasks involving categorical, heterogeneous data. Recent work across multiple disciplines illustrates CatBoost’s effectiveness and shortcomings in classification and regression tasks. Another important issue we expose in literature on CatBoost is its sensitivity to hyper-parameters and the importance of hyper-parameter tuning. One contribution we make is to take an interdisciplinary approach to cover studies related to CatBoost in a single work. This provides researchers an in-depth understanding to help clarify proper application of CatBoost in solving problems. To the best of our knowledge, this is the first survey that studies all works related to CatBoost in a single publication.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s40537-020-00369-8

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 15 references

Research works printed but not linked: 80

  1. Kotsiantis SB, Zaharakis I, Pintelas P. Supervised machine learning: a review of classification techniques. Emerg Artif Intellig Appl Comput Eng. 2007;160(1):3–24.
  2. Liudmila P, Gleb G, Aleksandr V, Anna Veronika D, Andrey G. Catboost: unbiased boosting with categorical features. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, 2018; pages 6638–6648. Curran Associates, Inc.
  3. Johnson JM, Khoshgoftaar TM. Deep learning and data sampling with imbalanced big data. In: 2019 IEEE 20th international conference on information reuse and integration for data science (IRI). 2019; p. 175–183. · DOI
  4. Johnson JM, Khoshgoftaar TM. Medicare fraud detection using neural networks. J Big Data. 2019;1:1. · doi:10.32604/jbd.2019.05899
  5. Yasunari M, Takuomi H, Anna O, Kouichi Y, Uesawa Y. Prediction model of aryl hydrocarbon receptor activation by a novel qsar approach, deepsnap-deep learning. Molecules. 2020;25(6):1317. · DOI
  6. Johnson JM, Khoshgoftaar TM. Survey on deep learning with class imbalance. J Big Data. 2019;6(1):1–54. · DOI · Paper page
  7. Spadon Gabriel, de Carvalho Andre C P L F, Rodrigues-Jr Jose F, Alves Luiz G A. Reconstructing commuters network using machine learning and urban indicators. Scientific Reports. 2019;9(1):N.PAG. · DOI
  8. Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY. Lightgbm: a highly efficient gradient boosting decision tree. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R, editors. Advances in neural information processing systems. New York: Curran Associates; 2017. p. 3146–54.
  9. Hasanin T, Khoshgoftaar TM, Leevy JL, Bauder RA. Investigating class rarity in big data. J Big Data. 2020;7(1):1–17. · DOI
  10. Herland M, Khoshgoftaar TM, Bauder RA. Big data fraud detection using multiple medicare data sources. J Big Data. 2018;5(1):29. · DOI
  11. Sheshasaayee A, Lakshmi JVN. An insight into tree based machine learning techniques for big data analytics using apache spark. 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT), July 2017; pp. 1740–1743.
  12. Matei Z, Reynold XS, Patrick W, Tathagata D, Michael a, A Dave, Xiangrui M, Josh R, Shivaram V, Michael FJ, Ghodsi A, Joseph G, Schenkert S, I Stoica. Apache spark: a unified engine for big data processing. Commun. ACM. 2016;59(11):56–65. · DOI · Paper page
  13. Hasanin T, Khoshgoftaar TM, Leevy JL. A comparison of performance metrics with severely imbalanced network security big data. In: 2019 IEEE 20th international conference on information reuse and integration for data science (IRI). IEEE. 2019; p 83–88. · DOI
  14. Sudha P, Gunavathi R. A survey paper on map reduce in big data. International Journal of Science and Research. 2016;5(9).
  15. Khramtsov V, Sergeyev A, Spiniello C, Tortora C, Napolitano NR, Agnello A, Getman F, Jong JTA, Kuijken K, Radovich M, Shan H, Shulga V. KiDS-SQuaD II. Machine learning selection of bright extragalactic objects to search for new gravitationally lensed quasars. Astonomy Astrophys. 2019;2019:632.
  16. Daoud EA. Comparison between xgboost, lightgbm and catboost using a home credit dataset. Int J Comput Inf Eng. 2019;13(1):6–10.
  17. Yufei Xia, Lingyun He, Yinguo Li, Nana Liu, Yanlin Ding. Predicting loan default in peer-to-peer lending using narrative data. J Forecasting. 2020;39(2):260. · DOI
  18. Zhang F, Fleyeh H. Short term electricity spot price forecasting using catboost and bidirectional long short term memory neural network. 2019 16th International Conference on the European Energy Market (EEM), Sep. 2019; pp. 1–6.
  19. Zhang Haichao, Zeng Ruishuang, Chen Linling, Zhang Shangfeng. Research on personal credit scoring model based on multi-source data. J Phys Conference Series. 2020;1437:012053. · DOI
  20. Adamović S, Miškovic V, Maček N, Milosavljević M, Šarac M, Saračević M, Gnjatović M. An efficient novel approach for iris recognition based on stylometric features and machine learning techniques. Fut Gener Comput Syst. 2020;107:144–57. · DOI
  21. Kong SH, Ahn D, Kim B, Srinivasan K, Ram S, Kim H, Hong AR, Kim JH, Cho NH, Shin CS. A novel fracture prediction model using machine learning in a community-based cohort. JBMR Plus. 2020;4(3):1. · DOI
  22. Saifur R, Muhammad I, Mohsin R, Khawaja M-G, Shumayla Y, Muhammad A. Performance analysis of boosting classifiers in recognizing activities of daily living. Int J Environ Res Public Health. 2020;17(3):1082. · DOI
  23. Yang H, Bath PA. The use of data mining methods for the prediction of dementia: evidence from the english longitudinal study of aging. IEEE J Biomed Health Inform. 2020;24(2):345–53. · DOI
  24. Kolesnikov AA, Kikin PM, Portnov AM. Diseases spread prediction in tropical areas by machine learning methods ensembling and spatial analysis techniques. ISPRS. 2019;8XLII–3/W:221–6.
  25. Lin F, Cui EM, Lei Y, Luo L. Ct-based machine learning model to predict the fuhrman nuclear grade of clear cell renal cell carcinoma. Abdominal Radiol. 2019;44(7):2528–34. · DOI
  26. Coma-Puig B, Carmona J. Bridging the gap between energy consumption and distribution through non-technical loss detection. Energies. 2019;12(9):1748. · doi:10.3390/en12091748
  27. Ghori KM, Ayaz A Rabeeh, Awais M, Imran M, Ullah A, Szathmary L. Impact of feature selection on non-technical loss detection. In: 2020 6th conference on data science and machine learning applications (CDMA). 2020; p 19–24. · doi:10.1109/cdma47397.2020.00009
  28. Punmiya R, Choe S. Energy theft detection using gradient boosting theft detector with feature engineering-based preprocessing. IEEE Trans Smart Grid. 2019;10(2):2326–9. · DOI · Paper page
  29. Fan Junliang, Wang Xiukang, Zhang Fucang, Ma Xin, Lifeng Wu. Predicting daily diffuse horizontal solar radiation in various climatic regions of china using support vector machine and tree-based soft computing models with local and extrinsic climatic data. J Clean Prod. 2020;248:119264. · DOI
  30. Huang G, Lifeng W, Ma X, Zhang W, Fan J, Xiang Y, Zeng W, Zhou H. Evaluation of catboost method for prediction of reference evapotranspiration in humid regions. J Hydrol. 2019;574:1029–41. · DOI · Paper page
  31. Abolfazli A, Brechmann A, Wolff S, Spiliopoulou M. Machine learning identifies the dynamics and influencing factors in an auditory category learning experiment. Sci Rep. 2020;10(1):1. · DOI
  32. Arkaprabha S, Ishita B. Screening of anxiety and depression among the seafarers using machine learning technology. Inform Med Unlocked. 2019;16:100149. · DOI
  33. Liu W, Deng K, Zhang X, Cheng Y, Zheng Z, Jiang F, Peng J. A semi-supervised tri-catboost method for driving style recognition. Symmetry. 2020;3:336. · DOI
  34. Bakhareva N, Shukhman A, Matveev A, Polezhaev P, Ushakov Y, Legashev L. Attack detection in enterprise networks by machine learning methods. In: 2019 international Russian automation conference (RusAutoCon). 2019; pages 1–6. · DOI
  35. Yi Hai-Cheng, You Zhu-Hong, Guo Zhen-Hao. Construction and analysis of molecular association network by combining behavior representation and node attributes. Front Genetics. 2019;10:1. · doi:10.3389/fgene.2019.01106
  36. Koehn D, Lessmann S, Schaal M. Predicting online shopping behaviour from clickstream data using deep learning. Expert Syst Appl. 2020;150:113342. · DOI
  37. Google.com. Google scholar. http://scholar.google.com , May 2020.
  38. Omer S, Lior R. Ensemble learning: a survey. WIREs Data Mining Knowl Discov. 2018;8(4):e1249. · DOI · Paper page
  39. Sujatha M, Prabhakar S, Lavanya GD. A survey of classification techniques in data mining. Int J Innovations Eng Technol (IJIET). 2013;2(4):1058–2319.
  40. Micci-Barreca Daniele. A preprocessing scheme for high-cardinality categorical attributes in classification and prediction problems. SIGKDD Explor Newsl. 2001;3(1):27–32. · DOI · Paper page
  41. Yin L, Mikhail O. Bdt: Gradient boosted decision tables for high accuracy and scoring efficiency. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 2017; pp. 1893–1901.
  42. Hancock JT, Khoshgoftaar TM. Survey on categorical data for neural networks. J Big Data. 2020;7:1–41. · DOI · Paper page
  43. Fisher WD. On grouping for maximum homogeneity. J Am Stat Assoc. 1958;53(284):789–98. · DOI · Paper page
  44. Diao L, Niu D, Zang Z, Chen C. Short-term weather forecast based on wavelet denoising and catboost. In: 2019 Chinese control conference (CCC). 2019; pp. 3760–4. · doi:10.23919/chicc.2019.8865324
  45. Ghori KM, Abbasi RA, Awais M, Imran M, Ullah A, Szathmary L. Performance analysis of different types of machine learning classifiers for non-technical loss detection. IEEE Access. 2020;8:16033–48. · DOI
  46. de Jong JTA, Kleijn GAV, Kuijken KH, Valentijn EA. The kilo-degree survey. Exp Astron. 2013;35(1–2):25–44. · DOI · Paper page
  47. Abolfathi B, et al. The fourteenth data release of the sloan digital sky survey: first spectroscopic data from the extended Baryon oscillation spectroscopic survey and from the second phase of the apache point observatory galactic evolution experiment. Astrophys J Suppl Series. 2018;235(2):42. · DOI · Paper page
  48. Leevy JL, Khoshgoftaar TM, Bauder RA, Seliya N. A survey on addressing high-class imbalance in big data. J Big Data. 2018;5(1):42. · DOI · Paper page
  49. Rajaraman A, Ullman JD. Mining massive datasets. Cambridge: Cambridge University Press; 2011. · DOI · Paper page
  50. Buitinck L, Louppe G, Blondel M, Pedregosa F, Mueller A, Grisel O, Niculae V, Prettenhofer P, Gramfort A, Grobler J, et al. API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD workshop: languages for data mining and machine learning. 2013; pp. 108–122.
  51. Hand DJ. Measuring classifier performance: a coherent alternative to the area under the roc curve. Mach Learn. 2009;77(1):103–23. · DOI · Paper page
  52. Johnson R, Zhang T. Learning nonlinear functions using regularized greedy forest. IEEE Trans Pattern Anal Mach Intellig. 2014;36(5):942–54. · DOI
  53. Steptoe Andrew, Breeze Elizabeth, Banks James, Nazroo James. Cohort profile: the english longitudinal study of ageing. Int J Epidemiol. 2013;42(6):1640–8. · DOI · Paper page
  54. He H, Bai Y, Garcia EA, Li S. Adasyn: adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence). IEEE, New York. 2008; pp. 1322–8.
  55. Herland M, Bauder RA, Khoshgoftaar TM. The effects of class rarity on the evaluation of supervised healthcare fraud detection models. J Big Data. 2019;6(1):1. · DOI
  56. Witten IH, Frank E. Data mining: practical machine learning tools and techniques with java implementations. ACM Sigmod Record. 2002;31(1):76–7. · DOI · Paper page
  57. Holte RC. Very simple classification rules perform well on most commonly used datasets. Mach Learn. 1993;11(1):63–90. · DOI · Paper page
  58. Webb GI. Multiboosting: a technique for combining boosting and wagging. Mach Learn. 2000;40(2):159–96. · DOI · Paper page
  59. Barua S, Islam MM, Yao X, Murase K. Mwmote-majority weighted minority oversampling technique for imbalanced data set learning. IEEE Trans Knowl Data Eng. 2012;26(2):405–25. · DOI · Paper page
  60. Deng H, Runger G. Gene selection with guided regularized random forest. Pattern Recogn. 2013;46(12):3483–9. · DOI · Paper page
  61. Friedman JH. Stochastic gradient boosting. Nonlinear methods and data mining. Comput Stat Data Anal. 2002;38(4):367–78. · DOI · Paper page
  62. Madalina-Mihaela B, Javier T-A, Pedro C-R, Antonio G. Detection of non-technical losses using smart meter data and supervised learning. IEEE Trans Smart Grid. 2018;10(3):2661–70.
  63. Bauder R, da Rosa R, Khoshgoftaar TM. Identifying medicare provider fraud with unsupervised machine learning. In: 2018 IEEE international conference on information Reuse and integration (IRI). · DOI
  64. Bauder RA, Khoshgoftaar TM. Medicare fraud detection using machine learning methods. In: 2017 16th IEEE international conference on machine learning and applications (ICMLA). · DOI
  65. Hancock J, Khoshgoftaar TM. Medicare fraud detection using catboost. In: 2020 IEEE 21st international conference on information reuse and integration for data science (IRI). IEEE. 2020: pp. 97–103. · doi:10.1109/iri49571.2020.00022
  66. Hancock J, Khoshgoftaar TM. Performance of catboost and xgboost in medicare fraud detection. In: 19th IEEE international conference on machine learning and applications (ICMLA); IEEE, New York. 2020. · doi:10.1109/iri49571.2020.00022
  67. Hochreiter S, Schmidhuber J. Lstm can solve hard long time lag problems. In: Advances in neural information processing systems. 1997; pp. 473–9.
  68. Ilya S, Oriol V, Le Quoc V. Sequence to sequence learning with neural networks. Advances in neural information processing systems. 2014; pp. 3104–3112.
  69. De Myttenaere A, Golden B, Le Grand B, Rossi F. Mean absolute percentage error for regression models. Neurocomputing. 2016;192:38–48. · DOI · Paper page
  70. Goodfellow I, Bengio Y, Courville A. Deep learning. New York: MIT Press; 2016. http://www.deeplearningbook.org .
  71. Shvai N, Hasnat A, Meicler A, Nakib A. Accurate classification for automatic vehicle-type recognition based on ensemble classifiers. IEEE Trans Intell Transportation Syst. 2020;21(3):1288–97. · DOI
  72. Maryam MN, Taghi MK, Clifford Kemp, Seliya N, Richard Z. Machine learning for detecting brute force attacks at the network level. In 2014 IEEE International Conference on Bioinformatics and Bioengineering. IEEE, New York. 2014; pp. 379–385.
  73. Najafabadi MM, Khoshgoftaar TM, Napolitano A. Detecting network attacks based on behavioral commonalities. Int J Reliability Quality Safety Eng. 2016;23(01):1650005. · doi:10.1142/s0218539316500054
  74. Wald Randall, Khoshgoftaar Taghi M. Richard Zuech, and Amri Napolitano. Network traffic prediction models for near-and long-term predictions. In 2014 IEEE International Conference on Bioinformatics and Bioengineering, 2014; IEEE, New York. pp. 362–368. · doi:10.1109/bibe.2014.69
  75. Tanase C, Ogrezeanu I, Badiu C. Molecular pathology of pituitary adenomas. Netherlands: Elsevier; 2011.
  76. Fang Y, Fullwood MJ. Roles, functions, and mechanisms of long non-coding rnas in cancer. Genom Proteom Bioinf. 2016;14(1):42–54. · DOI · Paper page
  77. Brawerman G. Eukaryotic messenger RNA. Ann Rev Biochem. 1974;43(1):621–42. · DOI
  78. Ou M, Cui P, Pei J, Zhang Z, Zhu W. Asymmetric transitivity preserving graph embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 2016; p. 1105–1114. · DOI · Paper page
  79. Wang Y, Ma K, Garcia-Hernandez L, Chen J, Hou Z, Ji K, Chen Z, Abraham A. A clstm-tmn for marketing intention detection. Eng Appl Artificial Intell. 2020;91:103595. · DOI
  80. Sharma N, Aggarwal LM. Automated medical image segmentation techniques. J Med Phys Assoc Med Phys India. 2010;35(1):3.

Cited by

See all 1,814

Metadata

Authors
Hancock, J. T., & Khoshgoftaar, T. M.
Venue
Journal of Big Data
Publisher
Springer Science and Business Media LLC
Published
2020-11-04
Volume
7
Issue
1
Pages
94-94
ISSN
2196-1115 (ISSN-L)
DOI
10.1186/s40537-020-00369-8
OpenAlex
W3094948551
Cited by
1,814
References
15
FWCI
37.03
Type
journal article
Language
en
Open access
gold
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: CatBoost for big data: an interdisciplinary review. https://app.cochat.ai/papers/10.1186/s40537-020-00369-8
The paper
Hancock, J. T., & Khoshgoftaar, T. M. (2020). CatBoost for big data: an interdisciplinary review. Journal of Big Data. https://doi.org/10.1186/s40537-020-00369-8

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.