CoChat
Sign Up
Cited by 54,569Statistical Methods and InferenceMachine Learning and Data ClassificationData Mining Algorithms and Applications

XGBoost: A Scalable Tree Boosting System

Open access
Chen, Tianqi, Guestrin, Carlos·2016·Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
DOI 10.1145/2939672.2939785
Open this paper in CoChat — free accountOpen PDF

Abstract

Tree boosting is a highly effective and widely used machine learning method. In this paper, we describe a scalable end-to-end tree boosting system called XGBoost, which is used widely by data scientists to achieve state-of-the-art results on many machine learning challenges. We propose a novel sparsity-aware algorithm for sparse data and weighted quantile sketch for approximate tree learning. More importantly, we provide insights on cache access patterns, data compression and sharding to build a scalable tree boosting system. By combining these insights, XGBoost scales beyond billions of examples using far fewer resources than existing systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/2939672.2939785

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • XGBoost is optimized for fast, parallel tree construction and designed to be fault tolerant under a distributed setting.

    “The XGBoost classifier is also optimized for fast, parallel tree construction, and designed to be fault tolerant under the distributed setting [29].”

    From Prediction of Employee Turnover in Organizations using Machine Learning Algorithms (2016) · cites this paper as [29]

    The Introduction describes a fast, scalable system with a novel algorithm for parallel tree learning, and §6.1 confirms that XGBoost runs in distributed settings. Neither passage establishes that the distributed system is designed to be fault tolerant; a search for fault tolerance found no supporting passage in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Compared to other gradient boosted machines, XGBoost uses a more regularized-model formalization to control over-fitting, which gives it better performance.

    “Compared to other gradient boosted machines, it uses a more regularized-model formalization to control over-fitting, which gives it better performance.”

    From Prediction of Employee Turnover in Organizations using Machine Learning Algorithms (2016) · cites this paper as [29]

    Section 2.1 says XGBoost adds a regularization term to avoid overfitting, unlike traditional gradient tree boosting when that term is set to zero. Section 6.3 reports better performance than R’s GBM, but comparable performance to scikit-learn; it does not establish that regularization causes a general performance advantage over other gradient boosting machines. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Although the same XGBoost algorithm was implemented, each of the three studies used different risk predictors and datasets, resulting in widely disparate AUC values of 0.91, 0.83, and 0.679-11.

    “Although the same algorithm was implemented, each study used different risk predictors and datasets, which resulted in widely disparate AUC values of 0.91, 0.83, and 0.679–11.”

    From AI-based diabetes care: risk prediction models and implementation concerns (2024) · cites this paper as [8]

    The Introduction identifies XGBoost as the tree-boosting system, so the paper supports identifying the algorithm. Section 6.2 describes this paper’s own datasets, and Table 3 reports an AUC of 0.8304 for a Higgs-dataset experiment. Neither passage establishes the three cited studies’ predictors, their AUC values of 0.91, 0.83 and 0.679, or the proposed reason those values differ. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • The XGBoost algorithm was used in three unimodal studies.

    “For example, the XGBoost algorithm—a machine learning algorithm that uses a sequentially iterative process to learn and correct previous errors—was used in three unimodal studies8.”

    From AI-based diabetes care: risk prediction models and implementation concerns (2024) · cites this paper as [8]

  • The top 10 predictors accounted for 43.7% of overall model performance based on the gain metric in the Extreme Gradient Boosting algorithm, the top 25 predictors for 58.1%, the top 50 predictors for 70.3%, and the top 100 predictors for 81.4% of model performance.

    “The top 10 predictors accounted for 43.7% of overall model performance based on the gain metric in the Extreme Gradient Boosting algorithm (77), the top 25 predictors for 58.1%, the top 50 predictors for 70.3%, and the top 100 predictors for 81.4% of model performance.”

    From Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System (2020) · cites this paper as [77]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 21 references couldn’t be matched to a known paper.

Research works printed but not linked: 6

  1. J. Bennett and S. Lanning. The netflix prize. In Proceedings of the KDD Cup Workshop 2007, pages 3-6, New York, Aug. 2007.
  2. T. Chen, H. Li, Q. Yang, and Y. Yu. General functional matrix factorization using gradient boosting. In Proceeding of 30th International Conference on Machine Learning (ICML'13), volume 1, pages 436-444, 2013.
  3. T. Chen, S. Singh, B. Taskar, and C. Guestrin. Efficient second-order gradient boosting for conditional random fields. In Proceeding of 18th Artificial Intelligence and Statistics Conference (AISTATS'15), volume 1, 2015.
  4. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28(2):337-407, 2000.
  5. P. Li. Robust Logitboost and adaptive base class (ABC) Logitboost. In Proceedings of the Twenty-Sixth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI'10), pages 302-311, 2010.
  6. B. Panda, J. S. Herbach, S. Basu, and R. J. Bayardo. Planet: Massively parallel learning of tree ensembles with mapreduce. Proceeding of VLDB Endowment, 2(2):1426-1437, Aug. 2009.

Cited by

Sorted by citations

Related

Authors
Chen, T., & Guestrin, C.
Affiliations
University of Washington (US)
ORCID
Guestrin, Carlos
Venue
Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Publisher
ACM
Published
2016-08-13
Pages
785-794
DOI
10.1145/2939672.2939785
OpenAlex
W2295598076
Cited by
54,569
References
21
Citations by year
2026: 12394 · 2025: 11769 · 2024: 8715 · 2023: 6731 · 2022: 5369 · 2021: 4238 · 2020: 2665 · 2019: 1560 · 2018: 779 · 2017: 294 · 2016: 48 · 2015: 1 · 2012: 4
FWCI
936.68
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor, 365 d)
Locations
  • ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
Funders
  • Office of Naval Research (N000141010672)
  • National Science Foundation (1258741)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
  • Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
Keywords
XGBoost, gradient boosting decision tree, gradient boosting, data sparsity, data compression, data sharding, scalability, large-scale machine learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review