CoChat
Sign Up
Cited by 669Privacy-Preserving Technologies in DataCryptography and Data SecurityCloud Data Security Solutions

SecureBoost: A Lossless Federated Learning Framework

Open access
Cheng, Kewei, Fan, Tao, Jin, Yilun, Liu, Yang, Chen, Tianjian, Papadopoulos, Dimitrios, Yang, Qiang·2021·IEEE Intelligent Systems
DOI 10.1109/mis.2021.3082561
Open this paper in CoChat — free accountOpen PDF

Abstract

The protection of user privacy is an important concern in machine learning, as evidenced by the rolling out of the General Data Protection Regulation (GDPR) in the European Union (EU) in May 2018. The GDPR is designed to give users more control over their personal data, which motivates us to explore machine learning frameworks for data sharing that do not violate user privacy. To meet this goal, in this article, we propose a novel lossless privacy-preserving tree-boosting system known as SecureBoost in the setting of federated learning. SecureBoost first conducts entity alignment under a privacy-preserving protocol and then constructs boosting trees across multiple parties with a carefully designed encryption strategy. This federated learning system allows the learning process to be jointly conducted over multiple parties with common user samples but different feature sets, which corresponds to a vertically partitioned dataset. An advantage of SecureBoost is that it provides the same level of accuracy as the non -privacy-preserving approach while at the same time, reveals no information of each private data provider. We show that the SecureBoost framework is as accurate as other nonfederated gradient tree-boosting algorithms that require centralized data, and thus, it is highly scalable and practical for industrial applications such as credit risk analysis. To this end, we discuss information leakage during the protocol execution and propose ways to provably reduce it.

AI snapshot

Written from the paper’s full text.

TL;DR

SecureBoost is a vertical federated gradient tree-boosting framework that uses entity alignment and Paillier homomorphic encryption to train jointly across parties without sharing raw data or labels. The authors prove it is lossless relative to centralized XGBoost under identical initialization and hyperparameters, and propose RL-SecureBoost to reduce leakage from the first tree.

Key findings

  • SecureBoost is lossless: Theorem 1 states that SecureBoost model \(M\) and XGBoost model \(M'\) behave identically if identically initialized and hyperparameterized.
  • On Credit 1 (150,000 instances, 10 attributes) and Credit 2 (30,000 instances, 25 attributes), overall SecureBoost achieved ACC 0.9345, F1 0.2576, AUC 0.8461 on Credit 1 and ACC 0.8180, F1 0.4634, AUC 0.7701 on Credit 2; RL-SecureBoost achieved ACC 0.9331, F1 0.2549, AUC 0.8423 on Credit 1 and ACC 0.8179, F1 0.4650, AUC 0.7682 on Credit 2. A pairwise Wilcoxon signed-rank test at significance level 0.05 indicated RL-SecureBoost is as accurate as SecureBoost.
  • RL-SecureBoost reduced leakage: mean leaf purity dropped from 0.8058 in the first tree to 0.66663 in the second tree on Credit 1, and from 0.7159 to 0.638 on Credit 2; the second tree’s mean leaf purity was just over 0.6 on both datasets.
  • Scalability experiments showed runtime increased almost linearly with maximum tree depth, and sample and feature numbers contributed equally to running time.

Methods

  • Vertical federated learning setting: data are partitioned by features among \(m\) parties, only the active party holds labels, and parties share a common set of users. SecureBoost first performs privacy-preserving entity alignment via inter-database intersection, then collaboratively builds gradient-boosted trees using Paillier additive homomorphic encryption; passive parties compute encrypted gradient statistics locally, and the active party decrypts aggregated values to find global splits.
  • Experiments used two public credit datasets, with 2/3 for training and the remainder for testing, vertically split into two halves across two parties. All methods used maximum tree depth 3, subsample fraction 0.8, learning rate 0.3, and Paillier encryption with 512-bit keys on a machine with 8 GB RAM and Intel Core i5-7200u CPU.
  • Analyses compared convergence with GBDT and XGBoost, varied maximum depth over {3,4,5,6,7,8}, varied feature number over {50,500,1000,5000} and sample number over {5000,10000,30000}, and evaluated RL-SecureBoost versus SecureBoost using accuracy, F1-score, AUC, and a Wilcoxon signed-rank test.

Limitations

  • The paper states that the active party learns the instance space for each split, which party is responsible for each decision, and all possible values of \(g_l, g_r, h_l, h_r\) during learning; the latter can be avoided with secure multi-party computation for encrypted comparison, but this significantly affects learning efficiency.
  • The paper states that passive-party leakage is directly related to the first tree’s leaf purity, and Theorem 2 shows that the first tree’s leaf purity can be inferred from leaf weights. RL-SecureBoost stores decision-tree leaves at the active party and builds the first tree independently using only active-party features, but residual information may still reveal information, though it decreases as first-tree purity increases.
  • Evident from the text: the empirical evaluation is limited to two public credit datasets and a two-party vertical split, with no multi-party or non-credit-domain experiments reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/mis.2021.3082561

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 15 cited sources checked · 0 supported · 0 flagged

  • Findingown data
    § 8.1 Scalability · p. 7

    Sample and feature numbers contribute equally to running time.

    “We make similar observations on both Figure 5 (b) and Figure 5 (c), which imply that sample and feature numbers contribute equally to running time.”
    Quote in the paperNot a substantial claim
  • Findingown data
    § 7 SECURITY DISCUSSION · p. 7

    RL-SecureBoost performs identically to SecureBoost in terms of prediction accuracy.

    “Moreover, as we experimentally demonstrate in Section 8, RL-SecureBoost performs identically as SecureBoost in terms of prediction accuracy.”
    • Also stated in: § 8.2 Performance of RL-SecureBoost · p. 7 · hedged
      “As observed, RL-SecureBoost performs equally well compared to SecureBoost in almost all cases.”
    • Also stated in: § 8.2 Performance of RL-SecureBoost · p. 7 · hedged
      “The comparison results indicate that RL-SecureBoost is as accurate as SecureBoost, with a significance level of 0 . 05 .”
    Quote in the paperSubstantial claim
  • Findingown data
    § 8.1 Scalability · p. 7

    The runtime increases almost linearly with the maximum depth of individual trees.

    “As depicted in Figure 5 (a), the runtime increases almost linearly with the maximum depth of individual trees”
    Quote in the paperSubstantial claim
  • Findingown data
    § 8.2 Performance of RL-SecureBoost · p. 7

    The mean leaf purity of the second tree is just over 0.6 on both datasets.

    “Moreover, the mean leaf purity of the second tree is just over 0 . 6 on both datasets”
    Quote in the paperSubstantial claim
  • Findingown data
    § 8.2 Performance of RL-SecureBoost · p. 7

    The mean leaf purity decreases significantly from the first to the second tree on both datasets.

    “According to Table 1, the mean leaf purity decreases significantly from the first to the second tree on both datasets”
    Quote in the paperSubstantial claim
  • May combine two claims. 10
  • Classified as probably not a substantial claim. 3

  • May combine two claims.

    Findingown data
    § 8.1 Scalability · p. 7

    SecureBoost shows a similar learning curve to other non-federated methods on the training dataset and even performs slightly better than others on the test dataset.

    “As can be observed from Figure 4, SecureBoost shows a similar learning curve with other non-federated methods on the training dataset and even performs slightly better than others on the test dataset.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 7 SECURITY DISCUSSION · p. 5

    Using secure multi-party computation for comparison of encrypted values, the active party learns only the optimal g_l, g_r, h_l, and h_r per party, and this significantly affects the efficiency during learning.

    “In this way, the active party learns only the optimal g l , g r , h l , h r per party; on the other hand, this significantly affects the efficiency during learning.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given
    § 1 INTRODUCTION · p. 1

    Data exchange under differential privacy still requires that data change hands between organizations, which may not be allowed by strict laws like GDPR.

    “In addition, data exchange under DP still requires that data change hands between organizations, which may not be allowed by strict laws like GDPR.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § SecureBoost: A Lossless Federated Learning Framework · p. 1

    SecureBoost is highly scalable and practical for industrial applications such as credit risk analysis.

    “We show that the SecureBoost framework is as accurate as other non-federated gradient tree-boosting algorithms that require centralized data and thus it is highly scalable and practical for industrial applications such as credit risk analysis.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 2 PRELIMINARIES AND RELATED WORK · p. 2

    SecureBoost can be easily applied to continuous data.

    “In contrast, our method guarantees better protection to the data and can be easily applied to continuous data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 1 INTRODUCTION · p. 1

    Differential privacy only prevents user-data leakage to a certain degree and cannot completely rule out the identity of an individual.

    “However, DP only prevents user-data leakage to a certain degree and cannot completely rule out the identity of an individual.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [7] [7] C. Dwork, A. Roth et al. , 'The algorithmic foundations of differential privacy,' Foundations and Trends® in Theoretical Computer Science , vol. 9, no. 3-4, pp. 211-407, 2014. -

      Not checked in the free run.

    • cites [8] [8] C. Dwork, 'Differential privacy: A survey of results,' in International Conference on Theory and Applications of Models of Computation . Springer, 2008, pp. 1-19. -

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 7 SECURITY DISCUSSION · p. 5

    The active party learning the instance space and which party is responsible for each decision seems unavoidable in this setting, unless one is willing to severely increase the overhead during the inference phase.

    “The former seems unavoidable in this setting, unless one is willing to severely increase the overhead during the inference phase.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 7 SECURITY DISCUSSION · p. 6

    With RL-SecureBoost, the active party learns the first tree independently based only on its own features, which fully protects the instance space of its leaves.

    “With RL-SecureBoost , the active party learns the first tree independently based only on its own features which fully protects the instance space of its leaves.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given
    § 7 SECURITY DISCUSSION · p. 5

    SecureBoost avoids revealing data records held by each party to others during training and inference, thus protecting the privacy of individual parties' data.

    “SecureBoost avoids revealing data records held by each of the parties to others during training and inference thus protecting the privacy of individual parties' data.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 1 INTRODUCTION · p. 1

    The differential privacy method is lossy in machine learning in that models built after noise is injected may perform unsatisfactorily in prediction accuracy.

    “Furthermore, the DP method is lossy in machine learning in that models built after noise is injected may perform unsatisfactorily in prediction accuracy.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § 7 SECURITY DISCUSSION · p. 5

    In SecureBoost, the active party learns the instance space for each split and which party is responsible for the decision at each node.

    “The active party is in an advantageous position with SecureBoost as it learns the instance space for each split and which party is responsible for the decision at each node.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 7 SECURITY DISCUSSION · p. 5

    There is some leakage that can be inferred during the protocol execution, and it is different for passive versus active parties.

    “However, we stress that there is some leakage that can be inferred during the protocol execution which is quite different for passive vs. active parties.”
    Quote in the paperSubstantial claim
12 of 12

  • The paper relies on this without support.

    Many data owners do not have sufficient amount of data to build high-quality models on their own.

    “In reality, many data owners do not have sufficient amount of data to build high-quality models.”

  • The paper relies on this without support.

    Different organizations need to collaborate while building machine learning models.

    “the need for different organizations to collaborate while building machine learning models still stays strong.”

  • The paper relies on this without support.

    It is still a challenge to allow different data owners to collaboratively build high-quality machine learning models while at the same time protecting user data privacy and confidentiality.

    “So far, it is still a challenge to allow different data owners to collaboratively build high-quality machine learning models while at the same time protecting user data privacy and confidentiality.”

  • The authors state this as an assumption.

    For Theorem 2, all initial predictions y_i(0) are initialized to a constant a with 0 < a < 1.

    “Suppose we initialize all ˆ y i (0) as a where 0 &lt; a &lt; 1 .”

  • The authors state this as an assumption.

    For Theorem 2, when the number of instances associated with a leaf n_j is relatively big, λ can be ignored in the leaf weight formula.

    “When n j is relatively big, we can ignore λ in -∑ i ∈ I j g i ∑ i ∈ I j h i + λ and rewrite the weight of leaf j as”

  • The authors state this as an assumption.

    For Theorem 3, when there are a large number of samples but a small number of leaf nodes k, Eq.(10) approximates Eq.(9).

    “When we have a large amount of samples but small number of leave nodes k , we can use the following equation to approximates Eq.( 9).”

  • The authors state this as an assumption.

    For Theorem 3, positive and negative samples are balanced (N_n = N_p).

    “When the positive samples and negative samples are balanced, N n = N p , we have”

  • The authors state this as an assumption.

    For Theorem 3, a is properly initialized, for instance a = 0.5.

    “When we have proper initialization of a , for instance a = 0 . 5 , θ j ∗ = 0 . 5 .”

  • The paper relies on this without support.

    Instances associated with the same leaf strongly indicate they belong to the same class.

    “Note that the instances that are associated with the same leaf strongly indicates they belong to the same class.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A recent work proposed a more practical data-free approach named FedSyn for one-shot FL framework with heterogeneity.

    “A recent work proposed a more practical data-free approach named FedSyn for one-shot FL framework with heterogeneity [191].”

    From Privacy and Robustness in Federated Learning: Attacks and Defenses (2024) · cites this paper as [191]

  • FedSyn is the first method that can be practically applied to various real-world applications because it requires no additional information except model parameters to be transferred, does not require an auxiliary dataset, and is the first to consider both model and statistical heterogeneities in FL.

    “FedSyn is the first method that can be practically applied to various real-world applications due to the following benefits: (1) FedSyn requires no additional information (except the model parameters) to be transferred between participants and the server; (2) FedSyn does not require any auxiliary dataset for training; (3) FedSyn is the first to consider both model and statistical heterogeneities”

    From Privacy and Robustness in Federated Learning: Attacks and Defenses (2024) · cites this paper as [191]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Denominator described as both negative and a sum of squares

    Minor

    The proof says the denominator must be negative, then identifies it as a sum of squared values, which cannot be negative.

  • Claim of revealing no provider information conflicts with acknowledged leakage

    Minor

    The abstract says SecureBoost reveals no information about each private data provider, while the security discussion states that the active party learns split-instance information and gradient statistics.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. yang, 2019, Federated Learning
  2. djatmiko, 2017, Privacy-preserving entity resolution and logistic regression on encrypted data, Private and Secure Machine Learning (PSML)
  3. mayer-schonberger, 2015, Regime change: Enabling big data through Europe&#x2019;s new data protection regulation, Columbia Sci Technol Law Rev, 17, 315
  4. 10.1007/11535706_11 · DOI · Paper page

Cited by

Sorted by citations

Related

Authors
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., & Yang, Q.
Affiliations
  • University of California, Los Angeles (US)
  • WeBank, Shenzhen, China
  • Hong Kong University of Science and Technology (HK)
ORCID
Venue
IEEE Intelligent Systems
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2021-11-01
Volume
36
Issue
6
Pages
87-98
ISSN
  • 1541-1672 (ISSN-L)
  • 1941-1294
DOI
10.1109/mis.2021.3082561
OpenAlex
W3164712068
Semantic Scholar
f71687796caa7d2a5d68406b53b51cb9ed57f586
Cited by
669
References
29
Citations by year
2026: 68 · 2025: 105 · 2024: 145 · 2023: 111 · 2022: 111 · 2021: 66 · 2020: 49 · 2019: 14
FWCI
38.11
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Intelligent Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Key Research and Development Program of China (2018AAA0101100)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Cryptography and Data Security
  • Physical Sciences › Computer Science › Information Systems › Cloud Data Security Solutions
Keywords
federated learning, privacy-preserving machine learning, gradient boosting decision tree, gradient boosting, entity alignment, vertical partitioning, information leakage, GDPR, credit risk assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review