CoChat
Sign Up
Cited by 669Privacy-Preserving Technologies in DataCryptography and Data SecurityCloud Data Security Solutions

The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index; none of the 15 sources its claims cite is checked for support yet.

SecureBoost: A Lossless Federated Learning Framework

Open access
Cheng, Kewei, Fan, Tao, Jin, Yilun, Liu, Yang, Chen, Tianjian, Papadopoulos, Dimitrios, Yang, Qiang·2021·IEEE Intelligent Systems
DOI 10.1109/mis.2021.3082561
Open this paper in CoChat — free accountOpen PDF

Abstract

The protection of user privacy is an important concern in machine learning, as evidenced by the rolling out of the General Data Protection Regulation (GDPR) in the European Union (EU) in May 2018. The GDPR is designed to give users more control over their personal data, which motivates us to explore machine learning frameworks for data sharing that do not violate user privacy. To meet this goal, in this article, we propose a novel lossless privacy-preserving tree-boosting system known as SecureBoost in the setting of federated learning. SecureBoost first conducts entity alignment under a privacy-preserving protocol and then constructs boosting trees across multiple parties with a carefully designed encryption strategy. This federated learning system allows the learning process to be jointly conducted over multiple parties with common user samples but different feature sets, which corresponds to a vertically partitioned dataset. An advantage of SecureBoost is that it provides the same level of accuracy as the non -privacy-preserving approach while at the same time, reveals no information of each private data provider. We show that the SecureBoost framework is as accurate as other nonfederated gradient tree-boosting algorithms that require centralized data, and thus, it is highly scalable and practical for industrial applications such as credit risk analysis. To this end, we discuss information leakage during the protocol execution and propose ways to provably reduce it.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

SecureBoost is a vertical federated gradient tree-boosting framework that uses entity alignment and Paillier homomorphic encryption to train jointly across parties without sharing raw data or labels. The authors prove it is lossless relative to centralized XGBoost under identical initialization and hyperparameters, and propose RL-SecureBoost to reduce leakage from the first tree.

Key findings

  • SecureBoost is lossless: Theorem 1 states that SecureBoost model \(M\) and XGBoost model \(M'\) behave identically if identically initialized and hyperparameterized.
  • On Credit 1 (150,000 instances, 10 attributes) and Credit 2 (30,000 instances, 25 attributes), overall SecureBoost achieved ACC 0.9345, F1 0.2576, AUC 0.8461 on Credit 1 and ACC 0.8180, F1 0.4634, AUC 0.7701 on Credit 2; RL-SecureBoost achieved ACC 0.9331, F1 0.2549, AUC 0.8423 on Credit 1 and ACC 0.8179, F1 0.4650, AUC 0.7682 on Credit 2. A pairwise Wilcoxon signed-rank test at significance level 0.05 indicated RL-SecureBoost is as accurate as SecureBoost.
  • RL-SecureBoost reduced leakage: mean leaf purity dropped from 0.8058 in the first tree to 0.66663 in the second tree on Credit 1, and from 0.7159 to 0.638 on Credit 2; the second tree’s mean leaf purity was just over 0.6 on both datasets.
  • Scalability experiments showed runtime increased almost linearly with maximum tree depth, and sample and feature numbers contributed equally to running time.

Methods

  • Vertical federated learning setting: data are partitioned by features among \(m\) parties, only the active party holds labels, and parties share a common set of users. SecureBoost first performs privacy-preserving entity alignment via inter-database intersection, then collaboratively builds gradient-boosted trees using Paillier additive homomorphic encryption; passive parties compute encrypted gradient statistics locally, and the active party decrypts aggregated values to find global splits.
  • Experiments used two public credit datasets, with 2/3 for training and the remainder for testing, vertically split into two halves across two parties. All methods used maximum tree depth 3, subsample fraction 0.8, learning rate 0.3, and Paillier encryption with 512-bit keys on a machine with 8 GB RAM and Intel Core i5-7200u CPU.
  • Analyses compared convergence with GBDT and XGBoost, varied maximum depth over {3,4,5,6,7,8}, varied feature number over {50,500,1000,5000} and sample number over {5000,10000,30000}, and evaluated RL-SecureBoost versus SecureBoost using accuracy, F1-score, AUC, and a Wilcoxon signed-rank test.

Limitations

  • The paper states that the active party learns the instance space for each split, which party is responsible for each decision, and all possible values of \(g_l, g_r, h_l, h_r\) during learning; the latter can be avoided with secure multi-party computation for encrypted comparison, but this significantly affects learning efficiency.
  • The paper states that passive-party leakage is directly related to the first tree’s leaf purity, and Theorem 2 shows that the first tree’s leaf purity can be inferred from leaf weights. RL-SecureBoost stores decision-tree leaves at the active party and builds the first tree independently using only active-party features, but residual information may still reveal information, though it decreases as first-tree purity increases.
  • Evident from the text: the empirical evaluation is limited to two public credit datasets and a two-party vertical split, with no multi-party or non-credit-domain experiments reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/mis.2021.3082561

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 of 15 cited sources checked · 0 supported · 0 flagged

Show all 32 claims

  • It may state two separate claims as one. 10
  • Probably not a substantial claim. 3
12 of 12

  • The paper relies on this without support.

    Many data owners do not have sufficient amount of data to build high-quality models on their own.

    “In reality, many data owners do not have sufficient amount of data to build high-quality models.”

  • The paper relies on this without support.

    Different organizations need to collaborate while building machine learning models.

    “the need for different organizations to collaborate while building machine learning models still stays strong.”

  • The paper relies on this without support.

    It is still a challenge to allow different data owners to collaboratively build high-quality machine learning models while at the same time protecting user data privacy and confidentiality.

    “So far, it is still a challenge to allow different data owners to collaboratively build high-quality machine learning models while at the same time protecting user data privacy and confidentiality.”

  • The authors state this as an assumption.

    For Theorem 2, all initial predictions y_i(0) are initialized to a constant a with 0 < a < 1.

    “Suppose we initialize all ˆ y i (0) as a where 0 &lt; a &lt; 1 .”

  • The authors state this as an assumption.

    For Theorem 2, when the number of instances associated with a leaf n_j is relatively big, λ can be ignored in the leaf weight formula.

    “When n j is relatively big, we can ignore λ in -∑ i ∈ I j g i ∑ i ∈ I j h i + λ and rewrite the weight of leaf j as”

  • The authors state this as an assumption.

    For Theorem 3, when there are a large number of samples but a small number of leaf nodes k, Eq.(10) approximates Eq.(9).

    “When we have a large amount of samples but small number of leave nodes k , we can use the following equation to approximates Eq.( 9).”

  • The authors state this as an assumption.

    For Theorem 3, positive and negative samples are balanced (N_n = N_p).

    “When the positive samples and negative samples are balanced, N n = N p , we have”

  • The authors state this as an assumption.

    For Theorem 3, a is properly initialized, for instance a = 0.5.

    “When we have proper initialization of a , for instance a = 0 . 5 , θ j ∗ = 0 . 5 .”

  • The paper relies on this without support.

    Instances associated with the same leaf strongly indicate they belong to the same class.

    “Note that the instances that are associated with the same leaf strongly indicates they belong to the same class.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Denominator described as both negative and a sum of squares

    Minor

    The proof says the denominator must be negative, then identifies it as a sum of squared values, which cannot be negative.

  • Claim of revealing no provider information conflicts with acknowledged leakage

    Minor

    The abstract says SecureBoost reveals no information about each private data provider, while the security discussion states that the active party learns split-instance information and gradient statistics.

2 of 2

Citations

Cites

Flagged first, then by citations

Show all 29 references

Research works printed but not linked: 4

  1. yang, 2019, Federated Learning
  2. djatmiko, 2017, Privacy-preserving entity resolution and logistic regression on encrypted data, Private and Secure Machine Learning (PSML)
  3. mayer-schonberger, 2015, Regime change: Enabling big data through Europe’s new data protection regulation, Columbia Sci Technol Law Rev, 17, 315
  4. 10.1007/11535706_11 · DOI · Paper page

Cited by

Sorted by citations

Items 201–300 of 668

Related

See all

Metadata

Authors
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., & Yang, Q.
Affiliations
  • University of California, Los Angeles (US)
  • WeBank, Shenzhen, China
  • Hong Kong University of Science and Technology (HK)
ORCID
Venue
IEEE Intelligent Systems
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2021-11-01
Volume
36
Issue
6
Pages
87-98
ISSN
  • 1541-1672 (ISSN-L)
  • 1941-1294
DOI
10.1109/mis.2021.3082561
OpenAlex
W3164712068
Semantic Scholar
f71687796caa7d2a5d68406b53b51cb9ed57f586
Cited by
669
References
29
Citations by year
2026: 68 · 2025: 105 · 2024: 145 · 2023: 111 · 2022: 111 · 2021: 66
FWCI
38.11
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Intelligent Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Key Research and Development Program of China (2018AAA0101100)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Computer Networks and Communications › Cryptography and Data Security
  • Physical Sciences › Computer Science › Information Systems › Cloud Data Security Solutions
Keywords
federated learning, privacy-preserving machine learning, gradient boosting decision tree, gradient boosting, entity alignment, vertical partitioning, information leakage, GDPR, credit risk assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review