CoChat
Sign Up
PreprintCited by 668Privacy-Preserving Technologies in DataCryptography and Data SecurityCloud Data Security Solutions

SecureBoost: A Lossless Federated Learning Framework

Open access
Cheng, Kewei, Fan, Tao, Jin, Yilun, Liu, Yang, Chen, Tianjian, Papadopoulos, Dimitrios, Yang, Qiang·2019·arXiv
DOI 10.48550/arxiv.1901.08755arXiv 1901.08755
Open this paper in CoChat — free accountOpen PDF

Abstract

The protection of user privacy is an important concern in machine learning, as evidenced by the rolling out of the General Data Protection Regulation (GDPR) in the European Union (EU) in May 2018. The GDPR is designed to give users more control over their personal data, which motivates us to explore machine learning frameworks for data sharing that do not violate user privacy. To meet this goal, in this paper, we propose a novel lossless privacy-preserving tree-boosting system known as SecureBoost in the setting of federated learning. SecureBoost first conducts entity alignment under a privacy-preserving protocol and then constructs boosting trees across multiple parties with a carefully designed encryption strategy. This federated learning system allows the learning process to be jointly conducted over multiple parties with common user samples but different feature sets, which corresponds to a vertically partitioned data set. An advantage of SecureBoost is that it provides the same level of accuracy as the non-privacy-preserving approach while at the same time, reveals no information of each private data provider. We show that the SecureBoost framework is as accurate as other non-federated gradient tree-boosting algorithms that require centralized data and thus it is highly scalable and practical for industrial applications such as credit risk analysis. To this end, we discuss information leakage during the protocol execution and propose ways to provably reduce it.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1901.08755

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Differential privacy only protects users to a limited degree and can reduce prediction accuracy because it is lossy.

    “However, DP only protects users from data leakage to a certain extent and may reduce performance in prediction accuracy because it is a lossy method [18].”

    From Federated Learning for Healthcare Informatics (2021) · cites this paper as [18]

  • Combining differential privacy with secure multi-party computation reduces noise growth as parties increase without sacrificing privacy.

    “Thus, some researchers combine DP with SMC to reduce the growth of noise injection as the number of parties increases without sacrificing privacy while preserving provable privacy guarantees, protecting against extraction attacks and collusion threats [18, 100].”

    From Federated Learning for Healthcare Informatics (2021) · cites this paper as [18]

    The Introduction (p. 1) describes noise injection in differential privacy, and the Security Discussion (p. 5) mentions secure multi-party computation as a way to limit information leakage. Neither passage says the methods are combined or that combining them reduces noise growth as the number of parties increases. Searches for that combination, noise growth, and party count found no supporting passage. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

  • Hyperparameter optimization (HPO) has a long history under AutoML, but it mainly concerns improving model accuracy rather than communication and computing efficacy for mobile devices.

    “Hyperparameter optimization (HPO) has a long history under the framework of AutoML [395, 273, 277], but it mainly concerns how to improve the model accuracy [64, 426, 374, 180] rather than communication and computing efficacy for mobile devices.”

    From Advances and Open Problems in Federated Learning (2021) · cites this paper as [64]

    The Introduction (pp. 1–2) and Preliminaries and Related Work (p. 2) discuss federated learning and mobile terminals, not the history or objectives of hyperparameter optimization under AutoML. Searches for HPO, model accuracy, and communication or computing efficiency for mobile devices found no passage addressing that comparison. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 30

  1. [2][2] J. P. Albrecht, 'How the gdpr will change the world,' Eur. Data Prot. L. Rev. , vol. 2, p. 287, 2016. -
  2. [3][3] V. Mayer-Schonberger and Y. Padova, 'Regime change: Enabling big data through europe's new data protection regulation,' Colum. Sci. & Tech. L. Rev. , vol. 17, p. 315, 2015. -
  3. [4][4] B. Goodman and S. Flaxman, 'European union regulations on algorithmic decision-making and a' right to explanation',' arXiv preprint arXiv:1606.08813 , 2016. - · arXiv
  4. [5][5] S. Hardy, W. Henecka, H. Ivey-Law, R. Nock, G. Patrini, G. Smith, and B. Thorne, 'Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption,' arXiv preprint arXiv:1711.10677 , 2017. - · arXiv
  5. [6][6] P. Mohassel and Y. Zhang, 'Secureml: A system for scalable privacypreserving machine learning,' in 2017 38th IEEE Symposium on Security and Privacy (SP) . IEEE, 2017, pp. 19-38. -
  6. [7][7] C. Dwork, A. Roth et al. , 'The algorithmic foundations of differential privacy,' Foundations and Trends® in Theoretical Computer Science , vol. 9, no. 3-4, pp. 211-407, 2014. -
  7. [8][8] C. Dwork, 'Differential privacy: A survey of results,' in International Conference on Theory and Applications of Models of Computation . Springer, 2008, pp. 1-19. -
  8. [9][9] J. Koneˇ cn` y, H. B. McMahan, F. X. Yu, P. Richt´ arik, A. T. Suresh, and D. Bacon, 'Federated learning: Strategies for improving communication efficiency,' arXiv preprint arXiv:1610.05492 , 2016. - · arXiv
  9. [10][10] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, 'Practical secure aggregation for privacy-preserving machine learning,' in CCS , 2017, pp. 1175-1191. -
  10. [11][11] Q. Yang, Y. Liu, T. Chen, and Y. Tong, 'Federated machine learning: Concept and applications,' ACM Transactions on Intelligent Systems and Technology , vol. 10, no. 2, pp. 12:1-12:19, 2019. -
  11. [12][12] Q. Yang, Y. Liu, Y. Cheng, Y. Kang, T. Chen, and H. Yu, 'Federated learning,' Synthesis Lectures on Artificial Intelligence and Machine Learning , vol. 13, no. 3, pp. 1-207, 2019. -
  12. [13][13] Y. Liu, Y. Kang, X. Zhang, L. Li, Y. Cheng, T. Chen, M. Hong, and Q. Yang, 'A communication efficient vertical federated learning framework,' CoRR , vol. abs/1912.11187, 2019. [Online]. Available: http://arxiv.org/abs/1912.11187 - · arXiv
  13. [15][15] J. Vaidya, 'A survey of privacy-preserving methods across vertically partitioned data,' in Privacy-preserving data mining . Springer, 2008, pp. 337-358. -
  14. [16][16] J. Vaidya and C. Clifton, 'Privacy-preserving decision trees over vertically partitioned data,' in IFIP Annual Conference on Data and Applications Security and Privacy . Springer, 2005, pp. 139-152. -
  15. [17][17] KDD '16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . New York, NY, USA: Association for Computing Machinery, 2016. -
  16. [18][18] R. Shokri and V. Shmatikov, 'Privacy-preserving deep learning,' in Proceedings of the 22nd ACM SIGSAC conference on computer and communications security . ACM, 2015, pp. 1310-1321. -
  17. [20][20] L. Zhao, L. Ni, S. Hu, Y. Chen, P. Zhou, F. Xiao, and L. Wu, 'Inprivate digging: Enabling tree-based distributed data mining with differential privacy,' in IEEE INFOCOM 2018-IEEE Conference on Computer Communications . IEEE, 2018, pp. 2087-2095. -
  18. [21][21] Q. Li, Z. Wen, and B. He, 'Practical federated gradient boosting decision trees,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, no. 04, 2020, pp. 4642-4649. -
  19. [22][22] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, 'Federated optimization in heterogeneous networks,' arXiv preprint arXiv:1812.06127 , 2018. - · arXiv
  20. [23][23] F. Hanzely, S. Hanzely, S. Horv´ ath, and P. Richt´ arik, 'Lower bounds and optimal algorithms for personalized federated learning,' arXiv preprint arXiv:2010.02372 , 2020. - · arXiv
  21. [24][24] M. Mohri, G. Sivek, and A. T. Suresh, 'Agnostic federated learning,' in International Conference on Machine Learning . PMLR, 2019, pp. 4615-4625. -
  22. [25][25] M. Yurochkin, M. Agarwal, S. Ghosh, K. Greenewald, N. Hoang, and Y. Khazaeni, 'Bayesian nonparametric federated learning of neural networks,' in International Conference on Machine Learning . PMLR, 2019, pp. 7252-7261. -
  23. [26][26] H. Wang, M. Yurochkin, Y. Sun, D. Papailiopoulos, and Y. Khazaeni, 'Federated learning with matched averaging,' arXiv preprint arXiv:2002.06440 , 2020. - · arXiv
  24. [27][27] Y. Liu, Y. Kang, C. Xing, T. Chen, and Q. Yang, 'A secure federated transfer learning framework,' IEEE Intelligent Systems , vol. 35, no. 4, pp. 70-82, 2020. -
  25. [28][28] J. Vaidya, C. Clifton, M. Kantarcioglu, and A. S. Patterson, 'Privacypreserving decision trees over vertically partitioned data,' ACM Transactions on Knowledge Discovery from Data (TKDD) , vol. 2, no. 3, p. 14, 2008. -
  26. [30][30] G. Liang and S. S. Chawathe, 'Privacy-preserving inter-database operations,' in International Conference on Intelligence and Security Informatics . Springer, 2004, pp. 66-82. -
  27. [31][31] T. Chen and C. Guestrin, 'Xgboost: A scalable tree boosting system,' in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining . ACM, 2016, pp. 785-794. -
  28. [32][32] P. Paillier, 'Public-key cryptosystems based on composite degree residuosity classes,' in International Conference on the Theory and Applications of Cryptographic Techniques . Springer, 1999, pp. 223-238. -
  29. [33][33] R. Bost, R. A. Popa, S. Tu, and S. Goldwasser, 'Machine learning classification over encrypted data,' in 22nd Annual Network and Distributed System Security Symposium, NDSS , 2015. -
  30. [34][34] F. Baldimtsi, D. Papadopoulos, S. Papadopoulos, A. Scafuro, and N. Triandopoulos, 'Server-aided secure computation with off-line parties,' in Computer Security - ESORICS 2017 - 22nd European Symposium on Research in Computer Security, Proceedings, Part I , 2017, pp. 103123.

Cited by

Sorted by citations

Authors
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., & Yang, Q.
Affiliations
  • University of California, Los Angeles (US)
  • Hong Kong University of Science and Technology (HK)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
Volume
36
Issue
6
Pages
87-98
ISSN
1541-1672 (ISSN-L)
DOI
10.48550/arxiv.1901.08755
arXiv
1901.08755
OpenAlex
W3164712068
Cited by
668
References
1
Citations by year
2026: 67 · 2025: 105 · 2024: 145 · 2023: 111 · 2022: 111 · 2021: 66 · 2020: 49 · 2019: 14
FWCI
38.25
Type
journal article
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • IEEE Intelligent Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Key Research and Development Program of China (2018AAA0101100)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Cryptography and Data Security
  • Physical Sciences › Computer Science › Information Systems › Cloud Data Security Solutions
Keywords
federated learning, privacy-preserving machine learning, gradient boosting decision tree, gradient boosting, entity alignment, vertical partitioning, information leakage, GDPR, credit risk assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review