Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
In a lot of real-world data mining and machine learning applications, data are provided by multiple providers and each maintains private records of different feature sets about common entities. It is challenging to train these vertically partitioned data effectively and efficiently while keeping data privacy for traditional data mining and machine learning algorithms. In this paper, we focus on nonlinear learning with kernels,and propose a federated doubly stochastic kernel learning (FDSKL) algorithm for vertically partitioned data. Specifically, we use random features to approximate the kernel mapping function and use doubly stochastic gradients to update the solutions, which are all computed federatedly without the disclosure of data. Importantly, we prove that FDSKL has a sublinear convergence rate, and can guarantee the data security under the semi-honest assumption. Extensive experimental results on a variety of benchmark datasets show that FDSKL is significantly faster than state-of-the-art federated learning methods when dealing with kernels, while retaining the similar generalization performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Vertical FL, on the other hand, can directly learn the global model from the partitioned data features among different edge devices without sharing them [156].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [156]
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · cited 6×
“For the oracle learners, we use state-of-the-art kernel classification solvers, including LIBSVM (Chang and Lin, 2011) and DSG (Dai et al., 2014).”
BackgroundMethodInfluentialReproducing Kernel Hilbert Spaces in Probability and Statistics
Berlinet, A., & Thomas-Agnan, C. (2004) · Springer eBooks · doi:10.1007/978-1-4419-9096-9 · Cited by 1,558 · cited 1×
“which is convex with respect to u2R. We give several common loss functions in Table 1. Given a positive de- nite kernel function K(x0;x) and the associated reproducing kernel Hilbert spaces (RKHS) H(Berlinet and Thomas-Agnan, 2011). A kernel method tries to nd a function f2H1 for solving: argmin f2H R(f) = E (x;y)2SL(…”
(2019) · SGEM International Multidisciplinary Scientific Conferences on SOCIAL SCIENCES and ARTS Proceedings · doi:10.5593/sgemsocial2019v/1.1 · Cited by 1,298
Hogwild: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Niu, F., Recht, B., Re, C., & Wright, S. J. (2011) · arXiv · doi:10.48550/arxiv.1106.5730 · Cited by 1,211 · cited 2×
“s. Note that, dierent from the proof of (Dai et al., 2014), we obtain the Lemma 3 by the summation formula of geometric progression and the Lemma 4 inspired by the proof of the recursive formula in (Recht et al., 2011). Firstly, we give the convergence analysis of error due to random feature. Let D t denote the subset…”
MethodInfluentialPrivacy preserving association rule mining in vertically partitioned data
Vaidya, J., & Clifton, C. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775142 · Cited by 1,001 · cited 2×
“…Du and Atallah (2001), linear regression (Gasc´on et al., 2016; Karr et al., 2009; Sanil et al., 2004; Gasc´on et al., 2017), association rule-mining (Vaidya and Clifton, 2002), k-means clustering (Vaidya and Clifton, 2003), logistic regression (Hardy et al., 2017; Nock et al., 2018), random…”
BackgroundSecureBoost: A Lossless Federated Learning Framework
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., & Yang, Q. (2019) · arXiv · doi:10.48550/arxiv.1901.08755 · Cited by 668 · cited 1×
“(2019), XGBoost Cheng et al. (2019) and support vector machine Yu et al.”
BackgroundPrivacy-preserving k -means clustering over vertically partitioned data
Vaidya, J., & Clifton, C. (2003) · Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/956750.956776 · Cited by 626
Parallel distributed computing using Python
Dalcin, L. D., Paz, R. R., Kler, P. A., & Cosimo, A. (2011) · Advances in Water Resources · doi:10.1016/j.advwatres.2011.04.013 · Cited by 590 · cited 2×
“We implemented our FDSKL in python, where the parallel computation was handled via MPI4py [6].”
MethodWeighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning
Rahimi, A., & Recht, B. (2008) · Cited by 558
Privacy-preserving distributed linear regression on high-dimensional data
Gascón, A., Schoppmann, P., Balle, B., Raykova, M., Doerner, J., Zahur, S., & Evans, D. (2017) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2017-0053 · Cited by 193 · cited 2×
“In the literature, there are many privacy-preserving federated learning algorithms for vertically partitioned data in various applications, for example, cooperative statistical analysis [7], linear regression [9, 10, 14, 21], association rule-mining [25], k-means clustering [26], logistic regression [13, 16], random f…”
Background
Research works printed but not linked: 10
- Nadeem Badshah . 2018. Facebook to contact 87 million users affected by data breach. The Guardian) Retrieved from https://www. theguardian. com/technology/2018/apr/08/facebook-to-contact-the-87-million-users-affected-by-data-breach ( 2018 ). Nadeem Badshah. 2018. Facebook to contact 87 million users affected by data breach. The Guardian) Retrieved from https://www. theguardian. com/technology/2018/apr/08/facebook-to-contact-the-87-million-users-affected-by-data-breach (2018).
- Wenliang Du and Mikhail J Atallah . 2001 . Privacy-preserving cooperative statistical analysis . In Seventeenth Annual Computer Security Applications Conference. IEEE, 102--110 . Wenliang Du and Mikhail J Atallah. 2001. Privacy-preserving cooperative statistical analysis. In Seventeenth Annual Computer Security Applications Conference. IEEE, 102--110.
- Bin Gu , Zhouyuan Huo , and Heng Huang . 2018 . Asynchronous Doubly Stochastic Group Regularized Learning. In International Conference on Artificial Intelligence and Statistics. 1791--1800 . Bin Gu, Zhouyuan Huo, and Heng Huang. 2018. Asynchronous Doubly Stochastic Group Regularized Learning. In International Conference on Artificial Intelligence and Statistics. 1791--1800.
- Stephen Hardy , Wilko Henecka , Hamish Ivey-Law , Richard Nock , Giorgio Patrini , Guillaume Smith , and Brian Thorne . 2017. Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. arXiv preprint arXiv:1711.10677 ( 2017 ). Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guillaume Smith, and Brian Thorne. 2017. Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. arXiv preprint arXiv:1711.10677 (2017). · arXiv
- Yang Liu , Yingting Liu , Zhijie Liu , Junbo Zhang , Chuishi Meng , and Yu Zheng . 2019. Federated Forest . arXiv preprint arXiv:1905.10053 ( 2019 ). Yang Liu, Yingting Liu, Zhijie Liu, Junbo Zhang, Chuishi Meng, and Yu Zheng. 2019. Federated Forest. arXiv preprint arXiv:1905.10053 (2019). · arXiv
- Richard Nock , Stephen Hardy , Wilko Henecka , Hamish Ivey-Law , Giorgio Patrini , Guillaume Smith , and Brian Thorne . 2018. Entity Resolution and Federated Learning get a Federated Resolution. arXiv preprint arXiv:1803.04035 ( 2018 ). Richard Nock, Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Giorgio Patrini, Guillaume Smith, and Brian Thorne. 2018. Entity Resolution and Federated Learning get a Federated Resolution. arXiv preprint arXiv:1803.04035 (2018). · arXiv
- Ali Rahimi and Benjamin Recht. 2008. Random features for large-scale kernel machines. In Advances in neural information processing systems. 1177--1184. Ali Rahimi and Benjamin Recht. 2008. Random features for large-scale kernel machines. In Advances in neural information processing systems. 1177--1184.
- Walter Rudin . 1962. Fourier analysis on groups. Vol ., Vol. 121967 . Wiley Online Library . Walter Rudin. 1962. Fourier analysis on groups. Vol., Vol. 121967. Wiley Online Library.
- Wanli Shi , Bin Gu , Xiang Li , and Heng Huang . 2020 . Quadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization. In The Thirty-Fourth AAAI Conference on Artificial Intelligence , New York, NY, USA , February 7-12, 2020. AAAI Press, 5734--5741. Wanli Shi, Bin Gu, Xiang Li, and Heng Huang. 2020. Quadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, New York, NY, USA, February 7-12, 2020. AAAI Press, 5734--5741.
- Gong-Duo Zhang , Shen-Yi Zhao , Hao Gao , and Wu-Jun Li. 2018. Feature-Distributed SVRG for High-Dimensional Linear Classification. arXiv preprint arXiv:1802.03604 ( 2018 ). Gong-Duo Zhang, Shen-Yi Zhao, Hao Gao, and Wu-Jun Li. 2018. Feature-Distributed SVRG for High-Dimensional Linear Classification. arXiv preprint arXiv:1802.03604 (2018). · arXiv
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“Vertical FL, on the other hand, can directly learn the global model from the partitioned data features among different edge devices without sharing them [156].”
BackgroundTrustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“In Reference [138], an efficient method of kernel learning was proposed.”
MethodVertical Federated Learning: Concepts, Advances, and Challenges
Liu, Y., Kang, Y., Zou, T., Pu, Y., He, Y., Ye, X., Ouyang, Y., Zhang, Y.-Q., & Yang, Q. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3352628 · Cited by 378
“FDSKL [61] integrates a non-linear kernel method into vertical federated learning.”
BackgroundMethodInfluentialDeep Model Poisoning Attack on Federated Learning
Zhou, X., Xu, M., Wu, Y., & Zheng, N. (2021) · Future Internet · doi:10.3390/fi13030073 · Cited by 170
“Owing to the aforementioned advantages, the federated learning paradigm has recently been transferred into many deep learning scenarios [25–28].”
BackgroundMembership Inference Attacks and Defenses in Federated Learning: A Survey
Bai, L., Hu, H., Ye, Q., Li, H., Wang, L., & Xu, J. (2024) · ACM Computing Surveys · doi:10.1145/3704633 · Cited by 243
“Furthermore, when it comes to the widely used VFL framework [162], previous MIAs prove to be generally ineffective due to the fact that each participant possesses only a portion of the feature space and has access to an incomplete target model.”
Metadata
- Authors
- Gu, B., Dang, Z., Li, X., & Huang, H.
- Affiliations
- JD Finance America Corporation, Mountain View, CA, USA
- Xidian University (CN)
- Western University (CA)
- Venue
- Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2020-08-20
- Pages
- 2483-2493
- OpenAlex
- W3080261024
- Cited by
- 68
- References
- 20
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 13 · 2023: 11 · 2022: 14 · 2021: 16 · 2020: 2
- FWCI
- 4.10
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › Cryptography and Data Security
- Keywords
- federated learning, vertically partitioned data, kernel learning, random features, data privacy, sublinear convergence, non-linear learning, privacy-preserving machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data. https://app.cochat.ai/papers/10.1145/3394486.3403298
- The paper
- Gu, B., Dang, Z., Li, X., & Huang, H. (2020). Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. https://doi.org/10.1145/3394486.3403298
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.