Secure and efficient parameters aggregation protocol for federated incremental learning and its applications
Abstract
Federated Learning (FL) enables the deployment of distributed machine learning models over the cloud and Edge Devices (EDs) while preserving the privacy of sensitive local data, such as electronic health records. However, despite FL advantages regarding security and flexibility, current constructions still suffer from some limitations. Namely, heavy computation overhead on limited resources EDs, communication overhead in uploading converged local models' parameters to a centralized server for parameters aggregation, and lack of guaranteeing the acquired knowledge preservation in the face of incremental learning over new local data sets. This paper introduces a secure and resource-friendly protocol for parameters aggregation in federated incremental learning and its applications. In this study, the central server relies on a new method for parameters aggregation called orthogonal gradient aggregation. Such a method assumes constant changes of each local data set and allows updating parameters in the orthogonal direction of previous parameters spaces. As a result, our new construction is robust against catastrophic forgetting, maintains the federated neural network accuracy, and is efficient in computation and communication overhead. Moreover, extensive experiments analysis over several significant data sets for incremental learning demonstrates our new protocol's efficiency, efficacy, and flexibility.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Wang et al. proposed a novel protocol for parameter aggregation in federated learning for edge devices with limited capacity, aiming to improve security while reducing communication and computational overhead, using orthogonal gradient aggregation to protect previously learned knowledge when related training samples are removed and ensure privacy.
“Wang et al. [52] proposed a novel protocol for parameter aggression in FL and its application for edge devices that have limited capacity. The goal is to improve security while reducing communication and computational overhead. The authors has proposed a novel approach of orthogonal gradient aggregation (OGA) that can protect previously learned knowledge when the related training samples are”
From A Review of Privacy Enhancement Methods for Federated Learning in Healthcare Systems (2023) · cites this paper as [52]
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Abdi, H., & Williams, L. J. (2010) · WIREs Computational Statistics · doi:10.1002/wics.101 · Cited by 11,104 · see citing works
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · see citing works
Understanding Machine Learning: From Theory to Algorithms
Shalev-Shwartz, S., & Ben-David, S. (2014) · doi:10.1017/cbo9781107298019 · Cited by 3,115 · see citing works
Federated Learning With Differential Privacy: Algorithms and Performance Analysis
Wei, K., Li, J., Ding, M., Ma, C., Yang, H. H., Farokhi, F., Jin, S., Quek, T. Q. S., & Vincent Poor, H. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2988575 · Cited by 2,400 · see citing works
Artificial intelligence and statistics
Yu, B., & Kumbier, K. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700813 · Cited by 2,329 · see citing works
Stochastic Gradient Descent Tricks
Bottou, L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_25 · Cited by 2,097 · see citing works
Improved Adam Optimizer for Deep Neural Networks
Zhang, Z. (2018) · International Workshop on Quality of Service · doi:10.1109/iwqos.2018.8624183 · Cited by 1,607 · see citing works
Memory Aware Synapses: Learning What (not) to Forget
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., & Tuytelaars, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_9 · Cited by 1,590 · see citing works
Chen, Y., Sun, X., & Jin, Y. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2953131 · Cited by 575 · see citing works
Continual learning of context-dependent processing in neural networks
Zeng, G., Chen, Y., Cui, B., & Yu, S. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0080-x · Cited by 309 · cited 3× · see citing works
“Therefore, we extend the scope of the work in References [34,35] to a multiparty training scenario, wherein the training data are distributed instead of being centralized.”
BackgroundOvercoming Catastrophic Forgetting With Unlabeled Data in the Wild
Lee, K., Lee, K., Shin, J., & Lee, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00040 · Cited by 214 · see citing works
Health care service delivery based on the Internet of things: A systematic and comprehensive study
Uşak, M., Kubiatko, M., Shabbir, M. S., Dudnik, O. V., Jermsittiparsert, K., & Rajabion, L. (2019) · International Journal of Communication Systems · doi:10.1002/dac.4179 · Cited by 202 · see citing works
A training-integrity privacy-preserving federated learning scheme with trusted execution environment
Chen, Y., Luo, F., Li, T., Xiang, T., Liu, Z., & Li, J. (2020) · Information Sciences · doi:10.1016/j.ins.2020.02.037 · Cited by 188 · see citing works
APPM: Adaptive Parallel Processing Mechanism for Service Function Chains
Cai, J., Huang, Z., Liao, L., Luo, J., & Liu, W. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3052223 · Cited by 69 · see citing works
On the Effective Parallelization and Near-Optimal Deployment of Service Function Chains
Luo, J., Li, J., Jiao, L., & Cai, J. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3043768 · Cited by 63 · see citing works
Secure and robust machine learning for healthcare: A survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3013489 · Cited by 59 · see citing works
Blockchain-Based Research Data Sharing Framework for Incentivizing the Data Owners
Shrestha, A. K., & Vassileva, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-94478-4_19 · Cited by 58 · see citing works
Advanced evasion attacks and mitigations on practical ML‐based phishing website classifiers
Song, F., Lei, Y., Chen, S., Fan, L., & Liu, Y. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22510 · Cited by 55 · see citing works
Porambage, P., Braeken, A., Gurtov, A., Ylianttila, M., & Spinsante, S. (2015) · IEEE World Forum on Internet of Things (WF-IoT) · doi:10.1109/wf-iot.2015.7389141 · Cited by 45 · see citing works
Al‐Rousan, N., Isa, N. A. M., Desa, M. K. M., & Al-Najjar, H. M. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22525 · Cited by 33 · see citing works
Rohan, T. I., Yusuf, M. S. U., Islam, M., & Roy, S. (2020) · IEEE Region 10 Symposium (TENSYMP) · doi:10.1109/tensymp50017.2020.9230731 · Cited by 22 · see citing works
Son, N. N., Kien, C. V., & Ánh, H. P. H. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22519 · Cited by 18 · see citing works
Securing Medical IoT Devices Using Diffie-Hellman and DES Cryptographic Schemes
Kester, Q.-A., & Xenya, M. C. (2019) · doi:10.1109/icsiot47925.2019.00025 · Cited by 12 · see citing works
An effective framework for semistructured document classification via hierarchical attention model
Zhao, W., Fang, D., Zhang, J., Zhao, Y., Xu, X., Jiang, X., Hu, X., & He, T. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22508 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Review of Privacy Enhancement Methods for Federated Learning in Healthcare Systems
Gu, X., Sabrina, F., Fan, Z., & Sohail, S. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20156539 · Cited by 139 · see citing works
Du, J., Qin, N., Huang, D., Zhang, Y., & Jia, X. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3238724 · Cited by 44 · see citing works
Energy trading scheme based on consortium blockchain and game theory
Chen, Y., Li, Y., Chen, Q., Wang, X., Li, T., & Tan, C. (2022) · Computer Standards & Interfaces · doi:10.1016/j.csi.2022.103699 · Cited by 44 · see citing works
Federated continual learning: Concepts, challenges, and solutions
Hamedi, P., Razavi‐Far, R., & Hallaji, E. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130844 · Cited by 44 · see citing works
Karras, A., Γιάνναρος, Α., Theodorakopoulos, L., Krimpas, G. A., Kalogeratos, G., Karras, C., & Sioutas, S. (2023) · Electronics · doi:10.3390/electronics12224633 · Cited by 36 · see citing works
Federated Incremental Learning algorithm based on Topological Data Analysis
Hu, K., Gong, S., Li, L., Luo, Y., Luo, Y., Jiang, S., & Xia, M. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111048 · Cited by 11 · see citing works
An effective and practical gradient inversion attack
Luo, Z., Zhu, C., Fang, L., Kou, G., Hou, R., & Wang, X. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22997 · Cited by 10 · see citing works
Decentralized federated meta‐learning framework for few‐shot multitask learning
Li, X., Li, Y., Wang, J., Chen, C., Yang, L., & Zheng, Z. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22951 · Cited by 10 · see citing works
Mining high utility pattern with negative items in dynamic databases
Han, M., Zhang, N., Wang, L., Li, X., & Cheng, H. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22794 · Cited by 8 · see citing works
Forward privacy multikeyword ranked search over encrypted database
Zhao, S., Zhang, H., Zhang, X., Li, W., Gao, F., & Wen, Q. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22884 · Cited by 7 · see citing works
A Universal Detection Method for Adversarial Examples and Fake Images
Lai, J., Huo, Y., Hou, R., & Wang, X. (2022) · Sensors · doi:10.3390/s22093445 · Cited by 3 · see citing works
- Authors
- Wang, X., Liang, Z., Koe, A. S. V., Wu, Q., Zhang, X., Li, H., & Yang, Q.
- Affiliations
- Third Affiliated Hospital of Sun Yat-sen University (CN)
- Guangzhou University (CN)
- Corresponding author
- Yang, Qintai
- ORCID
- Venue
- International Journal of Intelligent Systems
- Publisher
- Wiley
- Published
- 2021-11-08
- Volume
- 37
- Issue
- 8
- Pages
- 4471-4487
- ISSN
- 0884-8173 (ISSN-L)
- 1098-111X
- OpenAlex
- W3212533171
- Cited by
- 11
- References
- 26
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 3 · 2022: 5 · 2021: 1
- FWCI
- 0.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- International Journal of Intelligent Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- parameter aggregation, catastrophic forgetting, edge devices, communication overhead, computational overhead, privacy preservation, knowledge preservation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
