Communication-efficient hierarchical federated learning for IoT heterogeneous systems with imbalanced data
Open accessAbstract
Federated Learning (FL) is a distributed learning methodology that allows multiple nodes to cooperatively train a deep learning model, without the need to share their local data. It is a promising solution for telemonitoring systems that demand intensive data collection, for detection, classification, and prediction of future events, from different locations while maintaining a strict privacy constraint. Due to privacy concerns and critical communication bottlenecks, it can become impractical to send the FL updated models to a centralized server. Thus, this paper studies the potential of hierarchical FL in Internet of Things (IoT) heterogeneous systems. In particular, we propose an optimized solution for user assignment and resource allocation over hierarchical FL architecture for IoT heterogeneous systems. This work focuses on a generic class of machine learning models that are trained using gradient-descent-based schemes while considering the practical constraints of non-uniformly distributed data across different users. We evaluate the proposed system using two real-world datasets, and we show that it outperforms state-of-the-art FL solutions. Specifically, our numerical results highlight the effectiveness of our approach and its ability to provide 4–6% increase in the classification accuracy, with respect to hierarchical FL schemes that consider distance-based user assignment. Furthermore, the proposed approach could significantly accelerate FL training and reduce communication overhead by providing 75–85% reduction in the communication rounds between edge nodes and the centralized server, for the same model accuracy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A hierarchical federated learning approach permits aggregation at varying levels, facilitating collaboration among multiple parties.
“Hence few works concentrated on the development of a hierarchical FL approach that permits aggregation at varying levels, facilitating collaboration among multiple parties [46,47,48].”
From A Review of Privacy Enhancement Methods for Federated Learning in Healthcare Systems (2023) · cites this paper as [48]
Abdellatif et al. presented a hierarchical federated learning application to enhance privacy in healthcare, with End User, edge node, and centralized server layers; End Users train a deep learning model without sharing raw data by connecting to edge nodes, which connect to a centralized server; the method achieves efficiency considering varying data distributions and wireless conditions.
“Abdellatif et al. [48] presented a hierarchical FL application that aims to enhance privacy in healthcare applications. The proposed architecture comprises different layers such as End User (EU) layer, Edge node layer, and Centralized Server layer. In this approach, EUs collaborate to train a deep learning model without sharing raw data, by connecting to the edge nodes within their communication”
From A Review of Privacy Enhancement Methods for Federated Learning in Healthcare Systems (2023) · cites this paper as [48]
By leveraging hierarchical structures to organize clients and servers, privacy-preserving collaboration is facilitated.
“By leveraging hierarchical structures to organize clients and servers, privacy-preserving collaboration is facilitated [46,47,48].”
From A Review of Privacy Enhancement Methods for Federated Learning in Healthcare Systems (2023) · cites this paper as [48]
Hierarchical approaches are effective and relevant for enhancing privacy preservation in FL systems because they provide additional layers of control over data sharing among clients, edge and cloud servers.
“These approaches are effective and relevant for enhancing privacy preservation in FL systems as they provide additional layers of control over data sharing among clients, edge and cloud servers.”
From A Review of Privacy Enhancement Methods for Federated Learning in Healthcare Systems (2023) · cites this paper as [48]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Wireless Communications: Principles and Practice
Rappaport, T. S. (1996) · Cited by 17,494 · see citing works
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2005) · doi:10.1002/047174882x · Cited by 13,791 · see citing works
(2016) · Cited by 11,679 · see citing works
Information theory and statistics
(1959) · Journal of the Franklin Institute · doi:10.1016/0016-0032(59)90368-0 · Cited by 7,318 · see citing works
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607 · see citing works
The impact of the MIT-BIH Arrhythmia Database
Moody, G. B., & Mark, R. G. (2001) · IEEE Engineering in Medicine and Biology Magazine · doi:10.1109/51.932724 · Cited by 4,834 · see citing works
Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232 · see citing works
Federated Learning with Non-IID Data
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018) · arXiv · doi:10.48550/arxiv.1806.00582 · Cited by 2,077 · see citing works
Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
Nishio, T., & Yonetani, R. (2019) · ICC 2019 - 2019 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2019.8761315 · Cited by 1,875 · see citing works
Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data
Sattler, F., Wiedemann, S., Muller, K.-R., & Samek, W. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944481 · Cited by 1,725 · see citing works
On the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575 · see citing works
Optimizing Federated Learning on Non-IID Data with Reinforcement Learning
Wang, H., Kaplan, Z., Niu, D., & Li, B. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155494 · Cited by 1,054 · see citing works
Client-Edge-Cloud Hierarchical Federated Learning
Liu, L., Zhang, J., Song, S. H., & Letaief, K. B. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9148862 · Cited by 1,033 · see citing works
Federated Learning over Wireless Networks: Optimization Model Design and Analysis
Tran, N. H., Bao, W., Zomaya, A. Y., Nguyen, M. N. H., & Hong, C. (2019) · IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737464 · Cited by 943 · see citing works
Ghasempour, A. (2019) · Inventions · doi:10.3390/inventions4010022 · Cited by 685 · see citing works
Revisiting Distributed Synchronous SGDPreprint
Pan, X., Chen, J., Monga, R., Samy Bengio, A., & Józefowicz, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05800 · Cited by 608 · see citing works
Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment
Bosse, S., Maniry, D., Müller, K., Wiegand, T., & Samek, W. (2018) · MPG.PuRe (Max Planck Society) · Cited by 469 · see citing works
Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT
Mills, J., Hu, J., & Min, G. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2956615 · Cited by 436 · see citing works
Self-Balancing Federated Learning With Global Imbalanced Data in Mobile Systems
Duan, M., Liu, D., Chen, X., Liu, R., Tan, Y., & Liang, L. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3009406 · Cited by 434 · see citing works
Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks
Abad, M. S. H., Ozfatura, E., GUndUz, D., & Ercetin, O. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054634 · Cited by 424 · see citing works
Convergence Time Optimization for Federated Learning Over Wireless Networks
Chen, M., Poor, H. V., Saad, W., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3042530 · Cited by 403 · see citing works
Federated Learning in the Sky: Joint Power Allocation and Scheduling with UAV Swarms
Zeng, T., Semiari, O., Mozaffari, M., Chen, M., Saad, W., & Bennis, M. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9148776 · Cited by 218 · see citing works
Autonomic computation offloading in mobile edge for IoT applications
Alam, M. G. R., Hassan, M. M., Uddin, M. Z., Almogren, A., & Fortino, G. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2018.07.050 · Cited by 216 · see citing works
Agent-based Internet of Things: State-of-the-art and research challenges
Savaglio, C., Ganzha, M., Paprzycki, M., Bădică, C., Ivanović, M. D., & Fortino, G. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.09.016 · Cited by 216 · see citing works
Digital Communications Over Fading Radio Channels
Foschini, G. J., & Salz, J. (1983) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1983.tb03104.x · Cited by 199 · see citing works
Niedermeyer's Electroencephalography
Schomer, D. L., & da Silva, F. H. L. (2017) · doi:10.1093/med/9780190228484.001.0001 · Cited by 194 · see citing works
Distributed optimization with arbitrary local solvers
Ma, C., Konečný, J., Jaggi, M., Smith, V., Jordan, M. I., Richtárik, P., & Takáč, M. (2017) · Optimization methods & software · doi:10.1080/10556788.2016.1278445 · Cited by 191 · see citing works
Mhaisen, N., Abdellatif, A. A., Mohamed, A. M. S., Erbad, A., & Guizani, M. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3053588 · Cited by 159 · see citing works
Self-citationAccelerating Federated Learning over Reliability-Agnostic Clients in Mobile Edge Computing Systems
Wu, W., He, L., Lin, W., & Mao, R. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3040867 · Cited by 102 · see citing works
Semi-stochastic coordinate descent
Konečný, J., Qu, Z., & Richtárik, P. (2017) · Optimization methods & software · doi:10.1080/10556788.2017.1298596 · Cited by 91 · see citing works
Bandwidth Gain From Mobile Edge Computing and Caching in Wireless Multicast Systems
Sun, Y., Chen, Z., Tao, M., & Liu, H. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2979147 · Cited by 73 · see citing works
Socially-Aware-Clustering-Enabled Federated Learning for Edge Networks
Khan, L. U., Han, Z., Niyato, D. T., & Hong, C. S. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3090446 · Cited by 42 · see citing works
Feng, C., Wang, Y., Zhao, Z., Quek, T. Q. S., & Peng, M. (2020) · IEEE International Conference on Communications workshops · doi:10.1109/iccworkshops49005.2020.9145182 · Cited by 38 · see citing works
User-Centric Networks Selection With Adaptive Data Compression for Smart Health
Abdellatif, A. A., Mohamed, A. M. S., & Chiasserini, C. F. (2018) · IEEE Systems Journal · doi:10.1109/jsyst.2017.2785302 · Cited by 31 · see citing works
Self-citationPerformance Analysis and Adaptive DRX Scheme for Dual Connectivity
Sharma, L., Kumar, B. B., & Wu, S. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2937345 · Cited by 27 · see citing works
Yang, Y., Deng, X., He, D., et al. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2923314 · Cited by 22 · see citing works
Multi-Radio Dual Connectivity for 5G Small Cells Interworking
Monserrat, J. F., Bouchmal, F., Martín‐Sacristán, D., & Carrasco, O. (2020) · IEEE Communications Standards Magazine · doi:10.1109/mcomstd.001.1800030 · Cited by 18 · see citing works
Active Learning-based Classification in Automated Connected Vehicles
Abdellatif, A. A., Chiasserini, C. F., & Malandrino, F. (2020) · IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infocomwkshps50562.2020.9162964 · Cited by 18 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 8
- W. Samek, G. Montavon, A. Vedaldi, L.K. Hansen, K.-R. Müller, Explain- able AI: Interpreting, Explaining and Visualizing Deep Learning, Springer International Publishing, 2019.
- A.A. Abdellatif, A. Mohamed, C.F. Chiasserini, A. Erbad, M. Guizani, Edge computing for energy-efficient smart health systems: Data and application-specific approaches, in: Energy Efficiency of Medical Devices and Healthcare Applications, Elsevier, 2020, pp. 53-67.
- A.F. Aji, K. Heafield, Sparse communication for distributed gradient descent, 2017, ArXiv:1704.05021, URL https://arxiv.org/abs/1704.05021. · arXiv
- Cs, Eess], URL http://arxiv.org/abs/2007.14374. · arXiv
- H.B. McMahan, E. Moore, D. Ramage, S. Hampson, B.A. y Arcas, Communication-Efficient Learning of Deep Networks from Decentralized Data, in: Proceedings of the 20 Th International Conference on Artificial Intelligence and Statistics, 2017, pp. 1273-1282.
- Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, V. Chandra, Federated learning with non-iid data, 2018, arXiv preprint arXiv:1806.00582. · arXiv
- J. Chen, R. Monga, S. Bengio, R. Jozefowicz, Revisiting distributed syn- chronous SGD, in: International Conference on Learning Representations Workshop Track, 2016, URL https://arxiv.org/abs/1604.00981. · arXiv
- Jiří Matoušek, Bernd Gärtner, Understanding and using Linear Program- ming, first ed., Springer, Berlin, Heidelberg, 2007.
Cited by
Sorted by citations
A Review of Privacy Enhancement Methods for Federated Learning in Healthcare Systems
Gu, X., Sabrina, F., Fan, Z., & Sohail, S. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20156539 · Cited by 139 · see citing works
- Authors
- Abdellatif, A. A., Mhaisen, N., Mohamed, A., Erbad, A., Guizani, M., Dawy, Z., & Nasreddine, W.
- Affiliations
- Qatar University (QA)
- Hamad bin Khalifa University (QA)
- American University of Beirut (LB)
- Corresponding author
- Erbad, Aiman
- ORCID
- Venue
- Future Generation Computer Systems
- Publisher
- Elsevier BV
- Published
- 2022-03
- Volume
- 128
- Pages
- 406-419
- ISSN
- 0167-739X (ISSN-L)
- OpenAlex
- W3182946142
- Cited by
- 4
- References
- 41
- Citations by year
- 2024: 3 · 2023: 1
- FWCI
- 0.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Future Generation Computer Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Qatar Foundation (NPRP12S-0305-190231, NPRP12S-0305)
- Fonds National de la Recherche Luxembourg
- Qatar National Research Fund (# NPRP12S-0305-190231, NPRP12S-)
- Qatar National Library
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Information Systems › Internet of Things and AI
- Keywords
- hierarchical federated learning, Internet of Things, heterogeneous systems, class imbalance, user assignment, resource allocation, communication efficiency, privacy-preserving machine learning, remote patient monitoring, edge computing, communication cost reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
