A Survey on Federated Learning for Resource-Constrained IoT Devices
The DOI matches its Crossref record; 224 of the 224 references it cites were found in a registry or an index.
Abstract
Federated learning (FL) is a distributed machine learning strategy that generates a global model by learning from multiple decentralized edge clients. FL enables on-device training, keeping the client’s local data private, and further, updating the global model based on the local model updates. While FL methods offer several advantages, including scalability and data privacy, they assume there are available computational resources at each edge-device/client. However, the Internet-of-Things (IoT)-enabled devices, e.g., robots, drone swarms, and low-cost computing devices (e.g., Raspberry Pi), may have limited processing ability, low bandwidth and power, or limited storage capacity. In this survey article, we propose to answer this question: how to train distributed machine learning models for resource-constrained IoT devices? To this end, we first explore the existing studies on FL, relative assumptions for distributed implementation using IoT devices, and explore their drawbacks. We then discuss the implementation challenges and issues when applying FL to an IoT environment. We highlight an overview of FL and provide a comprehensive survey of the problem statements and emerging challenges, particularly during applying FL within heterogeneous IoT environments. Finally, we point out the future research directions for scientists and researchers who are interested in working at the intersection of FL and resource-constrained IoT environments.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Items 201–225 of 225
An incentive compatible reputation mechanism
Jurca, R., & Faltings, B. V. (2003) · IEEE Conference on Commerce and Enterprise Computing (CEC) · doi:10.1109/coec.2003.1210263 · Cited by 47 · cited 1×
“Some incentives or regulations schemes are implemented in blockchain [217]–[219], and some incentive mechanisms are proposed for high-quality federated data [220], [221].”
Federated Object Detection: Optimizing Object Detection Model with Federated Learning
Yu, P., & Liu, Y. (2019) · Proceedings of the 3rd International Conference on Vision, Image and Signal Processing · doi:10.1145/3387168.3387181 · Cited by 44 · cited 1×
“Smart Object Detection: Yu and Liu [213] designed an approach of optimizing object detection by considering Kullback–Leibler divergence (KLD) during the measurement of weights divergence of the client’s local models.”
Estimation of Individual Device Contributions for Incentivizing Federated Learning
Nishio, T., Shinkuma, R., & Mandayam, N. B. (2020) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps50303.2020.9367484 · Cited by 44 · cited 1×
“Besides, Nishio et al. [120] proposed a strategy of estimating the contributions of each client in an FL process and provide incentives accordingly, reducing the communication and computational overhead.”
GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning
Elgabli, A., Park, J., Bedi, A. S., Bennis, M., & Aggarwal, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00047 · Cited by 42
A VCG-based Fair Incentive Mechanism for Federated LearningPreprint
Cong, M., Yu, H., Weng, X., Qu, J., Liu, Y., & Yiu, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.06680 · Cited by 42 · cited 2×
“Similarly, Cong et al. [121] designed a client contribution-based incentive method for FL but using the concept of Vickrey–Clarke–Groves (VCGs) mechanism.”
No Peek: A Survey of private distributed deep learning
Vepakomma, P., Swedish, T. B., Raskar, R., Gupta, O., & Dubey, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.03288 · Cited by 42
Imteaj, A., & Amini, M. H. (2021) · Frontiers in Communications and Networks · doi:10.3389/frcmn.2021.657653 · Cited by 41 · cited 2×
“Imteaj and Amini [126] proposed the activity and resource-aware FL strategy that can handle straggler issues by examining resource status, labeling clients with trust values in accordance to their contributions toward model convergence, and accepting variable works from the participated clients.”
Self-citationCheckmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
Jain, P., Jain, A., Nrusimha, A., Gholami, A., Abbeel, P., Keutzer, K., Stoica, I. G., & Gonzalez, J. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.02653 · Cited by 34 · cited 1×
“Training on edge devices can bene-fit by adapting techniques that have served well in managing this problem in the server context, such as efficient gradient checkpointing [108], tensor rematerialization [109], and recompute [110].”
Compressing RNNs for IoT devices by 15-38x using Kronecker Products
Thakker, U., Beu, J., Gope, D., Zhou, C., Fedorov, I., Dasika, G., & Mattina, M. (2019) · arXiv · doi:10.48550/arxiv.1906.02876 · Cited by 30 · cited 1×
“The on-device inference problems are solved in [177] and [178], but the on-device training issues are not expounded.”
Self-citationGeneSys: Enabling Continuous Learning through Neural Network Evolution in Hardware
Samajdar, A., Mannan, P., Garg, K., & Krishna, T. (2018) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2018.00074 · Cited by 29 · cited 1×
“…this area can be categorized in the domain of novel instructions for executing ML in the CPU [188], [189], design of specialized accelerators [190]–[192], optimized software library [193], [194], development of new memory technologies [195], and near data processing to enable large storage using…”
Distilling On-Device Intelligence at the Network Edge
Park, J., Wang, S., Elgabli, A., Oh, S., Jeong, E., Cha, H., Kim, H., Kim, S., & Bennis, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05895 · Cited by 26 · cited 1×
“Jiang et al. proposed an efficient learning technique using pruned models, while Park et al. [107] proposed a strategy to generate a high-quality ML model through on-device model output, parameters, and data aggregation.”
Self-citationIncentives for Federated Learning: a Hypothesis Elicitation ApproachPreprint
Liu, Y., & Wei, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10596 · Cited by 26
Reaching Data Confidentiality and Model Accountability on the CalTrain
Gu, Z., Jamjoom, H. T., Su, D., Huang, H., Zhang, J., Ma, T., Pendarakis, D. E., & Molloy, I. (2019) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn.2019.00044 · Cited by 25 · cited 1×
“Gu et al. [112] proposed a framework to identify the mislabeled data which are injected through data poisoning attacks.”
Privacy is What We Care About: Experimental Investigation of Federated Learning on Edge Devices
Das, A., & Brunschwiler, T. (2019) · Proceedings of the First International Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things · doi:10.1145/3363347.3363365 · Cited by 25 · cited 2×
“Das and Brunschwiler [92] presented a solution to handle straggler clients by acknowl-edging their resource utilization (i.e., computation power) after each local update.”
Run-Time Efficient RNN Compression for Inference on Edge Devices
Thakker, U., Beu, J., Gope, D., Dasika, G., & Mattina, M. (2019) · 2019 2nd Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2) · doi:10.1109/emc249363.2019.00013 · Cited by 24 · cited 1×
“A potential direction can be found in [179].”
Self-citationMDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing
Zhang, Y., Gu, T., & Zhang, X. (2020) · 2020 19th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn48710.2020.00-45 · Cited by 23
Skipping RNN State Updates without Retraining the Original Model
Tao, J., Thakker, U., Dasika, G., & Beu, J. (2019) · Proceedings of the 1st Workshop on Machine Learning on Edge in Sensor Systems · doi:10.1145/3362743.3362965 · Cited by 19 · cited 1×
“DLS techniques are more specific to ResNet style architecture with skip connections or RNN-based models ([185], [186]).”
Self-citationMemory-Efficient, Limb Position-Aware Hand Gesture Recognition using Hyperdimensional Computing
Zhou, A., Muller, R., & Rabaey, J. (2021) · arXiv · doi:10.48550/arxiv.2103.05267 · Cited by 17 · cited 1×
“A lot of edge applications are now using DNN-based algorithms [102], [103] and there is an increasing focus on making DNN inference efficient on edge devices [104], [105].”
FedRec: Privacy-Preserving News Recommendation with Federated LearningPreprint
Qi, T., Wu, F., Wu, C., Huang, Y., & Xie, X. (2020) · arXiv (Cornell University) · Cited by 16
Deep neural network acceleration framework under hardware uncertainty
Imani, M., Wang, P., & Rosing, T. (2018) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed.2018.8357318 · Cited by 13
Measuring scheduling efficiency of RNNs for NLP applications
Thakker, U., Dasika, G. S., Beu, J. G., & Mattina, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03302 · Cited by 13 · cited 1×
“…in the domain of novel instructions for executing ML in the CPU [188], [189], design of specialized accelerators [190]–[192], optimized software library [193], [194], development of new memory technologies [195], and near data processing to enable large storage using smaller energy budget [196].”
Self-citationPrivacy Preserving Technique in Data Mining by Using Chinease Remainder Theorem
Pasupuleti, R., Narasimha, G., & Rupa, C. (2012) · Communications in computer and information science · doi:10.1007/978-3-642-32112-2_50 · Cited by 7
Pushing the Envelope of Dynamic Spatial Gating technologies
Huang, X., Thakker, U., Gope, D., & Beu, J. (2020) · Proceedings of the 2nd International Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things · doi:10.1145/3417313.3429380 · Cited by 7 · cited 1×
“DSG techniques ([181], [182]) identify spatial regions in the OFM that are deemed important and focus their attention only on those parts of the OFM.”
Self-citationUnderstanding the Impact of Dynamic Channel Pruning on Conditionally Parameterized Convolutions
Raju, R., Gope, D., Thakker, U., & Beu, J. G. (2020) · International Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things (AIChallengeIoT) · doi:10.1145/3417313.3429381 · Cited by 7 · cited 1×
“DCP techniques identify channels in the OFM that are deemed unimportant and skip computations for those channels ([183], [184]).”
Self-citationSucher, S. J. (2007) · Routledge eBooks · doi:10.4324/9780203940648-25 · Cited by 4
Research works printed but not linked: 10
- 10.5555/3295222.3295285 · doi:10.5555/3295222.3295285
- Tang, $D^{2}$ : Decentralized training over decentralized data, 4848
- 10.5555/2999611.2999647 · doi:10.5555/2999611.2999647
- Chai, Towards taming the resource and data heterogeneity in federated learning, 19
- 10.1109/comst.2020.2986024 · DOI · Paper page
- Thrun, 2012, Learning to Learn
- Kumar, Resource-efficient machine learning in 2 KB RAM for the Internet of Things, 1935
- Gupta, ProtoNN: Compressed and accurate kNN for resource-scarce devices, 1331
- Li, Federated optimization in heterogeneous networks, 2, 429
- Xu, MicroFL: A lightweight, secure-by-design edge network fabric for decentralized IoT systems
Cited by
Federated learning on non-IID data: A survey
Zhu, H., Xu, J., Liu, S., & Jin, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.098 · Cited by 1,062
Xu, W., Yang, Z., Ng, D. W. K., Levorato, M., Eldar, Y. C., & Debbah, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3239189 · Cited by 660
“Compared with these works [10], [19], [20], [21], [22], [23], the main focus of this paper is to provide a comprehensive overview of state-of-the-art signal processing techniques for EL over B5G networks.”
Federated learning for the Internet of Things: Applications, challenges, and opportunities
Zhang, T., Gao, L., He, C., Zhang, M., Krishnamachari, B., & Avestimehr, A. S. (2022) · IEEE Internet of Things Magazine · doi:10.1109/iotm.004.2100182 · Cited by 412
A state-of-the-art survey on solving non-IID data in Federated Learning
Ma, X., Zhu, J., Lin, Z., Chen, S., & Qin, Y. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.003 · Cited by 399
Decentralized Federated Averaging
Sun, T., Li, D., & Wang, B. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3196503 · Cited by 388
“The federated learningmethod has been applied to Internet of Things (IoT) researches [13].”
Method
Metadata
- Authors
- Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H.
- Affiliations
- Florida International University (US)
- IBM (United States) (US)
- Binghamton University (US)
- Venue
- IEEE Internet of Things Journal
- Published
- 2022-01-01
- Volume
- 9
- Issue
- 1
- Pages
- 1-24
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W3182125009
- Cited by
- 774
- References
- 223
- Citations by year
- 2026: 113 · 2025: 201 · 2024: 209 · 2023: 155 · 2022: 85
- FWCI
- 40.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Internet of Things Journal (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Information Systems › Internet of Things and AI
- Keywords
- federated learning, resource-constrained devices, Internet of Things, edge computing, on-device learning, data privacy, distributed learning, low-power devices, implementation barriers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.