Peer-to-peer Federated Learning on Graphs
Open accessAbstract
We consider the problem of training a machine learning model over a network of nodes in a fully decentralized framework. The nodes take a Bayesian-like approach via the introduction of a belief over the model parameter space. We propose a distributed learning algorithm in which nodes update their belief by aggregate information from their one-hop neighbors to learn a model that best fits the observations over the entire network. In addition, we also obtain sufficient conditions to ensure that the probability of error is small for every node in the network. We discuss approximations required for applying this algorithm to train Deep Neural Networks (DNNs). Experiments on training linear regression model and on training a DNN show that the proposed learning rule algorithm provides a significant improvement in the accuracy compared to the case where nodes learn without cooperation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Fully decentralized federated learning removes the central server and lets clients communicate only with one-hop neighbors.
“Therefore, some researches began to study fully decentralized framework where the central server is not required [41, 60, 85, 91]. The local clients are distributed over the graph/network where they only communicate with their one-hop neighbors.”
From Federated Learning for Healthcare Informatics (2021) · cites this paper as [60]
The performance of transfer learning varies with architectures and transfer techniques.
“The result varies with architectures and transfer techniques.”
From Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019) · cites this paper as [44]
Section 5.2.2 (p. 6) reports accuracy for one architecture under different distributed-training conditions, but does not compare architectures or transfer-learning techniques. Section 6 (p. 6) calls studies of other DNN architectures future work. Searches for transfer learning and variation across architectures or transfer techniques found no passage addressing the claim. (LLM, conf=0.76)
Experimental — accuracy not yet measuredA good performance improvement is obtained by transferring knowledge from both the intermediate layers and last layer of the teacher to a shallower student, while other architectures and transfer techniques do not fare so well and some even lead to negative performance impact.
“A good performance improvement is obtained by transferring knowledge from both the intermediate layers and last layer of the teacher to a shallower student while other architectures and transfer techniques do not fare so well and some of them even lead to negative performance impact.”
From Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019) · cites this paper as [44]
Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
Weight Uncertainty in Neural Networks
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1505.05424 · Cited by 1,342 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- DeGroot, M. H. Reaching a consensus. Journal of the American Statistical Association , 69(345):118-121, 1974. ISSN 01621459.
- Gal, Y. Uncertainty in Deep Learning. University of Cambridge , 2016.
- Koneˇ cn` y, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., and Bacon, D. Federated Learning: Strategies for Improving Communication Efficiency. arXiv preprint arXiv:1610.05492 , 2016. · arXiv
- Lalitha, A. and Javidi, T. Learning via Active Hypothesis Testing over Networks. In 2017 IEEE Information Theory Workshop (ITW) , pp. 374-378, Nov 2017. doi: 10.1109/ ITW.2017.8278014.
- Lalitha, A., Javidi, T., and Sarwate, A. D. Social Learning and Distributed Hypothesis Testing. IEEE Transactions on Information Theory , 64(9):6161-6179, Sept 2018. ISSN 0018-9448. doi: 10.1109/TIT.2018.2837050. · DOI
- Nedi´ c, A., Olshevsky, A., and Uribe, C. A. Non-asymptotic Convergence Rates for Cooperative Learning over Timevarying Directed Graphs. In 2015 American Control Conference (ACC) , pp. 5884-5889, July 2015. doi: 10. 1109/ACC.2015.7172262.
- Olfati-Saber, R., Franco, E., Frazzoli, E., and Shamma, J. S. Belief Consensus and Distributed Hypothesis Testing in Sensor Networks. In Workshop on Network Embedded Sensing and Control , Notre Dame University, South Bend, IN, October 2005.
- Shahrampour, S., Rakhlin, A., and Jadbabaie, A. Distributed Detection: Finite-Time Analysis and Impact of Network Topology. IEEE Transactions on Automatic Control , 61 (11):3256-3268, Nov 2016. ISSN 0018-9286. doi: 10. 1109/TAC.2015.2506903.
Cited by
Sorted by citations
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
The future of digital health with federated learning
Rieke, N., Hancox, J., Li, W., Milletarì, F., Roth, H. R., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B. A., Maier-Hein, K., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00323-1 · Cited by 3,119 · see citing works
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455 · see citing works
“[48] propose a Bayesian-based distributed algorithm, in which each device updates its belief by aggregating information from its one-hop neighbors to train a model that best fits the observations over the entire network.”
BackgroundFederated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2021) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-020-00082-4 · Cited by 1,542 · see citing works
“Therefore, some researches began to study fully decentralized framework where the central server is not required [41, 60, 85, 91].”
BackgroundA survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.007 · Cited by 1,480 · see citing works
A Survey on Federated Learning for Resource-Constrained IoT Devices
Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3095077 · Cited by 774 · see citing works
Yin, X., Zhu, Y., & Hu, J. (2022) · ACM Computing Surveys · doi:10.1145/3460427 · Cited by 638 · see citing works
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1909.13014 · Cited by 587 · see citing works
Federated Learning With Cooperating Devices: A Consensus Approach for Massive IoT Networks
Savazzi, S., Nicoli, M., & Rampa, V. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2964162 · Cited by 431 · see citing works
Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
So, J., Guler, B., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3054610 · Cited by 318 · see citing works
Vulnerabilities in federated learning
Bouacida, N., & Mohapatra, P. (2021) · IEEE Access · doi:10.1109/access.2021.3075203 · Cited by 248 · see citing works
Decentralized Federated Learning With Unreliable Communications
Ye, H., Liang, L., & Li, G. Y. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3152445 · Cited by 184 · see citing works
Lian, Z., Yang, Q., Wang, W., Zeng, Q., Alazab, M., Zhao, H., & Su, C. (2022) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2022.3175945 · Cited by 100 · see citing works
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications
Park, J., Samarakoon, S., Elgabli, A., Kim, J., Bennis, M., Kim, S.-L., & Debbah, M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3055679 · Cited by 25 · see citing works
- Authors
- Lalitha, A., Kilinc, O. C., Javidi, T., & Koushanfar, F.
- Affiliations
- University of California San Diego (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1901.11173
- OpenAlex
- W2913243576
- Cited by
- 146
- References
- 2
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 25 · 2023: 32 · 2022: 29 · 2021: 31 · 2020: 17 · 2019: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Control Multi-Agent Systems
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Sensor Networks and Detection Algorithms
- Keywords
- federated learning, peer learning, decentralized learning, graph networks, Bayesian belief updating, distributed learning algorithms, deep neural network, linear regression, error probability bounds, cooperative learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
