Model poisoning attacks against distributed machine learning systems
Abstract
Future military coalition operations will increasingly rely on machine learning (ML) methods to improve situational awareness. The coalition context presents unique challenges for ML: the tactical environment creates significant computing and communications limitations while also having to deal with an adversarial presence. Further, coalition operations must operate in a distributed manner, while coping with the constraints posed by the operational environment. Envisioned ML deployments in military assets must be resilient to these challenges. Here, we focus on the susceptibility of ML models to be poisoned (during training) or fooled (after training) by adversarial inputs. We review recent work on distributed adversarial ML, and present new results from our own investigations into model poisoning attacks on distributed learning systems without a central parameter aggregation node.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
If local algorithms are not encrypted, or the updates are not securely aggregated, data can leak or algorithms can be tampered with, reconstructed or stolen through parameter inference.
“In case the local algorithms are not encrypted, or the updates aren't securely aggregated, data can leak or algorithms can be tampered with 51 , reconstructed or stolen (parameter inference), which is unacceptable from the viewpoint of intellectual property, patent restrictions or asset protection.”
From Secure, privacy-preserving and federated machine learning in medical imaging (2020) · cites this paper as [51]
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,869 · see citing works
(2017) · Cited by 11,625 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Toward a Theory of Situation Awareness in Dynamic Systems
Endsley, M. R. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779049543 · Cited by 8,875 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108 · see citing works
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,189 · see citing works
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics 2016
Murray, I., & Graham, M. (2016) · Cited by 2,079 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · see citing works
Wild patterns: Ten years after the rise of adversarial machine learning
Biggio, B., & Roli, F. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.023 · Cited by 1,444 · see citing works
(2017) · neural information processing systems · Cited by 1,413 · see citing works
Poisoning Attacks against Support Vector Machines
Biggio, B., Nelson, B., & Laskov, P. (2012) · arXiv · doi:10.48550/arxiv.1206.6389 · Cited by 1,331 · see citing works
Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I. P., & Tygar, J. D. (2011) · Proceedings of the 4th ACM workshop on Security and artificial intelligence · doi:10.1145/2046684.2046692 · Cited by 1,194 · see citing works
Dalvi, N. N., Domingos, P. M., Mausam, Sanghai, S. K., & Verma, D. (2004) · Knowledge Discovery and Data Mining · doi:10.1145/1014052.1014066 · Cited by 1,085 · see citing works
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Chen, X., Liu, C., Li, B., Lu, K., & Song, D. (2017) · arXiv · doi:10.48550/arxiv.1712.05526 · Cited by 1,039 · see citing works
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017) · arXiv · doi:10.48550/arxiv.1701.06538 · Cited by 933 · see citing works
Lowd, D., & Meek, C. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081950 · Cited by 755 · see citing works
Analyzing Federated Learning through an Adversarial Lens
Bhagoji, A. N., Chakraborty, S., Mittal, P., & Calo, S. (2018) · arXiv · doi:10.48550/arxiv.1811.12470 · Cited by 703 · see citing works
Self-citationRevisiting Distributed Synchronous SGDPreprint
Pan, X., Chen, J., Monga, R., Samy Bengio, A., & Józefowicz, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05800 · Cited by 608 · see citing works
Mitigating Sybils in Federated Learning Poisoning
Fung, C., Yoon, C. J. M., & Beschastnikh, I. (2018) · arXiv · doi:10.48550/arxiv.1808.04866 · Cited by 385 · see citing works
Kott, A., Swami, A., & West, B. J. (2016) · Computer · doi:10.1109/mc.2016.355 · Cited by 215 · see citing works
When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8486403 · Cited by 209 · see citing works
Self-citationDistributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient DescentPreprint
Chen, Y., Su, L., & Xu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.05491 · Cited by 136 · see citing works
Analyzing the applicability of Internet of Things to the battlefield environment
Suri, N., Tortonesi, M., Michaelis, J., Budulas, P. P., Benincasa, G., Russell, S., Stefanelli, C., & Winkler, R. (2016) · International Conference on Military Communications and Information Systems (ICMCIS) · doi:10.1109/icmcis.2016.7496574 · Cited by 115 · see citing works
How to scale distributed deep learning?
Jin, P. H., Yuan, Q., Iandola, F., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1611.04581 · Cited by 85 · see citing works
Gossip training for deep learning
Blot, M., Picard, D., Cord, M., & Thome, N. (2016) · arXiv · doi:10.48550/arxiv.1611.09726 · Cited by 77 · see citing works
Distributed optimization for deep learning with gossip exchange
Blot, M., Picard, D., Thome, N., & Cord, M. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.11.002 · Cited by 50 · see citing works
A game-theoretic analysis of label flipping attacks on distributed support vector machines
Zhang, R., & Zhu, Q. (2017) · Conference on Information Sciences and Systems · doi:10.1109/ciss.2017.7926118 · Cited by 25 · see citing works
Deep learning for situational understanding
Chakraborty, S., Preece, A. D., Alzantot, M., Xing, T., Braines, D., & Srivastava, M. B. (2017) · International Conference on Information Fusion (FUSION) · doi:10.23919/icif.2017.8009785 · Cited by 17 · see citing works
Self-citationELASTIC GOSSIP: DISTRIBUTING NEURAL NETWORK TRAINING USING GOSSIP-LIKE PROTOCOLSPreprint
Pramod, S. (2018) · arXiv (Cornell University) · doi:10.13016/m2cl1x-yatq · Cited by 3 · see citing works
The Hidden Vulnerability of Distributed Learning in Byzantium
Mhamdi, E. M. E., Guerraoui, R., & Rouault, S. (2018) · arXiv · doi:10.48550/arxiv.1802.07927
Flagged first, then by citations
1 of the 37 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Secure, privacy-preserving and federated machine learning in medical imaging
Kaissis, G. A., Makowski, M. R., Rückert, D., & Braren, R. F. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0186-1 · Cited by 1,501 · see citing works
Federated learning in medical imaging: Part II: methods, challenges, and considerations
Darzidehkalani, E., Ghasemi-rad, M., & van Ooijen, P. M. A. (2022) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2022.03.016 · Cited by 103 · see citing works
Alzu’bi, A., Alomar, A., Alkhaza’leh, S., Abuarqoub, A., & Hammoudeh, M. A. A. (2024) · Tsinghua Science & Technology · doi:10.26599/tst.2023.9010080 · Cited by 95 · see citing works
Adversarial machine learning: a review of methods, tools, and critical industry sectors
Pelekis, S., Koutroubas, T., Blika, A., Berdelis, A., Karakolis, E., Ntanos, C., Spiliotis, E., & Askounis, D. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11147-4 · Cited by 75 · see citing works
A Framework for Edge-Assisted Healthcare Data Analytics using Federated Learning
Hakak, S., Ray, S., Khan, W. Z., & Scheme, E. (2020) · 2020 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata50022.2020.9377873 · Cited by 62 · see citing works
Threats on Machine Learning Technique by Data Poisoning Attack: A Survey
Ahmed, I. M., & Kashmoola, M. Y. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-8059-5_36 · Cited by 37 · see citing works
The FeatureCloud Platform for Federated Learning in Biomedicine: Unified Approach
Matschinske, J., Späth, J., Bakhtiari, M., Probul, N., Majdabadi, M. M. K., Nasirigerdeh, R., Torkzadehmahani, R., Hartebrodt, A., Orbán, B.-A., Fejér, S.-J., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/42621 · Cited by 35 · see citing works
Li, Y., Yuan, D., Sani, A. S., & Bao, W. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103319 · Cited by 32 · see citing works
Enhancing federated learning robustness through randomization and mixture
Nabavirazavi, S., Taheri, R., & Iyengar, S. S. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.04.009 · Cited by 30 · see citing works
Wainakh, A., Zimmer, E., Subedi, S., Keim, J., Grube, T., Karuppayah, S., Guinea, A. S., & Mühlhäuser, M. (2022) · Sensors · doi:10.3390/s23010031 · Cited by 24 · see citing works
VADAF: Visualization for Abnormal Client Detection and Analysis in Federated Learning
Meng, L., Wei, Y., Pan, R., Zhou, S., Zhang, J., & Chen, W. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3426866 · Cited by 20 · see citing works
Implicit model specialization through dag-based decentralized federated learning
Beilharz, J., Pfitzner, B., Schmid, R., Geppert, P., Bert Arnrich, A., & Polze, A. (2021) · International Middleware Conference · doi:10.1145/3464298.3493403 · Cited by 20 · see citing works
Defending Against Data Poisoning Attacks: From Distributed Learning to Federated Learning
Tian, Y., Zhang, W., Simpson, A. C., Liu, Y., & Jiang, Z. L. (2021) · The Computer Journal · doi:10.1093/comjnl/bxab192 · Cited by 16 · see citing works
The Convergence of AI/ML and Cybersecurity: Advancing Digital Forensic Techniques
Iyengar, S. S., Nabavirazavi, S., Hariprasad, Y., Prasad, H. B., & Mohan, C. K. (2025) · Signals and communication technology · doi:10.1007/978-3-031-89327-8_4 · Cited by 12 · see citing works
FedSlice: Protecting Federated Learning Models from Malicious Participants with Model Slicing
Zhang, Z., Li, Y., Liu, B., Cai, Y., Li, D., Guo, Y., & Chen, X. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00049 · Cited by 9 · see citing works
FLCom: Robust federated learning against strong model poisoning attacks
Li, Y., Xu, J., & Yang, D. (2025) · Computer Networks · doi:10.1016/j.comnet.2025.111442 · Cited by 7 · see citing works
Byzantine-robust federated learning with ensemble incentive mechanism
Shi-hai, Z., Pu, J., Fu, X., Liu, L., & Dai, F. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.05.017 · Cited by 5 · see citing works
Federated Learning: Solutions, Challenges, and Promises
Fotohi, R., Aliee, F. S., & Farahani, B. (2022) · Iranian Conference on Advances in Enterprise Architecture (ICAEA) · doi:10.1109/icaea57644.2022.10128790 · Cited by 4 · see citing works
Trustworthy Distributed Deep Neural Network Training in an Edge Device Network
Shubha, S. S., & Shen, H. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10021055 · Cited by 3 · see citing works
SecureTransfer: A Transfer Learning Based Poison Attack Detection in ML Systems
T, A., & Kartheeban, K. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.01507141 · Cited by 2 · see citing works
Bai, F., Zhao, Y., Shen, T., Zeng, K., Zhang, X., & Zhang, C. (2025) · Symmetry · doi:10.3390/sym17050782 · Cited by 1 · see citing works
Xu, L., Gao, Z., Fan, X., Chen, L., Kim, H., Suh, T., & Shi, W. L. (2020) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom50675.2020.00137 · Cited by 1 · see citing works
Oh, S., Sung, M., Rhee, Y., Hong, N., & Park, Y. R. (2021) · JMIR Medical Informatics · doi:10.2196/24940 · Cited by 1 · see citing works
Machine Learning and the Trusted Internet of Things (IoT)
MacDermott, Á., & Hung, P. C. K. (2021) · Springer eBooks · doi:10.1007/978-1-4899-7502-7_994-1
Soni, S., Binjola, R., & Mittal, K. (2025) · BENTHAM SCIENCE PUBLISHERS eBooks · doi:10.2174/9798898810030125040005
Counterfeit IC Detection via Federated Learning: Exposure to Byzantine Data Poisoning
Lodge, N., Tambe, N. D., Aklekar, D., & Saqib, F. (2025) · IEEE Physical Assurance and Inspection of Electronics (PAINE) · doi:10.1109/paine66113.2025.11320197
Roy, V., Jain, N., Maity, D., Kushwaha, V., Pacha, S., & Punia, S. (2025) · IEEE International Conference on Recent Trends in Computing and Smart Mobility (RCSM) · doi:10.1109/rcsm67767.2025.11507260
- Authors
- Tomsett, R., Chan, K. S., & Chakraborty, S.
- Affiliations
- IBM (United Kingdom) (GB)
- DEVCOM Army Research Laboratory (US)
- IBM (United States) (US)
- Venue
- Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications
- Publisher
- SPIE
- Published
- 2019-05-10
- Pages
- 46
- OpenAlex
- W2946559073
- Cited by
- 27
- References
- 37
- Citations by year
- 2025: 7 · 2024: 4 · 2023: 3 · 2022: 4 · 2021: 6 · 2020: 3
- FWCI
- 1.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- model poisoning, distributed learning, adversarial learning, adversarial training, federated learning, situational awareness, machine learning robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
