CoChat
Sign Up
PreprintCited by 1Privacy-Preserving Technologies in DataDistributed Control Multi-Agent SystemsStochastic Gradient Optimization Techniques

The DOI matches its DataCite record.

Privacy-Preserving Distributed Optimization and Learning

Open access
Chen, Ziqin, Wang, Yongqiang·2024·arXiv
DOI 10.48550/arxiv.2403.00157arXiv 2403.00157
Open this paper in CoChat — free accountOpen PDF

Abstract

Distributed optimization and learning has recently garnered great attention due to its wide applications in sensor networks, smart grids, machine learning, and so forth. Despite rapid development, existing distributed optimization and learning algorithms require each agent to exchange messages with its neighbors, which may expose sensitive information and raise significant privacy concerns. In this survey paper, we overview privacy-preserving distributed optimization and learning methods. We first discuss cryptography, differential privacy, and other techniques that can be used for privacy preservation and indicate their pros and cons for privacy protection in distributed optimization and learning. We believe that among these approaches, differential privacy is most promising due to its low computational and communication complexities, which are extremely appealing for modern learning based applications with high dimensions of optimization variables. We then introduce several differential-privacy algorithms that can simultaneously ensure privacy and optimization accuracy. Moreover, we provide example applications in several machine learning problems to confirm the real-world effectiveness of these algorithms. Finally, we highlight some challenges in this research domain and discuss future directions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2403.00157

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 81

  1. Abraham I, Dolev D, Gonen R and Halpern J (2006), Distributed computing meets game theory: robust mechanisms for rational secret sharing and multiparty computation, Proceedings of the 25th Annual Symposium on Principles of Distributed Computing, 53-62.
  2. Alexandru AB, Gatsis K, Shoukry Y, Seshia SA, Tabuada P and Pappas GJ (2020). Cloud-based quadratic optimization with partially homomor- phic encryption. IEEE Transactions on Automatic Control 66 (5): 2357-2364.
  3. Antwi-Boasiako E, Zhou S, Liao Y, Liu Q, Wang Y and Owusu-Agyemang K (2021). Privacy preservation in distributed deep learning: A survey on distributed deep learning, privacy preservation techniques used and interesting research directions. Journal of Information Security and Applications 61: 102949.
  4. Burbano-L DA, George J, Freeman RA and Lynch KM (2019), Inferring private information in wireless sensor networks, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 4310-4314.
  5. Chen Z and Wang Y (2023a). Locally differentially private distributed online learning with guaranteed optimality. arXiv preprint arXiv:2306.14094 . · arXiv
  6. Chen Z and Wang Y (2023b). Locally differentially private gradient tracking for distributed online learning over directed graphs. arXiv preprint arXiv:2310.16105 . · arXiv
  7. Chen W, Liu L and Liu GP (2022). Privacy-preserving distributed economic dispatch of microgrids: A dynamic quantization-based consensus scheme with homomorphic encryption. IEEE Transactions on Smart Grid 14 (1): 701-713.
  8. Chen X, Huang L, He L, Dey S and Shi L (2023b). A differentially private method for distributed optimization in directed networks via state decomposition. IEEE Transactions on Control of Network Systems 10 (4): 2165-2177.
  9. Cheng Z, Ye F, Cao X and Chow MY (2021). A homomorphic encryption-based private collaborative distributed energy management system. IEEE Transactions on Smart Grid 12 (6): 5233-5243.
  10. Cheng H, Liao X and Li H (2023). Distributed online private learning of convex nondecomposable objectives. IEEE Transactions on Network Science and Engineering (Early Access) .
  11. Cormode G, Jha S, Kulkarni T, Li N, Srivastava D and Wang T (2018), Privacy at scale: Local differential privacy in practice, Proceedings of the 2018 International Conference on Management of Data, 1655-1658.
  12. Ding T, Zhu S, He J, Chen C and Guan X (2021). Differentially private distributed optimization via state and direction perturbation in multiagent systems. IEEE Transactions on Automatic Control 67 (2): 722-737.
  13. Doan TVT, Messai ML, Gavin G and Darmont J (2023). A survey on implementations of homomorphic encryption schemes. The Journal of Supercomputing : 1-42.
  14. Dong Y, Chen X, Shen L and Wang D (2020), Privacy-preserving distributed machine learning based on secret sharing, Proceedings of the International Conference on Information and Communications Security, 684-702.
  15. Du W, Han YS and Chen S (2004), Privacy-preserving multivariate statistical analysis: Linear regression and classification, Proceedings of the 2004 SIAM International Conference on Data Mining, 222-233.
  16. Dwork C, Roth A and et al. (2014). The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science 9 (3-4): 211-407.
  17. Gade S, Winnicki A and Bose S (2020). On privatizing equilibrium computation in aggregate games over networks. IFAC-PapersOnLine 53 (2): 3272-3277.
  18. Gauthier F, Gratton C, Venkategowda NK and Werner S (2020), Privacy-preserving distributed learning with nonsmooth objective functions, 2020 54th Asilomar Conference on Signals, Systems, and Computers, 42-46.
  19. Gilad-Bachrach R, Dowlin N, Laine K, Lauter K, Naehrig M and Wernsing J (2016), CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy, Proceedings of the 33rd International Conference on Machine Learning, 201-210.
  20. Guo S, Zhang T, Xu G, Yu H, Xiang T and Liu Y (2021). Topology-aware differential privacy for decentralized image classification. IEEE Trans- actions on Circuits and Systems for Video Technology 32 (6): 4016-4027.
  21. Gupta N, Gade S, Chopra N and Vaidya NH (2020). Preserving statistical privacy in distributed optimization. IEEE Control Systems Letters 5 (3): 779-784.
  22. Hale MT and Egerstedt M (2017). Cloud-enabled differentially private multiagent optimization with constraints. IEEE Transactions on Control of Network Systems 5 (4): 1693-1706.
  23. Han S, Topcu U and Pappas GJ (2016). Differentially private distributed constrained optimization. IEEE Transactions on Automatic Control 62 (1): 50-64.
  24. Han D, Liu K, Lin Y and Xia Y (2022). Differentially private distributed online learning over time-varying digraphs via dual averaging. International Journal of Robust and Nonlinear Control 32 (5): 2485-2499.
  25. He Z, He J, Chen C and Guan X (2024). Private and robust distributed nonconvex optimization via polynomial approximation. IEEE Transactions on Control of Network Systems (Early Access) .
  26. Hou M, Li D, Wu X and Shen X (2019), Differential privacy of online distributed optimization under adversarial nodes, Proceedings of 2019 Chinese Control Conference, 2172-2177.
  27. Hu R and Zhang B (2021). A privacy-masking learning algorithm for online distributed optimization over time-varying unbalanced digraphs. Journal of Mathematics 2021: 1-12.
  28. Huang Z, Mitra S and Vaidya N (2015), Differentially private distributed optimization, Proceedings of the 16th International Conference on Distributed Computing and Networking, 1-10.
  29. Huang Z, Hu R, Guo Y, Chan-Tin E and Gong Y (2019). DP-ADMM: ADMM-Based distributed learning with differential privacy. IEEE Transactions on Information Forensics and Security 15: 1002-1012.
  30. Huang L, Wu J, Shi D, Dey S and Shi L (2024). Differential privacy in distributed optimization with gradient tracking. IEEE Transactions on Automatic Control (Early Access) .
  31. Huo X and Liu M (2022b). Privacy-preserving distributed multi-agent cooperative optimization-paradigm design and privacy analysis. IEEE Control Systems Letters 6: 824-829.
  32. Kasiviswanathan SP and Smith A (2008). A note on differential privacy: Defining resistance to arbitrary side information. CoRR abs/0803.3946 . · arXiv
  33. Kilbertus N, Gasc ón A, Kusner M, Veale M, Gummadi K and Weller A (2018), Blind justice: Fairness with encrypted sensitive attributes, International Conference on Machine Learning, 2630-2639.
  34. Li N, Zhang N, Das SK and Thuraisingham B (2009). Privacy preservation in wireless sensor networks: A state-of-the-art survey. Ad Hoc Networks 7 (8): 1501-1514.
  35. Li S, Zhu H, Gao Z, Guan X, Xing K and Shen X (2012), Location privacy preservation in collaborative spectrum sensing, Proceedings of the IEEE International Conference on Computer Communications, 729-737.
  36. Li C, Zhou P, Xiong L, Wang Q and Wang T (2018). Differentially private distributed online learning. IEEE Transactions on Knowledge and Data Engineering 30 (8): 1440-1453.
  37. Lin Y, Liu K, Han D and Xia Y (2023). Statistical privacy-preserving online distributed nash equilibrium tracking in aggregative games. IEEE Transactions on Automatic Control (Early Access) .
  38. Liu C, Johansson KH and Shi Y (2022). Private stochastic dual averaging for decentralized empirical risk minimization. IFAC-PapersOnLine 55 (13): 43-48.
  39. Lou X, Tan R, Yau DK and Cheng P (2017a), Cost of differential privacy in demand reporting for smart grid economic dispatch, Proceedings of 2017 IEEE International Conference on Computer Communications, 1-9.
  40. Lou Y, Yu L, Wang S and Yi P (2017b). Privacy preservation in distributed subgradient optimization algorithms. IEEE Transactions on Cybernetics 48 (7): 2154-2165.
  41. Lu Y and Zhu M (2015). Secure cloud computing algorithms for discrete constrained potential games. IFAC-PapersOnLine 48 (22): 180-185.
  42. L ü Q, Liao X, Xiang T, Li H and Huang T (2020). Privacy masking stochastic subgradient-push algorithm for distributed online optimization. IEEE Transactions on Cybernetics 51 (6): 3224-3237.
  43. L ü Q, Zhang K, Deng S, Li Y, Li H, Gao S and Chen Y (2023). Privacy-preserving decentralized dual averaging for online optimization over directed networks. IEEE Transactions on Industrial Cyber-Physical Systems .
  44. Marcolla C, Sucasas V, Manzano M, Bassoli R, Fitzek FH and Aaraj N (2022). Survey on fully homomorphic encryption, theory, and applications. Proceedings of the IEEE 110 (10): 1572-1609.
  45. Mohassel P and Rindal P (2018), ABY 3 : A mixed protocol framework for machine learning, Proceedings of the ACM Conference on Computer and Communications Security, 35-52.
  46. Mohassel P, Rosulek M and Zhang Y (2015), Fast and secure three-party computation: The garbled circuit approach, Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, 591-602.
  47. Nozari E, Tallapragada P and Cort és J (2016). Differentially private distributed convex optimization via functional perturbation. IEEE Transactions on Control of Network Systems 5 (1): 395-408.
  48. Phong LT, Aono Y, Hayashi T, Wang L and Moriai S (2018). Privacy-preserving deep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security 13 (5): 1333-1345.
  49. Pu S, Shi W, Xu J and Nedić A (2020). Push-pull gradient methods for distributed optimization in networks. IEEE Transactions on Automatic Control 66 (1): 1-16.
  50. Rivest RL, Adleman L, Dertouzos ML and et al. (1978). On data banks and privacy homomorphisms. Foundations of Secure Computation 4 (11): 169-180.
  51. Shokri R and Shmatikov V (2015), Privacy-preserving deep learning, Association for Computing Machinery, New York, USA, pp. 1310-1321.
  52. Shoukry Y, Gatsis K, Alanwar A, Pappas GJ, Seshia SA, Srivastava M and Tabuada P (2016), Privacy-aware quadratic optimization using partially homomorphic encryption, Proceedings of IEEE 55th Conference on Decision and Control, 5053-5058.
  53. Tang F, Wu W, Liu J, Wang H and Xian M (2019). Privacy-preserving distributed deep learning via homomorphic re-encryption. Electronics 8 (4): 411.
  54. Tian N, Guo Q, Sun H and Zhou X (2023). Fully privacy-preserving distributed optimization based on secret sharing. TechRxiv Preprint .
  55. Verbraeken J, Wolting M, Katzy J, Kloppenburg J, Verbelen T and Rellermeyer JS (2020). A survey on distributed machine learning. ACM Computing Surveys 53 (2): 1-33.
  56. Wagh GS, Gupta S and Mishra S (2020), A distributed privacy preserving framework for the smart grid, Innovative Smart Grid Technologies Conference, 1-5.
  57. Wang Y and Basar T (2022). Ensuring both accurate convergence and differential privacy in nash equilibrium seeking on directed graphs. arXiv preprint arXiv:2209.04938 . · arXiv
  58. Wang Y and Bas ¸ar T (2022). Quantization enabled privacy protection in decentralized stochastic optimization. IEEE Transactions on Automatic Control .
  59. Wang Y and Nedić A (2023). Decentralized gradient methods with time-varying uncoordinated stepsizes: Convergence analysis and privacy design. IEEE Transactions on Automatic Control (Early Access) .
  60. Wang Y and Nedić A (2024). Differentially-private distributed algorithms for aggregative games with guaranteed convergence. IEEE Transactions on Automatic Control .
  61. Wang Y and Nedić A (2024). Tailoring gradient methods for differentially private distributed optimization. IEEE Transactions on Automatic Control 69 (2): 872-887.
  62. Wang Y, Hale M, Egerstedt M and Dullerud GE (2016), Differentially private objective functions in distributed cloud-based optimization, Proceed- ings of 2016 IEEE 55th Conference on Decision and Control, 3688-3694.
  63. Wang W, Li D and Wu X (2019), Privacy-preservation in online distributed dual averaging optimization, Proceedings of 2019 Chinese Control Conference, 5709-5714.
  64. Wu T, Zhao C and Zhang YJA (2021). Privacy-preserving distributed optimal power flow with partially homomorphic encryption. IEEE Transac- tions on Smart Grid 12 (5): 4506-4521.
  65. Xie L, Liu J, Lu S, Chang TH and Shi Q (2022). An efficient learning framework for federated xgboost using secret sharing and distributed optimization. ACM Transactions on Intelligent Systems and Technology 13 (5): 1-28.
  66. Xiong Y, Xu J, You K, Liu J and Wu L (2020). Privacy-preserving distributed online optimization over unbalanced digraphs via subgradient rescaling. IEEE Transactions on Control of Network Systems 7 (3): 1366-1378.
  67. Yan F, Sundaram S, Vishwanathan S and Qi Y (2013). Distributed autonomous online learning: Regrets and intrinsic privacy-preserving proper- ties. IEEE Transactions on Knowledge and Data Engineering 25 (11): 2483-2493.
  68. Yan Y, Chen Z, Varadharajan V, Hossain MJ and Town GE (2021). Distributed consensus-based economic dispatch in power grids using the paillier cryptosystem. IEEE Transactions on Smart Grid 12 (4): 3493-3502.
  69. Yao AC (1982), Protocols for secure computations, 23rd annual symposium on foundations of computer science, 160-164.
  70. Ye M, Hu G, Xie L and Xu S (2021). Differentially private distributed nash equilibrium seeking for aggregative games. IEEE Transactions on Automatic Control 67 (5): 2451-2458.
  71. Ye M, Han QL, Ding L and Xu S (2023). Distributed nash equilibrium seeking in games with partial decision information: A survey. Proceedings of the IEEE 111 (2): 140-157.
  72. Zhang C and Wang Y (2019). Enabling privacy-preservation in decentralized optimization. IEEE Transactions on Control of Network Systems 6 (2): 679-689.
  73. Zhang T and Zhu Q (2016). Dynamic differential privacy for admm-based distributed classification learning. IEEE Transactions on Information Forensics and Security 12 (1): 172-187.
  74. Zhang C, Ahmad M and Wang Y (2018a). ADMM based privacy-preserving decentralized optimization. IEEE Transactions on Information Foren- sics and Security 14 (3): 565-580.
  75. Zhang C, Gao H and Wang Y (2018b). Privacy-preserving decentralized optimization via decomposition. arXiv preprint arXiv:1808.09566 . · arXiv
  76. Zhang D, Chen X, Wang D and Shi J (2018c), A survey on collaborative deep learning and privacy-preserving, 2018 IEEE Third International Conference on Data Science in Cyberspace, 652-658.
  77. Zhang X, Khalili MM and Liu M (2018d), Improving the privacy and accuracy of ADMM-based distributed algorithms, International Conference on Machine Learning, 5796-5805.
  78. Zhang M, Chen Y and Lin J (2021). A privacy-preserving optimization of neighborhood-based recommendation for medical-aided diagnosis and treatment. IEEE Internet of Things Journal 8 (13): 10830-10842.
  79. Zhang Q, He F, Gu J, Gu B, Deng C, Huang H and Tao D (2023). BAMBI: Vertical federated bilevel optimization with privacy-preserving and computation efficiency. https://openreview.net/forum?id=pO7KggcbMiP.
  80. Zhu J, Xu C, Guan J and Wu DO (2018). Differentially private distributed online algorithms over time-varying directed networks. IEEE Transactions on Signal and Information Processing over Networks 4 (1): 4-17.
  81. Zhu L, Liu Z and Han S (2019). Deep leakage from gradients. Advances in Neural Information Processing Systems 32.

Cited by

Sorted by citations

See all 1

Metadata

Authors
Chen, Z., & Wang, Y.
Affiliations
Clemson University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2403.00157
arXiv
2403.00157
OpenAlex
W4392425982
Cited by
1
References
0
Citations by year
2025: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Engineering › Control and Systems Engineering › Distributed Control Multi-Agent Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords
privacy-preserving distributed learning, differential privacy, cryptography, privacy-utility trade-off, distributed learning, communication complexity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Privacy-Preserving Distributed Optimization and Learning. https://app.cochat.ai/papers/10.48550/arxiv.2403.00157
The paper
Chen, Z., & Wang, Y. (2024). Privacy-Preserving Distributed Optimization and Learning. arXiv. https://doi.org/10.48550/arxiv.2403.00157