Gradient Obfuscation Gives a False Sense of Security in Federated Learning
Open accessAbstract
Federated learning has been proposed as a privacy-preserving machine learning framework that enables multiple clients to collaborate without sharing raw data. However, client privacy protection is not guaranteed by design in this framework. Prior work has shown that the gradient sharing strategies in federated learning can be vulnerable to data reconstruction attacks. In practice, though, clients may not transmit raw gradients considering the high communication cost or due to privacy enhancement requirements. Empirical studies have demonstrated that gradient obfuscation, including intentional obfuscation via gradient noise injection and unintentional obfuscation via gradient compression, can provide more privacy protection against reconstruction attacks. In this work, we present a new data reconstruction attack framework targeting the image classification task in federated learning. We show that commonly adopted gradient postprocessing procedures, such as gradient quantization, gradient sparsification, and gradient perturbation, may give a false sense of security in federated learning. Contrary to prior studies, we argue that privacy enhancement should not be treated as a byproduct of gradient compression. Additionally, we design a new method under the proposed framework to reconstruct the image at the semantic level. We quantify the semantic privacy leakage and compare with conventional based on image similarity scores. Our comparisons challenge the image data leakage evaluation schemes in the literature. The results emphasize the importance of revisiting and redesigning the privacy protection mechanisms for client data in existing federated learning algorithms.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Obfuscated gradients may become subject to reconstruction attacks and leak information.
“However, it needs to be considered that also obfuscated gradients may become subject to reconstruction attacks and leak information71,72.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [71]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1× · see citing works
“Three different model architectures are considered as the instances of shared model in federated learning, namely, LeNet [73], VGG-7 [55], and ResNet-18 [23].”
BackgroundU-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · cited 1× · see citing works
“We adopt the U-Net architecture [53] as the GAN generator, which is composed of an encoder that downsamples the images and a decoder that upsamples the feature maps back to the original size.”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 5× · see citing works
“The Adam optimizer is adopted for the GAN training, and the learning rate is set to 1 × 10 − 5 for both the generator and the discriminator.”
MethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · cited 3× · see citing works
“We choose the LeNet, VGG-7, and ResNet-18 and demonstrate the reconstruction results in Figure 16.”
BackgroundImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1× · see citing works
“In particular, we randomly apply one of the downsam-pling approaches, including nearest neighbor, bilinear scaling, and bicubic scaling, and add Gaussian noise to the training examples in the ImageNet dataset [14].”
MethodImage quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 1× · see citing works
“Pioneering work [17,61, 73] used a conventional image similarity score or the distance between original and reconstructed images as the evaluation metric, including the mean squared error (MSE), peak signal-to-noise ratio (PSNR) [20], and structural similarity index measure (SSIM) [59].”
MethodImage-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 1× · see citing works
“This architecture has been widely adopted in image-to-image translation [28] and image inpainting [46].”
BackgroundThe Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623 · cited 16× · see citing works
“Follow-up studies [26,65] further leveraged the neural network based image perceptual scores, such as learned perceptual image patch similarity (LPIPS) [69] as the attack evaluation metric.”
BackgroundMethodInfluentialCommunication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · cited 1× · see citing works
“Meanwhile, differentially private machine learning has been developed to achieve provable privacy guarantees [1].”
BackgroundTHE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE. <sup>1</sup>
Jaccard, P. (1912) · New Phytologist · doi:10.1111/j.1469-8137.1912.tb05611.x · Cited by 4,561 · see citing works
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023 · cited 12× · see citing works
“Generic solutions such as federated averaging (FedAvg) [43] have been proposed with different variants.”
BackgroundInfluentialLarge Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717 · cited 2× · see citing works
“Instead of focusing on the pixel-level error between the original image and the reconstructed counterpart, ROGS leverages a pretrained generative model G , such as BigGAN [11], to synthesize realistic images while maintaining the semantics of the original image.”
MethodKuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01316-z · Cited by 2,460 · cited 1× · see citing works
“Specifically, the classification network has been trained on OpenImage (V6) [33], which contains 9,600 classes, including the object category, color, season, etc.”
MethodZhu, L., & Han, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-63076-8_2 · Cited by 1,906 · see citing works
Measuring the Effects of Non-Identical Data Distribution for Federated Visual ClassificationPreprint
Hsu, T., Qi, & Brown, M. (2019) · arXiv.org · doi:10.48550/arxiv.1909.06335 · Cited by 1,757 · see citing works
Exploiting Unintended Feature Leakage in Collaborative Learning
Melis, L., Song, C., De Cristofaro, E., & Shmatikov, V. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00029 · Cited by 1,734 · cited 1× · see citing works
“Image reconstruction via the ROGS attack differs from the notion of property inference attack [44] that target sensitive information.”
BackgroundDesigning a Practical Degradation Model for Deep Blind Image Super-Resolution
Zhang, K., Liang, J., Gool, L. V., & Timofte, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00475 · Cited by 814 · cited 1× · see citing works
“This procedure approximates the effect of the image degradation at different levels [68].”
Inverting Gradients -- How easy is it to break privacy in federated learning?Preprint
Geiping, J., Bauermeister, H., Dröge, H., & Moeller, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.14053 · Cited by 749 · cited 26× · see citing works
“This method was further improved to recover high-resolution images with increased batch sizes [14, 57].”
BackgroundMethodInfluentialEdgeConnect: Generative Image Inpainting with Adversarial Edge LearningPreprint
Nazeri, K., Ng, E., Joseph, T., Qureshi, F. Z., & Ebrahimi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00212 · Cited by 747 · cited 2× · see citing works
“This architecture has been widely adopted in image-to-image translation [28] and image inpainting [46].”
BackgroundMethodFederated learning meets blockchain in edge computing: Opportunities and challenges
Nguyen, D. C., Ding, M., Pham, Q.-V., Pathirana, P. N., Le, L. B., Seneviratne, A., Li, J., Niyato, D., & Poor, H. V. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3072611 · Cited by 661 · cited 1× · see citing works
“We do not consider cryptography based defense [59] and other more advanced solutions, including blockchain-based decentralized optimization or federated network topology modification [40].”
MethodFedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1909.13014 · Cited by 587 · see citing works
Sparse Communication for Distributed Gradient Descent
Aji, A. F., & Heafield, K. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1045 · Cited by 585 · cited 4× · see citing works
“In Top-k compression [2], the client selects the k largest components of the gradient in absolute values and zeros out other entires.”
BackgroundMethodResultInfluentialiDLG: Improved Deep Leakage from GradientsPreprint
Zhao, B., Mopuri, K. R., & Bilen, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.02610 · Cited by 575 · cited 3× · see citing works
“Our work is closely related to those studies focused on gradient inversion attacks in federated learning [17,65,72,73].”
BackgroundsignSGD: Compressed Optimisation for Non-Convex Problems
Bernstein, J., Wang, Y.-X., Azizzadenesheli, K., & Anandkumar, A. (2018) · arXiv · doi:10.48550/arxiv.1802.04434 · Cited by 541 · cited 2× · see citing works
“[7] showed that the signSGD algorithm converges fast even though only the signs of the gradients are transmitted.”
BackgroundMethodStich, S. U., Cordonnier, J.-B., & Jaggi, M. (2018) · arXiv · doi:10.48550/arxiv.1809.07599 · Cited by 508 · see citing works
Xia, W., Zhang, Y., Yang, Y., Xue, J., Zhou, B., & Yang, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3181070 · Cited by 496 · cited 1× · see citing works
“From the technical perspective, this process is similar to inverting an input to the GAN latent space, which is also known as the GAN inversion task in the literature [26,55].”
MethodSee through Gradients: Image Batch Recovery via GradInversion
Yin, H., Mallya, A., Vahdat, A., Álvarez, J. M., Kautz, J., & Molchanov, P. A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01607 · Cited by 482 · cited 9× · see citing works
“Follow-up studies [26,65] further leveraged the neural network based image perceptual scores, such as learned perceptual image patch similarity (LPIPS) [69] as the attack evaluation metric.”
BackgroundMethodInfluentialThe Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks
Zhang, Y., Jia, R., Pei, H., Wang, W., Li, B., & Song, D. X. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00033 · Cited by 481 · cited 3× · see citing works
“For example, Zhang et al. [71] studied the model inversion problem in centralized learning.”
BackgroundInfluentialPractical Secure Aggregation for Federated Learning on User-Held DataPreprint
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04482 · Cited by 468 · cited 1× · see citing works
“Other works focused on the protection provided by a trusted shuffling server [11, 16] or secure aggregation protocols [8].”
BackgroundThe Non-IID Data Quagmire of Decentralized Machine LearningPreprint
Hsieh, K., Phanishayee, A., Mutlu, O., & Gibbons, P. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00189 · Cited by 337 · cited 1× · see citing works
“As the batch normalization layers are reported to cause accuracy drop and raise privacy concerns in federated learning [21, 23], we remove these layers in our study and leave them for future work.”
MethodUser-Level Privacy-Preserving Federated Learning: Analysis and Performance Optimization
Wei, K., Li, J., Ding, M., Ma, C., Su, H., Zhang, B., & Poor, H. V. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3056991 · Cited by 300 · cited 3× · see citing works
“To achieve a provable differential privacy guarantee, the client will inject additive Gaussian noise to the gradient [1,60].”
BackgroundA Survey of Privacy Attacks in Machine Learning
Rigaki, M., & Garcia, S. (2024) · ACM Computing Surveys · doi:10.1145/3624010 · Cited by 285 · cited 1× · see citing works
“The attacks concerning data privacy and model confidentiality in federated learning can be categorized into three types: membership inference attack, model inversion/data reconstruction attack, and property inference attack [28, 44].”
BackgroundcpSGD: Communication-efficient and differentially-private distributed SGDPreprint
Agarwal, N., Suresh, A. T., Yu, F., Kumar, S., & McMahan, H. B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10559 · Cited by 264 · cited 2× · see citing works
“Other defenses combine differential privacy with secure aggregation [2,33] to achieve stronger protection.”
BackgroundA Framework for Evaluating Client Privacy Leakages in Federated Learning
Wei, W., Liu, L., Loper, M. L., Chow, K.-H., Gürsoy, M. E., Truex, S., & Wu, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58951-6_27 · Cited by 236 · cited 8× · see citing works
“In addition, Wei et al. [61] proposed the attack success rate, namely, the percentage of successfully reconstructed training data.”
BackgroundMethodResultInfluentialThreats to federated learning A survey
Lyu, L., Yu, H., & Yang, Q. (2020) · arXiv · doi:10.48550/arxiv.2003.02133 · Cited by 221 · cited 1× · see citing works
“In spite of its popularity, FedAvg and its variants have exhibited vulnerabilities in protecting client data privacy [36].”
BackgroundA Taxonomy of Attacks on Federated Learning
Jere, M. S., Farnan, T., & Koushanfar, F. (2021) · IEEE Security & Privacy · doi:10.1109/msec.2020.3039941 · Cited by 217 · cited 1× · see citing works
“The attacks concerning data privacy and model confidentiality in federated learning can be categorized into three types: membership inference attack, model inversion/data reconstruction attack, and property inference attack [28, 44].”
BackgroundRobbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models
Fowl, L. H., Geiping, J., Czaja, W., et al. (2021) · arXiv.org · doi:10.48550/arxiv.2110.13057 · Cited by 201 · cited 1×
“It has been shown that when the integrity of the model/gradient is compromised, the data can be trivially reconstructed [16,63].”
BackgroundSun, J., Li, A., Wang, B., Yang, H., Li, H. H., & Chen, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00919 · Cited by 192 · cited 4× · see citing works
“It has been shown that Soteria is robust and does not impede the convergence of FedAvg [56].”
BackgroundMethodInfluentialEvaluating Gradient Inversion Attacks and Defenses in Federated LearningPreprint
Huang, Y., Gupta, S., Song, Z., Li, K., & Arora, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.00059 · Cited by 184 · cited 5× · see citing works
“Follow-up studies [26,65] further leveraged the neural network based image perceptual scores, such as learned perceptual image patch similarity (LPIPS) [69] as the attack evaluation metric.”
BackgroundMethodInfluentialA Field Guide to Federated OptimizationPreprint
Wang, J., Charles, Z., Xu, Z., Joshi, G., McMahan, H. B., y Arcas, B. A., Al-Shedivat, M., Andrew, G., Avestimehr, S., Daly, K., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.06917 · Cited by 181 · see citing works
On the "steerability" of generative adversarial networksPreprint
Jahanian, A., Chai, L., & Isola, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.07171 · Cited by 169 · cited 1× · see citing works
“From the technical perspective, this process is similar to inverting an input to the GAN latent space, which is also known as the GAN inversion task in the literature [26,55].”
MethodOn the Application of Symmetric Dirichlet Distributions and their Mixtures to Contingency Tables
Good, I. J. (1976) · doi:10.1214/aos/1176343649 · Cited by 167 · cited 1×
“[22], we simulate the non-IID data with the symmetric Dirichlet distribution [18].”
MethodTowards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images
Orekondy, T., Schiele, B., & Fritz, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.398 · Cited by 162 · cited 1× · see citing works
“Sensitive information is generally difficult to define and corresponding privacy relies on personal preference [48].”
BackgroundThe distributed discrete gaussian mechanism for federated learning with secure aggregation
Kairouz, P., Liu, Z., & Steinke, T. (2021) · arXiv · doi:10.48550/arxiv.2102.06387 · Cited by 117 · cited 2× · see citing works
“Other defenses combine differential privacy with secure aggregation [2,33] to achieve stronger protection.”
BackgroundGradient-Leakage Resilient Federated Learning
Wei, W., Liu, L., Wu, Y., Su, G., & Iyengar, A. (2021) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs51616.2021.00081 · Cited by 116 · cited 9× · see citing works
“FedCDP [62].”
BackgroundMethodResultInfluentialGradient Inversion with Generative Image PriorPreprint
Jeon, J., Kim, J., Lee, K., Oh, S., & Ok, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14962 · Cited by 95 · cited 2× · see citing works
“[27] updated the parameters of a generative model to obtain reconstructed results.”
BackgroundA Framework for Evaluating Gradient Leakage Attacks in Federated LearningPreprint
Wei, W., Liu, L., Loper, M. L., Chow, K. H., Gürsoy, M. E., Truex, S., & Wu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.10397 · Cited by 92 · cited 24× · see citing works
“[53] may not be a good indicator in some scenarios.”
BackgroundMethodResultInfluentialInstaHide: Instance-hiding Schemes for Private Distributed LearningPreprint
Huang, Y., Song, Z., Li, K., & Arora, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02772 · Cited by 86 · cited 1× · see citing works
“For example, Huang et al. [26, 27] proposed to encode the images by compositing several training examples and flipping the signs of the data randomly.”
MethodLam, M., Wei, G.-Y., Brooks, D., et al. (2021) · International Conference on Machine Learning · doi:10.48550/arxiv.2106.06089 · Cited by 84 · cited 1×
“Some studies have shown that the individual gradient can be recovered in secure aggregation [39].”
BackgroundGRACE: A Compressed Communication Framework for Distributed Machine Learning
Xu, H., Ho, C.-Y., Abdelmoniem, A. M., Dutta, A., Bergou, E. H., Karatsenidis, K., Canini, M., & Kalnis, P. (2021) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs51616.2021.00060 · Cited by 71 · see citing works
A Survey on Gradient Inversion: Attacks, Defenses and Future Directions
Zhang, R., Guo, S., Wang, J.-X., et al. (2022) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.2206.07284 · Cited by 66 · cited 1×
“Recent studies suggest that low-bit gradient quantization can be used as effective defenses [70].”
BackgroundTAG: Gradient Attack on Transformer-based Language Models
Deng, J., Wang, Y., Ji, L., et al. (2021) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.findings-emnlp.305 · Cited by 59 · cited 1× · see citing works
“Meanwhile, it has been shown that the modern Transformer model adopted in language tasks can also be compromised under the data reconstruction attack [13].”
MethodBayesian Framework for Gradient Leakage
Balunovic, M., Dimitrov, D. I., Staab, R., et al. (2021) · International Conference on Learning Representations · doi:10.48550/arxiv.2111.04706 · Cited by 56 · cited 2×
“The data reconstruction attack can also be interpreted from a Bayesian perspective [6].”
BackgroundMulti-label Classification with Partial Annotations using Class-aware Selective Loss
Ben-Baruch, E., Ridnik, T., Friedman, I., Ben-Cohen, A., Zamir, N., Noy, A., & Zelnik‐Manor, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00472 · Cited by 52 · see citing works
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private LearningPreprint
Yu, D., Zhang, H., Chen, W., & Liu, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.12677 · Cited by 49 · cited 3× · see citing works
“In this study, we perform attacks on various gradient obfus-cation defense schemes, including gradient compression [61], differentially private training [62], and representation perturbation [66].”
BackgroundMethodPRECODE - A Generic Model Extension to Prevent Deep Gradient Leakage
Scheliga, D., Mäder, P., & Seeland, M. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00366 · Cited by 44 · cited 3× · see citing works
“We treat the clipped gradient ˆ g as the raw signal and use the signal-to-noise ratio (SNR) [54], and FedCDP [62].”
BackgroundMethodCompression Boosts Differentially Private Federated Learning
Kerkouche, R., Ács, G., Castelluccia, C., & Genevès, P. (2021) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp51992.2021.00029 · Cited by 36 · cited 1× · see citing works
“More sophisticated schemes add noise to the manifold representation of gradients [30, 58], where the model accuracy can be greatly improved given the same privacy protection with reduced noise power.”
BackgroundIs Private Learning Possible with Instance Encoding?
Carlini, N., Deng, S., Garg, S., Jha, S., Mahloujifar, S., Mahmoody, M., Thakurta, A., & Tramèr, F. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00099 · Cited by 35 · cited 1× · see citing works
“The vulnerability of data encoding has been shown in recent work [12]: with a well-designed attack that formulated the problem as a noisy linear system, the original images can be recovered.”
BackgroundFishing for User Data in Large-Batch Federated Learning via Gradient MagnificationPreprint
Wen, Y., Geiping, J., Fowl, L., Goldblum, M., & Goldstein, T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.00580 · Cited by 33 · cited 1× · see citing works
“It has been shown that when the integrity of the model/gradient is compromised, the data can be trivially reconstructed [16,63].”
BackgroundA Quantitative Metric for Privacy Leakage in Federated Learning
Liu, Y., Zhu, X., Wang, J.-Z., et al. (2021) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp39728.2021.9413539 · Cited by 26 · cited 1× · see citing works
“Compared to other indicators, such as differential privacy budget of the algorithm or mutual information between the training examples and gradient [41,45], LPIPS is more intuitive and interpretable.”
BackgroundFederated Deep Learning with Bayesian Privacy
Gu, H.-L., Fan, L.-X., Li, B.-W., et al. (2021) · arXiv.org · doi:10.48550/arxiv.2109.13012 · Cited by 24 · cited 1×
“However, the vulnerabilities can still be exploited by model inversion attacks as differential privacy does not imply protection against reconstruction attack or Bayesian restoration targeting attribute privacy [19, 62].”
BackgroundLabel Leakage from Gradients in Distributed Machine Learning
Wainakh, A., Müßig, T., Grube, T., et al. (2021) · IEEE Consumer Communications & Networking Conference (CCNC) · doi:10.1109/ccnc49032.2021.9369498 · Cited by 23 · cited 2× · see citing works
“This attack has been further developed by revealing labels via analytical methods [50, 57, 63].”
BackgroundLebrun, T., Boutet, A., Aalmoes, J., & Baud, A. (2022) · International Middleware Conference · doi:10.1145/3528535.3565240 · Cited by 16 · cited 1× · see citing works
“Similar issues also exist in those defense methods that put attention on a particular type of attack [9].”
BackgroundSeparation of Powers in Federated Learning (Poster Paper)
Cheng, P.-C., Eykholt, K., Gu, Z., et al. (2021) · ResilientFL · doi:10.1145/3477114.3488765 · Cited by 9 · cited 1× · see citing works
“Other works focused on the protection provided by a trusted shuffling server [11, 16] or secure aggregation protocols [8].”
BackgroundExploring the Security Boundary of Data Reconstruction via Neuron Exclusivity AnalysisPreprint
Pan, X., Zhang, M., Yan, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13356 · Cited by 8 · see citing works
Minimal Model Structure Analysis for Input Reconstruction in Federated LearningPreprint
Jia, Q., Nassar, H., & Hansen, L. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15718 · Cited by 4 · see citing works
Separation of Powers in Federated LearningPreprint
Cheng, P.-C., Eykholt, K., Gu, Z., Jamjoom, H. T., Jayaram, K. R., Valdez, E., & Verma, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09400 · Cited by 3 · see citing works
Quantifying and Localizing Usable Information Leakage from Neural Network Gradients
Mo, F., Borovykh, A., Malekzadeh, M., Demetriou, S., Gündüz, D., & Haddadi, H. (2021) · arXiv · doi:10.48550/arxiv.2105.13929 · Cited by 1 · see citing works
Multi-label Classification with Partial Annotations using Class-aware Selective LossPreprint
Ben-Baruch, E., Ridnik, T., Friedman, I., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10955
Flagged first, then by citations
Research works printed but not linked: 9
- [2][2] Naman Agarwal, Ananda Theertha Suresh, Felix Xinnan X Yu, Sanjiv Kumar, and Brendan McMahan. cpSGD: Communication-efficient and differentiallyprivate distributed SGD. Advances in Neural Information Processing Systems , 31, 2018. -
- [4][4] Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. QSGD: Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems , pages 1709-1720, 2017. -
- [5][5] Pierre Baldi. Autoencoders, unsupervised learning, and deep architectures. In Proceedings of ICML Workshop on Unsupervised and Transfer Learning , pages 37-49. JMLR Workshop and Conference Proceedings, 2012. -
- [10][10] Antoine Boutet, Thomas Lebrun, Jan Aalmoes, and Adrien Baud. MixNN: Protection of federated learning against inference attacks by mixing neural network layers. arXiv preprint arXiv:2109.12550 , 2021. - · arXiv
- [18][18] Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy. In International Conference on Machine Learning , 2016. -
- [19][19] Antonious Girgis, Deepesh Data, Suhas Diggavi, Peter Kairouz, and Ananda Theertha Suresh. Shuffled model of differential privacy in federated learning. In International Conference on Artificial Intelligence and Statistics , 2021. -
- [20][20] Rafael C. Gonzalez and Richard E. Woods. Digital Image Processing, 3rd Edition . 2014. -
- [37][37] Alex Krizhevsky. Learning multiple layers of features from tiny images. Master thesis, Dept. of Comput. Sci., Univ. of Toronto, Toronto, Canada , 2009. -
- [67][67] Chengliang Zhang, Suyi Li, Junzhe Xia, Wei Wang, Feng Yan, and Yang Liu. BatchCrypt: Efficient homomorphic encryption for cross-silo federated learning. In USENIX Annual Technical Conference , pages 493-506, 2020. -
Cited by
Sorted by citations
Pahud de Mortanges, A., Luo, H., Shu, S. Z., Kamath, A., Suter, Y., Shelan, M., Pöllinger, A., & Reyes, M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01190-w · Cited by 93 · see citing works
“However, it needs to be considered that also obfuscated gradients may become subject to reconstruction attacks and leak information 71,72 .”
A Duty to Forget, a Right to be Assured? Exposing Vulnerabilities in Machine Unlearning Services
Hu, H., Wang, S., Chang, J., Zhong, H., Sun, R., Hao, S., Zhu, H., & Xue, M. (2024) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.24252 · Cited by 37 · see citing works
A Privacy Preserving System for Movie Recommendations Using Federated Learning
Neumann, D., Lutz, A., Müller, K., & Samek, W. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3634686 · Cited by 27 · see citing works
“[83], however call the effectiveness of gradient obfuscation into question, by proposing a novel data reconstruction attack scheme.”
BackgroundArtificial intelligence security and privacy: a survey
He, X., Xu, G., Han, X., Wang, Q., Zhao, L., Shen, C., Lin, C., Zhao, Z., Li, Q., Yang, L., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-025-4388-5 · Cited by 23 · see citing works
He, Z., Zhu, G., Zhang, S., Luo, E., & Zhao, Y. (2025) · Electronics · doi:10.3390/electronics14112183 · Cited by 11 · see citing works
Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention
Shen, L., Pu, Y., Ji, S.-L., Li, C., Zhang, X., Ge, C., & Wang, T. (2023) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.24115 · Cited by 23 · see citing works
“While Athalye et al. [1] and Yue et al. [70] demonstrated that gradient obfuscation could be circumvented by approximating the derivatives, their focus was primarily on the computer vision (CV) domain, which predominantly deals with continuous input spaces.”
BackgroundNoiseless Privacy-Preserving Decentralized Learning
Biswas, S., Even, M., Kermarrec, A.-M., Massoulié, L., Pires, R. P., Sharma, R., & de Vos, M. (2024) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2025-0043 · Cited by 8 · see citing works
Privacy Analysis of Federated Learning via Dishonest Servers
Jeter, T. R., & Thai, M. T. (2023) · IEEE International Conference on Big Data Security on Cloud, IEEE International Conference on High Performance and Smart Computing, and IEEE International Conference on Intelligent Data and Security (BigDataSecurity/HPSC/IDS) · doi:10.1109/bigdatasecurity-hpsc-ids58521.2023.00015 · Cited by 5 · see citing works
“FedAvg reports significantly lower PSNR values because gradients are updated at each epoch unlike in FedSGD [19, 20, 21, 22].”
BackgroundOn the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook
Fan, M., Wang, C., Chen, C., Liu, Y., & Huang, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02375-w · Cited by 17 · see citing works
“Given the above, we review recent progress revolving around privacy issues of diffusion models and LLMs.”
Erel-Özçevı̇k, M., Özçift, A., Özçevik, Y., et al. (2024) · Electronics · doi:10.3390/electronics13214261 · Cited by 4 · see citing works
A High-Dimensional Gradient Inversion Attack Based on Feature Distillation in Federated Learning
Cai, H., Ma, M., Wang, Z., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9812-7_15
A Privacy-Enhanced Multi-Stage Dimensionality Reduction Vertical Federated Clustering Framework
Wang, J., Zhang, J., & Chen, X. (2025) · Electronics · doi:10.3390/electronics14163182
- Authors
- Yue, K., Jin, R., Wong, C.-W., Baron, D., & Dai, H.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2206.04055
- OpenAlex
- W4281609047
- Cited by
- 26
- References
- 66
- Citations by year
- 2025: 11 · 2024: 11 · 2023: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Cryptography and Data Security
- Keywords
- federated learning, client privacy, data reconstruction attacks, gradient compression, gradient quantization, gradient sparsification, gradient perturbation, image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
