The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU
Abstract
We introduce CryptGPU, a system for privacy-preserving machine learning that implements all operations on the GPU (graphics processing unit). Just as GPUs played a pivotal role in the success of modern deep learning, they are also essential for realizing scalable privacy-preserving deep learning. In this work, we start by introducing a new interface to losslessly embed cryptographic operations over secret-shared values (in a discrete domain) into floating-point operations that can be processed by highly-optimized CUDA kernels for linear algebra. We then identify a sequence of "GPU-friendly" cryptographic protocols to enable privacy-preserving evaluation of both linear and non-linear operations on the GPU. Our microbenchmarks indicate that our private GPU-based convolution protocol is over 150× faster than the analogous CPU-based protocol; for non-linear operations like the ReLU activation function, our GPU-based protocol is around 10× faster than its CPU analog. With CryptGPU, we support private inference and training on convolutional neural networks with over 60 million parameters as well as handle large datasets like ImageNet. Compared to the previous state-of-the-art, our protocols achieve a 2× to 8× improvement in private inference for large networks and datasets. For private training, we achieve a 6× to 36× improvement over prior state-of-the-art. Our work not only showcases the viability of performing secure multiparty computation (MPC) entirely on the GPU to newly enable fast privacy-preserving machine learning, but also highlights the importance of designing new MPC primitives that can take full advantage of the GPU’s computing capabilities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Privacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356
SecureML: A System for Scalable Privacy-Preserving Machine Learning
Mohassel, P., & Zhang, Y. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.12 · Cited by 1,936
Stealing Machine Learning Models via Prediction APIs
Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016) · arXiv · doi:10.48550/arxiv.1609.02943 · Cited by 796
CrypTFlow: Secure TensorFlow Inference
Kumar, N., Rathee, M., Chandran, N., Gupta, D., Rastogi, A., & Sharma, R. (2020) · 2020 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40000.2020.00092 · Cited by 245
Parallel random numbers: as easy as 1, 2, 3
Salmon, J. K., Moraes, M. A., Dror, R. O., & Shaw, D. E. (2011) · Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/2063384.2063405 · Cited by 227
Research works printed but not linked: 48
- 10.1007/s11263-015-0816-y · DOI · Paper page
- 10.1145/2810103.2813677 · DOI · Paper page
- 10.1504/IJSN.2015.071829 · DOI · Paper page
- 10.1109/DAC.2018.8465894 · DOI · Paper page
- 10.14722/ndss.2020.23005 · DOI
- 10.2478/popets-2020-0036 · DOI
- lecun, 0, The MNIST database
- 10.46586/tches.v2018.i2.70-95 · DOI
- 10.1145/2976749.2978318 · DOI · Paper page
- badawi, 2020, Towards the alexnet moment for homomorphic encryption: HCNN, the first homomorphic cnn on encrypted data with gpus, IEEE Transactions on Emerging Topics in Computing
- simonyan, 2015, Very deep convolutional networks for large-scale image recognition, ICLRE
- 10.2478/popets-2020-0077 · DOI
- riazi, 2019, XONN: xnor-based oblivious deep neural network inference, USENIX Security Symposium, 1501
- 10.1145/3133956.3134056 · DOI · Paper page
- gilad-bachrach, 2016, Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy, ICML, 201
- 10.1145/3196494.3196522 · DOI · Paper page
- chandran, 2017, Ezpc: Programmable, efficient, and scalable secure two-party computation for machine learning, Cryptology ePrint Archive Report 2017/1109
- 0, PyTorch/CSPRNG
- gentry, 2009, A fully homomorphic encryption scheme, PhD thesis
- 10.2478/popets-2021-0011 · DOI · Paper page
- juvekar, 2018, GAZELLE: A low latency framework for secure neural network inference, USENIX Security Symposium, 1651
- ben-or, 1988, Completeness theorems for non-cryptographic fault-tolerant distributed computation (extended abstract), STOC, 1
- goldreich, 1987, How to play any mental game or A completeness theorem for protocols with honest majority, STOC, 218
- mishra, 2020, Delphi: A cryptographic inference service for neural networks, Usenix Security, 2505
- 10.2478/popets-2019-0035 · DOI · Paper page
- 10.2478/popets-2021-0011 · DOI · Paper page
- nair, 2010, Rectified linear units improve restricted boltzmann machines, ICML, 807
- beaver, 1991, Efficient multiparty protocols using circuit randomization, Crypto, 420
- 10.14722/ndss.2015.23113 · DOI · Paper page
- furukawa, 2017, High-throughput secure three-party computation for malicious adversaries and an honest majority, Eurocrypt, 225
- tramèr, 2019, Slalom: Fast, verifiable and private execution of neural networks in trusted hardware, ICLRE
- kamara, 2011, Outsourcing multi-party computation, IACR Cryptol ePrint Arch, 2011, 272
- 10.14722/ndss.2020.24202 · DOI
- 0, Cudnn
- 0, Cublas
- mohassel, 2018, ABY3: A mixed protocol framework for machine learning, ACM CCSW, 35
- goldreich, 2004, The Foundations of Cryptography - Volume 2 Basic Applications
- 10.1007/s001459910006 · DOI · Paper page
- knott, 2020, CrypTen: Secure multi-party computation meets machine learning, Proceedings of the NeurIPS Workshop on Privacy-Preserving Machine Learning
- 0, Cloud tensor processing units (tpus)
- 10.1145/2976749.2978331 · DOI · Paper page
- 10.1002/ecjc.4430720906 · DOI · Paper page
- 10.1162/NECO_a_00052 · DOI · Paper page
- paszke, 2019, PyTorch: An imperative style, high-performance deep learning library, NeurIPS, 8024
- yao, 1986, How to generate and exchange secrets (extended abstract), FOCS, 162
- 10.1145/3338466.3358922 · DOI
- jagielski, 2019, High-fidelity extraction of neural network models, CoRR
- goodfellow, 2016, Deep Learning
Cited by
CrypTen: Secure Multi-Party Computation Meets Machine Learning
Knott, B., Venkataraman, S., Hannun, A., Sengupta, S., Ibrahim, M., & van der Maaten, L. (2021) · arXiv · doi:10.48550/arxiv.2109.00984 · Cited by 257
Self-citationde Almeida, J. G., Messiou, C., Withey, S. J., Matos, C., Koh, D.-M., & Papanikolaou, N. (2025) · European Radiology · doi:10.1007/s00330-025-11654-6 · Cited by 23
Kaner, S., Garipcan, A. M., & Erdem, E. (2025) · Journal of King Saud University Computer and Information Sciences · doi:10.1007/s44443-025-00271-4 · Cited by 11
“…j + n ⋅ w m , n + b processing capabilities and deep learning strengths, all three models facilitate efficient analysis of large-scale datasets, offering innovative solutions aligned with the security requirements of modern cryptographic systems (Tan et al. 2021; Deebak and Hwang 2025; Tolba 2024).”
Metadata
- Authors
- Tan, S., Knott, B., Tian, Y., & Wu, D. J.
- Venue
- 2021 IEEE Symposium on Security and Privacy (SP)
- Publisher
- IEEE
- Published
- 2021-05
- Pages
- 1021-1038
- ISSN
- 1081-6011 (ISSN-L)
- OpenAlex
- W3155184874
- Cited by
- 159
- References
- 10
- FWCI
- 12.09
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU. https://app.cochat.ai/papers/10.1109/sp40001.2021.00098
- The paper
- Tan, S., Knott, B., Tian, Y., & Wu, D. J. (2021). CryptGPU: Fast Privacy-Preserving Machine Learning on the GPU. 2021 IEEE Symposium on Security and Privacy (SP). https://doi.org/10.1109/sp40001.2021.00098
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.