QUOTIENT: Two-Party Secure Neural Network Training and Prediction
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Recently, there has been a wealth of effort devoted to the design of secure protocols for machine learning tasks. Much of this is aimed at enabling secure prediction from highly-accurate Deep Neural Networks (DNNs). However, as DNNs are trained on data, a key question is how such models can be also trained securely. The few prior works on secure DNN training have focused either on designing custom protocols for existing training algorithms, or on developing tailored training algorithms and then applying generic secure protocols. In this work, we investigate the advantages of designing training algorithms alongside a novel secure protocol, incorporating optimizations on both fronts. We present QUOTIENT, a new method for discretized training of DNNs, along with a customized secure two-party protocol for it. QUOTIENT incorporates key components of state-of-the-art DNN training such as layer normalization and adaptive gradient methods, and improves upon the state-of-the-art in DNN training in two-party computation. Compared to prior work, we obtain an improvement of 50X in WAN time and 6% in absolute accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“We adopt the state of the art floating point convolutional neural network LeNet [34] (32C5-BN32C5-BN-MP2-64C5-BN-64C5-BN-MP2-512FC-BN-DO-10SM)2 into a fixed-point equivalent of the form 32C5-MP2-64C5-MP2-512FC10MSE for secure training & inference.”
MethodXNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114
Tschandl, P., Rosendahl, C., & Kittler, H. (2018) · Scientific Data · doi:10.1038/sdata.2018.161 · Cited by 3,628
Goldreich, O., Micali, S., & Wigderson, A. (1987) · Proceedings of the nineteenth annual ACM conference on Theory of computing - STOC '87 · doi:10.1145/28395.28420 · Cited by 3,590
Quinlan, J. R. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80053-6 · Cited by 2,553 · cited 1×
“Five of these are privacy sensitive in nature, as they include health datasets—Thyroid [45], Breast cancer [8], MotionSense [38], Skin cancer MNIST [53] and a financial credit dataset—German credit [12].”
SecureML: A System for Scalable Privacy-Preserving Machine Learning
Mohassel, P., & Zhang, Y. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.12 · Cited by 1,936 · cited 10×
“The work most similar to ours is SecureML [41].”
InfluentialOn the Convergence of Adam and Beyond
Reddi, S. J., Kale, S., & Kumar, S. (2019) · arXiv · doi:10.48550/arxiv.1904.09237 · Cited by 1,771
Hubara, I., Soudry, D., & Yaniv, R. E. (2016) · arXiv · doi:10.48550/arxiv.1602.02505 · Cited by 1,667
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364 · cited 1×
“However, for the task of training, there are few works that perform all operations in fixed-point [20, 23, 33, 39, 57].”
BackgroundsignSGD: Compressed Optimisation for Non-Convex Problems
Bernstein, J., Wang, Y.-X., Azizzadenesheli, K., & Anandkumar, A. (2018) · arXiv · doi:10.48550/arxiv.1802.04434 · Cited by 541
Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2021068 · Cited by 346
Mobile sensor data anonymization
Malekzadeh, M., Clegg, R. G., Cavallaro, A., & Haddadi, H. (2019) · Proceedings of the International Conference on Internet of Things Design and Implementation · doi:10.1145/3302505.3310068 · Cited by 298 · cited 5×
“For MNIST and Breast cancer datasets QUOTIENT[AMSgrad=Ours, Norm=Ours] training achieves near state of the art accuracy levels for MNIST and Breast Cancer, while we differ by at most ∼2% for German credit, MotionSense and Thyroid datasets.”
BackgroundMethodInfluentialPrivacy-preserving distributed linear regression on high-dimensional data
Gascón, A., Schoppmann, P., Balle, B., Raykova, M., Doerner, J., Zahur, S., & Evans, D. (2017) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2017-0053 · Cited by 193 · cited 3×
“encryption and garbled circuits, while previous works [14, 41] rely in part on OT for the same functionality.”
BackgroundMethodSelf-citationKim, M., & Smaragdis, P. (2016) · arXiv · doi:10.48550/arxiv.1601.06071 · Cited by 166
Cited by
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Self-citationTrustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“To reduce the high costs of computation and communication of the SMPC, parameter quantization and function approximation were used together with cryptographic protocols in References [8, 32].”
MethodCrypTFlow2: Practical 2-Party Secure Inference
Rathee, D., Rathee, M., Kumar, N., Chandran, N., Gupta, D., Rastogi, A., & Sharma, R. (2020) · Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3372297.3417274 · Cited by 326
VeriFL: Communication-Efficient and Fast Verifiable Aggregation for Federated Learning
Guo, X., Liu, Z., Li, J., Gao, J., Hou, B., Dong, C., & Baker, T. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.3043139 · Cited by 314
VFL: A Verifiable Federated Learning With Privacy-Preserving for Big Data in Industrial IoT
Fu, A., Zhang, X., Xiong, N., Gao, Y., Wang, H., & Zhang, J. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3036166 · Cited by 288
Metadata
- Authors
- Agrawal, N., Shahin Shamsabadi, A., Kusner, M. J., & Gascón, A.
- Venue
- Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security
- Publisher
- ACM
- Published
- 2019-11-06
- Pages
- 1231-1247
- OpenAlex
- W2984644945
- Cited by
- 196
- References
- 16
- FWCI
- 9.52
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.