CoChat
Sign Up
PreprintCited by 306Cryptography and Data SecurityPrivacy-Preserving Technologies in DataInternet Traffic Analysis and Secure E-voting

CryptoDL: Deep Neural Networks over Encrypted Data

Open access
Hesamifard, Ehsan, Takabi, Hassan, Ghasemi, Mehdi·2017·arXiv
DOI 10.48550/arxiv.1711.05189arXiv 1711.05189
Open this paper in CoChat — free accountOpen PDF

Abstract

Machine learning algorithms based on deep neural networks have achieved remarkable results and are being extensively used in different domains. However, the machine learning algorithms requires access to raw data which is often privacy sensitive. To address this issue, we develop new techniques to provide solutions for running deep neural networks over encrypted data. In this paper, we develop new techniques to adopt deep neural networks within the practical limitation of current homomorphic encryption schemes. More specifically, we focus on classification of the well-known convolutional neural networks (CNN). First, we design methods for approximation of the activation functions commonly used in CNNs (i.e. ReLU, Sigmoid, and Tanh) with low degree polynomials which is essential for efficient homomorphic encryption schemes. Then, we train convolutional neural networks with the approximation polynomials instead of original activation functions and analyze the performance of the models. Finally, we implement convolutional neural networks over encrypted data and measure performance of the models. Our experimental results validate the soundness of our approach with several convolutional neural networks with varying number of layers and structures. When applied to the MNIST optical character recognition tasks, our approach achieves 99.52\% accuracy which significantly outperforms the state-of-the-art solutions and is very close to the accuracy of the best non-private version, 99.77\%. Also, it can make close to 164000 predictions per hour. We also applied our approach to CIFAR-10, which is much more complex compared to MNIST, and were able to achieve 91.5\% accuracy with approximation polynomials used as activation functions. These results show that CryptoDL provides efficient, accurate and scalable privacy-preserving predictions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1711.05189

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Homomorphic encryption has successfully been applied to convolutional neural networks, and its benefits have been demonstrated in a machine learning as a service scenario.

    “Nevertheless, HE has successfully been applied to convolutional neural networks 70 , and its benefits demonstrated in a 'machine learning as a service' scenario 71 , whereby data is sent over the network to be processed on an off-site server (cloud computing).”

    From Secure, privacy-preserving and federated machine learning in medical imaging (2020) · cites this paper as [70]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 1 · Cited by 306

Research works printed but not linked: 35

  1. [2]2. L. J. M. ASLETT, P. M. ESPERANÇA, C. C. HOLMES, Encrypted statistical machine learning: new privacy preserving methods. CoRR abs/1508.06845 (2015). · arXiv
  2. [3]3. L. J. M. ASLETT, P. M. ESPERANCA, C. C. HOLMES, A review of homomorphic encryption and software tools for encrypted statistical machine learning. Tech. rep., University of Oxford, 2015.
  3. [4]4. K. ATKINSON, W. HAN, Theoretical Numerical Analysis: A Functional Analysis Framework. Texts in Applied Mathematics. Springer New York, 2009.
  4. [5]5. R. BOST, R. A. POPA, S. TU, S. GOLDWASSER, Machine learning classification over encrypted data. In 22nd Annual Network and Distributed System Security Symposium, NDSS, San Diego, California, USA (2015).
  5. [6]6. Y. CHEN, G. GONG, Integer arithmetic over ciphertext and homomorphic data aggregation. In Communications and Network Security (CNS), IEEE Conference on (Sept 2015), pp. 628-632.
  6. [10]10. T. ELGAMAL, A public key cryptosystem and a signature scheme based on discrete logarithms. In Proceedings of CRYPTO 84 on Advances in Cryptology (New York, NY, USA, 1985), Springer-Verlag New York, Inc., pp. 10-18.
  7. [11]11. C. GENTRY, A Fully Homomorphic Encryption Scheme. PhD thesis, Stanford, CA, USA, 2009. AAI3382729.
  8. [12]12. S. GOLDWASSER, S. MICALI, Probabilistic encryption & how to play mental poker keeping secret all partial information. In Proceedings of the Fourteenth Annual ACM Symposium on Theory of Computing (New York, NY, USA, 1982), STOC '82, ACM, pp. 365-377.
  9. [13]13. T. GRAEPEL, LAUTER, M. NAEHRIG, ML confidential: Machine learning on encrypted data. In Proceedings of the 15th International Conference on Information Security and Cryptology (Berlin, Heidelberg, 2013), ICISC'12, Springer-Verlag, pp. 1-21.
  10. [14]14. S. HALEVI, V. SHOUP, Algorithms in HElib. In Advances in Cryptology - CRYPTO - 34th Annual Cryptology Conference, Santa Barbara, CA, USA, Proceedings, Part I (2014), pp. 554- 571.
  11. [16]16. B. KARLIK, A. V. OLGAC, Performance analysis of various activation functions in generalized mlp architectures of neural networks. International Journal of Artificial Intelligence and Expert Systems 1, 4 (2011), 111-122.
  12. [17]17. M. KIM, K. E. LAUTER, Private genome analysis through homomorphic encryption. IACR Cryptology ePrint Archive 2015 (2015), 965.
  13. [19]19. F. LIU, W. K. NG, W. ZHANG, Secure scalar product for big-data in MapReduce. In Big Data Computing Service and Applications (BigDataService), IEEE First International Conference on (March 2015), pp. 120-129.
  14. [20]20. R. LIVNI, S. SHALEV-SHWARTZ, O. SHAMIR, On the computational efficiency of training neural networks. CoRR abs/1410.1141 (2014). · arXiv
  15. [21]21. W. -J. LU, Y. YAMADA, J. SAKUMA, Privacy-preserving genome-wide association studies on cloud environment using fully homomorphic encryption. BMC Medical Informatics and Decision Making 15, 5 (2015), 1-8.
  16. [22]22. P. PAILLIER, Public-key cryptosystems based on composite degree residuosity classes. In Proceedings of the 17th International Conference on Theory and Application of Cryptographic Techniques (Berlin, Heidelberg, 1999), EUROCRYPT'99, Springer-Verlag, pp. 223-238.
  17. [24]24. S. RANE, W. SUN, A. VETRO, Secure distortion computation among untrusting parties using homomorphic encryption. In ICIP (2009), IEEE, pp. 1485-1488.
  18. [25]25. R. RIVEST, L. ADLEMAN, M. DERTOUZOS, On data banks and privacy homomorphisms. In Foundations on Secure Computation, Academia Press (1978), pp. 169-179.
  19. [26]26. T. Shortell, A. Shokoufandeh, Secure signal processing using fully homomorphic encryption. In Advanced Concepts for Intelligent Vision Systems - 16th International Conference, ACIVS, Catania, Italy, October 26-29, Proceedings (2015), pp. 93-104.
  20. [27]27. M. Togan, C. Plesca, Comparison-based computations over fully homomorphic encrypted data. In Communications (COMM), 10th International Conference on (May 2014), pp. 1-6.
  21. [29]29. P. Xie, M. Bilenko, T. Finley, R. Gilda-Bachrach, K. E. Lauter, and M. Naehrig, Crypto-nets: Neural networks over encrypted data. CoRR abs/1412.6181 (2014). · arXiv
  22. [31]31. Y. Zhang, W. Dai, X. Jiang, H. Xiong, S. Wang, Foresee: Fully outsourced secure genome study based on homomorphic encryption. BMC Medical Informatics and Decision Making 15, 5 (2015), 1-11. Yuchen Zhang, Wenrui Dai contributed equally to this work.
  23. [37]37. M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, Deep Learning with Differential Privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS '16), Vienna, Austria. pp 308-318.
  24. [38]38. R. Shokri, V. Shmatikov, Privacy-Preserving Deep Learning. Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security (CCS '15). Denver, Colorado, USA. pp 1310-1321.
  25. [40]40. F. Tramèr and F. Zhang, A. Juels, M.K. ReiterT. Ristenpart, Stealing Machine Learning Models via Prediction APIs. 25th USENIX Security Symposium, USENIX Security 16, Austin, TX, USA, August 10-12, 2016. 601-618.
  26. [41]41. M. Fredrikson, S. Jha, T. Ristenpart, Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures. ACM Conference on Computer and Communications Security 2015, Denver, CO, USA, October 12-6, 2015, 1322-1333.
  27. [42]42. R. Shokri, M. Stronati, V. Shmatikov. Membership Inference Attacks against Machine Learning Models. arXiv preprint. arXiv: abs/1610.05820 (2016). · arXiv
  28. [44]44. L. J. M. Aslett, P. M. Esperanç, C. C. Holmes, Encrypted statistical machine learning: new privacy-preserving methods. CoRR, abs/1508.06845. 2015 · arXiv
  29. [46]46. J. Zhan, Using Homomorphic Encryption For Privacy-Preserving Collaborative Decision Tree Classification. 2007 IEEE Symposium on Computational Intelligence and Data Mining, 637-645.
  30. [47]47. Q. Zhang, L. T. Yang, Z. Chen, Privacy Preserving Deep Computation Model on Cloud for Big Data Feature Learning. IEEE Transactions on Computers, 2016.
  31. [48]48. F. Bu, Y. Ma, Z. Chen, H. Xu, Privacy Preserving Back-Propagation Based on BGV on Cloud, 17th IEEE International Conference on High Performance Computing and Communications, HPCC 2015, 7th IEEE International Symposiumon Cyberspace Safety and Security, CSS 2015, and 12th IEEE International Conference on Embedded Software and Systems, ICESS 2015, New York, NY, USA, August 24-26, 2015, 1791-1795, 2015.
  32. [50]50. B. D. Rouhani, M. S. Riazi, F. Koushanfar, DeepSecure: Scalable Provably-Secure Deep Learning, CoRR, abs/1705.08963, 2017. · arXiv
  33. [53]53. H. Takabi, E. Hesamifard, M. Ghasemi, Privacy Preserving Multi-party Machine Learning with Homomorphic Encryption, Private Multi-Party Machine Learning, NIPS 2016 Workshop, 2016.
  34. [54]54. E. Hesamifard, H Takabi, M. Ghasemi, CryptoDL: Towards Deep Learning over Encrypted Data, Annual Computer Security Applications Conference (ACSAC), 2016.
  35. [55]55. Z. Brakerski, C. Gentry, V. Vaikuntanathan, (Leveled) Fully Homomorphic Encryption Without Bootstrapping, Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS '12, 2012, 978-1-4503-1115-1, Cambridge, Massachusetts, 309-325, 17, ACM, New York, NY, USA,

Authors
Hesamifard, E., Takabi, H., & Ghasemi, M.
ORCID
Ghasemi, Mehdi
Venue
arXiv
Publisher
arXiv
Published
2017
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1711.05189
arXiv
1711.05189
OpenAlex
W2768347741
Cited by
306
References
1
Citations by year
2026: 1 · 2025: 14 · 2024: 37 · 2023: 45 · 2022: 30 · 2021: 67 · 2020: 49 · 2019: 43 · 2018: 19 · 2017: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Cryptography and Data Security
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Internet Traffic Analysis and Secure E-voting
Keywords
homomorphic encryption, deep neural network, privacy-preserving machine learning, convolutional neural networks, activation function approximation, polynomial approximation, ReLU, sigmoid, hyperbolic tangent function, encrypted data classification, MNIST, CIFAR-10

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review